Skip to content

feat: one-call failing-test debugging (test-runner presets) #985

Open

Description

@debugmcpdev

Problem

The most common debugging entry point is a failing test, and translating a test invocation into a debug launch takes runner-specific knowledge an agent (or human) shouldn't need: worker flags (--runInBand for Jest, single-thread flags for Vitest), module launch (python -m pytest), or locating hashed test binaries (cargo test --no-run). Today that translation is manual and error-prone; done wrong, breakpoints silently don't bind (worker pools) or the launch is refused.

Direction

A preset layer that maps (runner, target) → the adapter's launch config, so the intent is one call:

{ "testRunner": "pytest", "testTarget": "tests/test_billing.py::test_proration_rounding" }

With #982's snapshot, the launch answer is the autopsy: the one call ends paused at the failure with exception + frames + locals in the response.

In-repo precedent: packages/adapter-dart/src/runner.ts already auto-detects dart/dart-test/flutter/flutter-test from the pubspec and program path — this issue generalizes that idea. Prior art: .github/actions/debug-failing-test (#241, experimental).

Shape decision up front: new debug_test tool vs. preset args on start_debugging. A 30th tool touches the tool-count pins (server-required-arguments.test.ts, 24 tool-list snapshots, ~11 docs stating "29 tools") and must be decided before the 1.0 tool-schema freeze (ROADMAP). Preset args on start_debugging add no tool and keep the schema surface smaller — current lean, to be confirmed in M1.

Milestones (#759-style)

  • M0 — debugpy module launch. Blocked today: the launch path always sends program (proxy-launcher.ts sets it from scriptPath; dap-proxy-connection-manager.ts puts it on the wire), and debugpy rejects program+module ("mutually exclusive"). Allow a module-mode launch config through without program. (Workaround exists — launching pytest's __main__.py as the program — but carries a sys.path[0] difference vs python -m pytest.)
  • M1 — pytest. -m pytest <target> -x + the assertion-stop question: the uncaught default never pauses on an assert pytest catches; debugpy's userUnhandled filter exists but is deliberately unexposed ("noisy in practice", adapter-policy-python.ts). Spike: scoped userUnhandled for test-preset sessions only, measured against real pytest suites before any claim.
  • M2 — jest / vitest. ChildSessionManager adopts exactly ONE child (child-session-manager.ts releases the rest undebugged) → presets must force single-worker execution (--runInBand; vitest single-thread/no-file-parallelism). Ship with measured per-runner caveats, not promises.
  • M3 — cargo test / go test. cargo test --no-run + hashed-binary resolution (the dead cargo.test branch in rust-debug-adapter.ts:776-798 and the uncalled runCargoTest in cargo-utils.ts should be revived or deleted as part of this); go test -c.
  • M4 — rspec / dotnet test / minitest.

Each milestone lands with spike-notes-style measured facts (what binds, what doesn't, which flags are mandatory).

Activity

  1. debugmcpdev commented on Oct 26, 2026

    @debugmcpdev
    CollaboratorAuthor

    Status 2026-10-25: M0 merged, M1 in review

    Measured (docs/python/spike-notes-pytest.md in #994)

    • uncaught alone never pauses on a failing test (pytest catches it). debugpy's userUnhandled does: at the assert in the test frame with locals; for a raise in a helper, the raise-site frames sit above the live test frame (locals readable); never inside pytest.raises; once on pytest.skip(); in the fixture for a fixture error. Whole fixture file: 13 tests → 8 stops (6 failures + 1 fixture error + 1 skip), 0 on the 5 passing tests.
    • "all" (debugpy raised) is not a usable default: three false stops across two passing tests, one per user frame an expected exception crosses.
    • pydevd suffixes the exception name with (note: full exception trace is shown but execution is paused at: …) on text/exceptionId/typeName; the snapshot strips it. The frame name carries (Current frame).
    • Plugins: xdist dies at worker spawn (WinError 6: the debuggee runs under pythonw.exe, python (Windows): the debuggee runs under pythonw.exe, so a subprocess that inherits stderr fails with WinError 6 (pytest-xdist workers) #993) and would run the test undebugged anyway; cov coexists on 3.13 (unmeasured on ≤3.11). -p no:xdist -p no:cov is silent when absent, clean when present; the preset passes both unless the caller's args ask for the plugin.
    • pytest's capture holds a test's print until the report; -s is not forced (args: ["-s"] opts in).

    Shape M1 fixes for M2–M4

    1. testRunner/testTarget as optional start_debugging arguments, folded into adapterLaunchConfig at intake (no new tool; the facade and restart_debugging unchanged).
    2. AdapterModes.testRunners on the factory metadata (mirrored in the loader's list, shown by list_supported_languages, pinned in adapter-attach-contract.test.ts), and classifyTestRunner (src/session/launch/test-runner-gate.ts) refusing an undeclared runner by name before the session changes state.
    3. The adapter's transformLaunchConfig owns the expansion (packages/adapter-python/src/test-runners.ts is the template: node id, the runner's single-process flags, a rootdir walk for cwd, one launch-scoped note), with the keys in consumedLaunchKeys.
    4. AdapterPolicy.exceptionFiltersForLaunch(mode, launchConfig) for runner-specific filters; the launcher records what the launch armed beyond the mode, and the pending answer names it.
    5. A spike-notes file per milestone with the measured matrix before any claim.

    Open after M1

  2. debugmcpdev commented on Nov 2, 2026

    @debugmcpdev
    CollaboratorAuthor

    Status note (2026-11-02): the "shape decision up front" in the body is made — M1 (#994) landed testRunner/testTarget as start_debugging arguments, AdapterModes.testRunners on the factory metadata, the policy-owned expansion, no new tool. Two things for the remaining milestones:

  3. debugmcpdev commented on Nov 5, 2026

    @debugmcpdev
    CollaboratorAuthor

    The session tree's last step (#1091) lifts the gate this issue names: pytest-xdist workers are child debug sessions by default now — python -m pytest -n 2 yields two, and the worker running the test stops on a breakpoint set before the run (tests/adapters/python/integration/python-pytest-xdist.test.ts; pytest-xdist is in requirements/pytest.txt). The preset keeps -p no:xdist for now: a worker's stop is reported by wait_for_stop, not by the launch answer the one-call autopsy promises, so turning xdist back on in the preset is this issue's call, not the tree's.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions