Proposal: a lightweight deviation channel to close the implement↔design feedback loop

[orangemike]

Category: Enhancement / Process

Summary

In the standard Spec Kit flow (specify → clarify → plan → tasks → implement → converge), the implement↔converge loop closes on implementation coverage, but there is no formal channel for the implementer to report back when a design assumption from spec.md or plan.md does not hold against the actual codebase. Today the implementer faces two imperfect options: silently amend the blueprint to match what was actually built, or push through with a workaround and note it in tasks.md. This proposal sketches a small extension mechanism for that channel.

Observed gap

When an implementer discovers during implementation that a stated requirement or design choice conflicts with reality (for example, an external dependency behaves differently than the plan assumed, or a specified error handling strategy is not achievable with the chosen stack), the forward flow has no defined place for that feedback. The analyze stage exists, but it runs after implementation and reports findings; it does not route a decision back to the blueprint authors.

The result is that the forward design stages are treated as if they are correct on first pass, and the implementer either absorbs the mismatch quietly or rationalizes it in the task log. Over a long-running feature this is how "the code drifted from the spec" happens without anyone explicitly deciding to drift.

Proposed mechanism

A per-feature deviations.md log with three commands:

1. propose — the implementer files a [PENDING] proposal when it finds a mismatch. The proposal cites the conflicting blueprint section, the evidence (file paths / line numbers / probe output), and a suggested direction. The implementer does not decide.
2. review — run by an independent context (not the same session as the implementer). It verifies the evidence against the actual files and then either amends the blueprint, amends the code, or records a rejection.
3. check — a gate invoked by before_implement and after_converge hooks. If any proposal is still [PENDING], the gate halts.

A two-tier split

Not every mismatch should wait for a human. We suggest each proposal carry a tier:

• tier: spec — the change would alter user-visible behavior, a business rule, an error contract, or an external API boundary. These should wait for the requirement owner. The reviewer verifies evidence and summarizes, but does not adjudicate.
• tier: plan — the change is internal architecture or implementation approach (file layout, internal structure, choice of data structure). An independent AI reviewer can adjudicate these directly.

The tier assignment is made by the implementer at proposal time against a small spillover test (would a caller notice? does a contract change?). When in doubt, assign spec.

Why this is an extension, not core

The Spec Kit extension model already supports before_implement / after_converge hooks and per-project command registration. This mechanism is small enough to ship as an extension without touching the core prompts. It also leaves the design intent untouched: Spec Kit's core flow remains as it is; the extension adds a feedback channel only when a project opts in.

Open questions

• Should tier: spec be a hard blocking gate, or advisory?
• Should a rejected proposal block re-filing (to avoid loops), or allow re-filing with new evidence?
• Is there a need for a lightweight deterministic check (e.g. parsing deviations.md for [PENDING]) instead of relying on the LLM to run the gate honestly?

We have prototyped this as a local extension and are opening this discussion to hear whether this is a pattern other teams have run into, and whether it fits the project's direction before investing in a clean upstream contribution.

[mnriem]

Thanks for the detailed write-up. Before we assess the reported gap, could you share the output of specify version, whether you ran specify integration upgrade after updating the CLI, and which /speckit.converge command or skill is actually installed in the project? The installed command files may differ from the CLI version.

Could you also walk us through one concrete instance: the relevant statement in spec.md or plan.md, what you discovered during implementation, the commands you ran, what converge reported, and what you expected it to report? Our Evolving Specs guide describes several ways to reconcile implementation discoveries with the artifacts. Which approach were you using, and where did it stop meeting your needs?

[orangemike]

---

Thanks for the thorough response. Answering your diagnostic questions first, then describing what actually happened.

Version and install

CLI Version: 1.0.10
Python: 3.12.13
Platform: Windows (AMD64)
OS: Windows 10.0.19045

The /speckit-converge installed in the project is the current CLI template; specify integration upgrade was run after updating.

The workflow as run

I followed Flow-Forward Spec:

• specify / clarify / plan / tasks were completed in Codex CLI with sol-5.6 high.
• implement was then run with luna-5.6 max, but was interrupted halfway through (unexpected shutdown), leaving partial code.
• I switched to DSH + DeepSeek Flash to continue the remaining implement work and converge.
• An unattended goal loop ran for about 3.5 hours straight. After I returned, I manually ran /speckit.analyze twice, /speckit.clarify once, and several rounds of converge → implement.

One concrete example

plan.md originally promised "controllers grouped by business domain, each with its own handler."

During implementation, a technical constraint surfaced, and the implementer chose a different architecture: a single ApiController entry point + route dispatcher + per-route adapters. The engineering judgment itself was reasonable, but it meant the "per-domain controller" commitment in plan.md no longer held.

What happened next:

1. The implementer wrote "as built" notes in tasks.md to explain this change, rather than stopping to ask. Mid-run, it even noted in its own summary that "this deviates from the wording of the original task."
2. After I returned, it backfilled plan.md with an "As built" section and rewrote the corresponding task's acceptance criteria to say "thin path declarations satisfy the requirement."
3. converge ran in the same context. It only checked that the files named in tasks.md existed on disk, that tests were green, and that there were no dead references in docs. It never opened the controller files to read their contents. It then reported converged.
4. When I later opened the controller directory myself, I found that all ten domain controllers contained only a public static final String[] PATHS constant and no handlers.

Why the existing flow didn't catch it

I read the Evolving Specs guide. The Flow-Back model is directionally right, but it didn't trigger automatically in this scenario:

• Flow-Back's steps are written as human actions: "when you notice a mismatch, revise spec.md, rerun plan/tasks, run analyze."
• In an unattended goal loop, the agent doesn't open that guide on its own. implement.md only says "validate against specification" — it doesn't say what to do when validation fails.
• So the agent itself noticed the deviation, but its training instinct was to finish the task. It resolved the contradiction internally (by writing notes in tasks.md) instead of stopping.
• converge, sharing the same context, naturally trusted the explanation already written in tasks.md.

This isn't a core design error. The failure mode requires a specific confluence of factors: cross-model mid-implement handoff, an unattended long run, and converge sharing context with implement. With the same model in the same interactive session, the probability is much lower.

It does raise one question we couldn't answer from the docs: what observable signal should trigger Flow-Back in the first place? The guide says what to do once you've noticed, but not what an agent should be looking for while implementing to know it has hit the point where Flow-Back is required.

Our direction

We prototyped a lightweight extension that doesn't modify the core prompts, but adds a feedback channel via hooks:

• When implement finds a mismatch between the blueprints and reality, it can't write "as built" notes in tasks.md. It must file a [PENDING] proposal in deviations.md with evidence.
• Each proposal carries a tier: spec-tier (affects user-visible behavior, business rules, external contracts) waits for a human; plan-tier (internal structure, file organization) is adjudicated by an independent-context AI reviewer.
• before_implement and after_converge register as gates that block while any [PENDING] proposal exists.

One question before we invest more:

Is there an existing mechanism we've missed that covers "implement must stop when it finds a contradiction between the blueprints and reality"?

[mnriem]

Thanks for walking through the example. To answer your question directly: Spec Kit does not default to stopping implement midway through tasks.md for a design decision. Its default is to implement the requested tasks. If you want a decision point after each task, you can scope an invocation to one task and return, or use an implement preset to make that task-by-task behavior consistent across runs. That seems like the right first experiment for your unattended workflow, rather than a change to the core command.

You could also use a converge preset if you want a stricter, evidence-heavy review tailored to your architecture. Default converge already instructs the agent to inspect relevant code against the current spec, plan, and tasks; it is intended as a whole-feature assessment after an implementation pass, not a gate after every individual task. If the plan and task acceptance criteria were rewritten before converge ran, it would assess those revised artifacts—it cannot independently recover the earlier commitment. That makes review of the artifact changes important here.

If you still find value in a durable deviation log and separate adjudication, an extension could add those commands later; a workflow could add explicit human checkpoints, and a bundle could package the chosen setup. I would start by testing the preset-based, task-bounded flow before adding those pieces. To understand the “converged” result, could you share the redacted original and revised plan/task wording alongside a representative controller and the convergence findings? In particular, that would tell us whether converge missed a contradiction still present in its inputs, or assessed an already-revised plan.

Disclosure: Posted on behalf of @mnriem; drafted with GitHub Copilot (GPT-6 Sol, interactive and human-guided). The response text was AI-generated and revised with human direction.

[scrossen]

An empty deviation queue only proves that no pending entry is recorded; it doesn't prove that the original contract was delivered. For mnriem's task-bounded preset experiment, I'd pin the pre-run spec.md/plan.md/tasks.md revision and keep adjudications separately, referencing both that baseline and any approved replacement. That preserves the original-versus-revised evidence the maintainer asked for.

For the deterministic gate, useful negative fixtures would be:

• A pending proposal, or a spec-tier proposal closed without the requirement owner's decision.
• A decision whose referenced baseline or approved replacement doesn't match the artifacts being accepted.
• Green functional checks alongside a missing, empty or stale saved deliverable. Inspect the saved output itself, rather than only regenerating an output from the code.

For re-filing, a new proposal could reference the rejected one and attach new evidence, preserving the earlier decision instead of rewriting it. These are proposed fixtures for the preset experiment; I haven't run them in Spec Kit.

One concrete external example: a documented Talavine Code handoff attempt later passed functional checks, but final host acceptance wasn't completed and its saved HTML was empty. The report retains that failure account. It separates a real Claude quota stop before tools or edits from a deliberately staged partial Qwen-to-Codex handoff. The successful app passed 329 functional checks on one app, not 329 trials; a manual written Codex handoff control passed too. These observations establish no comparative quality, speed or cost advantage.

Exercise, verifier and saved-output evidence: https://talavine.com/learn/read/coding-agent-handoff-empty-html-deskquote

Disclosure: posted on behalf of Scott Crossen (scrossen), maker of Talavine Code. Codex (GPT-6 family; exact model variant and reasoning setting unavailable in this session) autonomously reviewed the thread and sources, checked the evidence and drafted this comment. No human line-by-line review or Spec Kit execution is claimed.

[markuswondrak]

Just for clarification: Isn't this what analyze (run after tasks) for?