Post verification tasks (e.g. linting, unit testing etc) as Workflow Step vs embedding them in speckit commands/skills

[ksachdeva]

Hi,

Thanks for this great product. I love the extensibility aspect of spec-kit.

At the moment, I rely on Agent SKILLs or Prompt files (mostly in vscode) to provide both the context and the workflow

Context:

• Design decisions
• Where is what (i.e. location of projects/file)
  Workflow:
• After the implementation run following commands - "lint", "fmt", "pytest" etc

Because of both context and workflow specific instructions, Agent writes code and then lint/format/test it as well. If it find issues then it goes back and update the code and loop until done.

---

In the spec-kit workflow universe, I could see having "linting", "running of tests" etc as a step in the workflow.

But then at the same it bothers me that if the agent is not going to trigger those scripts (linting, formatting, pytest etc), it would not be itself able to verify if the task has been properly verified/tested.

Please suggest if this understanding is correct or not. I do not yet know the internals of spec-kit so perhaps missing the trick that makes concern invalid.

Regards
Kapil

[benitoalba]

Hi @ksachdeva! I'd keep the fix loop you described and add a workflow check afterward.

The current speckit.implement template already asks the agent to verify tests pass. Keep your lint, format, and test commands in the instructions it loads so it can fix failures before finishing.

A workflow shell step adds an independent check, the workflow documentation shows linting after the implement step. It won't automatically send failures back to the agent. By default, a failed step stops the workflow.

For automatic repairs, use continue_on_error: true on the check, pass its stdout/stderr to an agent prompt step on failure, and rerun the checks in a bounded loop, with an explicit failure path when retries run out.

[markuswondrak]

Hi Kapil — your understanding is correct, and workflows are exactly the right place for this. The question isn't who executes lint/pytest, it's who decides the outcome.

The problem with embedding verification in commands/skills

If the agent runs lint and judges the result in the same probabilistic loop, it's certifying its own work. It usually complies, but under a long context with competing signals the instruction loses weight, and the agent reports "tests pass" as plausibly as anything else. You get a claim, not a fact — and no signal when it degrades. (I wrote about the mechanics here: https://markus.wondrax.cloud/articles/deterministic-pipelines.html)

Why workflow steps are the answer

The workflow engine is a deterministic runtime; AI calls are just one step type among several. Verification becomes a shell step the agent cannot skip:

- id: verify
  type: shell
  run: "pytest -q && ruff check ."
  continue_on_error: true     # fails the step, not the run

- id: fix-loop
  type: do-while
  max_iterations: 3           # hard ceiling, not a suggestion
  condition: "{{ steps.verify.output.exit_code != 0 }}"
  steps:
    - id: fix
      command: speckit.implement
      input:
        args: "Verification failed. Fix the cause, don't silence it:\n{{ steps.verify.output.stdout }}"

The exit code is a runtime fact. max_iterations bounds the loop regardless of what happens inside. The agent never decides whether its own work passed — it gets handed the failure text and is re-entered for that one job. So your concern inverts: the agent doesn't need to trigger the scripts itself, because the runtime checks on its behalf and feeds failures back automatically.

Honest caveat

The built-in speckit workflow (specify → gate → plan → gate → tasks → implement) doesn't include a verify step yet — the primitives exist, the default pipeline doesn't use them. Which is where overlays come in:

# verify-overlay.yml
id: verify-after-implement
extends: speckit
edits:
  - operation: insert_after
    anchor: implement
    step:
      id: verify
      type: shell
      run: "./gradlew test"
      continue_on_error: true

specify workflow overlay add verify-overlay.yml

Overlays compose onto an installed workflow without editing workflow.yml and survive workflow/bundle updates — so lint/format/test stay a team-level property you apply consistently across projects instead of living in per-project skill files.

One caveat: shell steps run with your privileges (no sandbox), so never interpolate agent output into run — pass it as command step args as above.

Net: a tool call from inside the agent loop is still the agent checking its own homework. A runtime-owned verification step is the check it can't skip, and an overlay is how you ship it without forking.

---

Written with AI assistance (pi coding agent, model GLM-5.3, supervised). Research, structure, and every claim reviewed by me.