[feat] Pluggable zero-token verifiers for verifyTaskflow / compileTaskflow

[hirisov]

What problem does this solve?

verifyTaskflow is a great zero-token structural audit (dead-ends, unreachable, gate-exhaustion, budget-overflow, concurrency, guard-contradictions, contract-ref mismatches) — but it stops at the graph. The contents of a script phase's run command are unchecked beyond shape validation in validateTaskflow (presence of run, no interpolation in the string form). So a flow can pass /tf compile clean and then fail at runtime for a reason that was statically knowable before any model was spawned — the opposite of the "verify before spend" thesis.

Concrete class of misses I hit: script gates that run a project's own test/build command. A few things that are 100% statically detectable, zero-token, and runner-agnostic:

• a grep whose pattern starts with - with no -- separator (grep parses it as flags → exit 2, a false RED);
• a grep/sed with an invalid regex under its dialect flags (exit 2);
• runner | grep … without pipefail / PIPESTATUS (the filter's exit masks the runner's — a failing runner can read as GREEN);
• a suite/smoke script that names a per-task test file the flow itself creates in an earlier phase → RED until that phase runs, RED-forever if it's skipped.

These are all "the plan is data, so it can be verified" — just at the command-string layer instead of the DAG layer.

Current workaround

Run my own verifier as a separate pre-flight step in my own extension before calling /tf run — i.e. re-implement verifyTaskflow's aggregation and call site wiring myself, outside the engine. It works, but it's a parallel verification path that doesn't flow into /tf compile's Mermaid overlay, /tf verify's report, or the runtime's pre-flight (runtime.ts calls verifyTaskflow before execution). The diagnostics live in a different place than the built-in ones, which is worse for users than having them in one report.

Proposed solution

A pluggable verifier seam on the existing public API, matching the pure-function, zero-dependency shape verify.ts already has:

1. Add an optional second argument to verifyTaskflow:

   export type Verifier = (flow: VerifiableFlow) => VerificationIssue[];
   export function verifyTaskflow(
     flow: VerifiableFlow,
     verifiers?: Verifier[],
   ): VerificationResult;

   Built-in detectors run first (unchanged), then each caller-supplied verifier runs against the same sanitized safeFlow; issues are merged into the singleVerificationResult. Verifiers are pure (no I/O), matching verify.ts's existing "no I/O" contract.

2. Thread it through CompileOptions so /tf compile overlays them on the Mermaid diagram like any other issue:

   export interface CompileOptions {
     direction?: "TD" | "LR";
     title?: string;
     verifiers?: Verifier[];   // forwarded to verifyTaskflow
   }

3. Thread an optional verifiers through the runtime pre-flight call sites (runtime.ts, the two verifyTaskflow(...) calls before execution) so a host embedding the engine can register domain verifiers that block spend.

The one open typing question is IssueCategory, which is a closed union today. Two options, happy with either:

• add a generic "custom" (or "plugin") category, or
• widen category to string (built-ins keep their union as the documented set).

Discovery / loading is the part I'd most like your call on. Minimum viable is programmatic-only (a host/extension passes verifiers when it calls the engine — this is all I personally need). If you want /tf verify and /tf compile to pick up project-local verifiers too, options are a convention dir (e.g. taskflow.verifiers/ auto-loaded), a named registry in a config file, or neither (programmatic only).
I'd suggest shipping programmatic-only first and deciding discovery later.

Alternatives considered

• gate.eval / gate.score: these judge a phase's output (or an upstream output target) with a fixed operator set (eval) or a fixed 6-type scorer set (score). Neither lints a script phase's run definition before execution, and neither allows arbitrary logic — which is what command-string lint needs. Wrong layer.
• A new phase type (lint): heavier than necessary and doesn't run at compile time. The checks belong next to the existing structural detectors, not in the runtime. External pre-flight only (the current workaround): works but fragments the verification surface and misses the runtime pre-flight + the Mermaid overlay.

Is this a good fit?

• It fits the declarative JSON-DSL model — it lints declarations, at the same zero-token compile stage as the existing structural detectors.
• It doesn't require new runtime dependencies — verifiers are plain pure functions; the built-in detectors stay zero-dependency.
• It's adjacent to "orchestration concern" — it's a verification/planningconcern rather than orchestration proper, but it lives in the verify/compile seam that already gates orchestration, so I think it belongs here.

If this lines up with the roadmap I'm happy to open a PR — small, focused (verify.ts and compile.ts and threading the two runtime call sites + tests under packages/taskflow-core/test/), one concern. Naming (verifiers vs validators vs checks) and the discovery question above I'll follow your preference on.

[heggria]

Yes — this is a good fit, and I’d be happy to review a focused PR.

For v1, please keep the scope to the programmatic Core API only: no project-local verifier discovery and no built-in shell-command lint rules yet.

I’d like to defer those two parts for separate reasons:

1. Project-local discovery requires a trust model. A verifier is executable code. Automatically loading verifier modules from a checked-out repository would mean that /tf verify or /tf compile could execute arbitrary project code. Before supporting that, we need to define explicit trust/consent, allowed locations, module loading, failure isolation, and consistent behavior across all hosts. With programmatic registration, the embedding host explicitly chooses which verifier code it trusts.

2. Shell linting is not reliably runner-agnostic. Rules involving grep, sed, pipefail, or PIPESTATUS depend on the operating system, shell, command implementation, and regex dialect. Some checks—such as whether an earlier phase creates a referenced test file—also require effect information that is not currently declared in the flow. Those rules may still be valuable, but they should first live in domain-specific verifiers so we can learn which checks are portable enough to promote into Core later.

The v1 seam is independently useful: callers can register trusted verifiers once and have their diagnostics merged into the normal verification and compile results, without committing Core to a discovery mechanism or shell-analysis policy.

For the API, I suggest:

• an extensible options shape such as verifyTaskflow(flow, { verifiers });
• a named TaskflowVerifier;
• a closed IssueCategory with a plugin category and verifier/source attribution;
• fail-closed handling for throwing or malformed verifiers, normalized into an error issue;
• propagation through compileTaskflow and runtime embedding paths, including imperative and event-kernel dynamic subflows;
• explicit tests for ordering, warning/error semantics, verifier failure, compile overlays, and no-spend runtime blocking.

One implementation note: the current runtime.ts verification calls cover dynamic inline definitions rather than the top-level flow, and the event-kernel path has its own call site. Please make any top-level preflight behavior explicit and test both execution paths.

We can design discovery/loading and consider portable built-in command checks in follow-up work, once their trust and portability boundaries are clear.

[hirisov]

Thanks @heggria — opened #84 with the v1 scope exactly as you scoped it: programmatic Core API only, no project-local discovery, no built-in shell-command lint.

The seam matches your guidance:

• verifyTaskflow(flow, { verifiers }) — extensible options shape.
• a named TaskflowVerifier ({ name, verify }).
• closed IssueCategory + a plugin category, with source attribution on every verifier issue.
• fail-closed: throwing/malformed verifiers normalize to one error/plugin issue, then continue.
• propagated through compileTaskflow and the runtime embedding paths — both imperative and event-kernel dynamic subflows.

On your implementation note: the two existing runtime.ts verify calls do cover dynamic inline definitions (as you said), and exec/step-kinds.ts is the event-kernel call site. For the top-level flow I added an explicit, opt-in preflight in executeTaskflow that runs before either engine dispatches and blocks only on error-severity plugin issues — built-in detectors stay advisory at the top level, so existing flows are unaffected. It's gated on deps.verifiers, so hosts that register none see no change. Both execution paths are tested.

Happy to adjust shape or naming on review.