[v2] Structured output package: schemas, sentinel files, JSON/YAML validation

[devzeebo]

Part of the Orchestrator v2 rebuild — a shared library in orchestrator-v2/packages/ that defines how agents declare, produce, and validate structured outputs. Complements the verification contract discussed in #28 by giving Task and Workflow Agents a concrete, machine-checkable output format.

Problem

Task Agents produce free-form LLM text today. Downstream steps (and Workflow Agent verify dispatches) have no shared way to know what shape of output was expected, where it should land (e.g. a sentinel file on disk), or whether the produced artifact is valid. Without this, workflows cannot reliably gate on structured results, and task prompts cannot be augmented with consistent output instructions.

Proposed solution

New @bifrost-ai/structured-output package (name TBD) providing:

Core concepts

• Output schema — a declarative description of the expected structure (fields, types, required/optional, constraints). Serializable to and from JSON and YAML.
• Sentinel files — convention for marker files (path + format) that tell a Task Agent where and how to write its structured result. The package should define the sentinel file format, discovery helpers, and prompt augmentation (turning a schema into agent-facing instructions).
• Validation — parse a candidate artifact (file contents or in-memory string) against a schema; return a structured pass/fail result with actionable errors (path, message, expected vs actual).

Consumers

Consumer	Use
Task Agent (agent-3-task)	Read sentinel/schema metadata from task inputs; augment the prompt with output instructions; optionally write and self-validate the result before returning completed.
Workflow Agent (agent-4-workflow)	On the verify dispatch, validate each child step's structured output against its declared schema before marking the workflow complete.
Task sources / scripts	Attach output expectations to task metadata without re-implementing parsing or validation.

Suggested API surface (illustrative)

// Define or load a schema (JSON or YAML)
const schema = StructuredOutputSchema.fromYaml(yamlText);

// Sentinel: e.g. `.bifrost/output/research.yaml` + embedded or referenced schema
const sentinel = SentinelFile.parse(path, contents);

// Augment a task prompt
const augmentedPrompt = augmentPrompt(basePrompt, sentinel);

// Validate produced output
const result = validate(contents, schema, { format: "yaml" });
// → { ok: true } | { ok: false, errors: [{ path, message, ... }] }

Exact API and sentinel on-disk layout are implementation details — the issue is the shared contract, not a specific file extension.

Alternatives considered

• Ad hoc per-agent validation — each agent parses and checks its own output. Rejected; duplicates logic and prevents Workflow Agent from validating generically.
• JSON Schema only, no YAML — rejected; YAML is a common agent output format and should be first-class (parse + validate, not just JSON).
• Bake validation into interfaces-task — rejected; structured output is an optional concern layered on top of the script task primitive, not part of the core execution contract.

Area

Orchestrator v2 / structured-output (supports #37, #39, and #28)

Acceptance criteria

• Package publishes types and runtime helpers for defining, loading, and serializing output schemas in JSON and YAML.
• Sentinel file format is documented; helpers exist to discover sentinel metadata and produce agent-facing output instructions from it.
• validate() (or equivalent) returns structured errors for missing fields, type mismatches, and parse failures.
• Unit tests cover round-trip schema serialization, valid/invalid YAML and JSON artifacts, and sentinel-driven prompt augmentation.
• No dependency on LLM engines or orchestrator runtime — usable from Task Agent, Workflow Agent, or plain scripts.

Dependencies

• [v2] Script task execution primitive ([v2] Script task execution primitive #32)
• [v2] Task Agent ([v2] Task Agent: LLM execution as a script #37) — primary consumer for prompt augmentation and self-validation
• [v2] Workflow Agent ([v2] Workflow Agent: DAG scheduling with dependencies + retries #39) — primary consumer for verify-dispatch validation

Can be implemented in parallel with agent packages; integration follows once agents land.

Verification

vp test + vitest-gwt in orchestrator-v2/packages/structured-output:

• Given a YAML schema → When serialized to JSON and back → Then equivalent.
• Given a valid JSON artifact + schema → When validated → Then ok: true.
• Given a malformed YAML artifact → When validated → Then ok: false with parse error details.
• Given a sentinel file + base prompt → When augmented → Then prompt includes format, path, and field expectations.

Related

• [feat] A trustworthy-completion / verification contract. what does pass mean? #28 — trustworthy-completion / verification contract (higher-level "what does pass mean?")
• [v2] Task Agent: LLM execution as a script #37 — Task Agent
• [v2] Workflow Agent: DAG scheduling with dependencies + retries #39 — Workflow Agent (verify dispatch checks child outputs)