Skip to content

feat(transcript): declarative extractor tier for flat event streams#148

Merged
slowdini merged 1 commit into
devfrom
feat/transcript-extract-tier
Jul 16, 2026
Merged

feat(transcript): declarative extractor tier for flat event streams#148
slowdini merged 1 commit into
devfrom
feat/transcript-extract-tier

Conversation

@slowdini

Copy link
Copy Markdown
Owner

Summary

Closes #139 (part of #122, data-driven harness descriptors phase 2).

Adds a declarative [transcript.extract] tier to harness descriptors: flat event streams (every tool event self-contained) now get full transcript ingest — transcript_check grading, parsed invocations in run.json, token/duration backfill into timing.json — from descriptor data alone, with zero Rust. Streams that need cross-event state (keyed tool_use/tool_result joins, content coercion) remain named code capabilities; claude-stream-json is untouched.

User-facing changes

  • New descriptor surface: [transcript] now takes exactly one of parser (named capability) or extract (declarative tier). The extract block holds exactly the ticket's five primitives — equality where filter, last-match final-text pick, flat tool-item mapping, token sum, duration via field pick or timestamp spread — all subtle rules fixed and non-configurable.
  • Codex flipped: harnesses/codex.toml ingests through the extract block. codex-items stays as the reference implementation and remains referenceable by BYOH descriptors; a differential test pins the extract spec byte-equivalent to the code parser on the full fixture corpus.
  • Validation: new load-time invariants (exactly-one of parser/extract, non-empty extract block, exactly-one duration variant) with actionable messages, including the layering trap: overlays can't delete an inherited parser, so moving a harness to extract needs a new label.
  • Docs: worked example + primitive reference in docs/byoh.md ("The declarative extractor"), two-tier story in docs/progressive-enhancements.md, updated harness init template scaffold, and the descriptor PR template now counts extract blocks as data-only.

Before/after (BYOH)

Before — a harness with a novel flat stream had no transcript ingest without a Rust parser PR:

--harness cool-custom-harness declares no transcript parser — transcript_check
assertions will grade as unverifiable ... tokens/duration go unrecorded.

After — pure descriptor data:

[transcript]
events_filename = "cool-events.jsonl"

[transcript.extract.tools]
where = { kind = "tool.call" }
name_field = "tool"
result_coalesce = ["output"]

→ no warning; run.json carries parsed tool_invocations, timing.json carries tokens/duration (proven end-to-end by the new tests/run/byoh.rs integration test).

Verification

  • cargo test — 619 tests pass (453 lib / 96 cli / 70 run), including the new engine units, the codex differential-equivalence test, validation rejects_* suite, BYOH end-to-end, and lint CLI tests; all existing codex pipeline/golden tests pass unchanged through the extract engine (no golden re-bless needed)
  • cargo fmt --check / cargo clippy --all-targets -- -D warnings — clean
  • Manual: harness lint codex passes; harness show codex round-trips the resolved extract block as loadable TOML; harness list still declares transcript for codex

Schema / docs callouts

  • schema/harness-descriptor.schema.json: transcript.required relaxed to events_filename; new fully-described extract sub-schema (exactly-one rules live in Rust invariants, matching the slug_template/slug_capability precedent)
  • Follow-up candidate: OpenCode has no transcript ingest today — if its stream is flat, it's now a pure-data descriptor change

🤖 Generated with Claude Code

Add a [transcript.extract] descriptor block so harnesses with flat event
streams (every tool event self-contained) get full transcript ingest —
transcript_check grading, parsed invocations in run.json, token/duration
backfill into timing.json — from descriptor data alone, no Rust.

- New generic engine (src/adapters/extract.rs) interpreting exactly five
  primitives: dotted-path string-equality `where` filter, last-match
  final-text pick, flat tool-item mapping, token sum, duration via field
  pick or timestamp spread. Cross-event joins and content coercion stay
  named code capabilities (claude-stream-json unchanged).
- TranscriptSection now declares exactly one of parser/extract; schema
  permits both, a load-time invariant rejects both/neither (plus empty
  extract blocks and duration with both/neither variants) with
  actionable messages.
- harnesses/codex.toml flips to the extract tier; the codex-items code
  parser stays as the reference implementation, pinned equivalent on the
  full fixture corpus by a differential test.
- BYOH integration test proves a project descriptor over a novel flat
  stream yields zero-code ingest end-to-end; lint tests cover the new
  invariants; template scaffold, byoh guide (worked example), and
  progressive-enhancements docs updated to the two-tier story.

Closes #139

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@slowdini
slowdini merged commit 12fb7b9 into dev Jul 16, 2026
6 checks passed
@slowdini
slowdini deleted the feat/transcript-extract-tier branch July 16, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(transcript): declarative extractor tier for flat event streams (stretch)

1 participant