feat(transcript): declarative extractor tier for flat event streams#148
Merged
Conversation
Add a [transcript.extract] descriptor block so harnesses with flat event streams (every tool event self-contained) get full transcript ingest — transcript_check grading, parsed invocations in run.json, token/duration backfill into timing.json — from descriptor data alone, no Rust. - New generic engine (src/adapters/extract.rs) interpreting exactly five primitives: dotted-path string-equality `where` filter, last-match final-text pick, flat tool-item mapping, token sum, duration via field pick or timestamp spread. Cross-event joins and content coercion stay named code capabilities (claude-stream-json unchanged). - TranscriptSection now declares exactly one of parser/extract; schema permits both, a load-time invariant rejects both/neither (plus empty extract blocks and duration with both/neither variants) with actionable messages. - harnesses/codex.toml flips to the extract tier; the codex-items code parser stays as the reference implementation, pinned equivalent on the full fixture corpus by a differential test. - BYOH integration test proves a project descriptor over a novel flat stream yields zero-code ingest end-to-end; lint tests cover the new invariants; template scaffold, byoh guide (worked example), and progressive-enhancements docs updated to the two-tier story. Closes #139 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #139 (part of #122, data-driven harness descriptors phase 2).
Adds a declarative
[transcript.extract]tier to harness descriptors: flat event streams (every tool event self-contained) now get full transcript ingest —transcript_checkgrading, parsed invocations inrun.json, token/duration backfill intotiming.json— from descriptor data alone, with zero Rust. Streams that need cross-event state (keyedtool_use/tool_resultjoins, content coercion) remain named code capabilities;claude-stream-jsonis untouched.User-facing changes
[transcript]now takes exactly one ofparser(named capability) orextract(declarative tier). The extract block holds exactly the ticket's five primitives — equalitywherefilter, last-match final-text pick, flat tool-item mapping, token sum, duration via field pick or timestamp spread — all subtle rules fixed and non-configurable.harnesses/codex.tomlingests through the extract block.codex-itemsstays as the reference implementation and remains referenceable by BYOH descriptors; a differential test pins the extract spec byte-equivalent to the code parser on the full fixture corpus.parser, so moving a harness to extract needs a new label.docs/byoh.md("The declarative extractor"), two-tier story indocs/progressive-enhancements.md, updatedharness inittemplate scaffold, and the descriptor PR template now counts extract blocks as data-only.Before/after (BYOH)
Before — a harness with a novel flat stream had no transcript ingest without a Rust parser PR:
After — pure descriptor data:
→ no warning;
run.jsoncarries parsedtool_invocations,timing.jsoncarries tokens/duration (proven end-to-end by the newtests/run/byoh.rsintegration test).Verification
cargo test— 619 tests pass (453 lib / 96 cli / 70 run), including the new engine units, the codex differential-equivalence test, validationrejects_*suite, BYOH end-to-end, and lint CLI tests; all existing codex pipeline/golden tests pass unchanged through the extract engine (no golden re-bless needed)cargo fmt --check/cargo clippy --all-targets -- -D warnings— cleanharness lint codexpasses;harness show codexround-trips the resolved extract block as loadable TOML;harness liststill declarestranscriptfor codexSchema / docs callouts
schema/harness-descriptor.schema.json:transcript.requiredrelaxed toevents_filename; new fully-describedextractsub-schema (exactly-one rules live in Rust invariants, matching theslug_template/slug_capabilityprecedent)🤖 Generated with Claude Code