Conversation
Merge pull request #111 from slowdini/dev
refactor(instant-magic): reduce plan mode to single harness-agnostic flow
…-batch subcommands These fixture-swap/barrier subcommands exist only to serve the in-session (interactive Claude Code) isolated-run loop: they are referenced solely by in-session runbook generation and their own test modules, never by any Cli dispatch path. Removing them is the first step of dropping the in-session dispatch mechanism (#119). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make CLI dispatch the only mechanism end-to-end and delete all in-session
(`DispatchMechanism::InSession` / `RunMode::Interactive`) code. Claude Code now
defaults to and runs via its `claude -p` stream-json path, exactly like Codex's
`codex exec`; hybrid and headless are the only run modes (kept as accepted
`--run-mode` values pending the vocabulary collapse in #C).
Removed:
- `DispatchMechanism` enum, `RunMode::Interactive`, `RunMode::mechanism()`;
`default_for(ClaudeCode)` now → hybrid, `resolve_run_mode` ClaudeCode →
[hybrid, headless]; `HarnessRunCapabilities.mechanism` field.
- The single-shared-`env/` layout: every run now stages one
`env-<group>-<condition>/` per (group, condition).
- `HarnessAdapter::{runbook_template, parse_transcript, parse_transcript_full}`;
`parse_cli_events*` are now required (Codex parses its events file directly,
OpenCode errors until ingest is wired). The in-session runbook + its profile.
- In-session transcript code (`parse_transcript*`, subagent resolution by
`agent_description`, `resolve_subagents_dir*`, `SubagentMeta`/`SubagentEntry`)
and the `--subagents-dir` / `--session-id` flags + `CLAUDE_CODE_SESSION_ID`
auto-resolution. Shared `extract_invocations`/`read_records`/`TranscriptSummary`
kept (the stream-json parser reuses them).
All mechanism-keyed branches collapse to the CLI path; surviving harness-specific
behavior keys on the harness/adapter. Pipeline record-runs/fill-transcripts now
read each task's `outputs/<harness>-events.jsonl` only. Broken prose referencing
the deleted flags/commands rewritten in the manifest, help, and stray-writes hint;
the run-mode vocabulary/docs rewrite (incl. docs/harness-parity.md) stays with #C.
Integration tests re-pointed from the single `env/` to the per-(group,condition)
layout; in-session-only tests removed. `cargo test`, `cargo fmt --check`, and
`cargo clippy --all-targets -- -D warnings` are green; Claude Code `claude -p`
run → ingest → finalize smoke produces run.json/timing.json/benchmark.json.
Part of #113. Closes #119. Likely supersedes #34; unblocks #120 and #122.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
refactor(run-mode): remove the in-session dispatch mechanism (#119)
feat(instant-magic): retire run mode, collapse docs
Restructure src/adapters/ into per-harness module trees (claude_code/, codex/, opencode/) so every harness-specific reference lives in a clearly-named file: - Adapter impls move out of harness.rs into <harness>/mod.rs; harness.rs keeps the trait, dispatch contexts, adapter_for(), and the cross-harness tests. - The shared TranscriptSummary moves out of the Claude transcript module into adapters/transcript.rs — it is the harness-neutral shape every parser produces. - Guard installers and verdict shaping move out of src/sandbox into <harness>/guard.rs (install_claude_guard/install_codex_guard, hook matchers, guard_decision/codex_guard_decision). sandbox/install.rs keeps only the shared marker/manifest/teardown machinery; the generic install_guard() wrapper (hardcoded to Claude) is deleted — production code already goes through adapter.install_guard(). - teardown_guard/guard_is_armed and the hook entry points' default marker paths derive from Harness::ALL + adapter skills_dir() instead of hardcoded .claude/.agents paths. - OpenCode slug/naming helpers move out of staging into adapters/opencode (taking the staged prefix as a parameter). No behavior change; unit tests move with their code. The guard / guard-codex subcommand names, marker/manifest filenames, and manifest fields are stable on-disk contracts and are unchanged. Part of #121. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ancements Reorganize the trait around the instant-magic contract (#113): a minimal baseline every harness must satisfy plus optional enhancements that raise fidelity. Behavior-neutral. - Baseline (required) is now just label() + skills_dir(); the module and trait docs state the contract, and every method's doc comment names its tier, the enhancement it belongs to, and the fallback. - OpenCode's hand-written "not wired" stubs become trait defaults: parse_cli_events/parse_cli_events_full and install_guard now default to Unsupported errors that name the enhancement and the harness. rewrites_frontmatter_name / advertises_staged_slug_name default to false; render_available_skills_block and the skill phrases get neutral generic defaults (all current overrides kept). - New defaulted seam methods absorb the remaining harness matches in generic code: - staged_slug + validate_stage_name (staging's OpenCode slug match and naming validation move behind the adapter) - run_capabilities (core::capabilities_for's harness match deleted; the struct stays in core, the values live with each adapter, and an invariant test keeps supports_guard in lockstep with guard_armed_message) - config_dir_names (staging's hardcoded .claude/.agents/.codex/ .opencode sibling-asset filter now unions the adapters' lists) - guard_hook_cleanup_dir (teardown's hardcoded .codex prune) - transcript_surfaces_skill_invocation (judge_tasks' Codex branch for the __skill_invoked code check) Part of #121. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review follow-up: the per-harness render_*_plan_mode_context session fns were four identical copies of the trait's <system-reminder> default with no production callers — production always went through the trait method. Delete them (their trim-behavior coverage moves into the cross-harness trait test), tighten the opencode slug helpers to private now that the staged_slug/validate_stage_name seams are their only callers, and drop a breadcrumb comment from the staging tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes #121. Reframes harness documentation around the instant-magic contract: a minimal baseline plus progressive enhancements, instead of a parity audit. - Delete docs/harness-parity.md — the codebase (tiered trait + invariant tests) now carries parity information; no agent-facing audit procedure or report template survives. - Add docs/progressive-enhancements.md (dev doc): the single dispatch mechanism, the baseline contract, one section per enhancement (why it needs harness code, what it unlocks, the fallback, its trait methods), an adding-a-new-harness checklist, and the guardrails salvaged from the parity doc. - Add docs/claude-notes.md, docs/codex-notes.md, docs/opencode-notes.md (dev docs): per-harness implementation notes harvested from the README's per-harness sections. - README: the Harnesses section slims to an enhancement-framed intro + "How dispatch works" + a support table declared the source of truth; the per-harness sections are removed — their dispatch recipes duplicated the generated RUNBOOK.md / dispatch-manifest.md, which the README now names as the runtime reference. Fixes the dangling #claude-code-fully-wired anchor and the stale "(Codex today)" model-flag note (Claude Code has --model wired). - Fold the three plugin-isolation recipes (--setting-sources, enabledPlugins, CLAUDE_CONFIG_DIR) into the plugin-shadow banner so operators see them exactly when contamination is detected; the banner and validity warnings now point at docs/claude-notes.md (test-driven: banner tests assert the recipes and the doc pointer). - AGENTS.md/CLAUDE.md: docs/ is now described as development docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…odules refactor(adapters): move harness-specific code into per-harness modules
refactor(adapters): tier the HarnessAdapter trait into baseline + enhancements
…ements docs(harness): replace the parity doc with progressive-enhancement docs
… stray-write lookbehind The guard's Bash tamper-protection rule denied create/copy/move verbs only for paths under .claude, so a Codex-guarded run could tamper with its own hook config (mkdir -p .codex, cp evil.json .codex/hooks.json) without the Bash classifier objecting. detect-stray-writes' staging-dir lookbehind likewise recognized only .claude/.agents, misclassifying OpenCode staged copies (.opencode/skills/...) as live-source reads. Both are now built from a new adapters::all_config_dir_names() — the sorted, deduplicated union of HarnessAdapter::config_dir_names() across Harness::ALL — so the deny surface and staging vocabulary grow automatically with any future adapter. Staging's is_harness_config_dir collapses onto the same helper. This deliberately broadens the guard's deny surface cross-harness: a Claude-guarded run now also denies creates under .codex/.agents/.opencode (union semantics; the guard has no harness identity at classify time). Reads of the protected dirs stay allowed, and creates scoped to a run's allowed roots still pass. Closes #130 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(sandbox): cover every harness config dir in guard tamper rule and stray-write lookbehind
Change the seven declarative getters on HarnessAdapter (label, config_dir_names, skill_surface_phrase, skill_unresolved_phrase, cli_events_filename, cli_model_flag, guard_armed_message) from &'static returns to owned types (String, Vec<String>, Option<String>), so descriptor-backed adapters built from runtime-loaded config can implement the trait without leaking. Prep for the descriptor registry (#122 phase 2); no behavior change, adapter_for() keeps its shape. all_config_dir_names() drops its LazyLock cache and computes the sorted, deduplicated union per call; detect_live_source_reads hoists the list out of the per-occurrence lookbehind loop. Closes #132 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
refactor(adapters): de-static the HarnessAdapter trait surface
The read-lines-and-skip-malformed loop was copy-pasted three times (claude_code/transcript.rs, codex/transcript.rs, and inlined in stream_json.rs). Replace all three with one read_jsonl helper in the shared adapters::transcript module. The stream-json full parse reads each line as a Value once and shapes records from it, so the terminal result event no longer needs a second from_str on the raw line. Behavior-identical: blank/whitespace-only and malformed lines are skipped exactly as before (whitespace-only lines previously failed the parse and were dropped; now they're filtered explicitly). Part of #133. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The `if ctx.harness == Harness::ClaudeCode` preflight branch in orchestrate/build.rs was the last variant-specific behavior branch outside adapter_for() and the guard CLI entry points. Replace it with a defaulted HarnessAdapter::detect_shadowed_skills enhancement method (None by default; Claude Code overrides via a thin env-reading wrapper over the new shadow_preflight fn, which owns the report-or-silence mapping and is unit-tested with an explicit config dir). The harness-neutral report types and formatters (PluginShadowReport, ShadowSource, format_shadow_banner, shadow_validity_warnings) move to a shared adapters::skill_shadow module; detection stays Claude-specific in claude_code/plugin_shadow.rs. detect_plugin_shadows/config_dir_from_env/ resolve_config_dir lose their crate-root re-export (no consumer outside claude_code/ remains). Ride-along: the staging Default impls use Harness::default() instead of naming ClaudeCode. Documents the new enhancement in docs/progressive-enhancements.md. Part of #133. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
WRITE_TOOLS / SHELL_TOOLS / READ_TOOLS and the inline apply_patch /
file_change / command_execution names in src/sandbox/ and src/pipeline/
hardcoded per-harness tool naming in generic code. Each adapter now
declares a ToolVocabulary (write/patch/shell/read tool names across its
guard-hook and transcript surfaces); consumers match against the
cross-harness union via all_tool_vocabulary(), mirroring
all_config_dir_names() but LazyLock-cached since the guard arbiter
consults it per hooked tool call.
The guard arbiter's deny messages interpolate {tool_name} instead of
naming apply_patch/Bash — byte-identical for every payload the current
hook matchers deliver. A hypothetical file_change/command_execution
guard payload would now be classified instead of falling through;
unreachable under both current HOOK_MATCHERs, so behavior-preserving.
Union semantics at guard time were already the effective behavior since
both harness hooks funnel into one decide().
Removes the pub WRITE_TOOLS const from the sandbox surface (data moves
into the Claude adapter; internal-only in practice). New cross-harness
invariant tests: exact union contents, HOOK_MATCHER names must be
declared in the vocabulary, transcript parsers imply stray-writes
vocabulary, and role lists stay disjoint.
Part of #133.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude's judge recipe spliced an optional $model_arg shell var while Codex duplicated its entire multi-line command in an if/else — two shell idioms for the same optional-model rule, blocking the recipes from templating as descriptor data. Both now render through one render_judge_dispatch_recipe template in adapters::cli_command, parameterized by the harness command line (containing $model_arg), the model flag, and the events/stderr capture prefix. The judge-worker prompt sentence now lives in one place. Claude output is byte-identical (locked by a new full-string snapshot test). Codex output is behavior-identical but not byte-identical: the if/else pair collapses to the $model_arg splice, -m "$model" (quoted) becomes -m $model inside the unquoted expansion (matching the Claude recipe's long-standing form; model IDs contain no whitespace), and command indentation drops from 6 to 4 spaces. Part of #133. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same cleanup as the staging Default impls: StepParams and DispatchTaskOpts named Harness::ClaudeCode as a field default. ClaudeCode is #[default] and every other field already uses its type's default, so both manual impls collapse into #[derive(Default)] (clippy derivable_impls) — and the variant-specific acceptance grep for #133 comes back clean. Part of #133. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uplings Refactor/absorb out of trait couplings
Capture RUNBOOK.md, dispatch-manifest.md, dispatch prompts, skills blocks, judge recipes, and opencode next-steps per harness as byte-exact golden fixtures rendered from fixed contexts, ahead of the descriptor refactor (#134). Re-bless with GOLDEN_BLESS=1 cargo test golden_. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…time validation Add schema/harness-descriptor.schema.json (embedded as SchemaName::HarnessDescriptor), the three built-in descriptor files under harnesses/, and src/adapters/descriptor.rs: serde model, TOML -> JSON -> schema-gate -> typed-struct loading, single-brace template substitution, and cross-field invariant validation with actionable messages — the load-time form of the old cross-harness adapter tests (guard/banner lockstep, slug/naming-rule agreement, config-dir coverage, matcher/ vocabulary coverage, role disjointness, dispatch-template contracts). Code-backed features become named capabilities (src/adapters/ capabilities.rs): transcript parsers, guard engines, the opencode slug generator, and the claude-plugins shadow preflight. Nothing is wired to adapter_for() yet; the descriptor path is exercised by its own tests (a negative test per invariant plus an embedded-files load test). The packaged-file test pins the new shipped files. Part of #134. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ptors Add src/adapters/descriptor_adapter.rs — one HarnessAdapter implementation reading every declarative value from a validated HarnessDescriptor and dispatching code-backed features through the named capabilities (which gain their dispatch methods here: parse/parse_full, install_guard/hook_cleanup_dir, detect). The skills-block render loop moves to a generic renderer (src/adapters/skills_block.rs) that the trait default also uses, and cli_command.rs gains the shared jq/xargs parallel scaffold (render_parallel_dispatch_recipe), mirroring the judge-recipe split. adapter_for() still returns the legacy hand-written adapters; a transitional equivalence test pins the descriptor-backed adapter against them across every observable method and the guard x model context matrix (manifest sections compared as joined text — the consumer joins with '\n', so element granularity is not observable). Deleted with the legacy impls in the cutover commit. Part of #134. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The registry becomes a Vec of (label, adapter) entries keyed by each loaded descriptor's own label instead of a three-field struct built positionally from EMBEDDED_DESCRIPTORS. This dissolves the positional index coupling and the label-to-variant lockstep assertion (the label is the identity now); duplicate labels panic at build. adapter_for bridges the enum to a label lookup via its serde form until #135's string-keyed identifier replaces the enum outright; the config-dir and tool-vocabulary unions iterate registry entries directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Harness is no longer a closed enum deriving clap::ValueEnum and serde: it becomes a Copy handle to a descriptor-registry entry, constructible only through Harness::resolve, so adapter lookup stays infallible and pass-through call sites keep their signatures. Resolution, iteration (Harness::known), the registry-level default (claude-code), artifact Deserialize, and the registry-driven --harness value parser live next to the registry; the parser keeps clap's [possible values] help, suggestions, and error text without a compile-time harness set. Guard teardown and the armed check discover skills dirs through the registry; the manifest-based restore and no-manifest stray-marker sweep are unchanged and now pinned by a sweep test. conditions.json keeps the exact kebab-case strings (fixture round-trip tests from the previous commit stay green); unknown names now fail with the known-harness list wherever they surface. Closes #135. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The registry and harness-identifier resolution (entries, build, resolve/ known/Default/Deserialize, the --harness value parser, adapter_for, the config-dir and tool-vocabulary unions) move from harness.rs into a new adapters::registry module with their tests. harness.rs had grown two jobs past readable size; it keeps only the HarnessAdapter trait surface. The adapters::* re-exports are unchanged, and #136's registry construction now has a dedicated home. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Feat/string keyed harness
Carve descriptor provenance into a new descriptor::layers module (Layer, DescriptorSource, embedded_sources) and rebuild the registry on top of it: entries now record their contributing sources, build_registry takes owned DescriptorSource values and returns a Result (duplicate labels error instead of panicking), and the static becomes a OnceLock whose lazy fallback loads the embedded built-ins only — the seam #136's user-supplied layers feed into next. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--harness now parses as a plain string and resolves against the descriptor registry in run_context_with_bootstrap, so runtime-loaded descriptor layers (#136) can name harnesses clap never saw. The PossibleValuesParser goes away with the compile-time-complete registry assumption: unknown names now surface the registry's own actionable error (unknown harness 'x'; known harnesses: ...) with exit 1 instead of clap's invalid-value rejection with exit 2, and run --help names the built-ins in the flag's doc text instead of [possible values:]. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A descriptor with just a label now loads: skills_dir leaves the schema's required set and becomes Option<String>, and HarnessAdapter::skills_dir returns Option<PathBuf>. Two new load-time invariants keep the combination coherent — [staging] configured without skills_dir and [guard] without skills_dir are both rejected, since staging copies into the skills dir and the guard keeps its marker/manifest there. Guard teardown and the armed-marker sweep skip skills_dir-less harnesses; staging call sites keep a PathBuf via expect, justified because the run preflight (upcoming) forces --no-stage for such harnesses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Descriptor sources now stack in precedence order — embedded built-ins, user-global ($EVAL_MAGIC_CONFIG_DIR, $XDG_CONFIG_HOME/eval-magic, or ~/.config/eval-magic, under harnesses/*.toml), project-local (.eval-magic/harnesses/*.toml), and a one-off --harness-file — with field-level merge: a later file overrides individual fields of an earlier descriptor with the same label, or defines a new harness outright. Every file passes the schema gate individually (only label is required, so partial overrides validate too) and merged results re-run the full invariant suite with a provenance chain naming every contributing file. Policy: broken discovered files are skipped with a warning pointing at harness lint — they never brick the CLI; a broken --harness-file is a hard error. User-layer files may not declare [guard] or flip run.supports_guard (fail-open safety until the guard engine opens up); unguarded runs fall back to the detect-stray-writes audit. When --harness is omitted, a --harness-file descriptor's label becomes the invocation's default harness. Test helpers set EVAL_MAGIC_CONFIG_DIR="" so a developer's user-global descriptors never leak into the suite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
harness list surveys the registry — label (with the session default marked), contributing layers (built-in + project when merged), and the enhancements each resolved descriptor declares. harness show prints one harness's resolved post-merge descriptor as authorable TOML behind a provenance comment header, ready to copy into a layer file; descriptor structs gain Serialize (defaults omitted) and the toml dependency turns on display for it. harness lint validates a descriptor file — TOML + schema, the user-layer guard restriction, and cross-field invariants merged onto the registered harness with the same label, so partial overrides are checked against their real merge target — or, given a name, strictly re-lints every discovered layer file, surfacing exactly what registry init skipped with a warning. ✓/✗ report lines follow the validate idiom; a reserved --probe live-dispatch check is documented as deferred. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The harness preflight now follows the #126 model: each enhancement the selected harness doesn't declare produces a warning naming the fallback that carries the run, and the run proceeds degraded. --guard on a guard-less harness (e.g. opencode) is forced off with a pointer at the detect-stray-writes audit instead of erroring; a harness without a skills_dir falls back to --no-stage with each SKILL.md inlined; a missing transcript parser warns that transcript_check grades as unverifiable with llm_judge carrying the grading; requested models without a descriptor model flag are recorded as provenance only. Only genuinely contradictory flag combinations (--bootstrap/--stage-name where the descriptor declares them incompatible with --no-stage) remain hard errors. ingest prints a matching note on transcript-less harnesses, which previously skipped transcript work silently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A descriptor with just a label and an exec_template now earns real dispatch guidance: cli_next_steps renders a generic iterate-dispatch recipe around the rendered exec command, and dispatch-manifest.md gains a generic '## Dispatch recipe' section with the final-message capture contract. A baseline descriptor (no exec template) gets the harness-agnostic handoff naming outputs/final-message.md and the ingest line. Built-in harnesses all declare their own templates, so the golden artifacts are untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three integration tests pin the bring-your-own-harness contract: a project-local descriptor with just a label and exec template carries a complete run — fallback warnings, generic recipes in RUNBOOK.md and dispatch-manifest.md, simulated dispatches, ingest with the stray-writes audit, and llm_judge grading; a --harness-file descriptor becomes the invocation's default harness; and a descriptor referencing the codex-items parser by name gets full transcript ingest (parsed tool invocations in run.json) with zero code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New docs/byoh.md — the user-facing bring-your-own-harness guide: the minimum viable descriptor, the exec-template contract (eval_root cwd, final-message recovery), layer order and field-level merge semantics, the named-capability reference, the guard restriction, the warning/fallback table, and the lint/show/list workflow. progressive-enhancements.md catches up with the layered registry (label is the only required field, warn-never-reject preflight, layers.rs and registry init in the code map) and its Adding-a-new-harness checklist now starts from a user descriptor instead of the retired enum variant step. README gains the BYOH paragraph, a docs-table row, and the guard-fallback wording; --help's examples cover harness lint/list/show and --harness-file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The hidden guard/guard-codex hooks fire on every PreToolUse in a dispatched session and only ever need the embedded descriptors, so cli::run() no longer runs init_registry for them — the lazy embedded-only registry fallback serves the hook path, and a broken user descriptor can't add a skip-warning (or discovery latency) per tool call. Also from review: refresh the three doc comments that still described the retired --harness value parser, caveat harness show's copy-paste claim for guarded built-ins (drop [guard] before reusing as a user layer), and reword lint's name-target failure summary to say the ✗ lines may belong to other discovered files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Feat/byoh layered descriptors
Full-string pins of the Claude/Codex deny-verdict bytes (CLI level) and the freshly installed hook-file contents (adapter level), captured against the current per-harness guard code. These are the compatibility lock for the upcoming generic-engine refactor (#137): armed hooks and their backups are an on-disk contract, so the refactor must reproduce these bytes exactly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collapse the per-harness write-guard modules (claude_code/guard.rs,
codex/guard.rs) into one engine (src/adapters/guard.rs) driven by the
descriptor's [guard] data block. The four per-harness axes — hooks_file,
matcher, hook_entry template, verdict_template — plus command_template move
into harnesses/{claude-code,codex}.toml; the GuardEngine capability enum is
gone. Templates are parsed to JSON and placeholder-substituted in string
values, so key order (and therefore verdict bytes and hook-file shape) is
authored data pinned by full-string tests. The teardown cleanup dir is now
derived from hooks_file's parent instead of being per-harness code.
Descriptor validation grows guard-template invariants (matcher⊆vocabulary now
reads guard.matcher; hook_entry/verdict_template must parse as JSON with
{command}/{matcher}/{reason} placeholders in string values; command_template
must carry {exe}/{marker}; hooks_file must stay relative). The guard↔banner
lockstep and guard-requires-skills_dir invariants carry over unchanged.
The hidden guard / guard-codex subcommands stay as frozen aliases over a new
generic run_guard_hook handler resolving the verdict through the adapter's
new guard_verdict method (default fails open). Verdict bytes verified
identical against a pre-change binary.
Part of #137.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Hidden `guard-hook --harness <name> <marker>` entry point resolving the verdict shape from the named harness's embedded descriptor; `guard` / `guard-codex` remain frozen aliases. The new subcommand joins the is_guard_hook gate so hook invocations stay off layered descriptor discovery, and it fails open on an unknown harness name. Part of #137. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--guard with a harness defined by user-supplied descriptors alone now stops the run in preflight instead of downgrading to a warning: the write guard is restricted to embedded built-ins (fail-open safety), and a run the user asked to guard must not continue silently unguarded. The error names the detect-stray-writes fallback. Built-in guardless harnesses (opencode) keep the warn-and-continue downgrade. Registry entries now expose embedded-layer provenance via has_embedded_layer(). Part of #137. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The user-layer restriction message and its doc comment drop the stale 'until the guard engine is opened up' framing — the engine is generic now; guard data stays embedded-only because the guard fails open. progressive-enhancements.md documents the six [guard] fields, the one engine at src/adapters/guard.rs, the frozen guard/guard-codex aliases, and the guard-hook entry point for future built-ins. byoh.md notes the hard --guard preflight rejection; the per-harness notes point at the shared engine; the --guard --help text names the user-descriptor rejection. Part of #137. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Feat/guard generic engine
…or PR template Closes #138 (descriptor epic #122, step 7): make the bring-your-own-harness path agent-followable end to end. - `eval-magic harness init <name>` scaffolds a commented descriptor template (every field explained inline, VERIFY-don't-guess prompts, lint-clean as written) plus a notes-file skeleton for the per-field verification record, into the project-local layer; `--stdout` prints the template, `--force` overwrites, already-registered names get an overlay note instead of an error. - Templates ship as embedded assets (harnesses/template.toml, harnesses/template-notes.md); drift guards pin that the template mentions every schema field and capability name and that the fully-uncommented examples form one coherent descriptor. - docs/byoh.md gains "Upstreaming your descriptor" (data-vs-code split, the four-file PR shape) and "Verify, don't guess"; progressive-enhancements.md, README, and the --help examples now route through `harness init`. - New .github/PULL_REQUEST_TEMPLATE/harness-descriptor.md scopes data-only descriptor PRs: diff shape, verification evidence, don't-guess attestation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(cli): harness init scaffold, BYOH upstreaming flow, and descriptor PR template
Add a [transcript.extract] descriptor block so harnesses with flat event streams (every tool event self-contained) get full transcript ingest — transcript_check grading, parsed invocations in run.json, token/duration backfill into timing.json — from descriptor data alone, no Rust. - New generic engine (src/adapters/extract.rs) interpreting exactly five primitives: dotted-path string-equality `where` filter, last-match final-text pick, flat tool-item mapping, token sum, duration via field pick or timestamp spread. Cross-event joins and content coercion stay named code capabilities (claude-stream-json unchanged). - TranscriptSection now declares exactly one of parser/extract; schema permits both, a load-time invariant rejects both/neither (plus empty extract blocks and duration with both/neither variants) with actionable messages. - harnesses/codex.toml flips to the extract tier; the codex-items code parser stays as the reference implementation, pinned equivalent on the full fixture corpus by a differential test. - BYOH integration test proves a project descriptor over a novel flat stream yields zero-code ingest end-to-end; lint tests cover the new invariants; template scaffold, byoh guide (worked example), and progressive-enhancements docs updated to the two-tier story. Closes #139 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(transcript): declarative extractor tier for flat event streams
Closes the #126 gaps: supported enhancements are now provided automatically, and the remaining silent degradations warn. - The write guard arms automatically on guard-capable built-in harnesses whenever staging is active; new --no-guard opts out. --guard stays as the explicit form (warns where auto-arm quietly stays off; still rejected on user-descriptor-only harnesses). RunOptions.guard is now tri-state (None = auto), resolved by the preflight, and the run plan prints the armed guard status. - The no-transcript-parser warning is scoped to eval configs that actually use transcript_check (resolve now runs before preflight). - A harness without a dispatch exec recipe now warns naming the generic-handoff fallback (new has_dispatch_recipes probe). - Docs updated across --help, README, progressive-enhancements, byoh, per-harness notes, and the descriptor template. Verification: cargo test (637 green), cargo fmt --check, cargo clippy --all-targets -- -D warnings, plus end-to-end drives (bare run auto-arms; opencode warns guard/transcript/dispatch; --guard --no-guard conflicts at parse). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat(run): auto-arm the write guard; scope preflight warnings
slowdini
added a commit
that referenced
this pull request
Jul 17, 2026
Merge pull request #150 from slowdini/dev
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release notes
Replace this paragraph with a short narrative for the release. If left unchanged, GitHub's auto-generated notes will be used instead.