Skip to content

feat: adapt offline Flare delivery evidence - #151

Merged
Dong1017 merged 2 commits into
mainfrom
codex/issue147-flare-adapter
Oct 26, 2026
Merged

Dong1017 merged 2 commits into
mainfrom
codex/issue147-flare-adapter

Conversation

@Dong1017

@Dong1017 Dong1017 commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Governing Issue and delivery slice

Refs #147

  • Accepted scope / delegation: bounded P0 offline adapter under the Issue-owner claim and plan, delegated through the explicitly invoked inferrsi-issue workflow.
  • What this PR delivers: one source-pinned Flare delivery-evidence adapter, a complete synthetic packet writer, contract tests and integration through the existing external executor and validation command.
  • What it does not deliver: [Task] Distill Flare preparation contracts and session-state lifetimes #148 source distillation, a serving engine, new runner/evaluator/backend, task-quality oracle, model/GPU experiments, ABI/preparation preflight or session-recovery scenarios.

Knowledge authorship declaration

Not applicable: no protected knowledge/policy path is changed. Material author: Codex (implementer). Model/effort settings are not asserted.

Changed artifacts and evidence

Flare summaries do not establish raw delivery semantics. The adapter recomputes median client delivery_ttft_ns for measured ttft-512 requests from complete raw packets, preserving SSE token groups and native/client clock separation. Both measurement commands validate their peer before exporting metrics, so missing or incomparable evidence remains INCONCLUSIVE in the existing evaluator.

Requirement Actual change Check / evidence and result Remaining gap
Frozen schema/readset, one delivery metric docs/flare-qualification.md, src/inferrsi/model/validators/flare.py; Flare contract bce032c014bae34502ba22d3304fa084d5b40afe Unit, clock, grouping, full corpus and comparison scope explicit None within P0
Raw recomputation and provenance Raw SSE/observations, package/corpus/input hashes and adapter/helper identities; separate receipts Recomputed metric ignores summary TTFT; originals unchanged None within P0
Negative evidence Missing raw/group/request, clock/coalescing/capacity/counter/resource mismatch, source/digest drift and client failure Rejections have no metric None within P0
Existing integration measure --packet --peer and validate-pair; unchanged RunSpec/runner/evaluator Real local processes cover valid synthetic pairing and incomplete-input INCONCLUSIVE None within P0
Complete synthetic sample tests/fixtures/flare_packet.py, tests/test_flare_adapter.py; operation-guide entry Complete one-repetition packet including coalesced events; explicit synthetic=true Real service evidence is outside P0

Checks and dependencies

  • Current head: aa196034c3b7250b0ea99c2f4b889e074b9570c3; reviewed base 65147a03e465dc838770474736a620e04d72337e.
  • Ready CI: current-head run — full repository checks and clean installed-wheel checks passed on Python 3.10 and 3.11.
  • Local checks: ruff check src tests tools and diff whitespace checks passed. The unchanged adapter's 19 contract cases passed locally; the real-runner integration passed after allowing Windows source-snapshot overhead (1 passed in 150.37s).
  • Retained failure: the earlier one-minute Windows integration envelope expired before the validator launched, despite successful metrics and verified source integrity. The follow-up changes only this ordinary test envelope to three minutes; output/verdict assertions and production code are unchanged.
  • Independent review: Dong1017 approval-agent review, LGTM aa196034c3b7250b0ea99c2f4b889e074b9570c3.
  • Dependencies: standard library plus existing artifact/executor contracts. [Task] Distill Flare preparation contracts and session-state lifetimes #148 is independently owned and not an implementation prerequisite.
  • Not run: model, server, GPU or agent-efficacy evaluation. Synthetic timings are test inputs.

Remaining work and Issue closure

  • Remaining Issue requirements: final live integration/CI gates and the Issue-level acceptance/closure card. Optional later research remains outside P0.
  • Goal-acceptance owner / closure delegation: invoking Issue-level owner under the explicit workflow delegation.
  • Goal ready for acceptance? Proposed with the linked implementation, current-head checks and independent review.

Real campaigns still need separately frozen task-quality and environment gates. Offline pair consistency does not establish service quality or a performance gain.

Recompute client delivery TTFT from complete source-shaped Flare packets,
preserve coalesced SSE groups and verify preparation/counter identities.
Use peer checks before metric output and the existing validation command
and evaluator to retain incomplete-evidence semantics.

Agent-assisted-by: Codex (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-26T01:22:10.017715Z aa19603 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Collaborator Author

Independent Dong1017 approval-agent review requested for head 019c6493f84e548ea7aa6f9cb6baa9a170bb81d3 against base 65147a03e465dc838770474736a620e04d72337e, implementing the accepted P0 scope in #147. Please inspect raw recomputation, native/client clocks, coalesced SSE preservation, complete group/resource/capacity checks, and the existing runner's incomplete-evidence behavior. Ruff passes; the refreshed 20-test adapter suite is running, and full Ready CI/installed-wheel checks are required. State the independent reviewer role and use LGTM <full-head-sha> for any comment-based approval. The Issue owner will assess all gates separately before integration; no real service-performance or quality result is claimed.

@Dong1017 Dong1017 added the area:workbench CLI workbench: profile, A/B loop, quality gates, evidence reports label Oct 26, 2026 — with ChatGPT Codex Connector
Give the real-command integration test a three-minute whole-run envelope.
On Windows the one-minute envelope expired during source snapshots before
the validation command could start; both metrics and source integrity passed.
Keep the same output, comparison and INCONCLUSIVE assertions.

Agent-assisted-by: Codex (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent reviewer: Codex subagent /root/issue147_review, separate from the Issue owner and implementer, acting under the explicitly invoked Issue-owner workflow.

LGTM aa19603

Reviewed head aa196034c3b7250b0ea99c2f4b889e074b9570c3 against base 65147a03e465dc838770474736a620e04d72337e. No blocking findings in the accepted Issue #147 P0 scope.

I checked all five changed files against the accepted criteria, pinned Flare bce032c014bae34502ba22d3304fa084d5b40afe performance/provenance code and native observation contract/producer, plus the existing executor/evaluator consumers. The adapter recomputes the selected TTFT from timestamped raw events, checks raw SSE and sidecar agreement, ignores null optional metadata, preserves coalesced delivery groups and separate clock domains, and gates export on complete groups, preparation capacity, actual reuse/counter agreement, resource conservation and matching pair identities. Invalid or incomparable inputs export no metric and retain the existing INCONCLUSIVE path.

I inspected the implementer's passing targeted Windows real-runner rerun (1 passed) and its artifacts: the valid synthetic pair is PASS with the expected fixture metric values; the incomplete peer is INCONCLUSIVE. The follow-up commit changes only the test's whole-run envelope from one to three minutes after the previous envelope expired before the validator launched; production code and all assertions are unchanged.

This is an offline contract review, with no service-quality, accelerator-performance or agent-efficacy qualification. Current-head CI and integration remain the Issue owner's separate merge gates.

@Dong1017
Dong1017 merged commit d99593f into main Oct 26, 2026
6 of 9 checks passed

Copy link
Copy Markdown
Collaborator Author

Squash-merged under the owner's explicit batch instruction. Head aa19603; merge d99593f.

Evidence verified before merge:

The current human merge instruction proceeds on scoped review, Ready tests and the actual required policy, not an assertion that every historical declarations job is green. Software/knowledge integration only; no Issue/campaign acceptance or model/hardware efficacy claim.

@Dong1017
Dong1017 deleted the codex/issue147-flare-adapter branch October 26, 2026 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:workbench CLI workbench: profile, A/B loop, quality gates, evidence reports

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant