Skip to content

fix: accept pinned Flare derived analysis after sidecar save - #155

Merged
Dong1017 merged 1 commit into
mainfrom
codex/issue147-observation-compat
Oct 26, 2026
Merged

Dong1017 merged 1 commit into
mainfrom
codex/issue147-observation-compat

Conversation

@Dong1017

@Dong1017 Dong1017 commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Governing Issue and delivery slice

Refs #147

  • Accepted scope / delegation: the user's corrective-PR instruction and reopening finding. Implementation/publication only; independent re-review and re-acceptance are required before any integration or Issue completion.
  • What this PR delivers: compatibility with pinned Flare's measurements-only derived-analysis augmentation while retaining sidecar integrity and raw metric recomputation.
  • What it does not deliver: new phases, runner/evaluator changes, [Task] Distill Flare preparation contracts and session-state lifetimes #148 work, model/GPU/service experiments, merge or Issue closure.

Flare saves observation.json before run() adds analysis.http_prefill_tokens_per_second for ttft-4096 and prefill-65536. Whole-object membership therefore rejected a normal producer packet. The adapter now accepts the unchanged observation or that exact addition using the pinned producer formula. Every original field and every other analysis field must still agree; raw SSE checks and selected client TTFT recomputation remain in place.

Knowledge authorship declaration

Not applicable: no protected knowledge path changes. Material agent authorship is retained in the commit trailer.

Changed artifacts and evidence

Recheck of the existing six P0 criteria
Existing P0 criterion Corrective evidence
Frozen schema/readset and one metric No schema, source pin, unit, clock, grouping or comparison-scope change.
Raw recomputation, immutable inputs and bound identities Integrity comparison allows only the documented addition; the recomputation test poisons both the aggregate and selected observation's TTFT summaries while requiring the raw result and unchanged input bytes.
Complete producer-shaped positive and semantic negatives Fixture writes raw SSE and sidecars first, then augments in-memory analysis before measurements. New negatives alter timestamps, HTTP status, request fingerprints and original/unknown analysis independently on either side with refreshed digests. Wrong-group/value additions reject; prior raw/group/clock/capacity/counter/resource/digest tests remain.
Invalid evidence exports no metric / INCONCLUSIVE Divergence CLI tests require rejection with metrics={}; existing real-command integration covers incomplete-peer INCONCLUSIVE.
Existing validation-command integration Existing RunSpec/external executor/runner/evaluator test uses the now producer-shaped fixture; no integration implementation changes.
Fixture-only boundary Synthetic offline contract coverage only; no service-quality, performance or agent-efficacy conclusion.

Checks and dependencies

  • Source: Flare bce032c014bae34502ba22d3304fa084d5b40afe, sidecar write, later augmentation and measurements write.
  • Base: a4a0fbd62defe2bc53ab090e1fa50bd7139663a6.
  • Head: 5a2d16eeaa02c63d2b355665db933d5cea576685.
  • Reproduced before production change: python -m pytest tests/test_flare_adapter.py::test_pinned_producer_augments_measurements_after_saving_sidecars -q with the new regression/fixture and base production code failed with ValueError: observation sidecar differs.
  • Focused: python -m pytest tests/test_flare_adapter.py -o addopts='' -q — 34 passed in 739.12s, including the unchanged real validation_command integration assertions.
  • Lint: ruff check src/inferrsi/model/validators/flare.py tests/fixtures/flare_packet.py tests/test_flare_adapter.py passed. git diff --check passed.
  • CI: Ready CI passed all four jobs: full-check and clean-wheel installed-package on Python 3.10/3.11. Draft CI, the PR declarations job, and the latest policy scan job passed. Current-head knowledge-model-policy status is success / Not applicable.
  • Dependencies / assumptions: supports only the existing pinned Flare packet contract; no [Task] Distill Flare preparation contracts and session-state lifetimes #148 dependency. Local checks run on Windows with the existing development environment; Ready CI checks Python 3.10/3.11 and clean wheel installation.

Remaining work and Issue closure

  • Independent re-review requested: @Dong1017, please review the exact head/base above, including the pinned write order, raw-field integrity, refreshed-digest negatives and raw metric recomputation. State the independent reviewer role; if approved, record LGTM 5a2d16eeaa02c63d2b355665db933d5cea576685.
  • Remaining Issue requirements / follow-up: independent current-head Dong1017 review and fresh P0 acceptance. Historical PR feat: adapt offline Flare delivery evidence #151 review/acceptance does not cover this correction.
  • Goal-acceptance owner: the user or separately authorized independent acceptance owner.
  • Goal ready for acceptance? Proposed for independent assessment; not accepted or complete. Keep [RFC] Flare beyond distillation: offline qualification evidence adapter #147 open. No automatic merge or closure is authorized.

Match the pinned producer augmentation without relaxing original observation integrity or raw metric recomputation. Exercise its write order and reject divergent fields even with refreshed packet digests.

Agent-assisted-by: Codex (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T08:05:26.436963Z 5a2d16e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 5a2d16e within the corrective software/source-compatibility scope of #147.

Independent reviewer role: this review was performed separately from the implementation/local-agent work. Inspected exact base a4a0fbd62defe2bc53ab090e1fa50bd7139663a6, all four changed files, pinned Flare bce032c014bae34502ba22d3304fa084d5b40afe, the reopening finding and existing PR discussion. No new actionable finding.

Verified clearance of the original blocker: Flare saves sidecars before measurements-only augmentation. The adapter now accepts the unchanged observation or exactly the documented analysis.http_prefill_tokens_per_second addition, for the two permitted groups and pinned formula. Every original field and other analysis entry remains in the comparison; this is not a blanket omission of analysis/integrity checks. The production file outside this assertion replacement is unchanged, including raw SSE validation, selected raw TTFT recomputation and failure behavior.

Regression review: producer-shaped fixture preserves write order; both augmented groups and unchanged groups are asserted. Five divergence types on both copies refresh digests and require rejection with empty metrics. Poisoned aggregate/selected-observation summaries still produce the raw metric with original bytes preserved. Existing actual validation_command integration retains valid-pair PASS and incomplete-peer INCONCLUSIVE/no metric.

CI: Ready 40289712837 has all four jobs successful; Python 3.11 logs record 989 passed, 9 skipped, 135 subtests and Ruff success. This CI ran synthetic merge ea1b7f9f98b2b03a99feb7a13540c39fa54e94a4 combining this head with the stated base. Latest declaration run 40292251455 and exact-head knowledge-model-policy both now read success; head was rechecked immediately before posting.

Limits: independent static/source/test-contract review plus eight standalone arithmetic JSON-round-trip checks; I did not rerun the full fixture/integration suite locally or run models/services/GPUs. The implementer's reported 34 focused tests are not relabeled as my execution. This supports the corrective PR's software readiness, not real-service/performance/agent efficacy, automatic integration or fresh whole-Issue acceptance. Keep #147's re-acceptance decision separate.

@Dong1017
Dong1017 merged commit 0c78a7e into main Oct 26, 2026
8 checks passed

Copy link
Copy Markdown
Collaborator Author

Owner-directed corrective integration completed: guarded squash of reviewed head 5a2d16eeaa02c63d2b355665db933d5cea576685 produced main 0c78a7e257dfaa3573b77581a39539e8a3a5da4c.

Green evidence: four Ready jobs, declarations, and refreshed exact-head knowledge-model-policy success from 40293232635. The first merge attempt was rejected while a new status became pending; no bypass was used. It was retried only after the head/status were re-read as unchanged/success and GitHub reported clean.

Actual squash tree 7645f47ead5ef29fd47c1f215838d12aeb01aca3 exactly equals the Ready-tested synthetic merge ea1b7f9f98b2b03a99feb7a13540c39fa54e94a4. Independent review remains applicable to the integrated content. Main merge lane is being verified. Separate main-push declarations run 40293527272 was cancelled; it is not claimed successful. Issue #147's bounded P0 re-acceptance is being recorded separately; no real-model/service/GPU efficacy claim.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant