[RFC] Flare beyond distillation: offline qualification evidence adapter

Conclusion / Minimal Increment

Beyond source-code distillation, the recommended first step is an offline evidence adapter for Flare qualification results, plus negative fixtures. Reuse InferRSI's external_benchmark / validation_command and existing evaluator; do not port the Rust serving engine or add an execution platform.

Status: P0 re-accepted after the producer-compatibility correction (2026-10-26, PRs #151 and #155; current acceptance decision). The original acceptance and reopening finding remain part of the history. The initial RFC's design and source-code observations are retained below. This issue does not authorize model/GPU runs, budget expenditure, or automatic integration. Distillation can proceed independently; the deliverable here is a verifiable experimental interface, not another knowledge summary.

Why This Is Worth Doing

NVIDIA/flare-local-inference is currently a local-inference research prototype targeting SM120/SM121, explicitly not a vLLM replacement. Its value goes beyond the division between Python preparation and Rust serving to include full session-state restoration, cost-constrained reuse, and layered measurement evidence.

Source baselines: Flare bce032c014bae34502ba22d3304fa084d5b40afe; InferRSI 65147a03e465dc838770474736a620e04d72337e. The following are source-code observations and engineering proposals, not local execution results.

flowchart LR
    A["Existing raw Flare qualification artifacts"] --> B["New: offline semantic adapter + fixtures"]
    B --> C["Existing validation_command"]
    C --> D["Existing evaluator / evidence packet"]
    D --> E["Preserve quality, completeness, and INCONCLUSIVE boundaries"]
    F["Optional follow-up: session restoration scenarios and prepared-artifact preflight checks"] -.-> B
    G["Independent: repository source-code distillation"] -.-> H["Existing knowledge base / Tracking #91"]

Three Directions, Starting with the First Slice

Priority	What Can Be Done	Minimal Deliverable and Boundaries
P0	Qualification evidence adaptation	One offline CLI, one minimal example packet, and positive/negative fixtures; read-only access to retained artifacts, without starting services/models
Optional follow-up	Full session-state restoration scenarios	Workloads and evidence requirements for miss / GPU hit / host restore / restart / branch / cancel; not a new offload backend
Optional follow-up	Preparation / serving compatibility preflight checks	Verify package, ABI, model/frontend, and actual capacity in the same adapter; not a new artifact platform

The Actual Gap for P0

InferRSI already has RunSpec, completeness/quality evaluation, and artifact utilities. Do not duplicate these capabilities.

The increment is in the semantics of the source results: the generic scalar metric reader is not responsible for understanding Flare's SSE grouping, clock domains, prepared capacity, or reuse-counter conservation. Flare qualification and provenance can serve as pinned-version references for the input contract.

Distinctions That Must Be Preserved	Semantics the Adapter Must Retain
SSE event vs. individual token	Coalesced tokens retain their shared group timestamp; do not fabricate per-token ITL
Client delivery vs. native completion	Do not subtract across clock domains; do not label HTTP TTFT as pure GPU prefill
Cache lookup hit vs. restoration actually adopted	Distinguish cost/capacity/enqueue rejection from admission; verify actual reused tokens
Measurement labels vs. actual prepared capacity	Cross-check against the preparation memory report; do not present different capacities as equivalent conditions
Summary vs. raw evidence	Recompute the first selected delivery metric; preserve missing groups, failures, and resource-counter inconsistencies

P0 Acceptance Checklist

Freeze the input schema/readset and select one delivery metric; specify its unit, clock domain, grouping, and comparison scope.
Recompute from raw observations rather than trusting only the summary; leave the original packet unchanged, and bind the derived receipt to source/package/corpus/adapter identities.
Cover a complete valid example as well as missing raw data/groups, incorrect clocks, incorrect expansion of coalesced events, capacity mismatch, and counter/resource mismatch.
Reject or explicitly flag invalid or incomplete inputs; do not fill with zeros or manufacture PASS results; hand off to the existing evaluator to preserve INCONCLUSIVE.
Integrate through the existing validation_command; do not add a global evaluator, runner, or parallel evaluation system.
Offline fixture PASS means only that adaptation/validation behavior passes; actual service quality, performance, and agent effectiveness remain unverified.

Follow-up Research Questions, Not to Be Implemented in Advance

The state to restore is more than KV. Checkpoints/fences in cache.rs and offload distinguish full endpoint state, publication/retirement, transfer tickets, and waiter lifecycles. Research should target "whether restoration is correct, how much prefill it actually replaces, and whether it interferes with concurrent decode," rather than hit rate alone.
Reuse has a cost. Upstream's estimated transfer < 0.5 × replaced prefill is a policy arm to be validated, not a universal threshold; comparisons must freeze the workload, capacity, and quality oracle.
Preparation and serving form a compatibility boundary. The runtime package design currently uses execution ABI 4 / prepared contract 2; do not directly compare results across different ABIs/packages/capacities. Python→Rust does not itself demonstrate a speedup.

Division of Responsibilities with Existing Work

[Roadmap] InferRSI: reusable knowledge, verifiable inference optimization #61: Workbench / Target System separation; Flare is an external research subject, not a reason to turn InferRSI into a serving engine.
[RFC] Benchmark selection by task for InferRSI #60 / [RFC] Bottleneck-driven optimization: scheduling, caching, and parallelism/pipelining #131: Reuse workloads, metric/quality thresholds, and the boundaries of cache-scheduling mechanisms.
Already merged feat: add model-aware inference research preparation #111: Reuse research on model preparation, format compatibility, and stage costs; do not build a separate preparation platform.
Skill Eval v1.1: automated evaluation for skill additions and updates #88 / Skill Eval: automate PR evaluation planning and feedback #101 / feat: plan and report skill PR evaluations with explicit authorization #118: Evaluation of skill-change effects; [RFC] Skill Indexer: better skill and context selection through collect #128: knowledge-context selection experiments; this issue's runtime state and measurement semantics do not replace them.
[RFC] Goal checkpoints: explicit progress and next actions across agent sessions #137: Agent goal checkpoints are not model-session/KV checkpoints.
[Tracking] SearchSpace distillation coverage and knowledge consolidation #91: Register new sources and follow-up deliverables; do not reopen historical [RFC] Repository knowledge distillation: source coverage, incremental updates and reusable mechanisms #48. Do not increase distillation/resource coverage counts before delivery.

Evidence and Limitations

The root LICENSE is Apache-2.0; copying source code requires retaining applicable notices, while model/dependency licenses must be checked separately. The test/serving qualification statements in the upstream latest commit are not local reproduction results, and the completed performance comparison is still unfinished. The measurement documentation requires distinguishing GPU replay, native scheduling/submission, and the HTTP layer. No tests, models, or GPU experiments were run for this work.

Search record: As of 2026-10-25, queries for Flare and flare-local-inference across this repository's titles/bodies/comments, without state restrictions, returned no matches; this is not absolute proof of absence in unindexed content.

Independent Distillation Task

Source-code distillation has been separated into #148 (a native sub-issue of #91): prepared-artifact boundaries and the full session-state lifecycle. That knowledge task and this RFC's offline evidence adapter have separate acceptance criteria; completing either one does not automatically close the other.

Current corrective slice — implementation and PR only

The user has explicitly authorized a narrow corrective PR under the reopening finding. I remain the accountable Codex owner in this chat. This instruction supersedes the historical integration/closure delegation below: no automatic merge or Issue closure; independent re-review and re-acceptance are required.

Current main is a4a0fbd62defe2bc53ab090e1fa50bd7139663a6. The pinned Flare source remains bce032c014bae34502ba22d3304fa084d5b40afe; its sidecar write precedes the measurements-only analysis.http_prefill_tokens_per_second augmentation for ttft-4096 and prefill-65536.

One corrective PR will: reproduce that write order in the offline fixture before changing production code; tolerate only the documented derived-analysis addition while preserving all original observation checks and raw TTFT recomputation; add genuine divergent-field negatives with refreshed manifest digests; run the focused adapter/real validation-command integration and Ready CI; report the exact base/head and results against the existing six P0 criteria. No model, server, GPU, new phase, runner, evaluator or #148 work is included. No existing Flare corrective PR was found during preflight.

Historical original owner plan (preserved)

Issue owner claim and accepted P0 plan

I am the Codex agent in the invoking owner session, acting as Issue-level owner under the user's explicit inferrsi-issue 147 delegation. This accepts the bounded P0 implementation, delivery PR/review/integration work and whole-Issue acceptance/closure responsibility, subject to the workflow's independent current-head Dong1017 review and all applicable CI gates.

Boundary: #148 is assigned to another agent. Its source distillation, knowledge registrations and #91 bookkeeping remain separately owned and separately accepted. This Issue does not depend on completing that knowledge slice. No model, GPU, cloud or paid execution is authorized or planned.

Preflight: upstream main is 65147a03e465dc838770474736a620e04d72337e; no existing Flare delivery PR was found. Existing unrelated local work will be preserved using an isolated task directory. Current RunSpec, external executor, metric reader, evaluator, artifact helpers and repository contribution/collaboration templates have been inspected.

One coherent delivery PR is planned:

Acceptance	Current gap / change	Decisive evidence
Frozen schema/readset and one delivery metric	Versioned offline packet contract pinned to Flare `bce032c014bae34502ba22d3304fa084d5b40afe`; select median client `delivery_ttft_ns` for measured `ttft-512` groups	Contract/readset documentation and complete synthetic packet
Raw recomputation and immutable source	Recompute from event observations, cross-check raw SSE/group coverage; receipt binds source, package, corpus, input and adapter identities	Recomputed metric independent of summary; original input bytes unchanged
Semantic rejection	Check clock/grouping, coalesced events, actual prepared capacity, cache-token counters and resource conservation	Positive fixture and each required negative case
Existing integration	Offline measurement CLI plus pair validation used by existing `external_benchmark` / `validation_command`; no new evaluator/runner	Real local executor/runner integration; incomplete input yields INCONCLUSIVE
Honest evidence boundary	Label fixtures as synthetic contract tests	No service-quality, performance, GPU or agent-efficacy claim

Next: implement the adapter, packet and focused tests; run applicable local checks; publish a Ready PR with exact revisions and request independent Dong1017 approval. Real service task-quality validation remains an additional campaign-owned requirement; adapter consistency is not a task-quality oracle.

P0 delivery ready; integration held by repository-wide CI

Delivery is in Ready PR #151 at aa196034c3b7250b0ea99c2f4b889e074b9570c3, with independent Dong1017 approval-agent LGTM. The current-head Ready run passed both Python 3.10/3.11 full-check and installed-wheel jobs. Local contract checks and the refreshed real-command Windows integration also passed; the earlier one-minute test-envelope timeout and bounded three-minute adjustment are retained in the PR evidence.

Integration remains held: the latest declarations job failed because the repository-wide sweep could not finish other PRs' status-history scans within its budget. Its own #151 result is success / Not applicable, but the whole applicable workflow job is not successful. The all-CI merge gate is therefore not met; #147 remains open pending that gate and final acceptance. No unrelated policy implementation is included in this PR.

#148 remains independently owned and separately accepted. This delivery uses synthetic offline fixtures only; no model, server, GPU or agent-efficacy experiment was run.

Accepted: Flare offline delivery evidence adapter (P0)

flowchart LR
    A["Existing Flare raw packet"] --> B["Offline raw and identity checks"]
    B --> C["Client TTFT and bound receipt"]
    C --> D["Existing runner and evaluator"]
    B -->|"Incomplete or inconsistent"| E["No metric exported"]
    E --> F["INCONCLUSIVE"]

Delivered: P0 in PR #151, accepted merge revision d99593fea76a6a8f6dc3cda7ee9fe108570a366b: one offline CLI, frozen packet/metric contract, a complete synthetic packet writer, negative fixtures and existing-runner integration.
Verified: independent current-head review; Ready checks on Python 3.10/3.11 including clean wheel installations; post-merge lane passed. On the actual merge revision, the real-command integration was rerun locally on Windows: 1 passed in 145.34s, covering a valid synthetic pair and missing-evidence INCONCLUSIVE.
Limits: acceptance is for offline adapter behavior. No model, server, GPU, service-quality, performance or agent-efficacy experiment was run. ABI/preparation preflight and recovery scenarios are future work; [Task] Distill Flare preparation contracts and session-state lifetimes #148 remains independently owned and separately accepted.

Full acceptance record

The accountable Issue owner accepts the six P0 criteria under the original scoped delegation. This is an Issue-level evidence decision after integration, not automatic closure from a merged PR.

P0 criterion	Accepted artifact and evidence
Frozen schema/readset; exactly one delivery metric	Contract and adapter: `inferrsi-flare-delivery-packet-v1`; Flare `bce032c014bae34502ba22d3304fa084d5b40afe`; median `metrics.delivery_ttft_ns`, nanoseconds, client elapsed clock, SSE event grouping and `flare_http_delivery_ttft_512` scope.
Recompute raw observations; preserve originals; bind identities	Raw SSE/observation/sidecar agreement, immutable input checks and source/package/corpus/input/adapter/helper identities are implemented and covered by the positive contract test. Summary TTFT is not used as the metric.
Complete normal sample and required negative cases	Complete synthetic packet writer plus tests: missing raw/group/request records, clock mismatch, invented coalesced-token grouping, prepared-capacity mismatch, request/reuse/resource mismatch, client failure and identity/digest drift.
Reject invalid/incomplete input without fabricated PASS	CLI rejection has no metric; peer validation occurs before either metric export. The real external-command test verifies INCONCLUSIVE for an incomplete peer.
Existing validation-command integration	Both measurement commands and `validate-pair` compose with existing RunSpec, external executor, runner and evaluator. No new evaluator, runner or backend was introduced. The actual merge was retested after inspecting the intervening stdin-isolation and report-generation changes.
Preserve fixture-only evidence boundary	Packets are explicitly synthetic, documentation/receipts retain the limits, and no real inference or efficacy is claimed.

Revisions and integration: reviewed PR head aa196034c3b7250b0ea99c2f4b889e074b9570c3, originally reviewed base 65147a03e465dc838770474736a620e04d72337e, squash merge d99593fea76a6a8f6dc3cda7ee9fe108570a366b, parent 2e9cb896d82a16047f06896b95efad434422ae4a. The adapter, contract document, fixture writer and test module are byte-identical between the reviewed head and merge revision. The added operation-guide entry is retained.

CI scope: the separate integrator's receipt records the merge. This owner's earlier aggregate-CI hold is preserved in the Issue history; failed repository-wide declarations scans are not relabelled as successful. The exact-head knowledge-model-policy status is SUCCESS / Not applicable (trusted-main scan). This acceptance does not claim that all repository-wide declaration jobs are green.

Preserved local history: the initial one-minute Windows test envelope expired before the validation command could start, with both metrics and source integrity already successful. Only the ordinary test envelope changed to three minutes; assertions and production behavior did not change. The refreshed head passed the focused integration, and the merge revision has now passed the same integration again. Synthetic metric values are test inputs, not a measured performance gain.

P0 compatibility correction — reopening after independent re-check

The owner explicitly requested reopening with this finding. This is a defect within the original offline-adapter P0 scope, not a new phase or a requirement to run models/GPUs. The earlier acceptance receipt and checked checklist are historical evidence, not proof that this newly identified compatibility gap is cleared.

Reproduced failure

Pinned Flare bce032c014bae34502ba22d3304fa084d5b40afe writes observation.json in stream_request(), then adds analysis.http_prefill_tokens_per_second to the in-memory observations for ttft-4096 / prefill-65536 before writing measurements.json.

InferRSI requires whole-object equality between the sidecar observation and an entry in measurements: flare.py:330–334. Consequently a normal packet produced by the pinned source is rejected with ValueError: observation sidecar differs.

Pure offline reproduction: the existing synthetic fixture validates; adding that producer-derived field only to measurements and refreshing its manifest digest makes validation fail. Existing fixture construction writes identical objects to both locations and misses the producer's real write order. No service, model or GPU execution was performed. Adapter/docs/tests/fixture are unchanged between the accepted merge d99593fea76a6a8f6dc3cda7ee9fe108570a366b and inspected main a4a0fbd62defe2bc53ab090e1fa50bd7139663a6.

Smallest corrective follow-up

Start from current main and preserve the original raw-observation integrity contract. Compare the producer-defined raw fields appropriately; explicitly handle its documented derived-analysis augmentation. Do not remove equality/integrity checks wholesale or silently accept arbitrary divergent data.
Add a faithful pinned-producer positive regression covering the sidecar/measurements write-order difference and retain negative tests for genuine raw-field divergence, digest mismatch, grouping/clock/capacity/counter errors. Recompute the selected metric from raw evidence, not the extra summary field.
Re-run focused adapter and existing validation_command integration tests, relevant repository CI, and re-check the original six P0 acceptance criteria. Report exact source/head SHAs, commands, results and remaining limits.
Deliver a narrowly scoped corrective PR referencing this issue. Keep this issue open until the correction is independently reviewed and re-accepted. No automatic closing keyword is needed in the PR body.

Optional restoration/preparation research, #148 distillation, real service performance and agent efficacy remain separate. No new runner/evaluator/platform, model/GPU experiment or acceptance claim is authorized by this correction.

P0 re-acceptance — 2026-10-26

Decision: ACCEPT the original offline-adapter P0 and complete this issue. Acting on the user's explicit request to take responsibility for the acceptance decision, I rechecked the original six criteria, the reopening defect, corrective #155, independent review and actual integration. No extra phase or real-model/GPU experiment is required by this scope.

Corrective head 5a2d16eeaa02c63d2b355665db933d5cea576685 is now integrated as 0c78a7e257dfaa3573b77581a39539e8a3a5da4c. The actual squash tree 7645f47ead5ef29fd47c1f215838d12aeb01aca3 exactly equals Ready-tested synthetic merge ea1b7f9f98b2b03a99feb7a13540c39fa54e94a4.

Original P0 criterion	Re-acceptance evidence
Frozen schema/readset and one metric	Pinned Flare contract and raw client TTFT definition unchanged.
Raw recomputation, immutable input, bound identities	Exact documented derived-field augmentation is allowed; all original fields remain compared. Poisoned summaries still yield the raw metric and unchanged input bytes.
Producer-shaped positive and required negatives	Sidecar-before-measurements augmentation is represented faithfully; both augmented groups, other groups and refreshed-digest divergence on both copies are covered. Existing grouping/clock/capacity/counter/digest negatives remain.
Invalid/incomplete inputs do not manufacture PASS	Divergence yields rejection/empty metrics; actual command integration retains incomplete-peer INCONCLUSIVE/no metric.
Existing validation_command integration	Corrected fixture passes through the existing RunSpec/executor/runner/evaluator. No parallel execution system.
Honest offline-only boundary	Acceptance is synthetic contract/source-compatibility evidence, not service quality, model/hardware performance or agent efficacy.

Independent exact-head LGTM, four successful Ready jobs, successful declarations, refreshed required status success and post-merge manifest/CPU smoke success support this decision. The separate main-push declaration run 40293527272 was cancelled, not passed; no all-run-green claim is made. Initial merge was rejected during pending status and retried only after current status success, without bypass.

flowchart LR
  A["Pinned producer: save sidecar"] --> B["Append known analysis to measurements"]
  B --> C["Verify original fields + exact allowed augmentation"]
  C --> D["Recompute raw metric / preserve rejection semantics"]
  D --> E["Existing validation_command + evaluator"]

The reopened P0 compatibility defect is resolved in main. Historical original acceptance and reopening evidence remain preserved. #148 source distillation, optional restoration/preparation research and any future real-service studies retain their own scope and authorization; none is silently accepted here.

Navigation Menu

Repository navigation

[RFC] Flare beyond distillation: offline qualification evidence adapter #147

Conclusion / Minimal Increment

Why This Is Worth Doing

Three Directions, Starting with the First Slice

The Actual Gap for P0

P0 Acceptance Checklist

Follow-up Research Questions, Not to Be Implemented in Advance

Division of Responsibilities with Existing Work

Evidence and Limitations

Independent Distillation Task

Current corrective slice — implementation and PR only

Issue owner claim and accepted P0 plan

P0 delivery ready; integration held by repository-wide CI

Accepted: Flare offline delivery evidence adapter (P0)

P0 compatibility correction — reopening after independent re-check

Reproduced failure

Smallest corrective follow-up

P0 re-acceptance — 2026-10-26

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[RFC] Flare beyond distillation: offline qualification evidence adapter #147

Description

Conclusion / Minimal Increment

Why This Is Worth Doing

Three Directions, Starting with the First Slice

The Actual Gap for P0

P0 Acceptance Checklist

Follow-up Research Questions, Not to Be Implemented in Advance

Division of Responsibilities with Existing Work

Evidence and Limitations

Independent Distillation Task

Activity

Dong1017 commented on Oct 25, 2026

Current corrective slice — implementation and PR only

Issue owner claim and accepted P0 plan

Dong1017 commented on Oct 26, 2026

P0 delivery ready; integration held by repository-wide CI

Dong1017 commented on Oct 26, 2026

Accepted: Flare offline delivery evidence adapter (P0)

Dong1017 commented on Oct 26, 2026

P0 compatibility correction — reopening after independent re-check

Reproduced failure

Smallest corrective follow-up

Dong1017 commented on Oct 26, 2026

P0 re-acceptance — 2026-10-26

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions