Repository navigation
[RFC] Goal checkpoints: explicit progress and next actions across agent sessions #137
Description
Proposal: make the next valid step explicit across sessions, without adding another agent runtime.
Inspected InferRSI main: 7218d486. LoopX reference: a6f28057. Static inspection only; no code imported or experiments run.
Why this belongs beyond knowledge distillation
InferRSI retains experiment plans and evidence, but a returning agent still has to reconstruct what remains, which gate is unresolved, and what it may propose next from issues and run artifacts.
LoopX suggests separating durable goal facts from the model's conversational memory. The minimum useful increment is a derived goal checkpoint plus a deterministic next-action recommendation, not adoption of its control platform.
| Layer | InferRSI already has | Proposed delta |
|---|---|---|
| Goal | Issue scope, criteria, owner decisions and acceptance responsibility | One bounded checkpoint showing remaining criteria, evidence and blockers |
| Execution | Frozen RunSpec / comparison plans, existing runner and explicit authorization | Reference these identities; do not create another runner or approval mechanism |
| Evidence | Run events/status, reports, verdicts and recorded usage | Read and reconcile them without changing PASS / FAIL / INCONCLUSIVE |
| Continuation | Agent reads the above and chooses what to do | A reason-coded recommendation that is reproducible from the same inputs |
Incremental architecture
flowchart TB
subgraph Existing["Existing InferRSI"]
G["Goal / decisions"]
P["Plan / authorization"]
E["Evidence / usage"]
R["Runner / gates"]
end
subgraph Delta["Small new layer"]
C["Checkpoint"]
D["Next decision"]
V["Markdown view"]
C --> D
C --> V
end
G --> C
P --> C
E --> C
D --> A["Agent / owner"]
A -->|"Existing authorized path"| R
R --> E
classDef existing fill:#eaf2fb,stroke:#6085aa,color:#172b4d;
classDef delta fill:#fff1dc,stroke:#c58b32,color:#553b13;
classDef external fill:#eeeeee,stroke:#888888,color:#333333;
class G,P,E,R existing;
class C,D,V delta;
class A external;
Blue: existing components. Orange: proposed increment. Grey: consumer.
There is deliberately no decision-to-runner execution edge. Restoring context is not resuming an interrupted experiment.
First slice: one small offline implementation
- Build a checkpoint from explicit inputs. Reference the goal/accepted decision, criterion IDs, frozen plan, result artifacts and explicit authorization evidence. Include source revisions/digests and observed usage. Do not invent approved criteria from issue prose or reinterpret a third-party comment as authority.
- Generate a compact local Markdown view. Show goal, remaining gates, evidence links, blockers and next action. It is a derived view, not an editable source of execution permission. Rebuild rather than maintain parallel Markdown, JSON and registry histories.
- Return a typed recommendation with reasons and input identity. Prefer one small Python decision function near the existing workbench/controller code. CLI wiring should be thin and follow repository conventions; settle its exact owner in preflight.
Illustrative decision cases, not an execution API:
| Observed facts | Recommendation |
|---|---|
| Stale/missing receipts, interrupted or ambiguous attempt | Repair evidence / prepare a new plan; do not retry execution automatically |
| Changed plan/revisions/settings or absent authorization | Request authorization for the exact new plan |
| Exhausted declared limits | Stop at the current budget boundary |
| External dependency or review outstanding | Wait, naming the dependency |
| Evidence supports a bounded next step | Propose that step through the existing authorized path |
| All criteria have explicit acceptance evidence | Report accepted completion; never infer it from merged code or an empty task list |
Use existing run/status artifacts and recorded usage; use comparison integrity receipts where available. Main's comparison command binds authorization to the exact plan hash, not a generic persisted authorization-receipt service. No new append-only accounting ledger in the first slice. Unknown token/cost data stays unknown; observed-token stopping is not a provider-side hard spending cap.
Reuse and boundaries
- Reuse RunSpec, runner artifacts, skill comparison controller and existing verdict/report contracts.
- Keep comparison execution fresh-only: its contract has no general resume/retry service. A checkpoint must not silently add one.
- No new registry, database, scheduler, distributed leases, daemon, dashboard, model runner, implicit authorization, automatic acceptance or knowledge promotion.
- Reading/rendering/deciding makes zero model calls and zero external mutations. Do not scrape private agent sessions or store secrets.
Offline acceptance
- Identical pinned inputs produce identical checkpoint and recommendation; reasons cite exact evidence.
- A merged PR with unmet goal criteria remains incomplete. Unknown owner acceptance remains unknown.
- Stale plans, missing/corrupt evidence and interrupted attempts cannot become runnable or accepted.
- Limit exhaustion stops; missing usage is not zero. Reporting does not debit, authorize or repeat a run.
- An agent can reconstruct the next step from the generated view without earlier conversation history in a deterministic fixture.
- Existing grader and paired-outcome semantics remain unchanged.
- Deliver a before/after example, focused tests and operation guidance. No paid pilot is required for this software slice; improved agent effectiveness needs separate evidence.
References: what to borrow, not what to install
| Source | Useful idea | Limit |
|---|---|---|
| LoopX typed run decision and Python adapter | Facts → ordered, reason-coded decisions | Current semantics are partly TypeScript, not a standalone Python quota library |
| Spend commit | Bind replay to immutable operation identity and request digest | Local settlement recovery is not exactly-once external execution; future reference only |
| Todo event retirement | Keep source authority distinct from projections | Historical registry/Markdown/events/task-lease layouts are not a current mutation protocol to copy |
| vLLM #54951 author's report | LoopX used to reduce goal drift within a larger optimization system | Reported adoption, not a controlled measurement of LoopX's contribution to performance |
LoopX's NOTICE credits huangruiteng and contributors; this RFC makes no corporate-ownership claim. Current source is Apache-2.0; older MIT releases do not make current code MIT. This proposal borrows design concepts only. Any later code reuse needs a pinned readset and applicable license/NOTICE retention.
Related work and next decision
- Skill Eval v1.1: automated evaluation for skill additions and updates #88 / Skill Eval: automate PR evaluation planning and feedback #101 / draft feat: plan and report skill PR evaluations with explicit authorization #118: consume existing evaluation/PR evidence; not a fourth Skill Eval implementation workstream. feat: plan and report skill PR evaluations with explicit authorization #118's duplicate-execution/publication controls are not yet main.
- [RFC] Skill Indexer: better skill and context selection through collect #128: collect chooses knowledge; this proposal explains goal progress. It does not rerank skills or replace collect.
- Proposal: controlled task-local harness improvement on top of Skill Eval #93: preserves its frozen harness-improvement gates, rather than introducing unlimited search.
- [Tracking] Knowledge PR admission: model and effort policy #50: model-declaration compliance is not execution authorization or task acceptance.
- [Tracking] SearchSpace distillation coverage and knowledge consolidation #91: optional LoopX knowledge distillation is separate from this runtime-facing design.
Next: confirm this checkpoint-only scope, select one existing workflow as the offline fixture, and identify the narrow code owner. Implement nothing broader merely to mirror LoopX.
Separate source-distillation task — 2026-10-25
LoopX source distillation is now tracked explicitly in #149, a native child of #91: goal authority, next-action and quota/accounting boundaries. It delivers knowledge, not this RFC's runtime checkpoint implementation; the two tracks have separate acceptance. #149 pins its newer source readset explicitly without replacing this RFC's historical source baseline.
Activity
Dong1017 commentedon Oct 25, 2026
Claim and implementation plan — goal-checkpoint owner
I am the Codex agent acting as the Issue-level owner for #137 under the user's explicit
inferrsi-issue 137delegation. I own the software slice, its acceptance evidence, PR/review follow-through and conditional closure. #149 is assigned to another agent and remains a separate knowledge-delivery track; this work will neither implement it nor depend on its completion.Preflight: current main is
65147a03e465dc838770474736a620e04d72337e. No existing delivery PR for this goal-checkpoint feature was found. The existing configured skill-comparison workflow provides frozen plans, integrity checks, retained status/trial receipts, usage and unchanged paired-outcome semantics.One focused PR
Verification: focused checkpoint and comparison regressions, lint, then the existing Ready CI lanes and independent Dong1017 reviewer-agent approval on the exact PR head. Reading/building/deciding must leave all source artifacts unchanged and make zero model calls.
Limits and dependencies: no new runner, registry, ledger, scheduler, dashboard, LoopX code import, private-session capture or model/GPU/cloud execution. No paid pilot is needed or authorized. This slice establishes software behavior only, not improved agent effectiveness. #149/#91 and the separate #118 execution work remain independently owned.
Next: implement this bounded offline slice in an isolated task branch, preserving unrelated local work.
Dong1017 commentedon Oct 25, 2026
Software slice delivered; integration held
Delivery: #150, current head
f1b8b87064e48750e60353a6d78ac41904abaf3d, assessed against main65147a03e465dc838770474736a620e04d72337e.Remaining integration gate: the serialized repository-wide declaration workflow has not completed successfully for all applicable checks. An earlier failed run reported #150 itself successful/Not applicable but failed on older PR status-history scan budgets. More recent PR-associated declaration attempts were cancelled while another global writer ran. A successful stable status alone does not replace the applicable workflow job. No checks will be bypassed or shared CI rewritten within this Issue.
#137 remains open and #150 unmerged until that gate passes and the current head/base, approval and all checks are refreshed. A single 30-minute owner follow-up will continue this delivery and stay quiet while unchanged. #149 remains with its separately assigned agent; this delivery does not implement or close #149/#91.
Integration refresh — 2026-10-26 02:48 UTC
Main now includes the shared policy-runner fix #146 and documentation update #144, assessed at
27328537d06898d6b87f3602548ed8ee8256f6a5. Their changed files do not alter the checkpoint/comparison implementation, its tests, or the Ready workflow definitions. The delivery head and exact-head independent LGTM remain unchanged; both inline threads remain resolved.After inspecting that external fix, I reran only the failed declaration job in run 40203618049. The latest attempt failed, using trusted policy code at
27328537d06898d6b87f3602548ed8ee8256f6a5:history-unavailable=0,budget-exhausted=0,budget-saturated=0.f1b8b87064e48750e60353a6d78ac41904abaf3dis stillsuccess / Not applicablein all three internal attempts.The whole applicable declarations job is therefore still unsuccessful. #150 remains unmerged and #137 remains open. I will not repeat this rerun without a relevant main/external-state change, bypass the failure, modify shared policy/credentials, or expand this Issue to repair other PRs. The 30-minute follow-up remains quiet while unchanged.
Integrated delivery refresh — 2026-10-26
Current delivery head is
34f2f2ac0f00eba51061913a55231b8d2667ca8f, integrating main0cffd20754e3202e5238a2878ac7653765e43d63by a normal merge commit. The sole conflict was adjacent operation-guide rows; both the checkpoint and campaign-chart entries are retained. The same five-file checkpoint diff remains (+915/-2); its implementation, tests and goal guide are unchanged from the preceding reviewed head. The task worktree is clean.34f2f2ac0f00eba51061913a55231b8d2667ca8fand has been read back by the Issue owner. The preceding LGTM was not reused as new-head approval. No unresolved inline threads remain; the PR is Ready and mergeable.d99593fea76a6a8f6dc3cda7ee9fe108570a366b: current feat: derive offline goal checkpoints and next actions #150 head succeeded/Not applicable in all three internal attempts, while Record MiniCPM-o throughput search, quality gates and peer baselines #51/Add Qwen3.8 FP8 correctness and service qualification workflows #55 invalidation returned HTTP 422. Old scan-budget error counts remain zero. The whole-job gate remains unsatisfied; superseded/cancelled PR-associated jobs are retained.#150 is still unmerged and #137 open. No manual CI rerun was issued during this refresh; normal synchronization checks ran for the new head. Wait for a relevant external fix, refresh current main/head/review/all CI, then follow the existing conditional squash/whole-Issue closure workflow. No shared CI/credentials changes, model pilot or work on #149/#91 is included.
Dong1017 commentedon Oct 26, 2026
Accepted: offline goal checkpoints and deterministic next proposals
flowchart LR G["Explicit goal and owner decisions"] --> C["Read-only checkpoint"] E["Existing plan, receipts and usage"] --> C C --> N["Reason-coded next recommendation"] C --> V["Disposable Markdown / JSON"] N --> O["Agent / acceptance owner"] V --> O classDef existing fill:#eaf2fb,stroke:#6085aa,color:#172b4d; classDef added fill:#fff1dc,stroke:#c58b32,color:#553b13; class G,E,O existing; class C,N,V added;621ea37f. An explicit goal plus existing comparison evidence now produces a reproducible checkpoint, typed next proposal and compact local view. The view adds no execution or acceptance authority.34f2f2ac0f00eba51061913a55231b8d2667ca8f; four Ready jobs passed, independent exact-head LGTM, and post-merge manifest/CPU smoke passed. Merged implementation, focused tests and goal guide match the reviewed head byte-for-byte. Zero model experiment calls.Full acceptance record
Acceptance responsibility and revisions
This Issue owner accepts the bounded software slice under the user's explicit
inferrsi-issue 137delegation and public owner plan. PR merge alone is not the acceptance evidence.34f2f2ac0f00eba51061913a55231b8d2667ca8f, integrating main0cffd20754e3202e5238a2878ac7653765e43d63.621ea37f4bde80ab95d157b7459fc3032e499e5f, parentd99593fea76a6a8f6dc3cda7ee9fe108570a366b; merged state and SHA read back. The PR was already merged by another session when this final acceptance pass began.Seven offline criteria — accepted
wait_acceptance; acceptance binding cases bind goal, scope, plan, criterion, evidence and named acceptance owner.read_reportfrom explicit report writes; grading is unchanged.Verification and preserved limitations
44 passed in 198.99s;ruff check src tests toolsand PR diff check passed. PR evidence records the tested scope. Existing fixed-output offline fixtures and real local subprocess/receipt paths were used; no real model experiment ran.