[huangruiteng]

Shared model throughput can exceed a tokens-per-minute quota even when benchmark case counts remain below their concurrency limits. This adds an optional, durable token-capacity envelope to the existing benchmark-toolkit and composes its decision with the existing case headroom receipt.

The goal-scoped helper provides a trailing 60-second usage window with disjoint uncached-input, cached-input and output counters; explicit measured/estimated/unmetered coverage; exact-run forecast reservations; evaluator and cold-start reserves; and bounded capacity-rejection cooldowns. Six explicit CLI commands configure, observe, inspect, admit, reject and release that aggregate state.

The runner owns complete provider observation, quota allocation among goals, credentials, job discovery, launch, rollback after case denial, liveness and termination. Task, score, evaluator and submission semantics are unchanged. This is a local admission-helper delivery; real provider integration, installed adoption and long-horizon net benefit are not qualified here.

Repair at 5e5ab17e375d144c0d59ca67eaa440a8e0f4586b:

• Sample implicit mutation time inside the existing locked build transition for configure, observe, reject, admit and release. Preserve explicit backward-time refusal.
• Scope token-only observation, reservation and release instructions to configured envelopes in both catalog and README; ordinary case admission remains usable without token configuration.
• Preserve the causal lock-order lesson in the self-repair pattern catalog.

Validation at that exact head:

• 210 passed, zero skipped across token capacity, concurrency envelope, adaptive concurrency and toolkit tests.
• Four reverse-lock regressions fail on the initial owner and pass after repair; eight preview/execute explicit-time rejection cases retain unchanged state.
• Real CLI, wall-clock and file-lock probe: original owner A exits 2/B exits 0 with one reservation; repaired owner both exit 0 with two exact reservations.
• Pinned main/head legacy configure/admit/status/release output is identical after normalizing generated timestamps only. Configured receipt composition, unrelated/new Goal isolation, exhausted-quota escape refusal and release-to-progress pass.
• Changed-Python Ruff, repository-declared Mypy (17 source files), compile, diff hygiene, bilingual RFC difference check and all eight public-path scans pass.
• Standard premerge: 19 selected checks passed, zero failures, plus six direct checks. Exact-scope change-quality receipt cqr_b82e2cf6ae3708c8f360 is valid.

Premerge reports manual_review_required only for benchmark-sensitive maintainer judgment. Review, current user merge authorization and final unchanged-head merge readiness remain separate gates. Remote CI was not consulted under the configured local-validation policy. Earlier setup/interrupted premerge attempts are retained as non-passing evidence; the complete frozen-head run supersedes them for these checks.

Placement: builtin benchmark-toolkit, provider loopx-core. Reuse existing file locks, atomic state writes, domain-state routing, timestamp validation and benchmark_resource_headroom_receipt_v0; no additional scheduler, generic control-plane owner or speculative compatibility layer. No rendered UI changes.

[loopx-agent]

Reviewer: model_agent; gpt-6.1-sol; OpenAI; runtime_reported; reasoning_effort=max

REQUEST_CHANGES：发现两个 P2：正常并发的锁获取顺序会误触时钟回退检查；未配置 token envelope 的通用指导增加了无法满足的必做步骤。

动机 / Motivation

共享模型吞吐额度的 benchmark runner 需要在新 run 启动前判断是否还有可用 token 容量。过去只有并发 case slot 和可选资源回执；本 PR 增加滚动一分钟用量、预测保留与冷却，以便在独立获准启动前挡住吞吐超额。已验证 token 回执可组合既有 case gate，但正常并发时序会被误判为时钟回退，通用指导也扩大了未配置场景的必做步骤。本次不实现 provider 采样器、跨 Goal 额度分配、runner 启动、真实存活检测或评分。实际 provider 的完整观测、runner 集成和长期效果仍由原消费者资格化；当前两处缺陷须先修复。

改动思路 / Approach

TPM 指每分钟 token 容量。新 owner 将未缓存输入、缓存输入和输出分开计量，用滚动 60 秒用量与 exact-run forecast 中较大者判断容量，并保留 evaluator/cold-start reserve 和 provider 冷却。它把判断转换为已有 headroom receipt，再交给 case gate；采样和 run 生命周期仍属 runner。

六个显式 CLI 和本地文件状态，不承担 provider/runner 调度。这个有界方案复用了现有文件锁、原子写和资源回执，边界合理；当前应局部修正时间顺序和可选指导，毋须另建 scheduler。此处选用下面的 base/head 对照表说明控制分支，未生成图示。

规格依据：loopx/capabilities/benchmark_toolkit/README.md，不可变 revision 843be494266774b63b3de61d0adf5dd7f4f877a2。以下 criterion_id 对应该版本已有条款：EC-atomic（独立获准启动前原子取得 case slot）已保留；CE-optional（按资源需要启用 headroom）被无条件 token 指导扩大；CE-shared-lock（共享文件锁协调并发）在新 TPM 路径出现下述反例。

具体改动 / Changes

已检查全部 7 个文件（+1990/-6）：1010 行 token owner、213 行 CLI、dispatch/exports、catalog、README 和 502 行测试。新增的是 benchmark 局部 policy/sample/state/decision schema，未修改通用控制面词汇；measured、estimated、unmetered 不混用。零请求 measured heartbeat 依赖 runner 提供完整观测，本次没有把它当实际 provider coverage 证明。

关键代码讲解

• admit_benchmark_token_capacity 在锁外取默认时间，再在锁内读状态并写 reservation。锁保证不会超额写入，却没有保证时间戳与提交顺序一致。
• _token_admission 先检查配置、时间、coverage 和 cooldown，再计算 solver/evaluator 预算；成功生成已有资源回执。重试同 run 不重复保留，显式拒绝保持 typed reason。
• agent_usage.required_sequence 是所有 catalog 读取方看到的指导。仅配置步骤有 shared-throughput 条件，后续 observe/reserve/release 没有对应条件。

独立检查	base / 预期	本次 head
未配置 token 的 case admission	原参数可用	仍成功；machine gate 未变
同一场景的 catalog	无 token-only 必做步骤	新增无条件 reserve/release；token-admit 返回 token_capacity_not_configured
A 先取时间、B 先取得锁	容量充足时两者各保留一次	B 成功、A 退出 2；新 A 重试及 replay 成功
typed cooldown / receipt composition	拒绝冷却中的新 run，已有 case gate 可消费	通过

PASS：token、concurrency、adaptive-concurrency、toolkit 四模块共 197 passed，0 skipped。真实隔离 CLI 验证 configure→observe→admit、replay、reject、release 与旧 case 路径；A 仅在真实 CLI main 的取时间与锁获取之间加入调度 barrier，B 使用正常源码 launcher，未替换时钟、文件状态或决策。全部 metering 为合成输入，无 provider、模型或 benchmark job。

对主干的风险 / Risks

P2：锁前采样默认时间造成合法并发误拒绝。 在 quota 10,000,000、target 9,000,000、零使用、每个 forecast 100 的条件下，让 A 在取时间后暂停，B 正常取得锁并提交，再恢复 A。A 在更新后的 state.updated_at 前被判为回退，CLI 只报 invalid benchmark token capacity input 并退出 2。随后状态仍 admission_ready=true、未超额且仅有 B；用新时间重试 A 即成功。现有并发测试为各请求传入同一固定时间，因而 197 项通过也不能排除此缺陷。最小修复是在锁内采样隐式时间，继续拒绝真正显式回退值；用真实时钟与反序锁 barrier 固定该回归。configure/reject/release 的相似顺序也应一并核对，当前独立复现限于 admit。

P2：将可选 token 路径写成所有 campaign 的必做步骤。 无 token policy 时读取实际 catalog，仍得到 atomically_reserve_token_capacity_before_case_slot_admission 等步骤；实际 token-admit 必然拒绝，而 base/head 的旧 case admission 均成功。这是 Agent 指令范围回归，不能因旧 machine gate 不变而视作隔离成功。应在 catalog 与 README 一致限定 observe/reserve/release 的启用条件，并加未配置与已配置两条真实入口对照。

我的整体评价 / Verdict

该局部能力有用，但上述缺陷会给持续并行工作增加无效重试和额外配置负担；long_horizon 与 user_experience 均存在回归，当前不宜批准。代码量主要用于独立 token 状态与测试，有界精简机会是统一变更锁内的默认时间采样，避免复制错误顺序。真实 provider、跨 Goal 分配、安装采用和长期净收益仍未验证。仅检查 CLI、catalog 和本地文件 owner，未改变渲染页面。无页面视觉改动；未验证真实 provider 或安装采用。

English verdict: REQUEST_CHANGES at af6a7e6; fix the admission clock race and conditional token guidance. 197 tests pass, but the independent CLI counterexamples fail.

[huangruiteng]

Reviewer: model_agent; gpt-6.1-sol; OpenAI; runtime_reported; reasoning_effort=max

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

APPROVE：完整复审 5e5ab17e375d144c0d59ca67eaa440a8e0f4586b，两个既有 P2 已独立验证修复，未发现新的阻塞项。

动机 / Motivation

共享模型吞吐额度的 benchmark runner 操作者会遇到这个问题。并发 case 数未超限时仍可能耗尽每分钟 token 额度；新增准入层在启动前保留吞吐，并在观测不完整或冷却中拒绝新工作。已验证本地滚动用量、原子 reservation、冷却与既有 case gate 的组合；反序拿锁不再误拒绝，未配置 token 的旧路径保持可用。本 PR 不实现 provider 采样器、跨 Goal 共享额度分配、runner 启动与终止、评分或真实部署采用。

改动思路 / Approach

TPM 是每分钟 token 吞吐额度。新 token owner 记录一分钟内互不重叠的未缓存输入、缓存输入和输出，结合 exact-run forecast 保留、solver/evaluator 余量和容量拒绝冷却，再生成既有 headroom receipt，交给原来的 case gate。

按现有 benchmark-toolkit 的资源准入边界保留该实现：吞吐状态不能从 case 数推导，已有 receipt 与 case gate 可以直接复用。当前交付边界是 provider-neutral 的本地准入 helper 与 CLI；实际 provider 观测完整性和 runner 生命周期仍由消费者承担。这区分了必须正确的计量/原子性、显式启用的容量政策、runner 的 provider IO 和只读状态投影。没有新增后台调度器或通用控制面决策。有限重构复用了现有 build 闭包来移动时间采样，无需另一套事务抽象或静默修正显式时间。

场景	旧观察	当前精确提交
A 先调用、B 先拿锁	原 token owner：B 成功，A 退出 2，只有一个 reservation	两者均成功，各保留一次
没有 token 配置	主干 case workflow 成功	同输入的完整输出在仅规范化时间戳后相等；token-only 指导有启用条件
显式回退时间	应拒绝且不改状态	configure/reject/admit/release 的 preview/execute 八例均保留拒绝
配置后耗尽容量	新 run/class 不应逃逸	均拒绝；释放 exact fixture reservation 后可继续

表格表达已执行的 CLI 对照，无需图示渲染。

具体改动 / Changes

规格：loopx/capabilities/benchmark_toolkit/README.md，不可变 revision 0cfbf11dcea4ba7bc8fb13f06f1c915b63ce8fad。沿用前次 review frame：EC-atomic 对应独立获准启动前原子取得 case slot；CE-optional 对应额外 headroom 按需启用；CE-shared-lock 对应配置/准入/释放的共享锁；CE-private 对应聚合信息与 launch authority 边界。本提交四项均 implemented：原 case gate/receipt 保持，token 增量另有真实 CLI 与负例验证；修改 README 的启用条件是恢复既有可选边界，并非重写规格以容纳缺陷。

全 PR 为 8 文件 +2163/-5：token owner、CLI adapter、exports/dispatch、catalog、README、回归测试与一条修复经验。相对上次评审的修复为 5 路径 +189/-15。没有生成日志、私有状态或 benchmark 原始材料。新增命令集合与 metering/workload 值属于 benchmark 局部合同，复用已有 headroom 词汇；未创建通用状态/权限模型。

关键代码讲解

• configure_benchmark_token_capacity 在现有锁内调用 build、取隐式时间并原子写状态；保留样本、reservation 和 cooldown。observe/reject/release 的相邻写路径也采用相同顺序，显式时间校验不被放宽。
• _token_admission 先判配置、时间、coverage/cooldown，再算全局与 solver 余量；拒绝不生成新状态，已有 exact run 不重复保留。返回的 receipt 仍不能证明真实进程存活或授予启动权限。
• admit_benchmark_token_capacity 将取时间放入锁内 build，消除调用顺序与提交顺序不一致的误拒绝。独立探针只延迟 A 的拿锁时机，使用真实时钟、B 的正常 CLI、真实文件锁与独立状态回读。
• agent_usage.required_sequence 将 token 观测、reserve、release 全部限定为已配置 envelope；实际 capability show 输出与 README 一致。无需为未配置的 campaign 增加配置轮次。

验证：210 passed，0 skipped（token capacity、concurrency envelope、adaptive concurrency、toolkit）；Ruff 通过，仓库声明的 Mypy 检查 17 个源文件无问题；八个公开路径边界扫描零错误；双语 RFC 差分和 diff hygiene 通过。精确范围质量回执 cqr_b82e2cf6ae3708c8f360 已 valid。预合并完成 19 项选择检查，零失败；另有直接 diff/compile/RFC 检查。覆盖 real CLI 的正常、拒绝、replay、反序并发、旧路径对照、Goal/run 隔离与释放恢复。

对主干的风险 / Risks

最强反例是充足容量下合法并发仍被时间检查误拒绝；旧 owner 已复现 A=2/B=0，当前为 A=0/B=0，并有四条写路径的固定拿锁顺序回归。另一反例是未配置 token 却被指导必须 reserve；当前 actual catalog 有条件，固定主干与 head 的旧 case 路径完整输出仅去除生成时间戳后相等。不同 Goal 与后来创建的未配置 Goal 不被激活；配置 Goal 的新 run 及 evaluator 不可绕过已耗尽额度，释放后确实恢复准入。

生产 caller 是已有 benchmark dispatch 和 runner helper API；它们复用 case receipt，计量与 forecast 状态无法从 case 数推导，因此保留独立资源状态的维护成本有依据。provider 样本全部为隔离合成输入，完整采集是 runner 前提，不能把单个 measured heartbeat 当真实 provider coverage 证明。没有运行模型或 benchmark job，也没有改变任务、评分、submission、凭据或控制面权限。

语义与 CI 对齐

diff-scoped semantic inventory 识别到六个局部 CLI 命令；完整 semantic drift smoke 已通过。初次预合并缺少 TypeScript 依赖，且在检查中补写文档导致 guard 快照改变；该未完成运行保留为失败/中断证据。补齐依赖、冻结提交后重新执行，当前相关检查通过，未放宽预算、删除断言或更改准入规则。Goal 设定 wait_for_ci=false，本次未查询、轮询或等待远端 CI。benchmark-sensitive manual hold 仍需维护者决策；本次已获请求者对 #7548 修复自合并的授权，review 结论与最终 merge-readiness 分别检查。

我的整体评价 / Verdict

该精确提交完成了本次本地准入能力与两个 P2 的修复。long_horizon=improved：持久化/replay/冷却恢复和反序并发减少无效重试；user_experience=improved：未配置路径无额外设置，启用路径可看到覆盖状态、拒绝及恢复结果。上述改进限于已执行的 helper 语义，不代表长期实验收益。

已检查 benchmark CLI、catalog、README 和修复经验条目；本 PR 没有渲染页面改动。真实 provider、安装采用和长期净收益未验证。架构判断 retain，当前能力边界与维护成本相称；不需要新增框架或整体语言迁移。现有私有临时探针不进入 PR，公开保留的是测试与通用修复经验。旧阻塞只在逐项核对当前代码与证据后协调关闭；该自评 COMMENTED approval 不能替代 GitHub 权限与合并前的精确 head readiness。

English verdict: APPROVE at 5e5ab17; both prior P2 findings are independently repaired. 210 tests and real CLI parity/concurrency/recovery checks pass; premerge has zero failures. Provider integration and long-horizon benefit remain unverified and outside this helper scope.

Both P2 findings in review 5679229003 are independently verified resolved at 5e5ab17. Implicit mutation clocks now sample inside the existing lock; four reversed-lock regressions and the real-clock/file-lock CLI probe pass while eight explicit backward-time refusals remain. Catalog and README token-only steps now require configured capacity; pinned main/head legacy CLI output matches and enabled scope/recovery passes. Exact-head full review: #7548 (review). 210 tests, 19 premerge checks and exact-scope quality receipt pass. The original review and discussion are preserved; maintainer-authorized repair and self-merge reconciliation.

[huangruiteng]

Maintainer merge decision for 5e5ab17: proceed with the explicitly authorized repair/self-merge using admin merge after unchanged-head readiness. Scope is the benchmark-toolkit token admission helper, CLI/exports/dispatch, conditional catalog/README, focused tests and the reusable clock-order lesson; no task, score, runner launch, permission or provider integration semantics change.

Validation: 210 tests pass with zero skips; real CLI clock-race, baseline/head legacy parity, Goal/run isolation and release recovery pass; Ruff, declared Mypy, compile, bilingual RFC/diff checks and eight-path public-boundary scan pass. Standard premerge completed 19 selected checks with zero failures, plus six direct checks; exact-scope quality receipt cqr_b82e2cf6ae3708c8f360 is valid. These checks cover the added resource invariant and both prior P2 failures. The earlier setup/interrupted run is not treated as passing. Actual provider coverage, installed adoption and long-horizon benefit remain unverified and out of this helper boundary.

The complete exact-head COMMENTED approval was published and read back: #7548 (review). Both prior findings have verified repair replies and resolved threads; the old blocking review was natively dismissed with evidence, preserving its discussion. Approval closeout is clear. Readiness returned ready=true for this unchanged head with zero unresolved threads and admin_bypass_required=true. Remote CI was not queried or waited for under the configured wait_for_ci=false policy. The remaining benchmark-sensitive manual hold is addressed by this authorized maintainer decision; admin bypass does not replace review, validation or the immediate readiness recheck.