feat(benchmark): add token capacity admission

loopx/capabilities/benchmark_toolkit/README.md

@@ -804,11 +804,14 @@ A countable experiment uses the toolkit in this order:
 1. When domain guidance is a factor, qualify the four-arm contract and freeze its
    task-goal hashes before preregistration.
 2. Read the project experiment board before selecting or launching another case.
-3. Read or configure the concurrency envelope, then reconcile its reservations with
-   exact runner liveness.
+3. Read or configure the concurrency envelope and, when provider throughput is
+   shared, the token-capacity envelope; reconcile configured reservations with exact
+   runner liveness.
 4. Declare a `run_permission_policy_v0` and preflight the runner boundary.
 5. Atomically admit a case slot immediately before the independently authorized
-   runner launch.
+   runner launch. When a token envelope is configured, first record token usage,
+   reserve token capacity, and pass its fresh provider-capacity receipt to case-slot
+   admission; launch only after both admissions succeed.
 6. Upsert the planned or running row, then launch one frozen case/arm; do not expose
    evaluator sources or official feedback.
 7. Capture ATIF tool evidence and a runner-owned runtime attestation.
@@ -821,7 +824,8 @@ A countable experiment uses the toolkit in this order:
 11. Run the independent verifier only after the agent phase.
 12. Reduce the official result through the benchmark-owned scoring path.
 13. Upsert terminal score, countability, effort, treatment fidelity, and insight
-    status; release its reservation; then read the matched-comparison projection.
+    status; release its case reservation and, when configured, its token reservation;
+    then read the matched-comparison projection.
 14. Apply attempt-countability, treatment-fidelity, and matched-pair gates before any
     comparison claim.
 
@@ -902,6 +906,84 @@ host default such as `/tmp`; if that binding cannot be proven, report `unresolve
 LoopX never records raw metrics, paths, provider logs, or the receipt in the
 envelope, and the receipt does not grant launch authority.
 
+### Token capacity
+
+Token capacity is optional. Campaigns without a configured token envelope retain
+case-slot admission without token observation, reservation, cooldown reporting, or
+release. For a provider quota expressed in tokens per minute, configure the
+goal-scoped token envelope as a second admission dimension; the following token
+steps apply to that configured campaign. The target is an integer fraction
+of the hard quota. Cold-start reserve absorbs short request bursts, and evaluator
+reserve prevents solver backfill from consuming capacity needed to close completed
+runs:
+
+```bash
+loopx benchmark token-capacity-configure \
+  --goal-id <goal-id> \
+  --quota-tokens-per-minute 10000000 \
+  --target-utilization-basis-points 9000 \
+  --cold-start-reserve-tokens-per-minute 500000 \
+  --evaluator-reserve-tokens-per-minute 1000000 \
+  --cooldown-seconds 90 \
+  --execute \
+  --format json
+```
+
+The runner converts provider usage into request-level
+`benchmark_token_usage_sample_v0` records. `uncached_input_tokens`,
+`cached_input_tokens`, and `output_tokens` are disjoint counters, so cached input is
+never counted twice. A measured zero-request sample is an explicit coverage
+heartbeat. A request whose usage cannot be obtained must be recorded as
+`metering_state=unmetered`; any such request in the exact trailing 60-second window
+makes new admission unresolved. Estimated but complete usage remains visible as
+`complete_estimated` and is counted at its supplied value.
+
+```bash
+loopx benchmark token-capacity-observe \
+  --goal-id <goal-id> \
+  --sample-json token-usage-sample.json \
+  --execute \
+  --format json
+
+loopx benchmark token-capacity-admit \
+  --goal-id <goal-id> \
+  --run-id <run-id> \
+  --workload-class <solver|evaluator> \
+  --tokens-per-minute <forecast-tpm> \
+  --execute \
+  --format json
+```
+
+Admission uses the greater of rolling observed tokens and current reservations. A
+solver is checked below the evaluator reserve; an evaluator may consume that
+reserve. The successful decision contains a fresh
+`benchmark_resource_headroom_receipt_v0` with a `provider_capacity=sufficient`
+check. Pass that receipt to `concurrency-admit`. If case-slot admission fails,
+release the token reservation before trying another run.
+
+Record an HTTP 429 or equivalent provider capacity rejection with
+`token-capacity-reject`. LoopX keeps the longer of the policy cooldown and a bounded
+provider `Retry-After`, and new token admissions remain closed until it expires.
+The usage window, cooldown, and exact run reservations are persisted, so a
+controller restart produces the same decision. Reservations are not process
+liveness: after a typed terminal or runner-invalid transition, release the same run
+from both envelopes. Reconcile orphaned reservations from authoritative runner
+liveness; LoopX never discovers or terminates jobs itself.
+
+```bash
+loopx benchmark token-capacity-reject \
+  --goal-id <goal-id> \
+  --retry-after-seconds 120 \
+  --execute \
+  --format json
+
+loopx benchmark token-capacity-release \
+  --goal-id <goal-id> \
+  --run-id <run-id> \
+  --execute \
+  --format json
+```
+
 Read back the gate with `concurrency-status`. To disable it, rerun
 `concurrency-configure` with the same capacity values and omit
 `--require-resource-headroom-receipt`; existing active reservations are preserved.

loopx/capabilities/benchmark_toolkit/__init__.py

@@ -228,6 +228,26 @@
     BENCHMARK_TREATMENT_CONTINUATION_RECEIPT_SCHEMA_VERSION,
     build_benchmark_treatment_continuation_receipt,
 )
+from .token_capacity import (
+    BENCHMARK_TOKEN_CAPACITY_DECISION_SCHEMA_VERSION,
+    BENCHMARK_TOKEN_CAPACITY_FILENAME,
+    BENCHMARK_TOKEN_CAPACITY_POLICY_SCHEMA_VERSION,
+    BENCHMARK_TOKEN_CAPACITY_STATE_SCHEMA_VERSION,
+    BENCHMARK_TOKEN_USAGE_SAMPLE_SCHEMA_VERSION,
+    admit_benchmark_token_capacity,
+    build_benchmark_token_capacity_policy,
+    build_benchmark_token_capacity_status,
+    configure_benchmark_token_capacity,
+    default_benchmark_token_capacity_path,
+    normalize_benchmark_token_capacity_policy,
+    normalize_benchmark_token_capacity_state,
+    normalize_benchmark_token_usage_sample,
+    observe_benchmark_token_usage,
+    read_benchmark_token_capacity_state,
+    record_benchmark_token_capacity_rejection,
+    release_benchmark_token_capacity,
+    render_benchmark_token_capacity_markdown,
+)
 
 __all__ = [
     "ATIF_SCHEMA_VERSION",
@@ -272,6 +292,11 @@
     "BENCHMARK_STUDY_MANIFEST_SCHEMA_VERSION",
     "BENCHMARK_TREATMENT_CONTINUATION_OBSERVATION_SCHEMA_VERSION",
     "BENCHMARK_TREATMENT_CONTINUATION_RECEIPT_SCHEMA_VERSION",
+    "BENCHMARK_TOKEN_CAPACITY_DECISION_SCHEMA_VERSION",
+    "BENCHMARK_TOKEN_CAPACITY_FILENAME",
+    "BENCHMARK_TOKEN_CAPACITY_POLICY_SCHEMA_VERSION",
+    "BENCHMARK_TOKEN_CAPACITY_STATE_SCHEMA_VERSION",
+    "BENCHMARK_TOKEN_USAGE_SAMPLE_SCHEMA_VERSION",
     "BENCHMARK_UPLOAD_ENVELOPE_SCHEMA_VERSION",
     "BENCHMARK_UPLOAD_READBACK_RECEIPT_SCHEMA_VERSION",
     "DEFAULT_RUN_PERMISSION_ALLOWED_ACTIONS",
@@ -321,6 +346,7 @@
     "RunnerOwnedProviderGateway",
     "StdioNativeGoalTransport",
     "admit_benchmark_case",
+    "admit_benchmark_token_capacity",
     "attach_native_goal",
     "benchmark_experiment_board_row_key",
     "build_benchmark_adaptive_concurrency_decision",
@@ -341,6 +367,8 @@
     "build_benchmark_runtime_observation",
     "build_benchmark_study_dashboard",
     "build_benchmark_treatment_continuation_receipt",
+    "build_benchmark_token_capacity_policy",
+    "build_benchmark_token_capacity_status",
     "build_benchmark_upload_envelope",
     "build_native_codex_isolation_envelope",
     "build_native_goal_public_trajectory_summary",
@@ -357,9 +385,11 @@
     "compact_native_goal_receipt",
     "compact_run_permission_policy_for_quota",
     "configure_benchmark_concurrency_envelope",
+    "configure_benchmark_token_capacity",
     "convert_traex_events_to_atif",
     "default_benchmark_concurrency_envelope_path",
     "default_benchmark_experiment_board_path",
+    "default_benchmark_token_capacity_path",
     "filter_public_benchmark_artifact_paths",
     "inspect_benchmark_source_revision_fence",
     "inspect_native_codex_profile",
@@ -378,20 +408,28 @@
     "normalize_benchmark_resource_headroom_receipt",
     "normalize_benchmark_rebalance_inventory",
     "normalize_benchmark_study_manifest",
+    "normalize_benchmark_token_capacity_policy",
+    "normalize_benchmark_token_capacity_state",
+    "normalize_benchmark_token_usage_sample",
     "normalize_benchmark_upload_envelope",
     "observe_native_goal_event",
+    "observe_benchmark_token_usage",
     "preview_benchmark_experiment_board_reconcile",
     "preview_benchmark_experiment_board_upsert",
     "probe_native_goal_process",
     "read_benchmark_concurrency_envelope",
     "read_benchmark_experiment_board_rows",
     "read_benchmark_local_upload_records",
     "read_benchmark_upload_receipt",
+    "read_benchmark_token_capacity_state",
     "rebase_native_codex_loopx_workspace_state",
     "refresh_native_goal_status",
     "release_benchmark_case",
+    "record_benchmark_token_capacity_rejection",
+    "release_benchmark_token_capacity",
     "render_benchmark_concurrency_markdown",
     "render_benchmark_rebalance_plan_markdown",
+    "render_benchmark_token_capacity_markdown",
     "render_benchmark_experiment_board_markdown",
     "render_native_codex_goal_prompt",
     "run_native_goal_process",

loopx/capabilities/benchmark_toolkit/catalog_entry.py

@@ -205,6 +205,68 @@
                 "remain runner-owned"
             ),
         },
+        {
+            "command": (
+                "loopx benchmark token-capacity-configure --goal-id <goal-id> "
+                "--quota-tokens-per-minute <tpm> "
+                "--target-utilization-basis-points <bps> "
+                "--cold-start-reserve-tokens-per-minute <tpm> "
+                "--evaluator-reserve-tokens-per-minute <tpm> "
+                "--execute --format json"
+            ),
+            "purpose": (
+                "Configure a durable rolling token-capacity envelope with explicit "
+                "burst and evaluator reserves."
+            ),
+            "write_boundary": (
+                "project-local aggregate policy and reservations only; provider "
+                "routes, credentials, request text, and launch authority remain "
+                "runner-owned"
+            ),
+        },
+        {
+            "command": (
+                "loopx benchmark token-capacity-observe --goal-id <goal-id> "
+                "--sample-json <token-usage-sample.json> --execute --format json"
+            ),
+            "purpose": (
+                "Record disjoint cached, uncached, and output token counts or an "
+                "explicit unmetered request in the exact trailing minute."
+            ),
+            "write_boundary": (
+                "typed aggregate usage only; no provider payload, prompt, response, "
+                "credential, endpoint, or raw log is accepted"
+            ),
+        },
+        {
+            "command": (
+                "loopx benchmark token-capacity-admit --goal-id <goal-id> "
+                "--run-id <run-id> --workload-class <solver|evaluator> "
+                "--tokens-per-minute <forecast-tpm> --execute --format json"
+            ),
+            "purpose": (
+                "Atomically reserve forecast throughput and emit a fresh provider-"
+                "capacity headroom receipt for case-slot admission."
+            ),
+            "write_boundary": (
+                "compact run reservation only; callers own usage collection, "
+                "case-slot rollback, launch, liveness reconciliation, and release"
+            ),
+        },
+        {
+            "command": (
+                "loopx benchmark token-capacity-reject --goal-id <goal-id> "
+                "[--retry-after-seconds <seconds>] --execute --format json"
+            ),
+            "purpose": (
+                "Persist a bounded provider-capacity cooldown after an HTTP 429 or "
+                "equivalent typed rejection."
+            ),
+            "write_boundary": (
+                "timestamp and bounded cooldown only; no error body, provider "
+                "identity, route, or credential is persisted"
+            ),
+        },
         {
             "command": (
                 "loopx benchmark concurrency-tune --goal-id <goal-id> "
@@ -394,18 +456,22 @@
             "read_experiment_board_before_launch_or_case_selection",
             "qualify_four_arm_contract_when_domain_guidance_is_a_factor",
             "configure_or_read_concurrency_envelope_before_launch",
+            "configure_or_read_token_capacity_when_provider_throughput_is_shared",
             "reconcile_admission_ledger_with_real_runner_liveness",
             "backfill_when_concurrency_status_reports_underfilled",
             "qualify_source_revision_before_each_new_run_admission",
             "qualify_resource_headroom_when_the_envelope_requires_a_receipt",
+            "record_complete_or_explicitly_unmetered_token_usage_when_token_capacity_is_configured",
             "tune_target_occupancy_from_fresh_runner_feedback_when_opted_in",
+            "atomically_reserve_token_capacity_before_case_slot_admission_when_token_capacity_is_configured",
             "atomically_admit_case_slot_before_runner_launch",
             "upsert_preregistered_or_running_row_when_a_run_starts",
             "classify_exact_runtime_observation_during_active_monitor_cycles",
             "require_runtime_continuity_before_terminal_closeout_write",
             "adjudicate_restricted_access_suspicion_after_solver_and_score_terminal",
             "upsert_terminal_score_countability_effort_and_insight_status",
             "release_case_slot_after_terminal_or_runner_invalid_transition",
+            "release_token_capacity_after_terminal_or_runner_invalid_transition_when_token_capacity_is_configured",
             "read_matched_comparisons_before_selecting_the_next_arm",
         ],
         "board_commands": {
@@ -460,6 +526,45 @@
             "and score transitions. When configured, admission requires a fresh "
             "public-safe resource-headroom receipt but never persists raw metrics."
         ),
+        "token_capacity_commands": {
+            "status": (
+                "loopx benchmark token-capacity-status --goal-id <goal-id> "
+                "--format json"
+            ),
+            "configure": (
+                "loopx benchmark token-capacity-configure --goal-id <goal-id> "
+                "--quota-tokens-per-minute <tpm> "
+                "--target-utilization-basis-points <bps> "
+                "--cold-start-reserve-tokens-per-minute <tpm> "
+                "--evaluator-reserve-tokens-per-minute <tpm> "
+                "--execute --format json"
+            ),
+            "observe": (
+                "loopx benchmark token-capacity-observe --goal-id <goal-id> "
+                "--sample-json <token-usage-sample.json> --execute --format json"
+            ),
+            "admit": (
+                "loopx benchmark token-capacity-admit --goal-id <goal-id> "
+                "--run-id <run-id> --workload-class <solver|evaluator> "
+                "--tokens-per-minute <forecast-tpm> --execute --format json"
+            ),
+            "reject": (
+                "loopx benchmark token-capacity-reject --goal-id <goal-id> "
+                "[--retry-after-seconds <seconds>] --execute --format json"
+            ),
+            "release": (
+                "loopx benchmark token-capacity-release --goal-id <goal-id> "
+                "--run-id <run-id> --execute --format json"
+            ),
+        },
+        "token_capacity_boundary": (
+            "Only campaigns with a configured token envelope require token usage "
+            "observation, reservation, cooldown reporting, and release; other "
+            "campaigns retain case-slot admission without token configuration. "
+            "The token envelope owns aggregate metering, forecast reservations, and "
+            "provider cooldown only. The runner owns provider adaptation, complete "
+            "coverage, job discovery, launch, rollback, liveness, and termination."
+        ),
         "selection_rule": (
             "Keep diagnostic-only explore rows separate and make paired claims "
             "only from matched_pair_countable comparisons."
@@ -987,6 +1092,26 @@
             "module": "loopx.capabilities.benchmark_toolkit.concurrency_envelope",
             "doc": "loopx/capabilities/benchmark_toolkit/README.md",
         },
+        {
+            "schema_version": "benchmark_token_capacity_policy_v0",
+            "module": "loopx.capabilities.benchmark_toolkit.token_capacity",
+            "doc": "loopx/capabilities/benchmark_toolkit/README.md",
+        },
+        {
+            "schema_version": "benchmark_token_usage_sample_v0",
+            "module": "loopx.capabilities.benchmark_toolkit.token_capacity",
+            "doc": "loopx/capabilities/benchmark_toolkit/README.md",
+        },
+        {
+            "schema_version": "benchmark_token_capacity_state_v0",
+            "module": "loopx.capabilities.benchmark_toolkit.token_capacity",
+            "doc": "loopx/capabilities/benchmark_toolkit/README.md",
+        },
+        {
+            "schema_version": "benchmark_token_capacity_decision_v0",
+            "module": "loopx.capabilities.benchmark_toolkit.token_capacity",
+            "doc": "loopx/capabilities/benchmark_toolkit/README.md",
+        },
         {
             "schema_version": "benchmark_adaptive_concurrency_policy_v0",
             "module": "loopx.capabilities.benchmark_toolkit.adaptive_concurrency",
@@ -1077,6 +1202,7 @@
         (
             "python -m pytest tests/capabilities/test_benchmark_toolkit.py "
             "tests/capabilities/test_benchmark_concurrency_envelope.py "
+            "tests/capabilities/test_benchmark_token_capacity.py "
             "tests/capabilities/test_benchmark_experiment_board.py "
             "tests/capabilities/test_benchmark_source_revision_fence.py -q"
         )
@@ -1094,6 +1220,7 @@
         "Integrity qualification establishes countability eligibility only; an independent official result and matched experiment contract are still required.",
         "Post-run analyst access never widens the solving agent's evidence boundary or grants feedback reuse in another scored run.",
         "Concurrency reservations are not runtime liveness: runners must reconcile exact processes before release or backfill.",
+        "Token reservations are not runtime liveness or provider attribution: runners must report complete metering coverage, roll back a token reservation after case-slot denial, and reconcile exact processes before release.",
         "The experiment board is a compact projection, not a score authority: benchmark-family runners and scoring adapters remain outside the active capability.",
     ],
     "next_real_step": (