feat: bound pending native execution work

.agents/skills/add-new-model/SKILL.md

@@ -50,12 +50,13 @@ redesign serving infrastructure or introduce a general model framework.
 | `recipe/<model>/` | Checkpoint preparation/export scripts, setup and launch instructions, and example requests. Reusable runtime implementation belongs in `src/`. |
 | `tests/<model>/` and `tests/<shared-component>/` | All test bodies, test helpers, and fixtures, including contract, tokenizer, checkpoint, HTTP, and kernel tests. |
 
-The [native runtime](../../../src/runtime/README.md) provides serial FIFO admission
+The [native runtime](../../../src/runtime/README.md) provides bounded serial FIFO admission
 and blocking dispatch per loaded executor. Integrate with available worker
 interfaces while separating processors and batch adapters from forward code.
 Models declare their execution unit, layouts and compatibility constraints;
-shared runtime code owns admission policy. Processing orchestration, queue/token
-budgets and dynamic batching remain planned. The native
+shared runtime code owns admission policy. Document pending-unit limits and the
+worker's overload response; preserve capacity after dispatched cancellation.
+Processing orchestration, token budgets and dynamic batching remain planned. The native
 target uses Rust host orchestration and bindings with CUDA/Metal device kernels.
 
 **Do not add test code or fixtures under `src/`.** This includes inline Rust test

.agents/skills/precheck-pr/SKILL.md

@@ -34,7 +34,8 @@ user instructions for those actions.
   processing and scheduling, model-specific adapters and executors, and hardware
   operations. In the native target, Rust orchestrates host work and dispatch;
   CUDA/Metal execute device kernels. Serial runtime admission and dispatch are
-  implemented; further orchestration and dynamic batching remain planned.
+  implemented with bounded pending units; further orchestration, token budgets
+  and dynamic batching remain planned.
   Consult the relevant `src/frontend/`, `src/runtime/`, `src/models/`, `src/backends/`, or
   `recipe/` documentation for the changed component's contract and current status.
 - For frontend changes, inspect affected request validation, error handling,
@@ -47,6 +48,9 @@ user instructions for those actions.
 - For processing/scheduling changes, apply the architecture contract's batching,
   output reconstruction, numerical, and lifetime invariants to the affected
   model's prepared-work and executor boundaries.
+  Verify overload rejects before dispatch, capacity recovers on queued cancellation
+  and completion, dispatched cancellation retains capacity, and worker status/body
+  mappings distinguish overload from inference failure.
 - Check that tests exercise the changed behavior. For bug fixes, look for a
   regression case that fails before the fix. Verify documentation and examples
   against the implementation.

.agents/skills/self-review/SKILL.md

@@ -25,6 +25,7 @@ For processing or batching changes, trace prepared inputs through executor
 layouts and output reconstruction. Apply the architecture contract's batching,
 numerical, and lifetime invariants to the affected model. For shared runtime
 admission changes, check FIFO execution units, cancellation before dispatch,
+pending-unit limits, overload/error response mapping, startup configuration,
 permit/resource retention after dispatch, and release on errors or panics against
 the [runtime contract](../../../src/runtime/README.md).
 

.agents/skills/system1-omni-review/SKILL.md

@@ -23,7 +23,7 @@ baseline; target code and configuration determine current paths and commands.
 - Check separate transport, processing, scheduling, model-executor, and backend ownership against the architecture contracts. Model-specific processors and batch adapters preserve semantics; shared runtime code owns orchestration and scheduling when implemented. Rust owns native host orchestration and bindings/dispatch; CUDA/Metal supply device operations, including optional processing transforms and packing. Serial runtime admission and dispatch are implemented; further orchestration and batching remain planned. CUDA and Metal need not have identical internals. A directory or pass-through HTTP support does not prove native support or batching.
 - For `src/frontend/`, inspect exact envelope/body and end-to-end header preservation, hop-by-hop filtering, backend path prefix/query handling, timeout/status mapping, health forwarding, cancellation and shutdown. The configured backend bypasses proxies, follows no redirects and retries no requests; changes need transport regression evidence, not a model benchmark.
 - For models, trace serialization → prompt/template → token IDs → one-pass prefill → scoring → response. Preserve the specific model's contract; do not impose Cua-S1's 26-letter choice restriction on a new model that implements score/noul. Verify pinned checkpoint/adapter/tokenizer/configuration identity, shapes/dtypes and readiness after real initialization.
-- For processing or batching changes, apply the architecture contract's compatibility, reconstruction, numerical, and lifetime invariants to the affected model. For admission changes, inspect the [runtime contract](../../../src/runtime/README.md), execution-unit granularity, cancellation before/after dispatch and permit/resource lifetimes. Require executor/kernel evidence for batch capability.
+- For processing or batching changes, apply the architecture contract's compatibility, reconstruction, numerical, and lifetime invariants to the affected model. For admission changes, inspect the [runtime contract](../../../src/runtime/README.md), execution-unit granularity, pending limits, overload versus inference-error responses, cancellation before/after dispatch and permit/resource lifetimes. Require executor/kernel evidence for batch capability.
 - For native/FFI/CUDA changes, inspect Rust and C ABI in lockstep, buffer and graph lifetimes, stream/device ownership, sizes/overflow, BF16 rounding locations, and error propagation/recovery. Compile success and ignored GPU tests do not validate kernels. Test changed inputs, reuse, growth, eviction and failure recovery when graph/cache behavior changes.
 - For performance or numerical claims, freeze baseline/candidate, hardware/toolchain, checkpoint, input manifest, precision, variables, tolerances and budget before comparing. Separate quality/parity from speed, native inference from frontend overhead, and cold/capture costs from warmed latency. Preserve errors and denominators; never relax tolerances or discard failures to produce a winner.
 

CONTRIBUTING.md

@@ -15,7 +15,7 @@ Keep processors and batch adapters separate from the model's forward
 implementation, even when they live in the same model directory. The README's
 target architecture shares processing orchestration and scheduling across
 models. Native workers already reuse
-[serial admission and dispatch](src/runtime/README.md); processing orchestration
+[bounded serial admission and dispatch](src/runtime/README.md); processing orchestration
 and GPU batching remain planned. Model executors own weights, forward passes,
 learned heads, and device state.
 

README.md

@@ -47,7 +47,7 @@ shared CUDA kernels in this repository.
   model workers.
 - **Model-owned execution.** Model executors own weights, forward passes,
   learned heads, device state, and kernel selection. Native workers have separate
-  processing and executor modules, with shared FIFO admission and blocking
+  processing and executor modules, with shared bounded FIFO admission and blocking
   dispatch in the [native runtime](src/runtime/README.md).
 - **Native CUDA workers.** The Cua-S1 4B 0.2 `text` adapter and
   Open-Jev-27B-v1.1 run as native workers with shared CUDA kernels. Cua-S1
@@ -68,7 +68,8 @@ Share processing and scheduling; let each model own its execution.
 The diagram shows the **target architecture**. Today the frontend forwards HTTP
 requests to separately running workers, whose handlers coordinate independent
 processors and executors. Both native workers use shared FIFO admission and
-blocking dispatch per loaded executor. Processing orchestration, batch budgets,
+blocking dispatch, with configurable pending-unit limits per loaded executor.
+Processing orchestration, token/batch budgets,
 compatibility grouping and dynamic batching remain planned.
 
 | Layer | Responsibility | Native target implementation |
@@ -161,7 +162,7 @@ requests and a matched comparison with raw HF Transformers and OpenJev-Fast.
 
 The current focus is the native Cua-S1 and Open-Jev CUDA workers and the serving
 benchmark harness. Planned work extends the shared runtime with processing
-orchestration, admission budgets, compatibility grouping and bounded dynamic
+orchestration, token budgets, compatibility grouping and bounded dynamic
 batching with batch-capable executors,
 the in-repository LAYA model engine, additional model engines and GPU backends
 including Metal, and per-model

docs/architecture.md

@@ -13,7 +13,8 @@ model-specific workers coordinate independent processor and executor modules
 through `prepare` → `execute` → `finish`. The shared Qwen executor accepts one
 prompt per forward call. Both workers use the
 [native runtime](../src/runtime/README.md) for FIFO admission and blocking dispatch
-per loaded executor. Shared processing orchestration, batch budgets,
+per loaded executor, with bounded pending execution units. Shared processing
+orchestration, token/batch budgets,
 compatibility grouping and dynamic batching are planned.
 
 The native workers currently compute their decision heads on the CPU after
@@ -92,7 +93,13 @@ executor, with FIFO waiting. Engines own scheduler instances; model-specific
 execution adapters declare the unit and submit owned closures. Waiting is async,
 so queued requests do not occupy blocking threads waiting for a model lock.
 Schedulers are local to each worker/executor; this does not coordinate separate
-processes. Queue-length limits and token budgets are not implemented yet.
+processes. Pending execution units (waiting plus dispatched) are bounded to 64 by
+default, configured by `OMNI_NATIVE_MAX_PENDING`. Full capacity rejects the unit
+before dispatch with a typed overload error; native workers map it to HTTP `503`
+using their model-specific error body. Normal inference failures remain `500`.
+The limit counts Cua-S1 question forwards and Open-Jev complete requests, not
+prepared tokens or ingress requests. A later rejected Cua-S1 unit fails the
+whole request without partial answers. Token budgets remain planned.
 
 - Batch only work accepted by the same loaded executor, with compatible
   checkpoint/adapter identity, device, dtype, and input layout. Models declare
@@ -120,7 +127,8 @@ Keep Rust declarations and the native ABI in sync, and rebuild all consumers
 when the ABI changes. Buffer handles and captured graphs must outlive queued
 device operations; synchronize before reading host results or reclaiming
 storage. Cancellation while waiting for admission removes the queued caller
-without dispatch. Once dispatched, the runtime retains admission and captured executor
+without dispatch and releases pending capacity. Once dispatched, the runtime retains
+both admission and capacity permits and captured executor
 resources until the blocking closure completes, even if its caller is cancelled.
 The executor must synchronize device work before returning; cancellation or
 timeout does not authorize freeing buffers still in use by submitted work. Keep

docs/benchmarks/native-admission/README.md

@@ -0,0 +1,151 @@
+# Bounded native admission validation
+
+Actual runs on 2026-10-05 compared merged main `9adb0701132d259a9fcc8a559472f1c3dc5590b0`
+with runtime candidate `b5c443c9310f2118289500b6088822e7bba1fb72`. All 1,536 measured
+requests succeeded with exact response/key-order parity, excluding Open-Jev
+`metadata.inference_seconds`. All six predeclared performance gates passed.
+The limit-one functional burst accepted one request and rejected seven with
+HTTP 503 for each model; a subsequent inference returned 200 with exact parity.
+
+The [protocol](protocol.json) records source/build/binary hashes, pinned models,
+controls, input hashes, budget and thresholds. The [compressed evidence](raw-results.json.gz)
+contains every measured/feasibility response, warmup, first-inference, near-limit,
+expected-error and overload/recovery record, plus fixed input manifests,
+per-run summaries, startup and raw device-memory samples.
+
+## Change and controls
+
+The independent variable is bounded pending execution and its overload mapping.
+Execution remains serial. Normal-load comparisons set `OMNI_NATIVE_MAX_PENDING=64`
+(the baseline ignores this new setting); candidate-only functional bursts use 1.
+The bound counts Cua-S1 question forwards and Open-Jev complete requests. It does
+not bound ingress requests or prepared-token memory; GPU batching remains planned.
+
+Controls: one reserved SM90 NVIDIA L20X, exact GPU5, driver 570.133.20, CUDA driver
+API 13000, NUMA memory node 1/CPUs 56–71, Rust 1.98.1,
+Python 3.12, httpx 0.28.1, tokenizers 0.22.2; BF16 backbone, existing FP32 CPU heads/f64
+normalization, unchanged CUDA ABI4 library, eager CUDA, 16 Tokio workers and disabled
+tokenizer parallelism. Checkpoint revisions and input hashes are in the protocol.
+No caches were dropped, models downloaded or exports repeated. Baseline binaries
+reuse an isolated build whose entire tree equals the merged baseline; all comparison
+measurements are fresh. Candidate uses its own Cargo target and matching package flags.
+
+The PR later integrated upstream documentation/recipes and added this report.
+All Rust sources, Cargo manifests/lockfile and CUDA build inputs still match the
+measured candidate; `native_source_hashes` lets reviewers verify that equivalence.
+
+## Warmed native HTTP comparison
+
+Per model/revision: one real-warmed server, first inference validation, one excluded
+64-request feasibility pass, then two measured 64-request passes at each concurrency
+1/8/16, with three excluded warmups per pass. There are 1,536 measured requests and
+256 excluded feasibility requests. All warmup and first-inference responses also
+passed parity. Each configuration then validated a 16,368-token input and four
+expected errors outside performance timing. No extra runs or discarded failures.
+
+Gates: mean of per-run mean HTTP latency ≤105% baseline, mean requests/s ≥95%, and
+mean per-run p95 ≤110%, for each model/concurrency. Timing is direct native HTTP;
+frontend overhead and individual kernel timings are not measured.
+
+| Model | Concurrency | Baseline mean ms | Candidate mean ms | Latency change | Throughput change | p95 change | Observed gate |
+|---|---:|---:|---:|---:|---:|---:|---|
+| cua | 1 | 23.47 | 23.53 | +0.27% | -0.27% | +0.39% | pass |
+| cua | 8 | 167.52 | 167.81 | +0.17% | -0.19% | +0.09% | pass |
+| cua | 16 | 320.86 | 321.56 | +0.22% | -0.25% | +0.43% | pass |
+| jev | 1 | 108.68 | 108.69 | +0.00% | -0.00% | -0.77% | pass |
+| jev | 8 | 812.37 | 812.04 | -0.04% | +0.04% | +3.42% | pass |
+| jev | 16 | 1515.66 | 1516.07 | +0.03% | -0.02% | -0.08% | pass |
+
+
+Relative ranges across the two repetitions, `(max - min) / mean`:
+
+| Model | Concurrency | Baseline mean run range | Candidate mean run range | Baseline p95 run range | Candidate p95 run range |
+|---|---:|---:|---:|---:|---:|
+| cua | 1 | 0.04% | 0.02% | 0.42% | 0.30% |
+| cua | 8 | 0.04% | 0.36% | 0.32% | 0.18% |
+| cua | 16 | 0.28% | 0.06% | 0.29% | 1.29% |
+| jev | 1 | 0.04% | 0.18% | 0.21% | 1.25% |
+| jev | 8 | 0.03% | 0.19% | 0.22% | 0.12% |
+| jev | 16 | 0.00% | 0.06% | 0.04% | 0.08% |
+
+## Functional overload and recovery
+
+After normal-load comparisons, one candidate server per model used limit 1, with
+real warmup and a validated first inference. One burst of eight concurrent
+16,368-token requests yielded one 200 and seven 503 responses per model. The
+successful response matched the ordinary reference; each 503 had the native worker
+error shape and `native execution capacity exhausted`. The next ordinary request
+returned 200 with exact parity; `/health` remained ready. These 16 burst requests
+and recovery/startup validation are outside measured performance denominators.
+Queued/dispatched cancellation and error/panic release are covered by CPU tests.
+
+## Startup and memory
+
+CPU builds/copies and existing checkpoint exports are preparation, excluded from
+process-to-readiness and replay. Startup and first inference are separate:
+
+| Configuration | Process to readiness (s) | First inference (ms) | Sampled GPU peak incl. near-limit (MiB) |
+|---|---:|---:|---:|
+| cua-baseline | 3.38 | 143.32 | 12417 |
+| cua-candidate | 3.67 | 50.16 | 12417 |
+| jev-baseline | 15.71 | 189.11 | 55465 |
+| jev-candidate | 15.31 | 199.80 | 55465 |
+
+Memory is device footprint sampled every 200 ms, including near-limit validation;
+it is not an allocator peak or a measured-only peak. All 40 task-owned processes
+exited and the scheduler reservation was released. Full cleanup/probes remain in
+the private original archive; machine/user locations are omitted from public records.
+
+## Reproduce and inspect
+
+Build each frozen source in a separate checkout/target using:
+
+```sh
+cargo build --release --locked -j 16 -p omni-cua-s1-native -p omni-open-jev-native
+```
+
+Prepare the pinned checkpoints with the
+[Cua-S1 recipe](../../../recipe/cua_s1/native.md) and
+[Open-Jev recipe](../../../recipe/open_jev/native.md). Reuse one unchanged CUDA
+library for both revisions and honor the execution host's GPU reservation rules.
+The archive can be inspected and its fixed input files extracted without GPU use:
+
+```python
+import gzip, json
+from pathlib import Path
+root = Path("docs/benchmarks/native-admission")
+evidence = json.loads(gzip.decompress((root / "raw-results.json.gz").read_bytes()))
+for name, text in evidence["inputs"].items():
+    Path(name).write_text(text)
+print(evidence["summary"]["comparisons"])
+```
+
+For each reserved model/revision server, use the
+[repository runner](../../../benchmarks/bench.py), metadata requirements and
+[comparison protocol](../../../benchmarks/README.md). Run a feasibility pass
+and then two passes at each concurrency, reusing that server:
+
+```sh
+python benchmarks/bench.py run cua-requests.jsonl --endpoint http://127.0.0.1:8000/v1/systemone --model cua-s1-4b-0.2 --metadata metadata.json --output run-c1-r1 --phase measured --concurrency 1 --warmup 3 --timeout 180
+```
+
+Use `jev-requests.jsonl` with model `open-jev` for Open-Jev. Create metadata from
+actual reserved hardware and the protocol's pinned revisions/precision; readiness
+requires the worker's successful real warmup. Preserve every repeat and failure.
+The private launcher used `gpu run --gpu-ids 5 --wait 10m --timeout 30m` and
+`numactl --membind=1 --physcpubind=56-71`. For the functional check, restart each
+candidate with limit 1, validate its first inference, send the archive's near-limit
+request eight times concurrently, and validate the following ordinary inference.
+
+## Limits and disclosure
+
+Two repetitions provide descriptive ranges, not a confidence interval or
+statistically proven equivalence. This synthetic workload tests regression/parity,
+not model accuracy or general serving capacity; no speed winner is claimed.
+Sequential configuration blocks leave time/order, CPU and thermal effects possible.
+Metal, CUDA Graphs, full frontend performance, aggregate token budgets and GPU
+batching are unverified here. Labelled accuracy metrics and demo video are N/A for
+this admission change. Inputs are task-generated synthetic data; the public archive
+retains original response bodies and timings while removing machine-local identity,
+paths and process/reservation details. No third-party images or private/customer
+inputs are included.