From 683c7c701e756aa41655250424270861c6e68d6b Mon Sep 17 00:00:00 2001 From: "Mr.Dong" <1042542469@qq.com> Date: Fri, 23 Oct 2026 12:49:13 +0800 Subject: [PATCH 1/2] vela: add the 2.0 0.8B hybrid reference worker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second member of the Vela 2.0 family: a pinned vela2-decoder executor (hybrid.py) with a 10-file gate (shard index and the side-loaded broad head hashed; the 2.02 GB shard by exact size), member-aware response finishing (span_heads with spans, keyed by span question ids), worker member detection from config.json model_type, and the per-member /v1/models entry. compare_reference.py serves either member; the hybrid oracle fixtures export under oracle-hybrid-. Evidence (author-run, x86 WSL2, CPU FP32, no fla — the runtime's CPU path never dispatches to it): nine-case parity vs the pinned upstream server exact-match (four 200s incl. key order, five agreed rejections incl. the 16,384-window 413); 0.3B nine-case regression; live worker smoke with span_heads and per-member thresholds; byte-identical Rust frontend forwarding; 133 CPU contract tests. --- README.md | 8 +- docs/supported-models.md | 10 +- mkdocs.yml | 2 +- recipe/README.md | 2 +- recipe/vela/compare_reference.py | 84 +++-- recipe/vela/reference.md | 144 ++++---- recipe/vela/test_compare_reference.py | 2 +- src/frontend/vela.py | 99 ++++-- src/models/vela/README.md | 122 ++++--- src/models/vela/engine.py | 11 +- src/models/vela/hybrid.py | 241 ++++++++++++++ src/models/vela/postprocess.py | 25 +- src/models/vela/protocol.py | 8 +- .../oracle-hybrid-example-request-cjk.json | 155 +++++++++ .../oracle-hybrid-example-request-score.json | 105 ++++++ ...-hybrid-example-request-set-threshold.json | 81 +++++ .../oracle-hybrid-example-request.json | 183 +++++++++++ tests/vela/test_hybrid.py | 310 ++++++++++++++++++ tests/vela/test_oracle_hybrid_fixtures.py | 64 ++++ tests/vela/test_worker_server.py | 48 +++ 20 files changed, 1528 insertions(+), 176 deletions(-) create mode 100644 src/models/vela/hybrid.py create mode 100644 tests/vela/fixtures/oracle-hybrid-example-request-cjk.json create mode 100644 tests/vela/fixtures/oracle-hybrid-example-request-score.json create mode 100644 tests/vela/fixtures/oracle-hybrid-example-request-set-threshold.json create mode 100644 tests/vela/fixtures/oracle-hybrid-example-request.json create mode 100644 tests/vela/test_hybrid.py create mode 100644 tests/vela/test_oracle_hybrid_fixtures.py diff --git a/README.md b/README.md index b2327ce0..88129980 100644 --- a/README.md +++ b/README.md @@ -164,9 +164,10 @@ Python screenshot worker, Valen-Preview-0923 has a Python reference worker for text and one-image state, CLM has a stub-encoder contract recipe, and OmniJev-4B v1.1 has its checkpoint export and CPU contract, with no worker yet. D3 (Decision 3.0) has a Python reference worker that runs the checkpoint's own -pinned runtime for text state, and Vela 2.0 0.3B has one that serves all five +pinned runtime for text state, and the Vela 2.0 members (0.3B and the 0.8B +hybrid) have one that serves all five SystemOne question kinds (`choice`/`noul`/`score` plus the `span`/`set` -extensions) on the checkpoint's own pinned runtime: +extensions) on each checkpoint's own pinned runtime: | Model | Status | | --- | --- | @@ -180,8 +181,9 @@ extensions) on the checkpoint's own pinned runtime: | CLM-v0.1-8B | [External worker with a CPU stub encoder](recipe/clm/README.md); contract checks only, real Qwen3-8B decisions unverified by this recipe | | LFM2.5-350M | [Transformers worker](recipe/lfm2/README.md); text state and independent choice questions | | OmniJev-4B v1.1 | [Checkpoint export and CPU contract](recipe/omnijev/README.md) only; the native worker is in progress ([#114](https://github.com/ThinkFlowLab/system1-omni/issues/114)) | -| D3 (Decision 3.0) | [Python reference worker](recipe/d3/reference.md) on the checkpoint's pinned runtime; `choice`/`noul`/`score` text state, media refused until stage 2; author-run on A100 (evidence in the integration PR) | +| D3 (Decision 3.0) | [Python reference worker](recipe/d3/reference.md) on the checkpoint's pinned runtime; `choice`/`noul`/`score` text state, media inputs refused; author-run on A100 (evidence in the integration PR) | | Vela 2.0 0.3B | [Python reference worker](recipe/vela/reference.md) on the checkpoint's pinned runtime; all five kinds (`choice`/`noul`/`score`/`span`/`set`) for text state; CPU FP32 validated (author-run, evidence in the integration PR) | +| Vela 2.0 0.8B hybrid | [Python reference worker](recipe/vela/reference.md) on the checkpoint's pinned runtime; same surface plus `span_heads`; CPU FP32 parity recorded in the integration PR | [Supported models and hardware](docs/supported-models.md) lists the devices and where each worker has been run. diff --git a/docs/supported-models.md b/docs/supported-models.md index 2f5967d9..52b31a0b 100644 --- a/docs/supported-models.md +++ b/docs/supported-models.md @@ -25,6 +25,7 @@ Models that are being added are also tracked in issues labeled [new model](https | OmniJev-4B v1.1 | None yet: [checkpoint export and CPU contract](../recipe/omnijev/README.md); the native worker is in progress ([#114](https://github.com/ThinkFlowLab/system1-omni/issues/114)) | Request contract, preparation, heads and finishing checked against the reference ([#120](https://github.com/ThinkFlowLab/system1-omni/pull/120)); no inference | Not yet | Not supported | Python 3.10 reference environment for the export and fixtures | | D3 (Decision 3.0) | [Python reference worker](../recipe/d3/reference.md) running the checkpoint's pinned runtime | Unverified | Author-run on NVIDIA A100 (sm_80): three-kind serving smoke with byte-identical frontend forwarding and reference parity (exact decisions, max delta 0.0, 348 == 348 tokens); evidence in the integration PR. Owner: @MrDongsls. | Unverified | Python 3.12, the [pinned runtime requirements](../recipe/d3/requirements.txt), the [pinned package](https://huggingface.co/vllm-sr/d3) (~52 GB) | | Vela 2.0 0.3B | [Python reference worker](../recipe/vela/reference.md) running the checkpoint's pinned runtime | **CPU FP32 validated** (author-run, x86 WSL2, 24 cores): startup with all-kind warmup, the four example requests plus five synthetic rejections, and exact-match reference parity against the pinned upstream server on all nine (four 200s byte-equal incl. key order, agreed 422/413); byte-identical frontend forwarding; evidence in the integration PR. Owner: @MrDongsls. | Unverified (bf16-autocast is an untested deployment option) | Not supported | Python 3.12, the [pinned requirements](../recipe/vela/requirements.txt), the [pinned package](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) (1.24 GB FP32 weights; ~4.4 GB with ONNX/assets) | +| Vela 2.0 0.8B hybrid | [Python reference worker](../recipe/vela/reference.md) running the checkpoint's pinned runtime (same wire layer as 0.3B) | Same surface as 0.3B plus the `span_heads` field (router/broad dispatch); **CPU FP32 parity** (author-run; evidence in the integration PR) | Unverified (bf16-autocast is the evaluated upstream setting but untested here) | Not supported | Python 3.12, the [pinned requirements](../recipe/vela/requirements.txt), the [pinned package](https://huggingface.co/vllm-sr/Vela-2.0-0.8B) (2.02 GB shard + 4.5 MB broad head; ~3 GB FP32 parameters) | - **Validated:** covered by the recipe on `main` or by the checks in the linked merged pull request. - **Author-run:** recorded by the recipe author on the stated hardware and not independently reproduced; the row names its owner. @@ -34,10 +35,11 @@ Models that are being added are also tracked in issues labeled [new model](https The Cua-S1 workers answer `choice` questions only. LAYA's English worker, Open-Jev and CLM support `choice`, `score`, and `noul` text questions. D3's reference worker serves the same three kinds for text state; its media -inputs stay refused for now and will be integrated soon. -Vela 2.0 0.3B's reference worker serves all five kinds (`choice`, `noul`, -`score`, plus the `span`/`set` extensions with their Noul views) for text -state on CPU FP32; the hybrid members (0.8B/4B/9B) are separate slices. +inputs stay refused. +Vela 2.0's reference workers (0.3B and the 0.8B hybrid) serve all five +kinds (`choice`, `noul`, `score`, plus the `span`/`set` extensions with +their Noul views, and `span_heads` on the hybrid) for text state on CPU +FP32; the larger hybrid members (4B/9B) are separate slices. CLM has a row per path: the merged `clm-serve` recipe exercises those answer shapes with stub embeddings and says nothing about decision quality, while the native engine's CUDA row is the path through a real Qwen3-8B encoder. Its engine-only figure comes from the same comparison run diff --git a/mkdocs.yml b/mkdocs.yml index 7e912aeb..8f2d39fe 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -76,5 +76,5 @@ nav: - CLM stub-encoder contract: recipe/clm/README.md - LFM2.5-350M choice worker: recipe/lfm2/README.md - D3 reference worker: recipe/d3/reference.md - - Vela 2.0 0.3B reference worker: recipe/vela/reference.md + - Vela 2.0 reference workers: recipe/vela/reference.md - Contributing: CONTRIBUTING.md diff --git a/recipe/README.md b/recipe/README.md index 3950951d..45594959 100644 --- a/recipe/README.md +++ b/recipe/README.md @@ -28,7 +28,7 @@ For a first real decision, follow the [complete CPU walkthrough](../docs/getting - [D3 reference worker](d3/reference.md): download the pinned vllm-sr/d3 package, verify its runtime manifest, and serve `choice`/`noul`/`score` text decisions through the Python worker and the Rust frontend (serving and parity validated on A100). -- [Vela 2.0 0.3B reference worker](vela/reference.md): download the pinned +- [Vela 2.0 reference workers (0.3B and the 0.8B hybrid)](vela/reference.md): download the pinned vllm-sr/Vela-2.0-0.3B package, pass its full-hash package gate, and serve all five SystemOne question kinds (`choice`/`noul`/`score` plus the `span`/`set` extensions) through the Python worker and the Rust frontend on CPU FP32. diff --git a/recipe/vela/compare_reference.py b/recipe/vela/compare_reference.py index fde4cd15..3588e06a 100644 --- a/recipe/vela/compare_reference.py +++ b/recipe/vela/compare_reference.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Compare the Vela 2.0 0.3B worker pipeline against the pinned upstream server. +"""Compare the Vela 2.0 worker pipeline against the pinned upstream server. Runs each fixture request through two paths that share one loaded pinned engine, so the comparison isolates this repository's admission and @@ -13,8 +13,8 @@ finishing). Both paths execute the same verified runtime, and the worker delegates -response construction to the same pinned code — so the criterion is the -charter's reference parity: exact match, including JSON key order, with +response construction to the same pinned code — so the criterion is +reference parity: exact match, including JSON key order, with rejections agreeing on status and body (``{"detail": [{"loc", "msg", "type"}]}``). ``--export`` additionally freezes the oracle fixture — request, SystemOne response and the predict()-level answers — for the @@ -24,11 +24,14 @@ The default fixture set is the recipe's four example requests plus five synthetic rejections: an invalid span question (whole-request -422), a schema that alone exceeds the 8,192-token window (413), and +422), a schema that alone exceeds the member's input window (413), and the three envelope 422s (missing field, extra field, wrong model name) that freeze the worker's preamble admission against the pinned -runtime's. CPU FP32 is the oracle setting; GPU bf16-autocast is a -deployment option, not this tool's default. +runtime's. The example files' ``model`` field is rewritten to the +served member's name, and the hybrid member's oracle fixtures export +under an ``oracle-hybrid-`` prefix, so both members share one tool. +CPU FP32 is the oracle setting; GPU bf16-autocast is a deployment +option, not this tool's default. """ from __future__ import annotations @@ -44,7 +47,7 @@ def parse_args(argv=None): parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) - parser.add_argument("--model", required=True, help="pinned Vela 2.0 0.3B package directory") + parser.add_argument("--model", required=True, help="pinned Vela 2.0 package directory") parser.add_argument("--device", default="cpu", help="torch device (default: the CPU FP32 oracle)") parser.add_argument( "--fixture", @@ -62,8 +65,8 @@ def parse_args(argv=None): return parser.parse_args(argv) -def over_limit_body(): - """A schema that alone cannot fit the 8,192-token window. +def over_limit_body(model=MODEL_NAME): + """A schema that alone cannot fit the member's input window. 255 choice options with ~300-character descriptions: far past ``max_length`` before any state is rendered, which is exactly the @@ -79,7 +82,7 @@ def over_limit_body(): for index in range(255) } return { - "model": MODEL_NAME, + "model": model, "state": "short", "questions": { "big": {"type": "choice", "instructions": "Pick one.", "criteria": criteria} @@ -87,11 +90,11 @@ def over_limit_body(): } -def invalid_span_body(): +def invalid_span_body(model=MODEL_NAME): """A span question whose criteria is a list, not a label mapping.""" return { - "model": MODEL_NAME, + "model": model, "state": "short", "questions": { "bad": { @@ -109,50 +112,63 @@ def missing_model_body(): return {"state": "short", "questions": {"q": {"type": "noul", "instructions": "any?"}}} -def extra_field_body(): +def extra_field_body(model=MODEL_NAME): """An envelope with a field the preamble does not permit ('extra_forbidden').""" return { - "model": MODEL_NAME, + "model": model, "state": "short", "questions": {"q": {"type": "noul", "instructions": "any?"}}, "images": [], } -def wrong_model_body(): +def wrong_model_body(served=MODEL_NAME): """Another member's name on this runtime ('value_error.model' 422).""" + other = "vllm-sr/Vela-2.0-0.3B" if served.endswith("0.8B") else "vllm-sr/Vela-2.0-0.8B" return { - "model": "vllm-sr/Vela-2.0-0.8B", + "model": other, "state": "short", "questions": {"q": {"type": "noul", "instructions": "any?"}}, } -def default_cases(): +def default_cases(served=MODEL_NAME): return [ ("example-request", HERE / "example-request.json"), ("example-request-set-threshold", HERE / "example-request-set-threshold.json"), ("example-request-cjk", HERE / "example-request-cjk.json"), ("example-request-score", HERE / "example-request-score.json"), - ("invalid-span-422", invalid_span_body()), - ("over-limit-413", over_limit_body()), + ("invalid-span-422", invalid_span_body(served)), + ("over-limit-413", over_limit_body(served)), ("missing-model-422", missing_model_body()), - ("extra-field-422", extra_field_body()), - ("wrong-model-422", wrong_model_body()), + ("extra-field-422", extra_field_body(served)), + ("wrong-model-422", wrong_model_body(served)), ] -def load_cases(paths): +def load_cases(paths, served=MODEL_NAME): if not paths: - cases = default_cases() + cases = default_cases(served) else: cases = [(Path(path).name, path) for path in paths] - return [ - (name, body if isinstance(body, dict) else json.loads(Path(body).read_text(encoding="utf-8"))) - for name, body in cases - ] + loaded = [] + for name, source in cases: + body = source if isinstance(source, dict) else json.loads(Path(source).read_text(encoding="utf-8")) + # The example files are member-agnostic apart from their model field; + # serving the other member rewrites it so the case exercises the 200 + # path it was written for, on both compared paths alike. Synthetic + # bodies (including the deliberate wrong-model one) are never touched. + if ( + not isinstance(source, dict) + and isinstance(body, dict) + and body.get("model") != served + and body.get("model") in (MODEL_NAME, "vllm-sr/Vela-2.0-0.8B") + ): + body = {**body, "model": served} + loaded.append((name, body)) + return loaded def reference_response(client, body): @@ -245,10 +261,10 @@ def main(argv=None): sys.path.insert(0, str(HERE.parents[1] / "src")) from fastapi.testclient import TestClient - from frontend.vela import decide - from models.vela.engine import Vela2Executor + from frontend.vela import decide, detect_member - executor = Vela2Executor(args.model, device=args.device) + executor_class, _default_name = detect_member(args.model) + executor = executor_class(args.model, device=args.device) # Both runtime imports stay inside a no-bytecode window: the package # gate refuses existing caches, and vela2_serve is a fresh import that # would otherwise write the __pycache__ the next worker start rejects. @@ -262,11 +278,15 @@ def main(argv=None): app = vela2_serve.create_app(executor.engine, model_name=executor.model_name, api_key=None) client = TestClient(app) + served = executor.model_name + # The hybrid member's oracle fixtures get their own prefix so both + # members' frozen baselines can live in one directory. + prefix = "oracle-hybrid-" if getattr(executor, "is_hybrid", False) else "oracle-" exported = 0 failed = 0 try: - for name, body in load_cases(args.fixture): + for name, body in load_cases(args.fixture, served): raw = json.dumps(body).encode("utf-8") worker = worker_response(raw, executor, decide) reference = reference_response(client, body) @@ -278,7 +298,7 @@ def main(argv=None): req, _plan = vela2_inference.system_one_to_predict(body["state"], body["questions"]) target = Path(args.export) if target.is_dir(): - target = target / f"oracle-{name}.json" + target = target / f"{prefix}{name}.json" export_oracle(name, body, worker[1], executor.engine.predict(req), target) exported += 1 print() diff --git a/recipe/vela/reference.md b/recipe/vela/reference.md index d65a0cda..b6f717fd 100644 --- a/recipe/vela/reference.md +++ b/recipe/vela/reference.md @@ -1,9 +1,13 @@ -# Vela 2.0 0.3B reference worker +# Vela 2.0 reference workers (0.3B, 0.8B hybrid) -This recipe serves Vela 2.0 0.3B through the Python reference worker and -the existing Rust frontend. The model-side implementation is under -[`src/models/vela/`](../../src/models/vela/), and the worker is -[`src/frontend/vela.py`](../../src/frontend/vela.py). +This recipe serves the Vela 2.0 members through the Python reference +worker and the existing Rust frontend: the 0.3B encoder and the 0.8B +hybrid, one pinned member per worker process. The model-side +implementation is under +[`src/models/vela/`](../../src/models/vela/) (0.3B in `engine.py`, the +0.8B hybrid in `hybrid.py`), and the worker is +[`src/frontend/vela.py`](../../src/frontend/vela.py), which picks the +executor from the package's declared `model_type`. Run the commands from the repository root with Python 3.12. The default and evidence-backed setting is CPU FP32 — the reference oracle; a CUDA @@ -14,33 +18,34 @@ description, not here. ## Scope -The slice is: +Scope: -- package: [`vllm-sr/Vela-2.0-0.3B`](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) - at the pinned revision below; -- the Python reference worker behind the Rust frontend, executing the +- packages: [`vllm-sr/Vela-2.0-0.3B`](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) + and [`vllm-sr/Vela-2.0-0.8B`](https://huggingface.co/vllm-sr/Vela-2.0-0.8B) + at the pinned revisions below; +- the Python reference worker behind the Rust frontend, executing each checkpoint's own pinned runtime (`vela2_inference.py` and siblings) after a full-hash package gate; - text/JSON state and all five typed kinds — `choice`, `noul`, `score`, `span`, `set` — in one request, with the span/set response extensions - (`spans`/`sets`/`thresholds` and their Noul views) served - forward-untouched; -- FP32 parameters, one bidirectional forward per assembled sequence, + (`spans`/`sets`/`thresholds`, `span_heads` on the hybrid, and their + Noul views) served forward-untouched; +- FP32 parameters, one bidirectional forward per assembled sequence + (0.3B) or one parts pass with per-question continuation (hybrid), calibrated readouts; zero generated tokens. -Out of scope for this slice: the hybrid members (0.8B/4B/9B — their -reference workers are a separate slice), the ONNX backend, any native -Rust/CUDA executor (the frozen RFC's ModernBERT-profile route), and -`noul_calibration` (a hybrid-only feature that does not exist on 0.3B). +Out of scope: the larger hybrid members (4B/9B), the ONNX backend of +the 0.3B, any native Rust/CUDA executor, and `noul_calibration` (off +by default; documented as a reference-server feature only). ## Architecture and ownership ~~~text Rust frontend - -> Python worker + -> Python worker (executor chosen by config.json model_type) -> admit: JSON transport + upstream envelope preamble (protocol) - -> verify: config identity pin, full SHA-256 of the 7-file table, - exact weights size, then import it (engine) + -> verify: config identity pin, full SHA-256 of the pinned table, + exact weights size, then import it (engine / hybrid) -> execute: the checkpoint's own vela2_inference system_one_request -> finish: response-order/key-set checks + formula mirrors (postprocess) ~~~ @@ -53,28 +58,37 @@ The ownership boundary follows triples; deeper validation (state parts, question semantics, the 413) is the pinned runtime's, verbatim; - `preprocess.py` maps the runtime's `SystemOneError` to 422/413; -- `engine.py` owns the pinned runtime and the package gate; -- `postprocess.py` owns the pure formulas and response finishing; -- `frontend.vela` owns HTTP lifecycle, the single-flight 503 lock, the - `VELA2_API_KEY` 401 gate and warmup-before-readiness; +- `engine.py` owns the 0.3B pinned runtime and its package gate; + `hybrid.py` mirrors it for the 0.8B (a 10-file pin table including + the shard index and the side-loaded `broad_head.safetensors`); +- `postprocess.py` owns the pure formulas and response finishing, + member-aware for the hybrid's `span_heads` field; +- `frontend.vela` owns HTTP lifecycle, member detection, the + single-flight 503 lock, the `VELA2_API_KEY` 401 gate and + warmup-before-readiness; - The Rust frontend forwards `/v1/systemone` and `/health` unchanged. ## Pinned artifacts -| Artifact | Value | -| --- | --- | -| Package | [vllm-sr/Vela-2.0-0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) | -| Pinned revision | `d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409` | -| Package gate | config structural identity + a 7-file SHA-256 table pinned per revision in `engine.py`, full hash before import, no unlisted importables or `__pycache__` in the package root, exact 1,236,473,252-byte weights check | -| License | Apache-2.0; tokenizer material under Gemma Terms of Use | +| Artifact | 0.3B | 0.8B hybrid | +| --- | --- | --- | +| Package | [vllm-sr/Vela-2.0-0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) | [vllm-sr/Vela-2.0-0.8B](https://huggingface.co/vllm-sr/Vela-2.0-0.8B) | +| Pinned revision | `d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409` | `326b01d81f61fff7fd4ae570e54b87a49119e490` | +| Package gate | config structural identity + a 7-file SHA-256 table in `engine.py` | config structural identity (incl. `model_name`) + a 10-file SHA-256 table in `hybrid.py`, cross-checked against the package's own SHA256SUMS | +| Weights check | exact 1,236,473,252 bytes | exact 2,022,189,488 bytes + hashed `broad_head.safetensors` | +| Served name | `vllm-sr/Vela-2.0-0.3B` | `vllm-sr/Vela-2.0-0.8B` (from `config.json`) | +| Input window | 8,192 tokens | 16,384 tokens | +| License | Apache-2.0; tokenizer material under Gemma Terms of Use | Apache-2.0; Qwen3.5-0.8B lineage notices in the package | -The gate's file table is checked in (not read from the package, which -ships no manifest), so a later snapshot — self-consistent or not — -cannot pass as the pinned revision. Re-check before changing the pin: +Each gate's file table is checked in (never read from the package), so +a later snapshot — self-consistent or not — cannot pass as the pinned +revision. Re-check before changing a pin: ~~~sh curl -sS "https://huggingface.co/api/models/vllm-sr/Vela-2.0-0.3B/revision/d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409" | jq -r '.sha' sha256sum weights/vela-2.0-0.3b/{vela2_inference.py,modeling_vela2.py,configuration_vela2.py,vela2_serve.py,config.json,calibration.json,tokenizer.json} +curl -sS "https://huggingface.co/api/models/vllm-sr/Vela-2.0-0.8B/revision/326b01d81f61fff7fd4ae570e54b87a49119e490" | jq -r '.sha' +sha256sum weights/vela-2.0-0.8b/{vela2_inference.py,modeling_vela2.py,configuration_vela2.py,vela2_serve.py,config.json,calibration.json,tokenizer.json,tokenizer_config.json,model.safetensors.index.json,broad_head.safetensors} ~~~ ## Install and prepare the artifacts @@ -88,15 +102,21 @@ python3.12 -m venv .venv ~~~sh export VELA_PKG="$PWD/weights/vela-2.0-0.3b" export VELA_REV=d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409 +export VELA_HYBRID_PKG="$PWD/weights/vela-2.0-0.8b" +export VELA_HYBRID_REV=326b01d81f61fff7fd4ae570e54b87a49119e490 mkdir -p weights .venv/bin/hf download vllm-sr/Vela-2.0-0.3B --revision "$VELA_REV" --local-dir "$VELA_PKG" +.venv/bin/hf download vllm-sr/Vela-2.0-0.8B --revision "$VELA_HYBRID_REV" --local-dir "$VELA_HYBRID_PKG" ~~~ -The full package is about 4.4 GB (the 1.24 GB FP32 safetensors, a 34 MB +The 0.3B package is about 4.4 GB (the 1.24 GB FP32 safetensors, a 34 MB tokenizer, the ONNX exports and doc assets); serving needs roughly 2 GB of RAM for the FP32 model plus working memory for the batch rows. -The worker refuses to start when the config is not the pinned member's, -when any of the 7 gated files fails its hash, when the package root +The 0.8B package is about 2.1 GB (a 2.02 GB shard plus the 4.5 MB +broad-head file); FP32 parameters take about 3 GB of RAM on CPU (about +3 GB of GPU memory on CUDA, where the backbone runs under bf16 +autocast). A worker refuses to start when the config is not its pinned +member's, when any gated file fails its hash, when the package root holds an importable entry the table does not name or an existing `__pycache__`, or when the weights file is not the pinned size — that gate runs before any package code is imported. @@ -105,12 +125,15 @@ gate runs before any package code is imported. ~~~sh PYTHONPATH=src .venv/bin/python -m frontend.vela \ - --model "$VELA_PKG" --device cpu --port 8000 + --model "$VELA_PKG" --device cpu --port 8000 # 0.3B +PYTHONPATH=src .venv/bin/python -m frontend.vela \ + --model "$VELA_HYBRID_PKG" --device cpu --port 8001 # 0.8B hybrid ~~~ -The worker verifies the package, loads the FP32 model, runs a real -warmup request covering every question kind, and only then binds the -port: +The worker detects the member from the package's `model_type`, verifies +the package, loads the model, runs a real warmup request covering every +question kind (the hybrid's warmup also drives both span heads), and +only then binds the port: ~~~sh curl -sS -i http://127.0.0.1:8000/health # {"status": "ok"} (upstream verbatim) @@ -198,12 +221,13 @@ the [model contract](../../src/models/vela/README.md). In short: transport errors follow the shared worker discipline; every invalid field rejects the whole request with the upstream 422 `{"detail": [{"loc", "msg", "type"}]}`; 413 `too_long` fires only when -questions plus schema alone exceed 8,192 tokens; and over-limit state -text is **truncated as the reference truncates it** (shrink part first, -floor 64 then 1, then protected parts; windowed re-coverage of cut -labelled parts) — a deliberate, loudly documented divergence from the -repository's no-silent-truncation convention, with the guardrail risk -recorded in the model contract. +questions plus schema alone exceed the member's input window (8,192 on +0.3B, 16,384 on the hybrid); and over-limit state text is **truncated +as the reference truncates it** (shrink part first, floor 64 then 1, +then protected parts; windowed re-coverage of cut labelled parts) — a +deliberate, loudly documented divergence from the repository's +no-silent-truncation convention, with the guardrail risk recorded in +the model contract. ## Repository-side validation @@ -228,26 +252,32 @@ invalid-span 422, an over-limit 413, and the missing-field / extra-field / wrong-model envelope 422s) — through two paths that share one loaded engine: the pinned `vela2_serve.py` app via FastAPI's test client, and the production -`frontend.vela.decide` pipeline. The criterion is the charter's -reference parity: **exact match** including JSON key order for 200s, +`frontend.vela.decide` pipeline. The criterion is reference parity: +**exact match** including JSON key order for 200s, and agreement on status and body for rejections (needs `pip install fastapi uvicorn httpx` on top of the requirements file). `--export tests/vela/fixtures/oracle-.json` freezes each passing 200 case — request, SystemOne response and predict()-level answers — as -the frozen comparison baseline for the frozen RFC's native encoder -slices, and writes it only after a passing comparison. +the frozen comparison baseline, and writes it only after a passing +comparison. ~~~sh PYTHONPATH=src .venv/bin/python recipe/vela/compare_reference.py \ --model "$VELA_PKG" --device cpu \ --export tests/vela/fixtures/ +PYTHONPATH=src .venv/bin/python recipe/vela/compare_reference.py \ + --model "$VELA_HYBRID_PKG" --device cpu \ + --export tests/vela/fixtures/ ~~~ Run it by hand and record the verbatim output, host, package revision and dependency versions in the pull request description. CPU FP32 is the oracle setting; keep flash-linear-attention out of the fixture path -(it does not exist on this encoder member anyway) and do not substitute -GPU bf16-autocast numbers for the baseline. +(it changes the hybrid's numbers and its parity against the fallback +path is not documented upstream) and do not substitute GPU +bf16-autocast numbers for the baseline. The tool rewrites the example +files' `model` field to the served member and exports the hybrid's +oracle fixtures under an `oracle-hybrid-` prefix. ## Troubleshooting and limits @@ -256,15 +286,15 @@ GPU bf16-autocast numbers for the baseline. `__pycache__` bytecode caches and wrong weights sizes at startup, before importing anything. - The 8 MiB body cap is slice-scoped (upstream sets no total-body - limit); a valid request bounded by the 8,192-token window is far - below it. -- `/health` and `/ready` are upstream's minimal bodies on this member, - not the richer shape of some other workers in this repository. + limit); a valid request bounded by either member's input window is + far below it. +- `/health` and `/ready` are upstream's minimal bodies on both + members, not the richer shape of some other workers in this + repository. - Long states are truncated (see Error behavior); if a consumer must never miss tail text, check `usage.input_tokens` against the expected budget or enforce limits upstream of the worker. - GPU bf16-autocast serving (`--device cuda`) is untested in this - slice's evidence; treat it as a deployment option to validate - separately. + evidence; treat it as a deployment option to validate separately. - GPU evidence for the supported-models table is recorded in the integration pull request, not in this recipe. diff --git a/recipe/vela/test_compare_reference.py b/recipe/vela/test_compare_reference.py index d97aaa45..f76fb69d 100644 --- a/recipe/vela/test_compare_reference.py +++ b/recipe/vela/test_compare_reference.py @@ -86,7 +86,7 @@ def check(name: str, condition: bool, detail: str = "") -> None: lines, ok = tool.compare_case("fixture", (200, response()), (422, rejection)) check("worker answering a rejected request fails", not ok, "\n".join(lines)) - # 3. The default case set carries the charter's fixture list shape. + # 3. The default case set carries the fixture list's shape. cases = tool.default_cases() names = [name for name, _ in cases] check( diff --git a/src/frontend/vela.py b/src/frontend/vela.py index c863a018..78f46631 100644 --- a/src/frontend/vela.py +++ b/src/frontend/vela.py @@ -1,10 +1,13 @@ -"""Python reference worker for Vela 2.0 0.3B. +"""Python reference worker for the Vela 2.0 members (0.3B, 0.8B hybrid). The Rust frontend remains the public transport layer. This worker owns only HTTP lifecycle and request orchestration; wire admission lives in models.vela.protocol, the pinned-runtime error mapping in -models.vela.preprocess, the pinned runtime and its package gate in -models.vela.engine, and response finishing in models.vela.postprocess. +models.vela.preprocess, the pinned runtimes and their package gates in +models.vela.engine (0.3B) and models.vela.hybrid (0.8B), and response +finishing in models.vela.postprocess. The executor is chosen by the +package's declared ``model_type``; one worker process serves one pinned +member. The error surface has two layers: transport errors follow the shared Python-worker discipline, and every request rejection the pinned runtime @@ -13,7 +16,8 @@ (or 413 ``too_long``), passed through verbatim. There are no per-question errors. Over-limit state text is truncated exactly as the pinned reference truncates it (see the model contract); only questions plus -schema that cannot fit the 8,192-token window are refused. +schema that cannot fit the member's input window (8,192 or 16,384 +tokens) are refused. """ from __future__ import annotations @@ -26,17 +30,21 @@ import socket import threading from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path from typing import Any from models.vela.engine import Vela2Executor +from models.vela.hybrid import EXPECTED_MODEL_NAME as HYBRID_MODEL_NAME, Vela2HybridExecutor from models.vela.postprocess import finish_response from models.vela.preprocess import map_runtime_error, to_request_error from models.vela.protocol import MODEL_NAME, MAX_BODY, RequestError, parse_body, admit_request LOG = logging.getLogger(__name__) -# Upstream vela2_serve.py /v1/models entry, kept verbatim so clients see -# the reference server's description and release date. +# Upstream vela2_serve.py /v1/models entry for the 0.3B member, kept +# verbatim so clients see the reference server's description and release +# date; the hybrid members source their entry from config.json, exactly +# as their vela2_serve.py does. MODELS_ENTRY = { "name": None, # the served model name, filled per request "release_date": "2026-09-29", @@ -48,7 +56,57 @@ UNAUTHORIZED_BODY = {"detail": "Missing or invalid API key"} -def decide(raw: bytes, executor: Vela2Executor) -> dict[str, Any]: +def detect_member(package: str | Path) -> tuple[type, str]: + """Pick the pinned executor for a package's declared member. + + Returns the executor class and the member's default served name; the + structural gate inside the executor is the real authority — this + only routes the right gate to the package. + """ + + config_path = Path(package) / "config.json" + if not config_path.is_file(): + raise ValueError(f"the package is missing config.json: {package}") + try: + config = json.loads(config_path.read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + raise ValueError(f"config.json in {package} is not readable JSON") from exc + if not isinstance(config, dict): + raise ValueError(f"config.json in {package} is not a JSON object") + model_type = config.get("model_type") + if model_type == "vela2-unified": + return Vela2Executor, MODEL_NAME + if model_type == "vela2-decoder": + return Vela2HybridExecutor, HYBRID_MODEL_NAME + raise ValueError( + f"config.json declares model_type {model_type!r}; this worker serves the " + "vela2-unified (0.3B) and vela2-decoder (0.8B) members of Vela 2.0" + ) + + +def models_entry(executor) -> dict[str, Any]: + """The /v1/models entry, verbatim per member's reference server.""" + + if getattr(executor, "is_hybrid", False): + # Upstream hybrid vela2_serve.py: name/description/release_date + # straight from config.json. + config = getattr(executor, "engine", None) + config = getattr(config, "config", None) if config is not None else None + config = config or {} + return { + "name": executor.model_name, + "release_date": config.get("release_date", ""), + "description": config.get( + "description", + "Vela 2.0: SystemOne Choice / Noul / Score with span and set extensions", + ), + } + entry = dict(MODELS_ENTRY) + entry["name"] = executor.model_name + return entry + + +def decide(raw: bytes, executor) -> dict[str, Any]: """Admit, execute, and finish one complete request.""" body = admit_request(parse_body(raw), executor.served) @@ -62,13 +120,13 @@ def decide(raw: bytes, executor: Vela2Executor) -> dict[str, Any]: # layer cannot foresee (state rendering, question semantics, the # window budget); its rejections are client errors, not failures. raise map_runtime_error(exc) from exc - return finish_response(questions, response) + return finish_response(questions, response, hybrid=getattr(executor, "is_hybrid", False)) class WorkerServer(ThreadingHTTPServer): daemon_threads = False - def __init__(self, address, executor: Vela2Executor, api_key: str | None = None): + def __init__(self, address, executor: Vela2Executor | Vela2HybridExecutor, api_key: str | None = None): self.executor = executor self.api_key = api_key self.inference_lock = threading.Lock() @@ -139,9 +197,7 @@ def do_GET(self): elif path == "/v1/models": if self._unauthorized(): return - entry = dict(MODELS_ENTRY) - entry["name"] = self.server.executor.model_name - self.send_json(200, {"models": [entry]}) + self.send_json(200, {"models": [models_entry(self.server.executor)]}) else: self.send_json(404, {"detail": "unknown route"}) @@ -190,12 +246,16 @@ def do_POST(self): def parse_args(argv=None): - parser = argparse.ArgumentParser(description="Python reference worker for Vela 2.0 0.3B") - parser.add_argument("--model", required=True, help="pinned Vela 2.0 0.3B package directory") - parser.add_argument("--device", default="cpu", help="torch device (reference contract: cpu)") + parser = argparse.ArgumentParser(description="Python reference worker for Vela 2.0 members") + parser.add_argument("--model", required=True, + help="pinned Vela 2.0 package directory (0.3B or the 0.8B hybrid)") + parser.add_argument("--device", default="cpu", + help="torch device (reference contract: cpu; cuda runs the hybrid " + "backbone under bf16 autocast with FP32 heads)") parser.add_argument("--batch-size", type=int, default=None) - parser.add_argument("--name", default=os.environ.get("VELA2_MODEL_NAME", MODEL_NAME), - help="model name accepted in the request's model field") + parser.add_argument("--name", default=os.environ.get("VELA2_MODEL_NAME"), + help="model name accepted in the request's model field " + "(default: the pinned member's served name)") parser.add_argument("--host", default="127.0.0.1") parser.add_argument("--port", type=int, default=8000) return parser.parse_args(argv) @@ -204,10 +264,11 @@ def parse_args(argv=None): def main(argv=None): args = parse_args(argv) logging.basicConfig(level=logging.INFO) - executor = Vela2Executor( + executor_class, default_name = detect_member(args.model) + executor = executor_class( args.model, device=args.device, - model_name=args.name, + model_name=args.name or default_name, batch_size=args.batch_size, ) server = None diff --git a/src/models/vela/README.md b/src/models/vela/README.md index d1ad1bda..9407d4fa 100644 --- a/src/models/vela/README.md +++ b/src/models/vela/README.md @@ -1,48 +1,64 @@ -# Vela 2.0 0.3B reference contract +# Vela 2.0 reference contracts (0.3B, 0.8B hybrid) -This directory contains the System1-Omni integration slice for +This directory contains the System1-Omni integration slices for [Vela 2.0 0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) -(`vela2-unified`). It is a reference-serving boundary only: a Python -worker that runs the checkpoint's own pinned runtime for text state and -all five typed question kinds (`choice`, `noul`, `score`, `span`, `set`) -on `/v1/systemone`. Native execution is not part of this slice; per the -frozen RFC this worker is the Phase-1 entry that freezes the -text/token_ids input, the position-level final-hidden readout contract, -the error/length boundaries and the parity baseline for later -ModernBERT-profile work. +(`vela2-unified`) and +[Vela 2.0 0.8B](https://huggingface.co/vllm-sr/Vela-2.0-0.8B) +(`vela2-decoder`, the first hybrid member): a Python worker that runs +the checkpoint's own pinned runtime for text state and all five typed +question kinds (`choice`, `noul`, `score`, `span`, `set`) on +`/v1/systemone`, one pinned member per worker process. The 0.3B +worker's contract freezes the text/token_ids input, the +position-level final-hidden readout contract, the error/length +boundaries and the parity baseline. The shared Vela 2.0 wire module is deliberately shaped for the family: the envelope admission (`protocol`), the pinned-runtime error mapping (`preprocess`) and response finishing (`postprocess`) are member-neutral; -this slice ships the 0.3B pinned executor (`engine`). The hybrid members -(0.8B/4B/9B) reuse the same wire layer with their own executors. +this directory ships the 0.3B pinned executor (`engine`) and the 0.8B +hybrid executor (`hybrid`). The worker picks the executor from the +package's declared `model_type`. The 4B/9B members are not +implemented. ## Pinned artifacts -| Artifact | Value | -| --- | --- | -| Package | [vllm-sr/Vela-2.0-0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) | -| Pinned revision | `d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409` | -| Identity | `vela2-unified` / `Vela2Model`, 309,114,369 parameters, all FP32 (1,236,473,252-byte safetensors) | -| Runtime | the checkpoint's own flat files: `vela2_inference.py`, `modeling_vela2.py`, `configuration_vela2.py`, `vela2_serve.py` | -| Calibration | the package's own `calibration.json` (never cross-copied between members) | -| License | Apache-2.0; the inherited tokenizer material carries Gemma Terms of Use | -| Runtime dependencies | [recipe/vela/requirements.txt](../../recipe/vela/requirements.txt) (torch, transformers, safetensors, tokenizers, numpy) | - -The package ships no manifest, so the executor's startup gate checks the -config's structural identity (model type, architecture, `max_length` -8,192, the 8 marker ids), then hashes every small file the runtime reads -or executes — the four runtime modules, `config.json`, -`calibration.json` and `tokenizer.json` — against the table checked in -for the pinned revision, before anything is imported, and refuses -importable entries the table does not name plus any existing -`__pycache__` bytecode cache (`model.safetensors` is -checked by exact size; the runtime's strict safetensors load rejects -truncated or structurally mismatched weights, but not same-size -bit-level corruption). +| Artifact | 0.3B | 0.8B hybrid | +| --- | --- | --- | +| Package | [vllm-sr/Vela-2.0-0.3B](https://huggingface.co/vllm-sr/Vela-2.0-0.3B) | [vllm-sr/Vela-2.0-0.8B](https://huggingface.co/vllm-sr/Vela-2.0-0.8B) | +| Pinned revision | `d6f03aa9baca4f9017fa8f8e9c2764d3bed3f409` | `326b01d81f61fff7fd4ae570e54b87a49119e490` | +| Identity | `vela2-unified` / `Vela2Model`, 309,114,369 parameters, all FP32 (1,236,473,252-byte safetensors) | `vela2-decoder` / `Vela2DecoderModel`, 754,565,954 parameters (2,022,189,488-byte shard + hashed 4,480,172-byte `broad_head.safetensors`) | +| Input window | 8,192 tokens | 16,384 tokens | +| Runtime | the checkpoint's own flat files: `vela2_inference.py`, `modeling_vela2.py`, `configuration_vela2.py`, `vela2_serve.py` | the same four modules, decoder-specific | +| Calibration | the package's own `calibration.json` (never cross-copied between members — shipped thresholds differ) | same rule; the side-loaded broad head needs its `broad_head` block | +| License | Apache-2.0; the inherited tokenizer material carries Gemma Terms of Use | Apache-2.0; Qwen3.5-0.8B lineage notices ship in the package | +| Runtime dependencies | [recipe/vela/requirements.txt](../../recipe/vela/requirements.txt) (torch, transformers, safetensors, tokenizers, numpy) | same | + +The 0.3B package ships no manifest, so its gate checks the config's +structural identity (model type, architecture, `max_length` 8,192, the +8 marker ids), then hashes every small file the runtime reads or +executes — the four runtime modules, `config.json`, `calibration.json` +and `tokenizer.json` — against the table checked in for the pinned +revision, before anything is imported, and refuses importable entries +the table does not name plus any existing `__pycache__` bytecode cache +(`model.safetensors` is checked by exact size; the runtime's strict +safetensors load rejects truncated or structurally mismatched weights, +but not same-size bit-level corruption). The 0.8B gate mirrors this +with a 10-file table — the same four runtime modules, `config.json` +(identity includes the served `model_name`, so a 4B or 9B package fails +with its own name), `calibration.json`, `tokenizer.json`, +`tokenizer_config.json`, `model.safetensors.index.json` (the shard map) +and the hashed `broad_head.safetensors` — with the 2 GB shard checked +by exact size. A later self-consistent snapshot rewrites its own files and fails the checked-in pin. +On either member the runtime loads parameters in FP32; on `cuda` the +hybrid runs its backbone under bf16 autocast with FP32 heads (the +evaluated setting), and on `cpu` it runs FP32 throughout. The hybrid's +served name comes from `config.json` (`vllm-sr/Vela-2.0-0.8B`); the +runtime's family-generic `DEFAULT_MODEL_NAME` is never used as a +default. + ## Current serving scope - The request envelope is upstream's three-field form, exactly: @@ -103,8 +119,14 @@ The response envelope is the pinned runtime's, passed through verbatim answers echo the value without a rule field). Serving-time temperatures divide the logits per question type; the learned `tau`/`tau_span` live inside the model. - The hybrid-only fields (`span_heads`, `noul_calibration`) do not - exist on this member. + The hybrid-only `noul_calibration` block is default-off on every + member (env `VELA2_NOUL_CALIBRATION` or the reference server's flag + enable it); the worker serves the default path only. On the 0.3B the + `span_heads` field does not exist; on the hybrid it is emitted + together with `spans`, keyed by the span question ids + (`{"router"|"broad"}` per question, after the runtime's six-rule + dispatch: the `"head"` override, trained ids, exact router label + sets, PII subsets, hallucination aliases, else broad). - `usage.input_tokens` is the sum over every rendered sequence — span window rows and extra span-question rows included, nothing deduplicated; `usage.output_tokens` is always 0. @@ -128,7 +150,8 @@ per-question errors on this path: upstream server would answer 500); no such path is reachable on the pinned runtime's request path. 3. HTTP 413 (`too_long`) only when the questions plus schema alone - exceed the 8,192-token window. + exceed the member's input window (8,192 on 0.3B, 16,384 on the + hybrid). 4. HTTP 401 (`{"detail": "Missing or invalid API key"}`) only when `VELA2_API_KEY` is set and the request does not carry exactly `Authorization: Bearer ` (applies to `/v1/systemone` and @@ -166,14 +189,17 @@ every question kind), so answering at all means ready. - `preprocess.py` owns the mapping of the pinned runtime's `SystemOneError` to HTTP (422, or 413 for `too_long`) and of its other ValueErrors to client errors. -- `engine.py` owns the pinned runtime, the pre-import package gate, the - served name, device state and the all-kind warmup. +- `engine.py` owns the 0.3B pinned runtime, the pre-import package gate, the + served name, device state and the all-kind warmup; `hybrid.py` mirrors + it for the 0.8B (its warmup also drives both span heads). - `postprocess.py` owns the pure confidence/score formulas (mirrored for the CPU tests) and response finishing: answer-order and key-set - reconstruction including set views, usage accounting, and refusal of - hybrid-only fields on this member. -- `src/frontend/vela.py` owns HTTP lifecycle, the single-flight lock, - the API-key gate and orchestration. + reconstruction including set views, usage accounting, the + member-aware `span_heads` check, and refusal of hybrid-only fields + on the 0.3B. +- `src/frontend/vela.py` owns HTTP lifecycle, member detection from the + package's `model_type`, the per-member `/v1/models` entry, the + single-flight lock, the API-key gate and orchestration. - The Rust frontend owns transport and forwarding, not model state. The server-side error surface and envelope semantics are recorded as @@ -184,16 +210,18 @@ the shared worker-contract issue, and nothing is posted upstream. - CPU contract tests, no weights, no torch: `PYTHONPATH=src python -m pytest tests/vela -q` (envelope triples, - transport discipline, package gate, formula mirrors, response - finishing, the HTTP surface with a contract fake). + transport discipline, both package gates, formula mirrors, response + finishing including the hybrid `span_heads` surface, member + detection, the HTTP surface with a contract fake). - Reference parity is exact match on recorded fixtures under the pinned versions: `recipe/vela/compare_reference.py` drives the recipe fixtures plus five synthetic rejections (invalid-span 422, over-limit 413, and the missing-field / extra-field / wrong-model envelope 422s) through both the pinned `vela2_serve.py` app and the - production worker pipeline over one shared engine, and freezes the - oracle fixture for the native encoder slices. CPU FP32 (no + production worker pipeline over one shared engine, for either member, + and freezes the oracle fixture for the native slices (`oracle-` for + 0.3B, `oracle-hybrid-` for 0.8B). CPU FP32 (no flash-linear-attention) is the oracle setting; GPU bf16-autocast is - a deployment option outside this slice's evidence. + a deployment option outside these slices' evidence. - The serving procedure, hardware requirements and the recorded evidence pointer are [recipe/vela/reference.md](../../recipe/vela/reference.md). diff --git a/src/models/vela/engine.py b/src/models/vela/engine.py index 19badaf9..9055d6c4 100644 --- a/src/models/vela/engine.py +++ b/src/models/vela/engine.py @@ -126,7 +126,7 @@ def verify_config(root: Path) -> dict[str, Any]: return config -def verify_files(root: Path, table: dict[str, str] | None = None) -> list[str]: +def verify_files(root: Path, table: dict[str, str] | None = None, revision: str = REFERENCE_REVISION) -> list[str]: """Hash every pinned file in full and return their names. Must run before any package code is imported: the table names the @@ -138,7 +138,8 @@ def verify_files(root: Path, table: dict[str, str] | None = None) -> list[str]: ``__init__`` in every module form, or a namespace package's nested modules — would shadow one of the pinned runtime's dependencies — and any ``__pycache__``, whose existing bytecode is not verifiable - against the pin. + against the pin. ``revision`` names the pin the table belongs to in + hash-failure messages (the hybrid member passes its own). """ table = dict(table or PINNED_FILES_SHA256) @@ -151,7 +152,7 @@ def verify_files(root: Path, table: dict[str, str] | None = None) -> list[str]: problems.append(f"missing pinned file {name}") continue if _sha256_safe(target) != digest: - problems.append(f"{name}: SHA-256 differs from the pin for {REFERENCE_REVISION}") + problems.append(f"{name}: SHA-256 differs from the pin for {revision}") for entry in sorted(Path(root).iterdir()): if entry.name in table: continue @@ -232,6 +233,10 @@ def verify_package(root: Path, table: dict[str, str] | None = None) -> None: class Vela2Executor: """Load and drive one immutable pinned Vela 2.0 0.3B runtime instance.""" + # Response finishing uses this to refuse the hybrid-only ``span_heads`` + # field (hybrid.py's executor sets it True). + is_hybrid = False + def __init__( self, package: str | Path, diff --git a/src/models/vela/hybrid.py b/src/models/vela/hybrid.py new file mode 100644 index 00000000..e02fa6fb --- /dev/null +++ b/src/models/vela/hybrid.py @@ -0,0 +1,241 @@ +"""Pinned executor for Vela 2.0 0.8B (``vela2-decoder``), the first hybrid member. + +Mirrors :mod:`models.vela.engine` for the compact hybrid member. The +``vela2-decoder`` package ships the same flat runtime modules as the +0.3B (with decoder-specific contents), so the gate reuses the same +helpers and the same one-package-per-process discipline; what differs +is the pin table, the structural identity of ``config.json``, and the +weights layout — one 2 GB shard checked by exact size plus the +side-loaded second span head. + +The runtime loads parameters in FP32 either way; on ``cuda`` it runs +the backbone under bf16 autocast with FP32 heads (the evaluated +setting), on ``cpu`` it runs FP32 throughout. The served model name +comes from ``config.json`` ``model_name`` (upstream +``DEFAULT_MODEL_NAME`` is the family-generic ``vllm-sr/Vela-2.0`` and +must not be used as a default here). +""" + +from __future__ import annotations + +import json +import sys +import threading +from pathlib import Path +from typing import Any + +from .engine import ( + put_first_on_sys_path, + verify_files, + verify_no_foreign_runtime, +) + +HYBRID_REPO = "vllm-sr/Vela-2.0-0.8B" +HYBRID_REVISION = "326b01d81f61fff7fd4ae570e54b87a49119e490" +# Structural identity of the pinned config.json. model_name is unique +# per member, so a 4B or 9B package fails with its own name. +EXPECTED_MODEL_TYPE = "vela2-decoder" +EXPECTED_ARCHITECTURE = "Vela2DecoderModel" +EXPECTED_MAX_LENGTH = 16384 +EXPECTED_MODEL_NAME = "vllm-sr/Vela-2.0-0.8B" +WEIGHTS_NAME = "model-00001-of-00001.safetensors" +EXPECTED_WEIGHTS_BYTES = 2022189488 +# The files this gate hashes, transcribed from the pinned revision and +# cross-checked against the package's own SHA256SUMS; repinning means +# updating this table together with HYBRID_REVISION. Every file the +# runtime reads or executes is here: the four runtime modules, +# ``vela2_serve.py`` (never imported, part of the pinned surface), +# ``config.json``, ``calibration.json``, ``tokenizer.json`` (the loaded +# tokenizer), ``tokenizer_config.json`` (listed by the runtime's own +# download manifest), ``model.safetensors.index.json`` (the shard map +# the loader reads) and ``broad_head.safetensors`` (the side-loaded +# second span head; 4.5 MB hashes in milliseconds). The 2 GB shard is +# checked by exact size only, as on 0.3B. +PINNED_FILES_SHA256 = { + "vela2_serve.py": "68e1ef278561f452aa0ca6afd6aa288636cecdb0b37fd7f7146a3c9bf17aa787", + "vela2_inference.py": "11de1322788942153a22db164a3fb26abcaa4c5bb4b057cd572e277457c17dc2", + "modeling_vela2.py": "b7ecd8dcacfb24dbe8249c7b96829a737f4e2b493fc2157497f1b62d9e02727d", + "configuration_vela2.py": "c9a41e434a9d2cfeb3ab6581c00679d241b3468c04d160a811d6fe2c85cd071f", + "config.json": "0637ca193b9159b6ece25837913956477164d3392d9b5dd1e6b74b95a484004a", + "calibration.json": "d96e447962d0dc1d58fa8422dddbae8b99960e21fd75a10ef9b096f700a211f5", + "tokenizer.json": "06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523", + "tokenizer_config.json": "bee8eba30f0eb4af73c0fe2cd06d0f89b657d7819941c438157ec42f7c80ea87", + "model.safetensors.index.json": "b53d1c79219677289bdb1541a1373dfe53119884f248b23c94a27aef7338919d", + "broad_head.safetensors": "687d7a435d6de846796274e417b6333f83df3085a6e7c3bca4c01cb4d75a30f0", +} +# A warmup that touches every question kind's readout path on both span +# heads: the CandidateHead option rows (choice/set share it, noul and +# score ride the same candidate readout), the span decoder and the +# threshold echo, one router-head span (the unsupported label is a +# trained alias) and one broad-head span (an open label with no trained +# match). Model text only; the result is discarded. +WARMUP_REQUEST = { + "state": { + "request": "Hello, I would like to check the status of my order.", + "answer": "Your order was delivered yesterday.", + }, + "model": None, # filled with the served name at warmup time + "questions": { + "kind": { + "type": "choice", + "instructions": "What is this message about?", + "criteria": {"order": "an order's status", "other": "anything else"}, + }, + "claims": { + "type": "set", + "instructions": "Which topics does the message mention?", + "criteria": {"delivery": "delivery of an order", "billing": "payments or charges"}, + }, + "tone": { + "type": "score", + "instructions": "How formal is the message?", + "criteria": ["casual", "neutral", "formal"], + }, + "polite": { + "type": "noul", + "instructions": "Is the message polite?", + }, + "unsupported": { + "type": "span", + "instructions": "Which spans of the answer are not supported by the request?", + "criteria": {"unsupported": "a claim not supported by the request"}, + }, + "products": { + "type": "span", + "instructions": "Which products or services are mentioned?", + "criteria": {"product": "a product or a service"}, + "over": "request", + }, + }, +} + + +def verify_config(root: Path) -> dict[str, Any]: + """Read config.json and check the pinned member's structural identity.""" + + path = Path(root) / "config.json" + if not path.is_file(): + raise ValueError(f"the package is missing config.json; this worker only serves {HYBRID_REPO}") + config = json.loads(path.read_text(encoding="utf-8")) + if config.get("model_type") != EXPECTED_MODEL_TYPE: + raise ValueError( + f"config.json declares model_type {config.get('model_type')!r}; this worker only " + f"serves the {EXPECTED_MODEL_TYPE} member {HYBRID_REPO} at {HYBRID_REVISION}" + ) + if config.get("architectures") != [EXPECTED_ARCHITECTURE]: + raise ValueError(f"config.json architectures must be [{EXPECTED_ARCHITECTURE!r}]") + if config.get("max_length") != EXPECTED_MAX_LENGTH: + raise ValueError(f"config.json max_length must be {EXPECTED_MAX_LENGTH}") + if config.get("model_name") != EXPECTED_MODEL_NAME: + raise ValueError( + f"config.json model_name must be {EXPECTED_MODEL_NAME!r}; another Vela 2.0 member " + "needs its own pinned executor" + ) + return config + + +def verify_weights(root: Path) -> None: + """Check the pinned shard is present at its exact pinned size.""" + + target = Path(root) / WEIGHTS_NAME + if not target.is_file(): + raise ValueError(f"the package is missing {WEIGHTS_NAME}") + size = target.stat().st_size + if size != EXPECTED_WEIGHTS_BYTES: + raise ValueError( + f"{WEIGHTS_NAME} is {size} bytes; the pinned revision's file is {EXPECTED_WEIGHTS_BYTES}" + ) + + +def verify_package(root: Path, table: dict[str, str] | None = None) -> None: + """The full pre-import gate for one hybrid package directory.""" + + verify_config(root) + verify_files(root, table or PINNED_FILES_SHA256, HYBRID_REVISION) + verify_weights(root) + + +class Vela2HybridExecutor: + """Load and drive one immutable pinned Vela 2.0 0.8B runtime instance.""" + + is_hybrid = True + + def __init__( + self, + package: str | Path, + *, + device: str = "cpu", + model_name: str | None = None, + batch_size: int | None = None, + token_budget: int | None = None, + ) -> None: + root = Path(package) + if not root.is_dir(): + raise ValueError(f"the Vela 2.0 0.8B package directory does not exist: {root}") + verify_package(root) + + # Same discipline as the 0.3B executor: the just-verified package + # root goes first on sys.path so the flat runtime modules resolve + # to the checkpoint's own copies, and a runtime module already + # imported from another directory is refused — one worker process + # serves one package. + verify_no_foreign_runtime(root) + put_first_on_sys_path(root) + # Import with bytecode writing off: the gate refuses existing + # bytecode caches, so a __pycache__ this import wrote would fail + # every later gate run of the same package. + previous_dont_write_bytecode = sys.dont_write_bytecode + sys.dont_write_bytecode = True + try: + import vela2_inference + finally: + sys.dont_write_bytecode = previous_dont_write_bytecode + + self.SystemOneError = vela2_inference.SystemOneError + self._system_one_request = vela2_inference.system_one_request + self.engine = vela2_inference.Vela2.from_pretrained( + root, backend="torch", device=device, + **({"batch_size": batch_size} if batch_size is not None else {}), + **({"token_budget": token_budget} if token_budget is not None else {}), + ) + # The decoder members declare their served name in config.json; + # the runtime's DEFAULT_MODEL_NAME is family-generic. + self.model_name = model_name or self.engine.model_name + self.max_length = int(self.engine.max_len) + self._lock = threading.RLock() + self._closed = False + + @property + def served(self) -> set[str]: + """The model names this executor answers to (upstream: one).""" + + return {self.model_name} + + def system_one(self, body: dict[str, Any]) -> dict[str, Any]: + """Run one admitted request body; the response is upstream's, verbatim. + + Hybrid responses carry one extra top-level field, ``span_heads``, + alongside ``spans``; response finishing checks it in + :func:`models.vela.postprocess.finish_response`. + """ + + with self._lock: + if self._closed: + raise RuntimeError("Vela 2.0 hybrid executor is closed") + return self._system_one_request(self.engine, body, served=self.served) + + def warmup(self) -> None: + """Run one real request of every question kind on both span heads.""" + + request = dict(WARMUP_REQUEST) + request["model"] = self.model_name + response = self.system_one(request) + if not isinstance(response, dict) or "answers" not in response: + raise RuntimeError("warmup did not produce a SystemOne response") + + def close(self) -> None: + with self._lock: + if self._closed: + return + self._closed = True + self.engine = None diff --git a/src/models/vela/postprocess.py b/src/models/vela/postprocess.py index 141b6938..907acb9b 100644 --- a/src/models/vela/postprocess.py +++ b/src/models/vela/postprocess.py @@ -91,26 +91,35 @@ def _question_ids_of(questions: Mapping[str, Any], *kinds: str) -> list[str]: ] -def finish_response(questions: Mapping[str, Any], response: dict[str, Any]) -> dict[str, Any]: +def finish_response( + questions: Mapping[str, Any], + response: dict[str, Any], + *, + hybrid: bool = False, +) -> dict[str, Any]: """Check the runtime response against the admitted questions and pass it through. The pinned runtime owns every value in this response; these checks only refuse a response that could not be reconstructed for the - request that produced it (wrong key set or order, a hybrid-only + request that produced it (wrong key set or order, a member-invented field, broken usage accounting, an answer of unexpected shape). + ``hybrid`` selects the member: the hybrid executor's responses carry + ``span_heads`` alongside ``spans`` (upstream emits both together, + keyed by the span question ids); the 0.3B member's never do. """ if not isinstance(response, dict): raise RuntimeError("the pinned runtime returned a non-object response") keys = set(response) foreign = keys & HYBRID_ONLY_FIELDS - if foreign: + if foreign and not hybrid: raise RuntimeError( f"the pinned runtime returned hybrid-only field(s) {sorted(foreign)}; " "this worker serves the 0.3B member" ) missing = [field for field in REQUIRED_FIELDS if field not in response] - if missing or not keys <= TOP_LEVEL_FIELDS: + allowed = TOP_LEVEL_FIELDS | HYBRID_ONLY_FIELDS if hybrid else TOP_LEVEL_FIELDS + if missing or not keys <= allowed: raise RuntimeError("the pinned runtime returned an unexpected response envelope") answers = response["answers"] expected = expected_answer_order(questions) @@ -126,6 +135,14 @@ def finish_response(questions: Mapping[str, Any], response: dict[str, Any]) -> d raise RuntimeError(f"{field} do not match the request's {field} questions") elif field in response: raise RuntimeError(f"the pinned runtime returned {field} without {field} questions") + if hybrid: + span_ids = _question_ids_of(questions, "span") + if span_ids: + heads = response.get("span_heads") + if not isinstance(heads, dict) or list(heads) != span_ids: + raise RuntimeError("span_heads do not match the request's span questions") + elif "span_heads" in response: + raise RuntimeError("the pinned runtime returned span_heads without span questions") usage = response["usage"] if ( not isinstance(usage, dict) diff --git a/src/models/vela/protocol.py b/src/models/vela/protocol.py index de5dd2db..d9341206 100644 --- a/src/models/vela/protocol.py +++ b/src/models/vela/protocol.py @@ -24,10 +24,10 @@ # hybrid members serve their config.json "model_name" instead. MODEL_NAME = "vllm-sr/Vela-2.0-0.3B" # Slice-scoped transport cap. Upstream vela2_serve.py sets no total-body -# limit (its only size boundary is the 8,192-token input window); a text -# request bounded by that window is orders of magnitude smaller, so this -# cap never binds a valid request. A deliberate difference recorded with -# the integration. +# limit (its only size boundary is the member's token input window); a +# text request bounded by that window is orders of magnitude smaller, so +# this cap never binds a valid request. A deliberate difference recorded +# with the integration. MAX_BODY = 8 * 1024 * 1024 # The envelope fields, in upstream's missing-field check order. REQUEST_FIELDS = ("state", "model", "questions") diff --git a/tests/vela/fixtures/oracle-hybrid-example-request-cjk.json b/tests/vela/fixtures/oracle-hybrid-example-request-cjk.json new file mode 100644 index 00000000..ddf6c71c --- /dev/null +++ b/tests/vela/fixtures/oracle-hybrid-example-request-cjk.json @@ -0,0 +1,155 @@ +{ + "case": "example-request-cjk", + "request": { + "model": "vllm-sr/Vela-2.0-0.8B", + "state": { + "request": "客户张伟（zhang.wei@example.cn）于2026年10月1日在上海市浦东新区世纪大道100号办理了业务，联系电话13800138000。", + "source": "客户的业务申请需要在三个工作日内审核完成。", + "answer": "客户的业务已经即时完成了。" + }, + "questions": { + "pii": { + "type": "span", + "instructions": "Which spans are personal information?", + "criteria": { + "PERSON": "a person's name", + "EMAIL_ADDRESS": "an e-mail address", + "PHONE_NUMBER": "a telephone number", + "STREET_ADDRESS": "a street address or postal address" + }, + "over": "request" + }, + "unsupported": { + "type": "span", + "instructions": "Which spans of the answer are not supported by the context?", + "criteria": { + "unsupported": "a claim not supported by the context" + } + }, + "billing": { + "type": "noul", + "instructions": "这个请求是否与账单有关？" + } + } + }, + "system_one": { + "model": "vllm-sr/Vela-2.0-0.8B", + "answers": { + "pii": { + "type": "noul", + "noul": 0.8401349388137276 + }, + "unsupported": { + "type": "noul", + "noul": 0.28838747148371713 + }, + "billing": { + "type": "noul", + "noul": 0.1674766566538726 + } + }, + "usage": { + "input_tokens": 597, + "output_tokens": 0 + }, + "spans": { + "pii": [ + { + "label": "PHONE_NUMBER", + "start": 32, + "end": 34, + "text": "10", + "probability": 0.4658691883087158 + }, + { + "label": "STREET_ADDRESS", + "start": 38, + "end": 52, + "text": "上海市浦东新区世纪大道100", + "probability": 0.4821831285953522 + }, + { + "label": "PHONE_NUMBER", + "start": 63, + "end": 74, + "text": "13800138000", + "probability": 0.4298103451728821 + } + ], + "unsupported": [] + }, + "span_heads": { + "pii": "router", + "unsupported": "router" + }, + "thresholds": { + "pii": 0.2914, + "unsupported": 0.5 + } + }, + "predict": { + "answers": [ + { + "id": "pii", + "type": "span", + "target_part": "user", + "spans": [ + { + "start": 32, + "end": 34, + "label": "PHONE_NUMBER", + "probability": 0.4658691883087158, + "text": "10" + }, + { + "start": 38, + "end": 52, + "label": "STREET_ADDRESS", + "probability": 0.4821831285953522, + "text": "上海市浦东新区世纪大道100" + }, + { + "start": 63, + "end": 74, + "label": "PHONE_NUMBER", + "probability": 0.4298103451728821, + "text": "13800138000" + } + ], + "threshold": 0.2914, + "threshold_rule": "pii:length", + "head": "router", + "head_reason": "id" + }, + { + "id": "unsupported", + "type": "span", + "target_part": "answer", + "spans": [], + "threshold": 0.5, + "threshold_rule": "span:*", + "head": "router", + "head_reason": "label_set" + }, + { + "id": "billing", + "type": "choice", + "answer": "no", + "probabilities": { + "no": 0.8325233433461274, + "yes": 0.1674766566538726 + }, + "abstain_probability": 0.0009211774164155532 + } + ], + "meta": { + "tokens": { + "user": 46, + "context": 10, + "answer": 6 + }, + "windows": 0, + "sequences": 2 + } + } +} diff --git a/tests/vela/fixtures/oracle-hybrid-example-request-score.json b/tests/vela/fixtures/oracle-hybrid-example-request-score.json new file mode 100644 index 00000000..9491e486 --- /dev/null +++ b/tests/vela/fixtures/oracle-hybrid-example-request-score.json @@ -0,0 +1,105 @@ +{ + "case": "example-request-score", + "request": { + "model": "vllm-sr/Vela-2.0-0.8B", + "state": "Hello, I would like to check the status of my order. It has been five days and I have heard nothing back.", + "questions": { + "tone": { + "type": "score", + "instructions": "How formal is the message?", + "criteria": [ + "casual", + "neutral", + "formal" + ] + }, + "urgency": { + "type": "score", + "instructions": "How urgently does the sender need a reply?", + "criteria": [ + "no rush", + "soon", + "today", + "immediately" + ] + } + } + }, + "system_one": { + "model": "vllm-sr/Vela-2.0-0.8B", + "answers": { + "tone": { + "type": "score", + "score": 1.1498890868381673, + "confidence": 0.3046350733874953, + "legend": { + "0": "casual", + "1": "neutral", + "2": "formal" + }, + "probabilities": { + "0": 0.16807713462834117, + "1": 0.5139566439051507, + "2": 0.3179662214665082 + } + }, + "urgency": { + "type": "score", + "score": 0.9787050638239541, + "confidence": 0.19233689920934172, + "legend": { + "0": "no rush", + "1": "soon", + "2": "today", + "3": "immediately" + }, + "probabilities": { + "0": 0.44128118285871365, + "1": 0.21311503083546, + "2": 0.2712213259289846, + "3": 0.07438246037684161 + } + } + }, + "usage": { + "input_tokens": 228, + "output_tokens": 0 + } + }, + "predict": { + "answers": [ + { + "id": "tone", + "type": "score", + "answer": "neutral", + "probabilities": { + "casual": 0.16807713462834117, + "neutral": 0.5139566439051507, + "formal": 0.3179662214665082 + }, + "expected_index": 1.1498890868381673, + "score": 0.5749445434190836 + }, + { + "id": "urgency", + "type": "score", + "answer": "no rush", + "probabilities": { + "no rush": 0.44128118285871365, + "soon": 0.21311503083546, + "today": 0.2712213259289846, + "immediately": 0.07438246037684161 + }, + "expected_index": 0.9787050638239541, + "score": 0.3262350212746513 + } + ], + "meta": { + "tokens": { + "user": 25 + }, + "windows": 0, + "sequences": 1 + } + } +} diff --git a/tests/vela/fixtures/oracle-hybrid-example-request-set-threshold.json b/tests/vela/fixtures/oracle-hybrid-example-request-set-threshold.json new file mode 100644 index 00000000..bab3d048 --- /dev/null +++ b/tests/vela/fixtures/oracle-hybrid-example-request-set-threshold.json @@ -0,0 +1,81 @@ +{ + "case": "example-request-set-threshold", + "request": { + "model": "vllm-sr/Vela-2.0-0.8B", + "state": "My card was charged twice and the parcel never arrived.", + "questions": { + "issues": { + "type": "set", + "instructions": "Which issues does the customer report?", + "criteria": { + "billing": "payments, charges, refunds or invoices", + "shipping": "delivery of an order or a parcel", + "login": "signing in, passwords or account access" + }, + "threshold": 0.15 + } + } + }, + "system_one": { + "model": "vllm-sr/Vela-2.0-0.8B", + "answers": { + "issues.billing": { + "type": "noul", + "noul": 0.6211446881608428 + }, + "issues.shipping": { + "type": "noul", + "noul": 0.8205402483597802 + }, + "issues.login": { + "type": "noul", + "noul": 0.00012988178108716601 + } + }, + "usage": { + "input_tokens": 133, + "output_tokens": 0 + }, + "sets": { + "issues": { + "selected": [ + "billing", + "shipping" + ], + "probabilities": { + "billing": 0.6211446881608428, + "shipping": 0.8205402483597802, + "login": 0.00012988178108716601 + } + } + }, + "thresholds": { + "issues": 0.15 + } + }, + "predict": { + "answers": [ + { + "id": "issues", + "type": "set", + "answer": [ + "billing", + "shipping" + ], + "probabilities": { + "billing": 0.6211446881608428, + "shipping": 0.8205402483597802, + "login": 0.00012988178108716601 + }, + "threshold": 0.15 + } + ], + "meta": { + "tokens": { + "user": 11 + }, + "windows": 0, + "sequences": 1 + } + } +} diff --git a/tests/vela/fixtures/oracle-hybrid-example-request.json b/tests/vela/fixtures/oracle-hybrid-example-request.json new file mode 100644 index 00000000..801ef7aa --- /dev/null +++ b/tests/vela/fixtures/oracle-hybrid-example-request.json @@ -0,0 +1,183 @@ +{ + "case": "example-request", + "request": { + "model": "vllm-sr/Vela-2.0-0.8B", + "state": { + "request": "Hi, I'm Tom Baker (tom.baker@example.com). What is the maximum daily dose of paracetamol for an adult?", + "source": "For adults, the maximum dose of paracetamol is 4 grams in 24 hours, taken as 500 mg to 1 g every 4 to 6 hours.", + "answer": "Adults can take up to 6 grams of paracetamol in 24 hours, in doses of 500 mg to 1 g every 4 to 6 hours." + }, + "questions": { + "pii": { + "type": "span", + "instructions": "Which spans are personal information?", + "criteria": { + "AGE": "a person's age", + "CREDIT_CARD": "a payment card number", + "DATE_TIME": "a date, time or date of birth", + "DOMAIN_NAME": "an internet domain name or website address", + "EMAIL_ADDRESS": "an e-mail address", + "GPE": "a country, city, state or other geopolitical place", + "IBAN_CODE": "an international bank account number (IBAN)", + "IP_ADDRESS": "an IPv4 or IPv6 address", + "NRP": "a nationality, religious or political group", + "ORGANIZATION": "the name of a company, institution or other organisation", + "PERSON": "a person's name", + "PHONE_NUMBER": "a telephone number", + "STREET_ADDRESS": "a street address or postal address", + "TITLE": "a personal or professional title such as Dr. or Mrs.", + "US_DRIVER_LICENSE": "a US driver's licence number", + "US_SSN": "a US social security number", + "ZIP_CODE": "a postal or ZIP code" + }, + "over": "request" + }, + "halu": { + "type": "span", + "instructions": "Which spans of the answer are not supported by the context?", + "criteria": { + "unsupported": "a claim not supported by the context" + } + }, + "domain": { + "type": "choice", + "instructions": "Which subject area is this request about?", + "over": "request", + "criteria": { + "health": "medicine, clinical practice, nutrition, ageing or sexual health", + "math": "arithmetic, algebra, geometry, statistics or other mathematics", + "other": "a subject that fits none of the listed areas" + } + } + } + }, + "system_one": { + "model": "vllm-sr/Vela-2.0-0.8B", + "answers": { + "pii": { + "type": "noul", + "noul": 0.9965421673633966 + }, + "halu": { + "type": "noul", + "noul": 0.9972382713052719 + }, + "domain": { + "type": "choice", + "choice": "health", + "confidence": 0.9288764270674694, + "probabilities": { + "health": 0.9621185645193945, + "math": 0.004639298028680393, + "other": 0.03324213745192508 + } + } + }, + "usage": { + "input_tokens": 1041, + "output_tokens": 0 + }, + "spans": { + "pii": [ + { + "label": "PERSON", + "start": 8, + "end": 17, + "text": "Tom Baker", + "probability": 0.983367919921875 + }, + { + "label": "EMAIL_ADDRESS", + "start": 19, + "end": 40, + "text": "tom.baker@example.com", + "probability": 0.9851056337356567 + } + ], + "halu": [ + { + "label": "unsupported", + "start": 19, + "end": 29, + "text": "to 6 grams", + "probability": 0.7782317996025085 + } + ] + }, + "span_heads": { + "pii": "router", + "halu": "router" + }, + "thresholds": { + "pii": 0.2914, + "halu": 0.35 + } + }, + "predict": { + "answers": [ + { + "id": "pii", + "type": "span", + "target_part": "user", + "spans": [ + { + "start": 8, + "end": 17, + "label": "PERSON", + "probability": 0.983367919921875, + "text": "Tom Baker" + }, + { + "start": 19, + "end": 40, + "label": "EMAIL_ADDRESS", + "probability": 0.9851056337356567, + "text": "tom.baker@example.com" + } + ], + "threshold": 0.2914, + "threshold_rule": "pii:length", + "head": "router", + "head_reason": "id" + }, + { + "id": "halu", + "type": "span", + "target_part": "answer", + "spans": [ + { + "start": 19, + "end": 29, + "label": "unsupported", + "probability": 0.7782317996025085, + "text": "to 6 grams" + } + ], + "threshold": 0.35, + "threshold_rule": "halu", + "head": "router", + "head_reason": "id" + }, + { + "id": "domain", + "type": "choice", + "answer": "health", + "probabilities": { + "health": 0.9621185645193945, + "math": 0.004639298028680393, + "other": 0.03324213745192508 + }, + "abstain_probability": 0.04287965117790319 + } + ], + "meta": { + "tokens": { + "user": 27, + "context": 39, + "answer": 39 + }, + "windows": 0, + "sequences": 2 + } + } +} diff --git a/tests/vela/test_hybrid.py b/tests/vela/test_hybrid.py new file mode 100644 index 00000000..ee3fb300 --- /dev/null +++ b/tests/vela/test_hybrid.py @@ -0,0 +1,310 @@ +"""Tests for the Vela 2.0 0.8B hybrid member: gate, worker dispatch, response finishing. + +Gate tests mirror test_engine_manifest: temp directories with an +injectable pin table and recomputed hashes — no weights, no torch, no +network. Worker tests drive FakeVela2Hybrid (from test_worker's fake, +plus the hybrid response surface) through decide/detect_member/ +models_entry: no weights, no torch, no GPU. + + PYTHONPATH=src python -m pytest tests/vela -q +""" + +from __future__ import annotations + +import hashlib +import json +import re +from pathlib import Path + +import pytest + +from frontend.vela import decide, detect_member, models_entry +from models.vela.engine import Vela2Executor, verify_files +from models.vela.hybrid import ( + EXPECTED_ARCHITECTURE, + EXPECTED_MAX_LENGTH, + EXPECTED_MODEL_NAME, + EXPECTED_MODEL_TYPE, + EXPECTED_WEIGHTS_BYTES, + HYBRID_REPO, + HYBRID_REVISION, + PINNED_FILES_SHA256, + WEIGHTS_NAME, + Vela2HybridExecutor, + verify_config, + verify_package, + verify_weights, +) +from models.vela.postprocess import finish_response + +from .test_protocol import body +from .test_worker import FakeVela2, five_kind_body + +CONFIG = { + "model_type": EXPECTED_MODEL_TYPE, + "architectures": [EXPECTED_ARCHITECTURE], + "max_length": EXPECTED_MAX_LENGTH, + "model_name": EXPECTED_MODEL_NAME, +} + + +def table_of(files: dict[str, bytes]) -> dict[str, str]: + return {name: hashlib.sha256(content).hexdigest() for name, content in files.items()} + + +def small_files(): + return { + "vela2_serve.py": b"upstream server stub\n", + "vela2_inference.py": b"upstream engine stub\n", + "modeling_vela2.py": b"upstream model stub\n", + "configuration_vela2.py": b"upstream config stub\n", + "calibration.json": b'{"temperature": {}}', + "tokenizer.json": b'{"version": "stub"}', + "tokenizer_config.json": b'{"model_type": "stub"}', + "model.safetensors.index.json": b'{"metadata": {"total_size": 1}}', + "broad_head.safetensors": b"second span head stub\n", + } + + +def build_package(root: Path, files: dict[str, bytes], config: dict | None = None) -> dict[str, str]: + for name, content in files.items(): + (root / name).write_bytes(content) + (root / "config.json").write_text( + json.dumps(config if config is not None else CONFIG), encoding="utf-8" + ) + with (root / WEIGHTS_NAME).open("wb") as stream: + stream.truncate(EXPECTED_WEIGHTS_BYTES) + files = dict(files) + files["config.json"] = (root / "config.json").read_bytes() + return table_of(files) + + +def hybrid_body(): + value = five_kind_body() + value["model"] = EXPECTED_MODEL_NAME + return value + + +class FakeVela2Hybrid(FakeVela2): + """The pinned hybrid runtime's contract, driven without its code.""" + + is_hybrid = True + + def __init__(self, model_name=EXPECTED_MODEL_NAME, too_long=False): + super().__init__(model_name, too_long) + self.max_length = 16384 + + class _Engine: + config = { + "model_name": EXPECTED_MODEL_NAME, + "description": ( + "Vela 2.0 0.8B: SystemOne Choice / Noul / Score " + "with span and set extensions, 16,384-token input" + ), + "release_date": "", + } + + self.engine = _Engine() + + def system_one(self, body): + response = super().system_one(body) + if "spans" in response: + # Upstream emits span_heads together with spans, keyed by the + # span question ids; router stands in for both heads here. + response["span_heads"] = {qid: "router" for qid in response["spans"]} + return response + + +# --------------------------------------------------------------------- gate + + +def test_the_checked_in_hybrid_pin_table_shape_is_locked(): + assert HYBRID_REPO == "vllm-sr/Vela-2.0-0.8B" + assert HYBRID_REVISION == "326b01d81f61fff7fd4ae570e54b87a49119e490" + assert set(PINNED_FILES_SHA256) == { + "vela2_serve.py", "vela2_inference.py", "modeling_vela2.py", + "configuration_vela2.py", "config.json", "calibration.json", "tokenizer.json", + "tokenizer_config.json", "model.safetensors.index.json", "broad_head.safetensors", + } + for digest in PINNED_FILES_SHA256.values(): + assert re.fullmatch(r"[0-9a-f]{64}", digest) + + +def test_verification_accepts_a_package_matching_its_table(tmp_path): + table = build_package(tmp_path, small_files()) + assert verify_files(tmp_path, table) == sorted(table) + verify_config(tmp_path) + verify_weights(tmp_path) + verify_package(tmp_path, table) + + +def test_a_unified_package_is_refused_with_its_own_name(tmp_path): + build_package( + tmp_path, small_files(), + config={**CONFIG, "model_type": "vela2-unified"}, + ) + with pytest.raises(ValueError, match="vela2-unified.*only serves the vela2-decoder member"): + verify_config(tmp_path) + + +@pytest.mark.parametrize( + ("override", "fragment"), + [ + ({"architectures": ["OtherModel"]}, "architectures"), + ({"max_length": 8192}, "max_length"), + ({"model_name": "vllm-sr/Vela-2.0-4B"}, "needs its own pinned executor"), + ], +) +def test_foreign_config_structures_are_refused(tmp_path, override, fragment): + build_package(tmp_path, small_files(), config={**CONFIG, **override}) + with pytest.raises(ValueError, match=fragment): + verify_config(tmp_path) + + +def test_tampered_files_are_rejected(tmp_path): + table = build_package(tmp_path, small_files()) + victim = tmp_path / "broad_head.safetensors" + victim.write_bytes(victim.read_bytes() + b"tampered\n") + # The shared gate must name the hybrid's pin, not the 0.3B's. + with pytest.raises( + ValueError, match=f"broad_head.safetensors: SHA-256 differs from the pin for {HYBRID_REVISION}" + ): + verify_files(tmp_path, table, HYBRID_REVISION) + + +def test_a_missing_broad_head_is_rejected(tmp_path): + table = build_package(tmp_path, small_files()) + (tmp_path / "broad_head.safetensors").unlink() + with pytest.raises(ValueError, match="missing pinned file broad_head.safetensors"): + verify_files(tmp_path, table) + + +def test_a_later_self_consistent_snapshot_is_refused_by_the_checked_in_pin(tmp_path): + build_package(tmp_path, small_files()) + with pytest.raises(ValueError, match="SHA-256 differs from the pin"): + verify_files(tmp_path, PINNED_FILES_SHA256) + + +def test_weights_are_checked_by_exact_pinned_size(tmp_path): + build_package(tmp_path, small_files()) + verify_weights(tmp_path) + + with (tmp_path / WEIGHTS_NAME).open("wb") as stream: + stream.truncate(EXPECTED_WEIGHTS_BYTES - 1) + with pytest.raises(ValueError, match="is 2022189487 bytes"): + verify_weights(tmp_path) + + (tmp_path / WEIGHTS_NAME).unlink() + with pytest.raises(ValueError, match=f"missing {WEIGHTS_NAME}"): + verify_weights(tmp_path) + + +# ------------------------------------------------------------- worker wiring + + +def test_detect_member_routes_each_declared_member(tmp_path): + unified = tmp_path / "unified" + unified.mkdir() + (unified / "config.json").write_text(json.dumps({"model_type": "vela2-unified"})) + decoder = tmp_path / "decoder" + decoder.mkdir() + (decoder / "config.json").write_text(json.dumps({"model_type": "vela2-decoder"})) + + cls, name = detect_member(unified) + assert cls is Vela2Executor and name == "vllm-sr/Vela-2.0-0.3B" + cls, name = detect_member(decoder) + assert cls is Vela2HybridExecutor and name == EXPECTED_MODEL_NAME + + +def test_detect_member_refuses_packages_without_a_usable_config(tmp_path): + empty = tmp_path / "empty" + empty.mkdir() + with pytest.raises(ValueError, match="missing config.json"): + detect_member(empty) + foreign = tmp_path / "foreign" + foreign.mkdir() + (foreign / "config.json").write_text(json.dumps({"model_type": "vela2-unified-xl"})) + with pytest.raises(ValueError, match="vela2-unified-xl"): + detect_member(foreign) + broken = tmp_path / "broken" + broken.mkdir() + (broken / "config.json").write_text("{not json") + with pytest.raises(ValueError, match="not readable JSON"): + detect_member(broken) + array = tmp_path / "array" + array.mkdir() + (array / "config.json").write_text("[]") + with pytest.raises(ValueError, match="not a JSON object"): + detect_member(array) + + +def test_models_entry_is_verbatim_per_member(): + hybrid = models_entry(FakeVela2Hybrid()) + assert hybrid == { + "name": EXPECTED_MODEL_NAME, + "release_date": "", + "description": ( + "Vela 2.0 0.8B: SystemOne Choice / Noul / Score " + "with span and set extensions, 16,384-token input" + ), + } + assert models_entry(FakeVela2())["release_date"] == "2026-09-29" + + +# -------------------------------------------------------- response finishing + + +def test_decide_answers_all_five_kinds_with_span_heads(): + response = decide(json.dumps(hybrid_body()).encode(), FakeVela2Hybrid()) + assert set(response) == {"model", "answers", "usage", "spans", "sets", "thresholds", "span_heads"} + assert list(response["span_heads"]) == ["unsupported"] + assert response["model"] == EXPECTED_MODEL_NAME + + +def test_decide_accepts_a_hybrid_response_without_span_questions(): + value = body(model=EXPECTED_MODEL_NAME) + response = decide(json.dumps(value).encode(), FakeVela2Hybrid()) + assert "span_heads" not in response and "spans" not in response + + +def test_a_hybrid_response_missing_span_heads_is_refused(): + class Headless(FakeVela2Hybrid): + def system_one(self, body): + response = super().system_one(body) + response.pop("span_heads", None) + return response + + with pytest.raises(RuntimeError, match="span_heads do not match"): + decide(json.dumps(hybrid_body()).encode(), Headless()) + + +def test_a_hybrid_response_with_stray_span_heads_is_refused(): + class Stray(FakeVela2Hybrid): + def system_one(self, body): + response = super().system_one(body) + response["span_heads"] = {"ghost": "router"} + return response + + with pytest.raises(RuntimeError, match="span_heads without span questions"): + decide(json.dumps(body(model=EXPECTED_MODEL_NAME)).encode(), Stray()) + + +def test_the_unified_member_still_refuses_span_heads(): + class UnifiedLeak(FakeVela2): + def system_one(self, body): + response = super().system_one(body) + if "spans" in response: + response["span_heads"] = {qid: "router" for qid in response["spans"]} + return response + + with pytest.raises(RuntimeError, match="this worker serves the 0.3B member"): + decide(json.dumps(five_kind_body()).encode(), UnifiedLeak()) + + +def test_finish_response_keeps_the_hybrid_envelope_tight(): + questions = hybrid_body()["questions"] + response = FakeVela2Hybrid().system_one(hybrid_body()) + assert finish_response(questions, response, hybrid=True) is response + del response["answers"] + with pytest.raises(RuntimeError, match="unexpected response envelope"): + finish_response(questions, response, hybrid=True) diff --git a/tests/vela/test_oracle_hybrid_fixtures.py b/tests/vela/test_oracle_hybrid_fixtures.py new file mode 100644 index 00000000..6f3fc1c6 --- /dev/null +++ b/tests/vela/test_oracle_hybrid_fixtures.py @@ -0,0 +1,64 @@ +"""Ties the frozen hybrid oracle fixtures to the recipe examples they came from. + +Mirrors test_oracle_fixtures (the 0.3B tie) for the 0.8B hybrid fixtures +recorded by ``recipe/vela/compare_reference.py`` under the +``oracle-hybrid-`` prefix. Each fixture's ``request`` is the recipe's +example request with the ``model`` field rewritten to the hybrid's +served name (the comparison tool does exactly this rewrite for file +cases), and its recorded ``system_one`` response passes the worker's +own response gate on the hybrid setting. No weights, no torch. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from models.vela.hybrid import EXPECTED_MODEL_NAME +from models.vela.postprocess import finish_response + +TESTS = Path(__file__).parent +RECIPE = TESTS.parent.parent / "recipe" / "vela" + +CASES = [ + ("oracle-hybrid-example-request", "example-request.json"), + ("oracle-hybrid-example-request-cjk", "example-request-cjk.json"), + ("oracle-hybrid-example-request-set-threshold", "example-request-set-threshold.json"), + ("oracle-hybrid-example-request-score", "example-request-score.json"), +] + + +def oracle(case: str) -> dict: + return json.loads((TESTS / "fixtures" / f"{case}.json").read_text()) + + +def recipe_body(example: str) -> dict: + body = json.loads((RECIPE / example).read_text()) + # The tool's load_cases rewrite for file cases: the examples are + # member-agnostic apart from the model field. + return {**body, "model": EXPECTED_MODEL_NAME} + + +@pytest.mark.parametrize("case,example", CASES) +def test_the_oracle_request_is_the_recipe_example(case: str, example: str): + recorded = oracle(case) + served = recipe_body(example) + assert recorded["request"] == served + # Same insertion order, so byte-for-byte serialization holds too. + assert list(recorded["request"]) == list(served) + + +@pytest.mark.parametrize("case,example", CASES) +def test_the_recorded_response_passes_the_worker_gate(case: str, example: str): + recorded = oracle(case) + response = recorded["system_one"] + assert response["model"] == EXPECTED_MODEL_NAME + assert finish_response(recorded["request"]["questions"], response, hybrid=True) is response + if any(q.get("type") == "span" for q in recorded["request"]["questions"].values()): + assert set(response["span_heads"]) == { + qid for qid, q in recorded["request"]["questions"].items() if q.get("type") == "span" + } + # The frozen case label still names the recipe file it belongs to. + assert recorded["case"] == example.removesuffix(".json") diff --git a/tests/vela/test_worker_server.py b/tests/vela/test_worker_server.py index ce2d106b..9221a933 100644 --- a/tests/vela/test_worker_server.py +++ b/tests/vela/test_worker_server.py @@ -292,3 +292,51 @@ def test_the_correct_bearer_answers_normally(keyed_server): headers=(("Authorization", "Bearer secret-key"),), ) assert status == 200 and json.loads(raw)["answers"]["billing"] == {"type": "noul", "noul": 0.75} + + +@pytest.fixture() +def hybrid_server(): + from .test_hybrid import FakeVela2Hybrid, hybrid_body + + instance = start(FakeVela2Hybrid()) + yield instance, hybrid_body() + stop(instance) + + +def test_the_hybrid_member_serves_its_own_models_entry(hybrid_server): + server, _ = hybrid_server + status, _, raw = request(server, "GET", "/v1/models") + assert status == 200 + assert json.loads(raw) == { + "models": [ + { + "name": "vllm-sr/Vela-2.0-0.8B", + "release_date": "", + "description": ( + "Vela 2.0 0.8B: SystemOne Choice / Noul / Score " + "with span and set extensions, 16,384-token input" + ), + } + ] + } + + +def test_the_hybrid_member_answers_all_five_kinds_with_span_heads(hybrid_server): + server, hybrid_body = hybrid_server + status, ctype, raw = post(server, json.dumps(hybrid_body).encode("utf-8")) + assert status == 200 and ctype == "application/json" + response = json.loads(raw) + assert response["model"] == "vllm-sr/Vela-2.0-0.8B" + assert set(response) == { + "model", "answers", "usage", "spans", "sets", "thresholds", "span_heads" + } + assert list(response["span_heads"]) == ["unsupported"] + assert response["answers"]["unsupported"] == {"type": "noul", "noul": 0.9} + + +def test_the_hybrid_member_rejects_another_model_name(hybrid_server): + server, _ = hybrid_server + wrong = body(model="vllm-sr/Vela-2.0-0.3B") + status, _, raw = post(server, json.dumps(wrong).encode("utf-8")) + assert status == 422 + assert json.loads(raw)["detail"][0]["type"] == "value_error.model" From 78fe14378a008b65e8cad83445efbd1dd84adcc7 Mon Sep 17 00:00:00 2001 From: "Mr.Dong" <1042542469@qq.com> Date: Mon, 26 Oct 2026 13:36:08 +0800 Subject: [PATCH 2/2] vela: name both members in the recipe index and fix the oracle-fixture docstring --- recipe/README.md | 3 ++- tests/vela/test_oracle_hybrid_fixtures.py | 13 ++++++------- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/recipe/README.md b/recipe/README.md index cab14625..f7ea12a2 100644 --- a/recipe/README.md +++ b/recipe/README.md @@ -51,7 +51,8 @@ For a first real decision, follow the [complete CPU walkthrough](../docs/getting - [Vela 2.0 reference workers (0.3B and the 0.8B hybrid)](vela/reference.md): download the pinned vllm-sr/Vela-2.0-0.3B package, pass its full-hash package gate, and serve all five SystemOne question kinds (`choice`/`noul`/`score` plus the `span`/`set` - extensions) through the Python worker and the Rust frontend on CPU FP32. + extensions) through the Python worker and the Rust frontend on CPU FP32; + or vllm-sr/Vela-2.0-0.8B at revision `326b01d8` behind its own 10-file gate. - [Clef reference worker](clef/reference.md): load pinned Clef-flash (9B) or Clef (27B) and serve text `choice`/`noul`/`score`. Image, video and native CUDA are not served. Operator-reported flash Public231 concurrency-1 diff --git a/tests/vela/test_oracle_hybrid_fixtures.py b/tests/vela/test_oracle_hybrid_fixtures.py index 6f3fc1c6..3114a904 100644 --- a/tests/vela/test_oracle_hybrid_fixtures.py +++ b/tests/vela/test_oracle_hybrid_fixtures.py @@ -1,12 +1,11 @@ """Ties the frozen hybrid oracle fixtures to the recipe examples they came from. -Mirrors test_oracle_fixtures (the 0.3B tie) for the 0.8B hybrid fixtures -recorded by ``recipe/vela/compare_reference.py`` under the -``oracle-hybrid-`` prefix. Each fixture's ``request`` is the recipe's -example request with the ``model`` field rewritten to the hybrid's -served name (the comparison tool does exactly this rewrite for file -cases), and its recorded ``system_one`` response passes the worker's -own response gate on the hybrid setting. No weights, no torch. +The 0.8B hybrid fixtures recorded by ``recipe/vela/compare_reference.py`` +under the ``oracle-hybrid-`` prefix keep the recipe's example requests +(with the ``model`` field rewritten to the hybrid's served name) and their +recorded ``system_one`` responses, which must pass the production +``finish_response`` gate on the hybrid setting; the 0.3B oracles have no +equivalent consumer yet. No weights, no torch. """ from __future__ import annotations