Repository navigation
Conversation
added 8 commits
October 25, 2026 02:56
Second member of the Vela 2.0 family: a pinned vela2-decoder executor (hybrid.py) with a 10-file gate (shard index and the side-loaded broad head hashed; the 2.02 GB shard by exact size), member-aware response finishing (span_heads with spans, keyed by span question ids), worker member detection from config.json model_type, and the per-member /v1/models entry. compare_reference.py serves either member; the hybrid oracle fixtures export under oracle-hybrid-. Evidence (author-run, x86 WSL2, CPU FP32, no fla — the runtime's CPU path never dispatches to it): nine-case parity vs the pinned upstream server exact-match (four 200s incl. key order, five agreed rejections incl. the 16,384-window 413); 0.3B nine-case regression; live worker smoke with span_heads and per-member thresholds; byte-identical Rust frontend forwarding; 133 CPU contract tests.
- recipe/vela/export_native.py: gate-verified export into a native bundle — 320 backbone tensors re-keyed backbone. -> model. with per-tensor dtypes preserved (204 BF16 Linear/conv1d + 116 F32), the 37 F32 heads incl. the side-loaded broad span head in one file, backbone_config at the config root (the family Config::load shape), and a manifest pinning the source revision, the gate table, the tensor inventory and every bundle file's SHA-256. - src/models/vela/native (omni-vela-native): the bundle manifest gate and the wire admission mirroring protocol.py and the pinned runtime's preamble with byte-equal 422/400 bodies. - recipe/vela/export_token_plan.py and five frozen token plans under tests/vela/fixtures/: the predict mapping, the row decomposition and the rendered records (p_ids, blocks, readout positions, truncation, a windowed long-span case); example-request totals 1,041 tokens, matching the oracle's usage. Checks: cargo test (15 tests, no weights, no GPU), fmt/clippy clean, mkdocs --strict.
The Rust processing of the native path, mirrored from the pinned runtime and checked field-for-field against the five frozen token plans (opt-in render parity with the exported bundle's tokenizer): - state.rs: the state -> typed-parts mapping (role grouping, field ranges, canonical DEV2.0 JSON) with the pinned rejection triples; - questions.rs: the SystemOne -> predict mapping — the validation walk and order, Noul folded into a two-option choice (choice renders [ABS], the reference's abstain default), over resolution, threshold/head carried, the set-label collision check; - words.rs: the GLiNER2/CJK _WORD splitter and the first-token word mapping; - render.rs: the encoder (char-offset tokens, memoized), the six-rule span-head dispatch with the hallucination-alias label replacement, the truncation budget, the DEV2.0 rendering (parts pass, question blocks with option ends and query positions, span layout a/c with word positions) and the window rows (head only, as the reference); - processing.rs: the Processor over the exported bundle. Also: the envelope error locs are flat arrays, matching the reference triples exactly. Checks: cargo test (26 tests) + fmt/clippy clean + opt-in render parity (5/5 fixtures).
The Rust processing of the native path, mirrored from the pinned runtime and checked field-for-field against the five frozen token plans (opt-in render parity with the exported bundle's tokenizer): - state.rs: the state -> typed-parts mapping (role grouping, field ranges, canonical DEV2.0 JSON) with the pinned rejection triples; - questions.rs: the SystemOne -> predict mapping — the validation walk and order, Noul folded into a two-option choice (choice renders [ABS], the reference's abstain default), over resolution, threshold/head carried, the set-label collision check; - words.rs: the GLiNER2/CJK _WORD splitter and the first-token word mapping; - render.rs: the encoder (char-offset tokens, memoized), the six-rule span-head dispatch with the hallucination-alias label replacement, the truncation budget, the DEV2.0 rendering (parts pass, question blocks with option ends and query positions, span layout a/c with word positions) and the window rows (head only, as the reference); - processing.rs: the Processor over the exported bundle. Also: the envelope error locs are flat arrays, matching the reference triples exactly. Checks: cargo test (26 tests) + fmt/clippy clean + opt-in render parity (5/5 fixtures).
- heads.rs: CandidateHead (option-end + query hiddens; bilinear + scalar-MLP, [ABS] last), both SpanHeadV2 instances (router and the side-loaded broad head) and set_bias, loaded from the exported bundle's heads.safetensors — plain float32, the reference's own autocast-off setting (erf-based GELU via libm). - scoring.rs: the calibration mirror — per-type temperatures, set thresholds, the PII length rule (log-space interp) and sparse gate, span thresholds by id/label set, softmax/sigmoid, span_decode_r4 (word units, unit vote, fill, edge/URL trim), label aliasing, score value normalization (i/(n-1)), and the window aggregation (mean over covering windows, -30 uncovered). - recipe/vela/export_head_inputs.py and the recorded head-input fixtures: the reference's consumed hidden states, its head logits and its calibrated predict answers for the five frozen cases. Opt-in parity (VELA_NATIVE_BUNDLE): five cases, head-logit drift <= 3.8e-5, predict-answer drift <= 7.3e-7 (the windowed long-span case exact), discrete picks, span structures and threshold rules equal.
- heads.rs: CandidateHead (option-end + query hiddens; bilinear + scalar-MLP, [ABS] last), both SpanHeadV2 instances (router and the side-loaded broad head) and set_bias, loaded from the exported bundle's heads.safetensors — plain float32, the reference's own autocast-off setting (erf-based GELU via libm). - scoring.rs: the calibration mirror — per-type temperatures, set thresholds, the PII length rule (log-space interp) and sparse gate, span thresholds by id/label set, softmax/sigmoid, span_decode_r4 (word units, unit vote, fill, edge/URL trim), label aliasing, score value normalization (i/(n-1)), and the window aggregation (mean over covering windows, -30 uncovered). - recipe/vela/export_head_inputs.py and the recorded head-input fixtures: the reference's consumed hidden states, its head logits and its calibrated predict answers for the five frozen cases. Opt-in parity (VELA_NATIVE_BUNDLE): five cases, head-logit drift <= 3.8e-5, predict-answer drift <= 7.3e-7 (the windowed long-span case exact), discrete picks, span structures and threshold rules equal.
scoring.rs system_one_response: the SystemOne envelope from the predict answers and the response plan — Noul views (span = the range-filtered max word probability, set fan-out <qid>.<label>), span clipping to the plan's field ranges with re-trim, span_heads, thresholds, the choice top-two margin and the score variance confidence, and usage. Replaying the recorded reference logits into the frozen oracle responses matches within 1.5e-6 on the four oracle cases (opt-in with VELA_NATIVE_BUNDLE).
executor.rs: the backbone through the shared qwen3_5 crate. The family Model exposes only last-position hidden states, so every readout position runs as its own full sequence [parts .. position] — causality makes that hidden identical to the reference's in-tree readout — packed into forward_batch. The backbone runs from the uniform-BF16 export (export_native.py --bf16-out, the shared loader's format); the heads stay F32 on the faithful tensors. Opt-in GPU parity (VELA_NATIVE_BUNDLE + VELA_CUDA_LIB): four cases replay the recorded reference logits, worst drift 0.285 (the bf16-vs-FP32 scale; gate 0.35). First recorded execution of these kernels on sm_120 (RTX 5060, CUDA 13.1).
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
added 4 commits
October 25, 2026 14:26
…processing # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…reference-serving # Conflicts: # README.md # docs/supported-models.md # recipe/README.md
…heads # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
…executor # Conflicts: # Cargo.toml # README.md # docs/supported-models.md # recipe/README.md
MrDongsls
force-pushed
the
model/vela2-native-executor
branch
from
October 25, 2026 06:48
382d46d to
b6dc583
Compare
added 16 commits
October 25, 2026 18:34
…ember docs Keep the Clef and pplx-decider rows from main alongside the Vela 0.8B hybrid additions; the package-gate descriptions now carry the directory-symlink refusal from main.
…ows alongside the Clef/pplx-decider additions
… add the Clef/pplx rows
…native-processing # Conflicts: # recipe/README.md
added 3 commits
October 26, 2026 13:46
…heads # Conflicts: # recipe/vela/native.md
# Conflicts: # recipe/vela/native.md # src/models/vela/native/Cargo.toml # src/models/vela/native/src/bundle.rs # tests/vela/native_heads.rs # tests/vela/native_processing.rs
This was referenced Oct 26, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #151 + #174 + #175 + #176 — review the last two commits only.
Required human contributor answers
What outcome are you trying to achieve?
Complete the native library: the SystemOne response envelope from predict answers (Noul views, set fan-out, span clipping, confidence, usage), and backbone execution on CUDA through the shared qwen3_5 crate. (Source: issue [New Model]: Support the Vela 2.0 series #137.)
What is the highest-leverage variable?
Backbone readout parity — the causality argument (full-sequence-per-position ≡ in-tree readout) must hold, and the bf16-vs-FP32 gap must stay within declared gates.
How will this work move that variable?
The opt-in GPU test replays the recorded reference logits through the Rust heads fed by the CUDA backbone — four cases, worst drift 0.285, gate 0.35 — independently reproduced by the precheck.
Who is the customer or user?
Contributors wiring the HTTP worker around Processor + Executor; ultimately users needing GPU-accelerated Vela decisions.
Is a PR needed? Yes — completes the native library.
For a bug fix: N/A.
Purpose
Two commits:
scoring.rs system_one_response— Noul views (span = range-filtered max word probability via per-label sigmoid, set fan-out), span clipping with re-trim,span_heads, thresholds, confidence formulas, usage. Replays recorded logits into frozen oracle responses.executor.rs— full-sequence-per-readout-position throughforward_batch(causality makes each hidden identical to the reference's in-tree readout; layout-a/c both handled). Uniform-BF16 backbone (pinned in the manifest); heads stay F32.Test Plan
System1-Omni Version / Commit:
model/vela2-native-executor@064244dcargo test -p omni-vela-nativecargo clippy -p omni-vela-native --all-targets -- -D warningscargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warningsmkdocs build --strictrecipe/vela/test_compare_reference.pyVELA_NATIVE_BUNDLE=… --test vela_native_response -- --ignoredVELA_NATIVE_BUNDLE=… VELA_CUDA_LIB=… --test vela_native_executor -- --ignored149a9bab)Test Result
All passed. GPU parity is opt-in (~5.7 min on tested hardware).
Demo / evidence
Author-run, x86 WSL2, RTX 5060 (sm_120), CUDA 13.1, CPU-FP32 fixtures:
Response parity (4/4 oracle, drift ≤1.5e-6):
GPU executor parity (4/4, logit drift vs CPU-FP32 reference, gate 0.35):
Independently reproduced by the precheck (212.6s vs author's 206s, identical drifts). First recorded execution of the shared qwen3_5 CUDA kernels on sm_120 (Blackwell).
Precision note: the uniform-BF16 backbone is a slightly lower-precision profile than the reference's GPU autocast (which keeps norms and GDN gate math in FP32); drift concentrates in span heads (0.05–0.28), q blocks at 0.0066. Long-span and broad-head GPU cases not measured.
Usage note:
input_tokenspasses through from the caller; the reference'ssum(n_input)is not reimplemented in native code. The response test feeds the oracle's value — usage parity rests on the reference until engine wiring.Not measured: serving latency, throughput, memory.
Self-review