[New Model][RFC]: Support vllm-sr/d3-turbo #190

The model to consider.

Add support for vllm-sr/d3-turbo, starting with this immutable package:

vllm-sr/d3-turbo@a0f3e9e59ebd1202fe1d4de1c4c0f9a986758cbb

It is a 27B-class multimodal Decision 3.0 checkpoint from the vLLM Semantic Router team, licensed under Apache-2.0. The inspected package declares attention_mode: prefix_bidir and a separate FP32 255-code readout.

This is an independent model-support issue. Link it from the model tracker #83 and roadmap #125; retain #136 as related existing D3 work rather than replacing this model's own tracking.

First supported scope

• Text or JSON state; optionally one top-level inline PNG/JPEG image.
• choice, noul and score through the existing /v1/systemone frontend/worker boundary.
• The pinned package's default per-question context policy, explicit transport/media budgets, and no silent truncation.
• An explicit turbo profile, with one selected package/profile per worker process.

Box/Track, images inside questions/options (assets), multiple images, videos, long_context, quantization and additional hardware backends are outside the first slice. Reject unsupported inputs explicitly. Keep the existing legacy D3 profile's supported behavior and media paths intact.

The closest planned or implemented model.

System1-Omni already has a pinned D3 reference integration:

• #142: the reference worker.
• #148: image/video serving.
• #156: package identity and runtime-import checks.
• #168: legacy D3 native prerequisites and FP32 answer-code readout.

At the inspected main snapshot, the worker pins vllm-sr/d3@cbff3a3cb0b074b6585fe9f747b9f9f5557a0e75. It is not an integration of the turbo package. Reuse src/models/d3/, the existing worker adapter and its verification discipline; do not add a competing HTTP worker or weaken the legacy profile's package gate.

The important execution difference is the full-attention visibility:

• Legacy D3's noncausal_full_attention lets each prompt's state and question attend across the whole prompt.
• Turbo's prefix_bidir lets the state attend only to the complete state, while each question attends to that state and its own complete question block.
• Gated DeltaNet remains causal; questions must not read one another.

Main already has a whole-prompt bidirectional Qwen entry point from #164. Audit and reuse the applicable implementation; it does not by itself implement turbo's block mask or state-once continuation.

Coordinate model preparation/head reuse with the existing D3 work, and shared execution interfaces with #85 and #150. This proposal does not assign new work to their current contributors.

What's your difficulty of supporting the model you want?

1. Pinned reference profile

Use the fixed package's public AutoModel.system_one as the model-answer oracle. Audit its high-level composition, validation and error handling before adapting it; the legacy prepare/run/respond contract cannot simply be assumed unchanged.

The reference stage delegates to the bundled turbo runtime, including its state-once behavior. It does not implement a second shared state-once engine in the model PR.

Bind the selected repo/revision, runtime/wrapper/compose file closure, configuration, tokenizer/processor and readout to that profile. Verify the files that will actually execute before import, and prevent old/new packages' same-named Python modules from being mixed within a process.

Freeze prompt text, token IDs, exact state/question boundaries, legal code order, image tensors/grids/positions, probabilities, typed answers, errors and usage. Preserve the model's own confidence, calibration, ordering and token-accounting semantics. Record any intentional wire restrictions.

Use two separate fixed-package reference processes where needed: the default state-once mode and D3_STATE_ONCE=0, which evaluates full question rows under the same mask. Keep their numerical and usage differences separate.

2. Native correctness as a later milestone

After reference serving and fixtures are accepted, add model-specific preparation, checkpoint export, the FP32 code head and native text/image execution using qualified shared-backend interfaces.

Establish a full-row, same-prefix_bidir eager baseline before enabling native reuse. Compare exact preparation first, then hidden states, readout, probabilities and decisions under tolerances declared before native comparison. Report near-boundary decision changes explicitly.

The semantic state boundary must follow the pinned tokenizer/runtime. Applying the existing causal prefix planner's 64-token rounding without proving equivalence can change which tokens attend to the question.

3. Shared features remain separate work

Shared block-attention/continuation operations, request-local observation reuse and question branch batching must be coordinated on their own existing or focused follow-up issues/PRs. This model issue records dependencies and model integration acceptance; it does not implement a general prefix cache, serving contract, scheduler or shared batcher.

Keep public wire-contract changes with #61, observation reuse with #85, and shared admission/batching/scheduling with #150. Cross-request batching or scheduling requires its own follow-up.

Delivery and acceptance

Reference serving, native correctness and qualified optimized execution are separate milestones, not a precommitted PR count.

• Add an explicit turbo profile without changing the legacy D3 profile's behavior or weakening package verification.
• Serve the first-slice question types and text/JSON/single-image inputs, with explicit unsupported-input rejections, fixed limits and reference-defined answers/errors/usage.
• Freeze small deterministic preparation and answer fixtures with provenance and generation commands; put every new test body, helper and fixture under root tests/.
• Complete real warmup before readiness; verify first/repeated requests, failure recovery, shutdown and direct/frontend-proxied behavior against the pinned reference.
• Include the operator recipe in the model draft before calling the integration runnable: artifact pins, dependency/setup requirements, worker/frontend launch, health, one real request, validation commands, resource requirements and limitations.
• Update the model README, root README, recipe/README.md, supported-model status and mkdocs.yml alongside implemented scope.
• Once text serving works, establish this model's own frozen Public231 C1 baseline: contract smoke, warmup 5, concurrency 1, three measured rounds, complete measured SHA, actual device/compute capability, serving pass rate and pooled p50/p95. Keep image/many-question experiments as supplementary evidence.
• For native bring-up, pass the frozen preparation and numerical checks and revalidate actual consumers of changed shared operations; CPU fixtures/builds do not establish GPU parity.
• Record reference, native and optimized support separately; retain unrun checks and unsupported hardware explicitly.

Start the model implementation on a branch targeting this independent issue, with its PR kept draft. Complete the repository's human contributor-intent gate before implementation and PR submission; proposed use cases below are not a substitute for explicit human answers or a PR-necessity decision.

Once a measured primary exists, model-specific latency work stays on that model's draft branch: one optimization tip, then smoke/screen and the required same-tip Public231 C1 remeasure before another tip. Follow the repository's KEEP/DROP/inconclusive and quality rules. Shared-feature implementations stay outside that model diff.

Ready-for-review and merge each require separate explicit owner approval; CI success or a favorable latency result does not supply that approval.

Use case and motivation

Proposed workloads include routing/classifying a ticket with several typed checks, and asking several independent questions about one agent screenshot. A pinned turbo profile would make the model available behind System1-Omni's existing serving interface and establish a reproducible basis for later native integration.

The first success criterion is correct, reproducible serving within the declared scope. Any local speed, memory, quality or hardware-support claim must come from the measured implementation revision. This proposal does not claim that upstream multi-question measurements reproduce on System1-Omni.

Current validation status

This is a proposal based on source/configuration inspection. No turbo model inference, native parity, Public231 C1 measurement or performance optimization has been run for this issue draft.

When naming the intended production target, use H200 / SM90. Every actual result must report the device string and compute capability measured on its host; another reported identity is not by itself an invalid run. No GPU allocation or execution is implied by this proposal.

Reference links

• Pinned decision configuration
• Pinned runtime
• Pinned AutoModel wrapper
• Existing D3 contract
• Contribution guide
• New-model workflow

Before submitting a new issue...

• Searched existing/past Issues, related PRs and the model documentation. On 2026-10-27, the repository's Issues search for turbo and all-state PR search for d3-turbo found no dedicated match. This search result is not maintainer acceptance; recheck before publication.