[Integration]: Freeze Intern-Decision-4B HF oracle and multi-field contract #167

Parent and scope

Follow-up to #133, scoped to the reference contract needed before the first native Intern-Decision-4B PR. The human contributor's requested outcome is reproducible native support for System1-Omni users, preserving the official reference through backend reuse and small independent PRs.

At main cd302b6e3d7723ebfc33bba9698da2c1e881f203, there is no Intern-Decision worker. #133 documents two official backends with different numerical behavior. Comparing a new worker to whichever backend happens to pass would not establish parity.

Proposed first slice

Use the official HF BF16 4B path as the normative oracle, with its source commit, checkpoint revision, tokenizer/processor, calibration checksum, Transformers and FLA versions frozen together. Treat XTuner as a separate comparison with a separately recorded environment; do not require bitwise equality or mix its expected values into HF fixtures. This is a proposed implementation choice, not a claim that a maintainer has already approved it.

Keep text and online image preparation, choice/noul/score, and multi-field readout in the first native milestone. Defer 2B/0.8B, quantization, Metal, Graphs and prefix reuse to separately validated slices.

Acceptance

• Record immutable reference/checkpoint/processor/tokenizer/calibration pins, licenses, and a reproducible reference command; verify frozen vision tensors against the named Qwen base.
• Preserve exact schema-to-prompt rendering and field/option order. Verify <decision> and all 62 answer symbols are single tokens under the selected tokenizer.
• Capture prompt text, IDs, T/H/W positions, expanded image counts and the logit row immediately before every placeholder. Cover text, one/multiple images, mixed three-type requests, 1/16 fields and 62 options.
• Record the /v1/decisions to /v1/systemone mapping, reference confidence, usage, score and tie behavior. Count the single prefill once and generated tokens as zero where the pinned reference does so; verify rather than infer these semantics from sibling workers.
• Reject malformed and over-budget inputs without silent truncation; test the 8192-token budget including image tokens and the up-to-8-image limit described in [New Model]: Support internlm/Intern-Decision (4B / 2B / 0.8B) — third Qwen3.5 decision family, single-prefill multi-field readout #133 against the pinned implementation.
• Declare intermediate-logit, probability and decision-agreement tolerances before native comparison. Include near ties and per-checkpoint calibration; keep fixture generation separate from the implementation under test.
• Record HF/XTuner differences and the selected oracle explicitly. A disagreement is evidence to report, not a reason to change the oracle after the fact.

Implementation boundary

Model-owned processors compile the fields and preserve request identity; the executor reads selected positions and selected LM-head symbols; postprocessing calibrates complete fields. Reuse the shared Qwen/vision implementation and serial admission. #129 already carries selected-position multimodal readouts for OmniJev: inspect and reuse the compatible part rather than creating a competing shared API. Intern-Decision's selected-symbol head is not OmniJev's full-vocabulary LM-feature computation.

Coordination: #133 includes an earlier implementation plan from linear3735 and a later owner assignment to psv666. Keep this reference/fixture work separate from any existing native implementation, and reconcile ownership before duplicating it. No inference results or performance claim are attached to this issue.