[RFC] Drama3 NPU task readiness: execution contracts, validation and knowledge supplementation #27

Status: Proposed
Goal-acceptance account: @Dong1017
Accountable parent agent: Cursor agent (Claude Opus 5.5) acting under @Dong1017 — original session 2026-10-18; continuation session 2026-10-19 (same agent/model, separate session)
Initial delivery: bounded knowledge/instrument supplementation; no hardware campaign or standalone-engine implementation is authorized by this proposal.

Delivery follows the accepted collaboration contract. The parent owns the Issue outcome and delegates bounded implementation and independent revision-scoped review. Record acting agents and applicable implementation, publication, integration and closure authority. Review readiness, authorized squash integration and goal acceptance remain distinct.

1. Goal

Prepare the smallest useful supplement that lets an agent undertake a Drama3 NPU qualification and optimization task using existing InferRSI (formerly OmniRSI) knowledge and experiment interfaces.

Use the GPU implementation history to identify concrete questions, correctness obligations and measurement requirements. Do not assume its failures, remedies, hardware primitives or performance gains transfer to NPU.

Three outcomes must remain separate:

1. Supplement accepted: scoped knowledge, contracts and any justified adapters are delivered and reviewed.
2. Target qualified: a named model/NPU/software configuration passes its required execution and quality checks.
3. Optimization demonstrated: a candidate improves the agreed service objective against eligible baselines.

This RFC's initial scope delivers outcome 1 and an actionable handoff for outcome 2. Outcomes 2–3 require a separately authorized execution scope, resources and frozen acceptance criteria. They may be tracked as linked work; their success is not implied by closing the supplementation work.

2. Baseline and reuse

Original inspection baseline: OmniRSI (now InferRSI) d51c941dc57807c1e16ea698cd9291f414110c45. Reconcile with current main and existing work before implementation.

Reconciliation revision (recorded separately, 2026-10-19): InferRSI main bd5e356ac9267362c80f27adba61d76e047583c7, merged into PR #28; it renames the maintained package/CLI/environment protocol to inferrsi / INFERRSI_* and adds the scoped Drama3 diagnostic case reused by the supplement.

• KB #17 is accepted: reuse stable identities, typed sources, revision-addressable Collect and candidate Create/Update. Repository-owned knowledge changes use scoped PRs; runtime store operations do not edit matrices or promote candidates.
• PR #9 supplies Strata and both Omni frameworks' architecture/audio guidance; PR #12 supplies candidate NPU/performance skills. Imported knowledge is not target qualification.
• Reuse CANN attribution, Ascend model adaptation, ACLGraph, operator tuning, TorchNPU allocator, benchmark and evidence skills. Add or amend content only for a demonstrated gap.
• Existing Qwen-Image Ascend experience is another model's imported evidence, not Drama3 NPU validation.
• GPU-derived experience already imported through PRs Record mixed audio scheduling and codec workspace isolation lessons #13–Record production prefix service outcome and paused results #22 shares lineage with the source below. Relink or amend it rather than counting duplicate summaries as independent corroboration.
• Issue #15 remains a proposal. Its fixed skill layout, thresholds and orchestration are not prerequisites or automatically adopted policy.

Exactly two primary matrices remain: docs/reference-matrix.md and docs/searchspace-matrix.md. Task-local contracts/test tables are scoped artifacts, not a third global inventory.

3. GPU history motivating the supplement

Source: Drama3 single-B300 repository, pinned at 58a1882a4d941f8d21bfce6be37d3d5e7c9fde80.

The inspected implementation is a Python engine with persistent request state, CUDA Graphs and incremental audio decoding. It is not a Rust implementation. KDA is a workflow reference, not evidence of executed kernel generation.

These are source-backed cases and repository-reported results, not newly reproduced experiments. The GPU campaign's historical 1.5× target and README ratios do not set this NPU task's objective. A ratio against SGLang alone is not a ratio against the best eligible baseline.

4. Target declaration

The following can remain TBD during proposal review. Resolve them before any dependent execution or performance claim; missing information is a blocker or explicit unknown, never a successful check.

Single physical NPU is the initial intended scenario. If it cannot accommodate the required model/precision/state, report the capacity gap; do not silently switch devices, quantize or offload.

5. Deliverables

A. Scoped knowledge delta and execution map

Map existing material to Drama3's reference preparation, backbone, GDN/full attention, MoE, codebook generation, codec and PCM delivery stages.

For each relevant path record:

• existing knowledge/source revision and what was actually inspected;
• implementation entry, operator/backend, dtype/layout, shapes and persistent state;
• supported/unsupported/unverified status, fallback and missing evidence;
• whether the missing piece is knowledge, code integration, environment information or a measurement.

Preserve GPU/source/import lineage. Do not fabricate NPU equivalence or duplicate an existing authoritative case. Mapping a full pipeline does not require implementing every path in this RFC.

B. State and execution correctness contract

Define target-independent invariants and the target-dependent checks needed to establish them:

• Live requests, padding and scratch cannot overwrite one another's recurrent, KV, convolution or codec state.
• Partial batches, bucket boundaries, cancellation, EOS, maximum length and slot reuse preserve state and output ownership.
• Every asynchronous read has a valid producer dependency and buffer lifetime; graph outputs survive until their consumers finish.
• Capture/replay uses compatible shapes, addresses, workspace and fallback behavior.
• Arithmetic checks cover precision/layout conversion, accumulation/rounding and complete state updates against an appropriate reference.
• Audio completion includes queued PCM drain; model-slot release is not automatically response completion.

Begin with one representative prefill/decode path and the partial-batch noninterference case. Reuse source-supported validators/assertions where appropriate. CPU fixtures can check contracts, parsing and reference logic; they do not validate NPU execution, allocator behavior or graph safety.

C. Shared measurement and quality contract

Reuse the existing experiment interfaces and GPU benchmark client where applicable. Add only missing adapters; preserve callable inputs/outputs and explicit failure behavior.

Freeze before target comparison:

• Identical requests, model inputs and reference-code integers where supported; reference encoding and cold inline-audio workloads have separately stated costs.
• Requested concurrency versus achieved active work; arrival/closed-loop policy, connection policy, warmup/cache state and server-session boundaries.
• Throughput denominator: complete measured makespan, including fill/drain and failure accounting. Define acoustic frames from the selected codec contract; do not blindly inherit a samples-per-frame constant.
• TTFA: dispatch to first usable audio, with transport/encoding boundary stated. Report p50/p95 from enough request samples, not a percentile of a few run means.
• RTF: per-request dispatch-to-last-audio time divided by returned audio duration; state how run means/repetitions are aggregated.
• Quality: reference/operator checks plus complete audio-quality evaluation, including invalid/empty/truncated or excessive-silence outputs. WER/speaker similarity tools and tolerances need target/workload approval, not inherited GPU thresholds.
• Keep failed requests, incomplete repetitions, unknown finish reasons and output-length differences. Never replace them with fabricated successful measurements.

Prefer incremental persistence of request outcomes for long-running batches. Separate source revision, model revision, knowledge revision, run identity and validator identity. Raw audio/evidence storage and redaction follow existing conventions.

D. Minimal instrument supplementation and handoff

For every proposed adapter/helper, identify the existing callable being reused, the concrete gap, input/output, dependencies, tests and unsupported modes. Keep valid source locations and APIs; no mandatory new directory tree.

The handoff must let the next authorized agent determine:

1. What can be reused immediately?
2. Which exact target checks remain blocked or unrun?
3. Which commands/validators are executable, versus specifications only?
4. What authority, hardware and evidence are needed next?

No placeholder script is a completed instrument. If no new code is needed, demonstrate the existing interface mapping instead of adding a wrapper.

6. Optimization reasoning carried forward

Future hypotheses should state:

observed bottleneck → mechanism → applicability → expected benefit → memory/compute/transfer/planning cost → correctness and quality gates → service measurement → counter-signals/stop condition.

• Separate physical bounds, calibrated predictions, local timings and service outcomes.
• Kernel/graph gains do not establish service gains. Measure eligible coverage and admission/codec/drain costs.
• A throughput increase may worsen TTFA; the owner's objective decides whether that trade-off is acceptable.
• Rust, custom operators, mixed prefill, prefix reuse and multistream execution are options selected by evidence, not mandatory architecture.
• CUDA Graph, PTX, PDL, SM partitioning and FlashInfer details do not become NPU implementations by renaming APIs.
• An unsupported baseline is not zero throughput. A future 2× target must name eligible baselines, concurrency points, latency/quality constraints and a bounded budget.

Instrument roles remain optional and composable: an Evaluator measures real outcomes, an Inspector checks evidence, and Spark may fill a concrete research gap. A Simulator requires a qualified target model; this RFC does not require one or create four agents/services.

7. Delivery and acceptance

Acceptance of this supplementation scope requires:

• Relevant existing knowledge and GPU cases are reused with source/import lineage; unknowns remain explicit.
• Stage/state ownership and the partial-batch noninterference case are concrete enough to implement or run without guessing semantics.
• The benchmark contract distinguishes timing, workload, failure and quality boundaries; baseline eligibility is explicit.
• Any new adapter is callable and tested within its claimed scope; source-only specifications and CPU fixtures carry no NPU validation credit.
• No generic KB/collaboration framework is duplicated; changes use existing authoritative locations and the two matrices.
• Target-dependent gaps have an owner/action and blocking condition. A blocked qualification is not relabeled “ready to run.”
• Independent revision-scoped review is complete; limitations and the next authorization boundary are documented.

Closing this scope may state “knowledge/instrument supplement accepted; target qualification pending.” It must not state “Drama3 NPU supported,” “NPU validated,” “performance improved” or “task ready for unrestricted execution” without corresponding evidence.

Hardware qualification and optimization should proceed only after the owner authorizes their bounded scope. Do not run them merely to satisfy this RFC's documentation checklist.

8. Boundaries and owner decisions

Out of initial scope: hardware runs, environment installation, standalone-engine implementation, a Rust rewrite, new kernel generation, a broad source survey, bulk knowledge migration, automatic promotion, a third primary matrix or a new orchestration/job framework.

Before assignment, confirm the preparation scope and accountable agent. Before target execution, confirm the hardware/software tuple, required service features, precision/quality policy, comparator eligibility and resource budget. Before optimization acceptance, freeze the service objective and latency/quality trade-offs.

The parent should propose the smallest remaining slice, preserve failed or inconclusive evidence, and ask for scope changes when needed. Repository placement of new runtime code remains undecided. Source-matching approval and goal acceptance remain with the owner or explicitly authorized delegate.