Compile the DSL Moonshine encoder for the Torq NPU (batchless + prune wiring) - #7
Merged
Merged
Conversation
… wiring) Wire the encoder export so the full 6-layer DSL Moonshine encoder compiles to a Torq NPU vmfb (validated on the mpact simulator): - ENC_BATCHLESS=1 feeds the module a rank-2 [frames,dim] input so the residual stream / LayerNorm / o_proj are 2D like the hand-written enc6 (avoids the batch dim that the Torq layout solver chokes on). - Apply TorqPruneOutputsPass last (drops the dangling per-layer [1,H,S,D] Q/K/V outputs that trip MatMulPattern:57) and pass modelDim to TorqPlugin.install. - build.gradle.kts / settings.gradle.kts: wire the synaptics-torq vendor plugin (host export-time only). Requires the matching synaptics-torq passes (TorqRopeSeqMajorPass + TorqAttentionTilingPass K^T coordination) on the encoder-npu-rope-seq branch there. Note: correct execution on the SL2610 *hardware* is still blocked by a runtime multi-dispatch limit (see docs/torq.md / the version-safety harness); this compiles and simulates correctly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MoonshineDecoder already defaults its encoder to the vendor prebuilt encoder.vmfb on --device=torq. Add TorqRunModule.enableNpuClock() (devmem 0xf7e104b0 32 0x216, the vendor demo's utils/npu.py step) and call it from MoonshineDecoder.init when the encoder targets the NPU. Defensive, not a correctness fix: the vendor encoder was verified to run bit-identically at the boot clock (0x298) and 0x216, so the clock is not why our self-compiled encoder zeroed (that is the dispatch-fusion gap — vendor = 1 fused dispatch, our compiler emits 40). This guards a clock-off boot state and matches the vendor's startup sequence. Verified e2e on the board (voicecc asr): test.wav -> "One, two, three.", c1 -> "Turn the light on", c4 -> "Set an alarm for 7 a.m.". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a ready-to-send Torq support package under docs/synaptics-support/: - encoder_1layer.mlir — one Moonshine-tiny encoder layer authored in the SKaiNET DSL and emitted as StableHLO, self-contained (weights as inputs). Verified: iree-compile (Torq v2.0.0) fragments it into 41 dispatches, which return all-zeros on the SL2610 NPU, whereas the vendor's shipped encoder.vmfb for the same model is a single fused dispatch and runs. - README.md — the support message + where to file (GitHub issue on synaptics-torq/torq-compiler, or the Astra Support Portal), the exact compile/run commands, the dispatch-count evidence table, and 5 questions (fusion flags, runtime dispatch-count limit, StableHLO-vs-ONNX import, the css_linalg.generic un-tiled codegen crash, and access to the compiler build that fused their moonshine encoder). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…thon Add scripts/compile-gemma.sh: one command, no Python, drives the kgemma FunctionGemmaExport (DSL gemmaNetwork() with the argMax tail) -> gemma-gen.mlir + bf16 gemma.safetensors -> iree-convert-parameters (gemma-gen.irpa) -> iree-compile llvm-cpu (gemma-gen.vmfb). TARGET host|aarch64; IREE_IMAGE selects the stock (validation) or Torq-fork (board) compiler. Delete the superseded rewrite scripts — the argmax tail is now the DSL argMax op and weights are emitted bf16 by the exporter: - add_argmax_perpos.py / add_argmax_tail.py (folded into ops.argMax) - make_f16.py (folded into the bf16 export) Pipeline.kt already loads gemma-gen.vmfb + gemma-gen.irpa, so no runtime change is needed; the artifacts are just regenerated by the clean path. Verified end to end on host (vmfb first token = 262146 <tool_0>). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- TARGET=board compiles gemma-gen.mlir with the local g165 Torq-fork iree-compile (aarch64+NEON, board-compatible bytecode). REQUIRED for the SL2610: stock IREE 3.x emits a "Ch" bytecode feature the board's iree-run-module rejects (verified). - docker runs now use --user so the emitted .irpa/.vmfb are not root-owned. Verified end to end on the board (192.168.3.26): the self-compiled bf16 gemma-gen vmfb+irpa generates "turn the light on" -> [262146, 236769, 3255, 718, 498, 1373, 262152, 106] = <tool_0>(state="on")<end> token-for-token identical to the llama.cpp reference (and to the host run). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat(gemma): self-compile FunctionGemma — compile-gemma.sh, retire Python
michalharakal
added a commit
that referenced
this pull request
Jul 23, 2026
…le multi-target Elevate three first-class goals into docs/PLAN.md (done-when #5-7) + two phases: - #5 Moonshine ASR on the Torq NPU (was buried in P3; today blocked — encoder zeros on NPU, vendor vmfb is the stopgap). - #6 Streaming ASR via Moonshine v2 (position-free sliding-window causal encoder + adapter; huggingface.co/papers/2602.12241). v1's bidirectional encoder blocks streaming. - #7 Moonshine + Gemma reusable standalone across IREE targets (host CPU proven; GPU roadmap). Strategy = v2-first leapfrog: v2's bounded O(Tw) local attention both streams AND tiles onto the NPU where v1's O(T^2) full attention can't — so P6 unifies #5 and #6 and subsumes the v1-on-NPU items (3N-c/3D-e/3DEC-d/3SC-d), which are deprioritized (vendor encoder.vmfb stays the NPU stopgap). P7 = reuse hardening: decouple the bf16-Torq default, prove the same DSL on host llvm-cpu. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wire the encoder export so the full 6-layer DSL Moonshine encoder compiles to a Torq NPU vmfb (validated on the mpact simulator).
Changes
ENC_BATCHLESS=1feeds the module a rank-2[frames,dim]input so the residual stream / LayerNorm / o_proj are 2D like the hand-written enc6 (avoids the batch dim the Torq layout solver chokes on).TorqPruneOutputsPasslast (drops the dangling per-layer[1,H,S,D]Q/K/V outputs that tripMatMulPattern:57) and passmodelDimtoTorqPlugin.install.build.gradle.kts/settings.gradle.kts: wire thesynaptics-torqvendor plugin (host export-time only).Pairs with
SKaiNET-embedded-vendors#1 (the seq-major RoPE + prune Torq passes): SKaiNET-developers/SKaiNET-embedded-vendors#1 — both must merge together, they're interdependent.
Note: correct execution on the SL2610 hardware is still blocked by a multi-dispatch runtime limit (see the Torq version-safety harness PR #6). This compiles and simulates correctly.
🤖 Generated with Claude Code