Torq passes: seq-major RoPE + output prune for the DSL Moonshine encoder - #1
Open
michalharakal wants to merge 1 commit into
Open
Torq passes: seq-major RoPE + output prune for the DSL Moonshine encoder#1michalharakal wants to merge 1 commit into
michalharakal wants to merge 1 commit into
Conversation
Two new Torq-target passes + a tiling tweak that let the DSL Moonshine encoder compile for the SL2610 NPU (root-caused this session, validated on the mpact sim): - TorqRopeSeqMajorPass: sinks the attention head-transpose BELOW RoPE so K reaches the QK^T as reshape/projection-sourced seq-major [Sk,H,D]. A head-major transpose-sourced K^T mis-seeds the Torq global-layout solver and trips MatMulPattern.cpp:57; enc6 (no RoPE) avoids it, so we make RoPE seq-major. - TorqAttentionTilingPass: when the K producer is the seq-major restore permute, build the K^T with a single [1,2,0] (skip the double transpose). - TorqPruneOutputsPass: drop the dangling per-layer [1,H,S,D] Q/K/V graph outputs (the DSL trace leaves them; the Torq layout solver trips :57 on returned attention intermediates). - TorqPlugin: register TorqRopeSeqMajorPass before tiling; add modelDim; drop the --torq-disable-slices recommendation (it breaks the encoder's CSS stack). Pairs with the encoder-npu-rope-seq branch in SKaiNET-embedded (ENC_BATCHLESS + prune wiring). Hardware execution is still blocked by a multi-dispatch runtime limit, tracked in the SL2610 version-safety harness. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two new Torq-target passes + a tiling tweak that let the DSL Moonshine encoder compile for the SL2610 NPU (root-caused this session, validated on the mpact simulator).
Changes
TorqRopeSeqMajorPass(new): sinks the attention head-transpose BELOW RoPE so K reaches the QK^T as reshape/projection-sourced seq-major[Sk,H,D]. A head-major transpose-sourced K^T mis-seeds the Torq global-layout solver and tripsMatMulPattern.cpp:57; enc6 (no RoPE) avoids it, so we make RoPE seq-major.TorqAttentionTilingPass: when the K producer is the seq-major restore permute, build the K^T with a single[1,2,0](skip the double transpose).TorqPruneOutputsPass(new): drop the dangling per-layer[1,H,S,D]Q/K/V graph outputs (the DSL trace leaves them; the layout solver trips:57on returned attention intermediates).TorqPlugin: registerTorqRopeSeqMajorPassbefore tiling; addmodelDim; drop the--torq-disable-slicesrecommendation (it breaks the encoder's CSS stack).Pairs with
SKaiNET-embedded#7(encoder-npu-rope-seq:ENC_BATCHLESS+ prune wiring). Both must merge together — they're interdependent.Note: correct execution on the SL2610 hardware is still blocked by a multi-dispatch runtime limit (tracked in the SL2610 version-safety harness). This compiles and simulates correctly.
🤖 Generated with Claude Code