perf: LTX-2.3 SPEED on FA4 attention (10.4s to 7.3s)#16
Merged
Conversation
Flips the SPEED profile from TORCH_SDPA (a stand-in shipped because FA4 never built) to FLASH_ATTN -- FastVideo's actual streaming-recipe attention, the last recipe-level deviation from their published fast path. Root cause FA4 didn'\''t build: the XOR-op/flash-attention fork FastVideo'\''s June-26 pyproject pinned (f4c68c7f) is stale against cutlass-dsl 4.5+ API moves (cute.core.ThrMma, cute.make_fragment renamed/relocated). FastVideo hit the same break and fixed it 2026-07-06 (hao-ai-lab/FastVideo#1564): dropped the fork, pinned flash-attn-4 from Dao-AILab/flash-attention upstream at a CuTe-DSL-4.6-compatible commit (82d6441, = upstream PR ai-dynamo#2648). This reproduces that exact pin + FastVideo'\''s own matching wrapper fix (the pinned _flash_attn_fwd now returns a 4-tuple; their wrapper takes [:2]). Changes: - Dockerfile.dreamverse: uninstall the baked XOR-op flash-attn-cute, install flash_attn/cute from Dao-AILab upstream @82d6441 (ARG FA4_CUTE_REF). - patches/flash-attn-cute-fa4-tuple-fix.patch: [:2] on both _flash_attn_fwd callers in fastvideo/attention/utils/flash_attn_cute.py. - config.py profile_attention_backend: SPEED -> FLASH_ATTN. - bake_bench.py: derive attention backend from the profile (was hardcoded TORCH_SDPA -> would bake wrong kernels); latency gate skips the i2v cold pass; default gate 14s -> 10s for the FA4 baseline. Validated (di-slc-47 B200, fresh-boot from the image alone, dual-mode Mega-Cache blob): boot-warm 460s, warm gen 7.3-7.4s (vs SDPA 10.4s), preset ltx2_distilled, FLASH_ATTN selected, latency gate pass. Quality OK'\''d by Johan (5 seed-locked prompts vs SDPA): on par or better. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Johan-de-R
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Flips the LTX-2.3 SPEED profile from TORCH_SDPA to FLASH_ATTN (FA4) — the last recipe-level deviation from FastVideo's published fast path. 10.4s -> 7.3s/clip (~30%), quality on par or better.
Why FA4 didn't build before
FastVideo's June-26 pyproject pinned
flash-attn-cutefrom theXOR-op/flash-attentionfork (f4c68c7f, Feb 2026), which is stale againstcutlass-dsl4.5+ API moves (cute.core.ThrMma,cute.make_fragmentrenamed/relocated) — it crashes on the 4.6.0.dev0 our stack pulls. FastVideo hit the identical break and fixed it 2026-07-06 (hao-ai-lab/FastVideo#1564): they dropped the fork and now pinflash-attn-4straight from the real upstreamDao-AILab/flash-attentionat a CuTe-DSL-4.6-compatible commit (82d6441= upstream PR ai-dynamo#2648). This PR reproduces that exact pin, plus FastVideo's own matching wrapper fix (the pinned_flash_attn_fwdreturns a 4-tuple now; their wrapper takes[:2]).Changes
flash-attn-cute, installflash_attn/cutefromDao-AILab/flash-attention@82d6441(ARG FA4_CUTE_REF, pinned — do not bump without re-validating).[:2]on both_flash_attn_fwdcallers infastvideo/attention/utils/flash_attn_cute.py(git-am style, applied like the existing patches).profile_attention_backend: SPEED -> FLASH_ATTN.Validation (di-slc-47 B200, fresh boot from the image alone, dual-mode Mega-Cache blob)
Ship image built:
2.1.5-ltx23-fa4-d333ce5e(shape hash matches prod). Cutover (registry push + model repoint) is a separate step, to be done with explicit go-ahead; rollback stays the current2.1.5-ltx23-speed-v2-d333ce5e.🤖 Generated with Claude Code