ltx23 SPEED: 25s→10.9s — distilled-preset path fix, exact from_file recipe, dreamverse-based image, LTX23_PROFILE pinning by Johan-de-R · Pull Request #12 · deepinfra/dynamo

Johan-de-R · 2026-06-30T23:06:48Z

What

Ships the LTX-2.3 SPEED profile (~10.9s/clip warm, 1920×1088@121f, NVFP4 + TORCH_SDPA + 5+2 steps) through the dynamo serving path. Four pieces:

Distilled-preset path fix (the root cause of 25s): FastVideo selects its pipeline preset by string-matching the weights path (fastvideo/registry.py: needs ltx-2/ltx2 AND distilled). Our serving mounts weights at /data/default → detector misses → silent fallback to the base preset → extra guidance forward passes per denoising step (mod=3.0/rescale/STG) → ~2.4× slower with visually fine output. Fix: /models/ltx-2.3-distilled-diffusers → /data/default symlink in the image; the speed yaml references the alias. factory.py hard-fails the worker at boot if the distilled preset doesn't resolve, so any future mount change is an error, not a silent regression.
Exact recipe, no re-derivation: the speed profile builds the generator via VideoGenerator.from_file(ltx23/streaming_speed.yaml) — the literal validated recipe. The previous factory re-derived compile/quant/inductor settings by hand, which is where deviations kept creeping in.
Dockerfile.dreamverse: serving image builds FROM FastVideo's own validated dreamverse env (their prerelease FP4 kernel stack; our from-scratch build silently resolved older stable packages because uv's prerelease=allow only applies from the FastVideo CWD). ai-dynamo layered on top with kernel/torch versions constraint-frozen. Environment fidelity, not a measured package speedup — the measured cause was the preset.
LTX23_PROFILE pins the full recipe into the image (earlier commits): profile → attention backend + NVFP4 + compile mode, Mega-Cache guard, RUNBOOK.

Validation (di-slc-47, B200, prod weights, prod mount topology)

	before	after
warm clip (dynamo pool)	25–26s	10.6–10.9s
warm refine stage	15.4s	5.3s
preset (log proof)	`mod_video=3.0` (base)	`mod_video=1.0` (distilled)

Bisect chain proving the root cause (each single-variable): golden context reproduces 10.7s deterministically; only-weights-path-swap flips 10.7↔25.1s; full-content-via-anonymous-path still slow; sha256 audit shows weight files bit-identical; registry resolves ltx2_base for /data/default and ltx2_distilled for the alias.

Remaining before merge

Mega-Cache t2v blob baked (452MB) + i2v extension pass (in progress)
Final ship image assembly + fresh-boot validation (kill + restart from image alone)
Quality sign-off on fixed-path clips (Johan)

Rollback: current prod quality image 2.1.5-ltx23-mc-i2v3 (untouched by this PR).

🤖 Generated with Claude Code

…o the image Shipping the validated ~10s SPEED recipe to prod needs the recipe to travel with the image, not a fragile redis deploy-config flag. Two defaults that the SPEED path needs were not coupled to the profile: - attention backend: hardcoded FLASH_ATTN in worker.py + warmup.py. We validated SPEED end-to-end on TORCH_SDPA (~10s, quality OK'd); FastVideo's streaming_demo SPEED path uses FLASH_ATTN (~4.5s) but we have NOT validated SPEED+FLASH_ATTN, so ship what we tested. - --enable-optimizations: store_true / default off, so NVFP4 only applied if a launch arg was passed (redis). Couple both to LTX23_PROFILE via config.py helpers (profile_attention_backend, profile_default_optimizations), used by BOTH the warmup bake and the serving worker so the compile cache always matches. Now a SPEED ship image only needs `ENV LTX23_PROFILE=speed` baked in -> NVFP4 + TORCH_SDPA + speed kwargs, no deploy-config flags. QUALITY is unchanged (FLASH_ATTN + bf16, what prod runs). Both still overridable via --attention-backend / --[no-]enable-optimizations. factory.load_model still hardware-gates NVFP4 to Blackwell, so non-Blackwell hosts safely fall back to bf16 regardless of profile. RUNBOOK: document the SPEED bake (LTX23_PROFILE=speed pins recipe+attention). ltx23/test_config.py: 5 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

A ship image needs the per-shape *.megacache.bin blobs, not just the inductor /cache -- without them a fresh pod cold-compiles the torch.compile front-end (~33 min/shape SPEED) on every boot instead of the ~12 min Mega-Cache boot-warm. The trigger is LTX_MEGACACHE_DIR being set during warmup, and it's easy to forget (I did). Two safeguards so the next person can't: - warmup.py: print a loud WARNING at driver start if LTX_MEGACACHE_DIR is unset. - RUNBOOK: the SPEED bake now sets LTX_MEGACACHE_DIR=/cache/megacache at warmup (under /cache so the existing cache.tar captures the blobs -- no separate megacache.tar) and bakes `ENV LTX_MEGACACHE_DIR` into the image so the serving side doesn't need a redis extra_env flag either. Cross-refs ltx23/CACHING.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…amverse-based image Root cause of the 10s-vs-25s serving gap: FastVideo picks the pipeline preset by string-matching the weights path (registry.py). /data/default misses the detector -> BASE preset -> extra guidance forwards per denoise step (~2.4x). Fix: detector-matching alias path + yaml. Also: speed profile now builds via VideoGenerator.from_file(streaming_speed.yaml) (the exact validated recipe, no re-derivation), and Dockerfile.dreamverse builds the serving image FROM the dreamverse env (prerelease FP4 kernel stack) with ai-dynamo layered on top. Validated: dynamo pool + prod weights = 10.9s/clip warm (was 25s). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Johan-de-R · 2026-07-03T00:34:45Z

Root cause of the 10s-vs-25s serving gap found and fixed (validated end-to-end tonight):

FastVideo selects its pipeline preset by string-matching the weights path (fastvideo/registry.py: needs ltx-2/ltx2 AND distilled in the path). Our serving mounts weights at /data/default → detector misses → silent fallback to the base preset → extra guidance forward-passes per denoising step (mod=3.0, rescale, STG) → ~2.4x slower with visually fine output.

Fix in this branch: /models/ltx-2.3-distilled-diffusers symlink alias (Dockerfile.dreamverse) + the speed profile now loads the exact validated recipe via VideoGenerator.from_file(ltx23/streaming_speed.yaml) — no re-derived config anywhere. Also: serving image now builds FROM the dreamverse env, inheriting FastVideo's validated prerelease FP4 kernel stack (prerelease=allow was silently not applied in the from-scratch build).

Validation: dynamo pool + production weights = 10.9s/clip warm (musician), 11.0s (dog); warm refine 5.29s; preset proof in logs (mod_video=1.0). Was 25-26s.

Remaining before merge: Mega-Cache bake + fresh-boot validation + quality sign-off (clips with Johan).

🤖 Generated with Claude Code

…v bake helper The preset comes from FastVideo path-string detection; a mount/rename that misses the detector must be a boot error, not a silent 2.4x regression. bake_i2v.py extends the Mega-Cache blob with i2v graphs (t2v-only bake gap). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…oot-cause attribution Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

… a warm-latency gate The bake now FAILS if a warm generation exceeds LTX23_MAX_WARM_S (default 14s), so any future silent slowdown (preset drift, package/env change) goes red at bake time instead of reaching prod. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Johan-de-R · 2026-07-03T01:40:36Z

Final validation PASSED — ship image 2.1.5-ltx23-speed-695d2ebc (built from branch tip 7fa23eb4), fresh boot, image alone (no host cache/code mounts, prod mount topology):

preset check OK: /models/ltx-2.3-distilled-diffusers -> ltx2_distilled
[megacache] LOADED 588151772 bytes (t2v+i2v blob)
BOOT_WARM_S = 451.3s      (vs ~30 min cold-compile)
GEN[musician] = 10.7s     GEN[dog] = 10.8s
LATENCY GATE (LTX23_MAX_WARM_S=14): PASSED

All technical gates on the checklist are green: t2v+i2v Mega-Cache blob baked, fresh-boot validated, boot assertion + bake latency gate exercised end-to-end in the shipped artifacts. Remaining: quality sign-off (clips with Johan) → merge → registry push + model repoint (separate cutover step, rollback = 2.1.5-ltx23-mc-i2v3).

🤖 Generated with Claude Code

Johan-de-R · 2026-07-03T03:24:52Z

Quality sign-off: Johan approved the fixed-path clips (musician + dog, distilled preset, 10.7-10.8s renders). All merge gates are now green.

🤖 Generated with Claude Code

github-actions Bot added the documentation Improvements or additions to documentation label Jun 30, 2026

Johan-de-R and others added 2 commits June 30, 2026 23:24

Johan-de-R and others added 2 commits July 3, 2026 00:55

ltx23: RUNBOOK speed-bake flow + dreamverse base provenance; honest r…

8bc8c0b

…oot-cause attribution Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Johan-de-R changed the title ~~ltx23: pin the SPEED recipe (NVFP4 + TORCH_SDPA) into the image via LTX23_PROFILE~~ ltx23 SPEED: 25s→10.9s — distilled-preset path fix, exact from_file recipe, dreamverse-based image, LTX23_PROFILE pinning Jul 3, 2026

Johan-de-R merged commit a198e87 into deep-main-v1.1.1-videogen Jul 3, 2026
10 of 15 checks passed

This was referenced Jul 3, 2026

refactor: rename LTX_MEGACACHE_* env to generic DI_MEGACACHE_* (backward-compat) #13

Open

perf: skip x264 scene-cut analysis on LTX-2.3 PyAV save path (env-configurable) #15

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ltx23 SPEED: 25s→10.9s — distilled-preset path fix, exact from_file recipe, dreamverse-based image, LTX23_PROFILE pinning#12

ltx23 SPEED: 25s→10.9s — distilled-preset path fix, exact from_file recipe, dreamverse-based image, LTX23_PROFILE pinning#12
Johan-de-R merged 6 commits into
deep-main-v1.1.1-videogenfrom
johan/ltx23-speed-bake

Johan-de-R commented Jun 30, 2026 •

edited

Loading

Uh oh!

Johan-de-R commented Jul 3, 2026

Uh oh!

Johan-de-R commented Jul 3, 2026

Uh oh!

Johan-de-R commented Jul 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Johan-de-R commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Validation (di-slc-47, B200, prod weights, prod mount topology)

Remaining before merge

Uh oh!

Johan-de-R commented Jul 3, 2026

Uh oh!

Johan-de-R commented Jul 3, 2026

Uh oh!

Johan-de-R commented Jul 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Johan-de-R commented Jun 30, 2026 •

edited

Loading