ltx23 SPEED: 25s→10.9s — distilled-preset path fix, exact from_file recipe, dreamverse-based image, LTX23_PROFILE pinning

examples/diffusers/Dockerfile.dreamverse

@@ -0,0 +1,63 @@
+# SPDX-License-Identifier: Apache-2.0
+#
+# LTX-2.3 SPEED serving image, built FROM FastVideo's validated dreamverse env
+# (fastvideo @ 4f3ad3f6 with its intended prerelease kernel stack) + the Dynamo
+# serving layer (ai-dynamo, deepinfra patches, worker app). Rebuild the base:
+#   git clone https://github.com/hao-ai-lab/FastVideo && cd FastVideo
+#   git checkout 4f3ad3f6df9327257f08f4c45d24154b52c06616
+#   docker build -f apps/dreamverse/docker/Dockerfile -t dreamverse:johan-ltx .
+FROM dreamverse:johan-ltx
+
+# dreamverse venv is /opt/venv (python 3.12); uv is on PATH (/root/.local/bin).
+ENV VIRTUAL_ENV=/opt/venv
+ENV PATH="/opt/venv/bin:${PATH}"
+
+# Freeze the proven prerelease FP4-kernel stack + torch. These versions are
+# ALREADY installed in dreamverse; the constraint just prevents the ai-dynamo
+# resolve below from moving them. --prerelease=allow (below) is required because
+# nvidia-cutlass-dsl==4.6.0.dev0 is a prerelease.
+RUN printf 'torch==2.11.0\nnvidia-cutlass-dsl==4.6.0.dev0\nquack-kernels==0.5.3\nflashinfer-python==0.6.13\n' > /opt/dynamo-constraint.txt
+ENV UV_CONSTRAINT=/opt/dynamo-constraint.txt
+
+# Dynamo runtime needs libucx0 (already present in dreamverse's CUDA base in
+# most cases, but install defensively).
+RUN apt-get update \
+ && apt-get install -yq libucx0 \
+ && apt-get clean
+
+# Install Dynamo with /v1/videos support. --prerelease=allow so the constraint's
+# cutlass-dsl dev pin is honored; the constraint keeps torch/kernels frozen.
+RUN . /opt/venv/bin/activate \
+ && uv pip install --prerelease=allow ai-dynamo==1.1.1
+
+# deepinfra patches, applied to the INSTALLED (non-editable) fastvideo package.
+# The patches are git-format (a/fastvideo/... b/fastvideo/...); -p1 from
+# site-packages strips a/ -> fastvideo/... . dreamverse's fastvideo is @4f3ad3f6,
+# the exact commit these patches were authored against, so they apply clean.
+COPY patches/ltx23_gpu_worker_megacache.patch /tmp/megacache.patch
+COPY patches/x264-threads-cap.patch /tmp/x264-threads-cap.patch
+# NOTE: verify by grepping the patched source, NOT by importing fastvideo --
+# importing pulls in triton, which fails to init its driver in a GPU-less build
+# ("0 active drivers").
+RUN SP=/opt/venv/lib/python3.12/site-packages \
+ && patch -p1 -d "$SP" --forward --fuzz=0 < /tmp/megacache.patch \
+ && patch -p1 -d "$SP" --forward --fuzz=0 < /tmp/x264-threads-cap.patch \
+ && grep -q 'torch.compiler.load_cache_artifacts' "$SP/fastvideo/worker/gpu_worker.py" \
+ && grep -q 'FASTVIDEO_X264_THREADS' "$SP/fastvideo/entrypoints/video_generator.py" \
+ && echo "patches applied + verified"
+
+ENV FASTVIDEO_VIDEO_CODEC=libx264
+ENV FASTVIDEO_X264_PRESET=ultrafast
+
+# Worker app tree + entrypoint (page-cache warming; unsets LD_LIBRARY_PATH,
+# which is a no-op for refine speed -- Experiment X showed torch loads its
+# bundled cuBLAS regardless of LD, refine time is set by the FP4-kernel packages).
+WORKDIR /opt/app
+COPY . /opt/app/
+ENTRYPOINT ["/opt/app/entrypoint.sh"]
+
+# LOAD-BEARING: FastVideo picks the DISTILLED preset by string-matching the
+# weights path (needs "ltx-2"+"distilled"; miss = silent base preset, ~2.4x
+# slower). Prod mounts weights at /data/default, so alias it. factory.py
+# hard-fails at boot if the preset does not resolve.
+RUN mkdir -p /models && ln -s /data/default /models/ltx-2.3-distilled-diffusers

examples/diffusers/ltx23/RUNBOOK.md

@@ -161,6 +161,80 @@ docker run --rm localhost:30500/fastvideo-runtime:<version>-ltx2-$HASH \
 
 Should report sizes matching the host's `/cache`.
 
+### Step 4b: profile selection (quality vs speed) — pin it into the IMAGE
+
+LTX-2.3 has two recipes, selected by the `LTX23_PROFILE` env var, which
+`ltx23/factory.py` reads at load time (`quality`=bf16/8-step default,
+`speed`=NVFP4/max-autotune/5-step — see `PROFILES.md`). The two recipes
+produce **different torch.compile cache keys**, so the warmup bake and the
+serving worker MUST agree on the profile, or the serving path misses the
+baked cache and cold-compiles 10-15 min per shape.
+
+**Ship rule: bake `LTX23_PROFILE` into the image ENV; never set it in the
+model/deploy config.** Making the image tag the single source of truth means
+the recipe travels with the image — a `-speed-` image always serves speed, a
+`-quality-` image always serves quality. A `LTX23_PROFILE` set in the redis
+model-config env would *override* the image ENV and can silently drift away
+from the baked cache (wrong recipe + cache miss, no error). Leave it out of
+the deploy config so the image wins.
+
+To bake the **SPEED** (fast / ~10s) image (flow validated 2026-07-03: dynamo pool
++ prod weights = 10.6–10.9s/clip warm):
+
+0. **Base image = `Dockerfile.dreamverse`** (not the old `warmupbase`): FastVideo's
+   validated dreamverse env + the dynamo layer. It creates the load-bearing
+   `/models/ltx-2.3-distilled-diffusers -> /data/default` symlink: FastVideo picks
+   the DISTILLED preset by string-matching the weights path; a miss silently
+   serves the ~2.4x-slower base preset. `factory.py` hard-fails boot if the
+   preset does not resolve. The recipe is `ltx23/streaming_speed.yaml`, loaded
+   verbatim (`VideoGenerator.from_file`) -- never re-derive it in code.
+   ```bash
+   docker build -f Dockerfile.dreamverse -t localhost:30500/fastvideo-runtime:<version>-ltx23-dreamverse-base .
+   ```
+
+1. Step 3 warmup — two passes on ONE persistent `/cache`, both with
+   `-e LTX23_PROFILE=speed -e LTX_MEGACACHE_DIR=/cache/megacache
+   -e VIDEO_POOL_GEN_TIMEOUT_S=5400` and weights at `/data/default`:
+   ```bash
+   # pass 1: t2v compile + blob (~25 min cold)
+   ... localhost:30500/fastvideo-runtime:<version>-ltx23-dreamverse-base \
+     python3 ltx23/bake_bench.py t2v
+   # pass 2: extend the blob with i2v graphs (move the pass-1 blob aside first —
+   # a present blob suppresses saving — and set LTX_MEGACACHE_SAVE_EVERY=1)
+   ... -e LTX_MEGACACHE_SAVE_EVERY=1 ... python3 ltx23/bake_bench.py i2v
+   ```
+   The pool runs `enable_optimizations=True`, so SPEED applies NVFP4 (QUALITY
+   stays bf16 regardless).
+
+   **`LTX_MEGACACHE_DIR` is NOT optional for a ship image.** Without it the bake
+   produces only the inductor `/cache`, not the per-shape `*.megacache.bin`
+   blobs — and a fresh pod then cold-compiles the torch.compile front-end (~33
+   min/shape SPEED) on *every* boot instead of the ~12 min Mega-Cache boot-warm.
+   `warmup.py` prints a loud WARNING if it's unset. Putting it **under `/cache`**
+   means the existing `cache.tar` step below captures the blobs automatically —
+   no separate `megacache.tar`. Full rationale: `ltx23/CACHING.md`.
+2. Step 4 bake — add `ENV LTX23_PROFILE=speed` **and `ENV LTX_MEGACACHE_DIR=/cache/megacache`**
+   to the bake Dockerfile and use a distinct tag so quality and speed images are
+   never conflated (`IMAGE_SHAPE_HASH` only encodes the shape menu, not the recipe):
+   ```bash
+   printf 'FROM localhost:30500/fastvideo-runtime:<version>-ltx23-dreamverse-base\nADD cache.tar /\nENV IMAGE_SHAPE_HASH=%s\nENV LTX23_PROFILE=speed\nENV LTX_MEGACACHE_DIR=/cache/megacache\n' "$HASH" > Dockerfile
+   docker build -t localhost:30500/fastvideo-runtime:<version>-ltx23-speed-$HASH .
+   ```
+   Pinning `LTX_MEGACACHE_DIR` in the image ENV (rather than the model's redis
+   `extra_env`) keeps the serving side from depending on a deploy-config flag —
+   the image alone knows where its Mega-Cache lives.
+
+The QUALITY image is the default (no `LTX23_PROFILE` needed at warmup; omit the
+`ENV LTX23_PROFILE` line at bake, or set it to `quality` explicitly). It still
+needs `LTX_MEGACACHE_DIR` at both bake and serve for the same boot-warm reason.
+
+The attention backend follows the profile automatically (`config.py`
+`profile_attention_backend`: `quality`→`FLASH_ATTN`, `speed`→`TORCH_SDPA`) in
+both the warmup bake and the serving worker, so you do **not** set
+`FASTVIDEO_ATTENTION_BACKEND` separately — the one `LTX23_PROFILE` env pins it.
+Note SPEED ships `TORCH_SDPA` (the config we validated, ~10s), not FastVideo's
+`FLASH_ATTN` speed path; override with `worker.py --attention-backend` to revisit.
+
 ### Step 5: validate with `benchmark.py`
 
 ```bash

examples/diffusers/ltx23/bake_bench.py

@@ -0,0 +1,74 @@
+"""Bake + bench driver for the LTX-2.3 SPEED image (see RUNBOOK.md).
+
+  python3 ltx23/bake_bench.py t2v   # pass 1: t2v compile + Mega-Cache blob + warm bench
+  python3 ltx23/bake_bench.py i2v   # pass 2: extend the blob with i2v graphs
+                                    # (move the pass-1 blob aside; LTX_MEGACACHE_SAVE_EVERY=1)
+
+Exits non-zero if a warm generation exceeds MAX_WARM_S: a silently-slowed bake
+(wrong preset, package drift, upstream change) must fail here, not reach prod.
+"""
+import asyncio, logging, os, sys, time
+
+logging.basicConfig(level=logging.INFO, format="%(message)s")
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+from lib.pool import SubprocessPool
+
+OUT = os.environ.get("BENCH_OUT", "/tmp/bench-out")
+os.makedirs(OUT, exist_ok=True)
+W, H, NF, FPS, STEPS, GS = 1920, 1088, 121, 24, 5, 1.0
+SHAPE = "%dx%d@%df" % (W, H, NF)
+MAX_WARM_S = float(os.environ.get("LTX23_MAX_WARM_S", "14"))
+I2V_IMAGE = os.environ.get("LTX23_I2V_IMAGE", "/work/joy_nordic_woman_mid.png")
+T2V_PROMPTS = [
+    ("musician", "A street musician in her thirties sings and strums an acoustic guitar on a sunny city sidewalk, natural human face, photorealistic, sharp focus."),
+    ("dog", "A close-up tracking shot of a golden retriever sprinting through a sunlit alpine meadow at golden hour, photorealistic"),
+]
+
+
+def req(tag: str, prompt: str, **extra) -> dict:
+    d = {
+        "request_id": "bake_" + tag, "prompt": prompt, "width": W, "height": H,
+        "num_frames": NF, "fps": FPS, "num_inference_steps": STEPS,
+        "guidance_scale": GS, "seed": 42, "negative_prompt": None,
+        "output_path": os.path.join(OUT, tag + ".mp4"),
+    }
+    d.update(extra)
+    return d
+
+
+async def main(mode: str) -> int:
+    pool = SubprocessPool(
+        model_path=os.environ.get("LTX23_MODEL_PATH", "/models/ltx-2.3-distilled-diffusers"),
+        num_gpus=1, enable_optimizations=True, attention_backend="TORCH_SDPA",
+        model_factory_dotted="ltx23.factory:load_model", model_label="ltx23-bake",
+    )
+    warm_times: list[tuple[str, float]] = []
+    try:
+        t = time.perf_counter()
+        r = await pool.route(SHAPE, req("warm", "a calm test warmup clip"))
+        print("BOOT_WARM_S=%.1f status=%s" % (time.perf_counter() - t, r.get("status")), flush=True)
+        if mode == "t2v":
+            gens = [(tag, req(tag, prompt)) for tag, prompt in T2V_PROMPTS]
+        else:
+            gens = [("i2v", req(
+                "i2v", "The scene comes alive with gentle natural motion and a slow cinematic push-in.",
+                ltx2_images=[(I2V_IMAGE, 0, 1.0)], ltx2_image_crf=0.0))]
+        for tag, r_ in gens:
+            t = time.perf_counter()
+            r = await pool.route(SHAPE, r_)
+            dt = time.perf_counter() - t
+            warm_times.append((tag, dt))
+            print("GEN[%s]=%.1fs status=%s" % (tag, dt, r.get("status")), flush=True)
+        print("BAKE_BENCH_DONE mode=%s" % mode, flush=True)
+    finally:
+        await pool.shutdown()
+    slow = [(tag, dt) for tag, dt in warm_times[1:] or warm_times if dt > MAX_WARM_S]
+    if slow:
+        print("LATENCY GATE FAILED (> %.0fs warm): %s -- do NOT ship this bake; "
+              "check the preset boot log and recent env/package changes." % (MAX_WARM_S, slow), flush=True)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(asyncio.run(main(sys.argv[1] if len(sys.argv) > 1 else "t2v")))

examples/diffusers/ltx23/config.py

@@ -25,9 +25,16 @@
 
 Both profiles share: model FastVideo/LTX-2.3-Distilled-Diffusers, 1920x1088
 landscape, 121 frames @ 24fps, guidance 1.0, negative "", refine upsampler =
-<model>/spatial_upscaler, FLASH_ATTN, vae_tiling False, the Blackwell Inductor
+<model>/spatial_upscaler, vae_tiling False, the Blackwell Inductor
 knobs (factory.py: shape_padding=False etc.), LD_LIBRARY_PATH unset, cu128 env.
 
+Attention backend is per-profile (see profile_attention_backend): QUALITY uses
+FLASH_ATTN (FastVideo's SM100/SM103 fast kernels, what prod runs); SPEED uses
+TORCH_SDPA -- the config we validated end-to-end on B200 (~10s, quality OK'd by
+Johan). FastVideo's own streaming_demo speed path uses FLASH_ATTN (faster,
+~4.5s), but SPEED+FLASH_ATTN is NOT something we have validated, so we ship the
+attention we tested. Revisit with worker.py --attention-backend.
+
 Changing any of these requires a coordinated re-bake (the compile cache is keyed
 on the kwargs + shape). Do NOT hand-edit values away from the FastVideo source.
 """
@@ -118,3 +125,29 @@ def profile_kwargs(profile: str) -> dict[str, Any]:
 def profile_uses_nvfp4(profile: str) -> bool:
     """SPEED uses NVFP4; QUALITY is bf16."""
     return (profile or "quality").strip().lower() == "speed"
+
+
+def profile_attention_backend(profile: str) -> str:
+    """Attention backend per profile.
+
+    QUALITY -> FLASH_ATTN (FastVideo's SM100/SM103 fast kernels; what prod runs).
+    SPEED   -> TORCH_SDPA: the config validated end-to-end on B200 (~10s,
+    quality OK'd). FastVideo's streaming_demo speed path actually uses FLASH_ATTN
+    (faster, ~4.5s), but we have NOT validated SPEED+FLASH_ATTN ourselves, so we
+    ship the attention we tested. Bake (warmup) and serve (worker) both derive
+    the default from here so the compile cache always matches; override with
+    worker.py --attention-backend to revisit FLASH_ATTN.
+    """
+    return "TORCH_SDPA" if (profile or "quality").strip().lower() == "speed" else "FLASH_ATTN"
+
+
+def profile_default_optimizations(profile: str) -> bool:
+    """Whether NVFP4 + torch.compile optimizations default ON for this profile.
+
+    SPEED needs them ON (NVFP4 is the whole point); QUALITY is bf16, OFF. Coupling
+    this to the profile means a SPEED ship image (LTX23_PROFILE=speed baked in)
+    serves NVFP4 with no extra launch arg -- the recipe lives in the image, not a
+    redis deploy-config flag. factory.load_model still hardware-gates NVFP4 to
+    Blackwell, so a non-Blackwell host safely falls back to bf16 regardless.
+    """
+    return profile_uses_nvfp4(profile)

examples/diffusers/ltx23/factory.py

@@ -22,6 +22,31 @@
 logger = logging.getLogger(__name__)
 
 
+def _assert_distilled_preset(yaml_path: str) -> None:
+    """Fail boot if FastVideo would not resolve the DISTILLED preset.
+
+    FastVideo picks the preset by string-matching the weights path (needs
+    "ltx-2"+"distilled"); a miss silently serves the BASE preset ~2.4x slower.
+    """
+    import yaml as _yaml
+
+    with open(yaml_path) as fh:
+        model_path = _yaml.safe_load(fh)["generator"]["model_path"]
+
+    from fastvideo.registry import _get_config_info
+
+    info = _get_config_info(model_path, raise_on_missing=False)
+    preset = getattr(info, "default_preset", None) if info else None
+    if preset != "ltx2_distilled":
+        raise RuntimeError(
+            f"LTX23_PROFILE=speed but FastVideo resolved preset {preset!r} for "
+            f"model_path={model_path!r}; the path must contain 'ltx-2' and "
+            "'distilled' (base preset is ~2.4x slower). Fix the weights alias "
+            "(Dockerfile symlink /models/ltx-2.3-distilled-diffusers)."
+        )
+    logger.info("LTX-2.3 preset check OK: %s -> ltx2_distilled", model_path)
+
+
 def load_model(
     model_path: str,
     num_gpus: int,
@@ -52,6 +77,22 @@ def load_model(
     # See ltx23/config.py / ltx23/PROFILES.md. The denoise step count (8 vs 5)
     # lives in the shapes file / per-request num_inference_steps, not here.
     profile = os.environ.get("LTX23_PROFILE", "quality").strip().lower()
+
+    # SPEED loads the validated recipe verbatim via from_file -- do NOT re-derive
+    # compile/quant/inductor settings in code (hand-set knobs are where the
+    # recipe drifted before). Attention backend comes from the env (TORCH_SDPA).
+    if profile == "speed":
+        yaml_path = os.path.join(
+            os.path.dirname(os.path.abspath(__file__)), "streaming_speed.yaml"
+        )
+        _assert_distilled_preset(yaml_path)
+        logger.info(
+            "LTX-2.3 profile=speed: VideoGenerator.from_file(%s) "
+            "[exact 10s recipe, no re-derivation]",
+            yaml_path,
+        )
+        return VideoGenerator.from_file(yaml_path)
+
     optimization_kwargs = profile_kwargs(profile)
 
     # Inductor knobs from FastVideo's LTX-2.3 reference (basic_ltx2_3_distilled).