Clockwork Diffusion: Efficient Generation With Model-Step Distillation

[remyx-ai]

What does this PR do?

Adds Clockwork Diffusion (arXiv:2312.08128) as a new caching hook in the hooks/ gallery, alongside FBC, FasterCache, MAG, and TaylorSeer — wired in exactly as apply_clockwork() + Config + a stateful ModelHook.

Clockwork reuses cross-step computation through the existing StateManager mechanism (the same head_block_output / tail_block_residuals caches FBC holds across steps), and as a cache hook it attaches to every DiT/UNet pipeline rather than introducing a single net-new pipeline. The one adaptation needed: existing caches are transformer-block-centric while Clockwork targets UNet low-resolution resblocks, but the fqn mechanism already supports UNet (per the layer_skip docstring: "For UNet models, you must provide the correct fqn").

Before submitting

• This PR fixes a typo or improves the docs
• Did you read the contributor guideline?
• Did you read our philosophy doc?
• Was this discussed/approved via a GitHub issue or the forum?
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

Tests: the suite did not run in CI — the runner is missing the repo's dependencies (ModuleNotFoundError: No module named 'accelerate', surfacing during collection on examples/advanced_diffusion_training, examples/consistency_distillation, and examples/controlnet). This is an environment/import error, not a code failure; run locally to validate.

Who can review?

@sayakpaul @yiyixuxu

Drafted by Outrider — paper: arXiv:2312.08128.

Discovery context

How this candidate was selected. Surfaced by an Outrider deep-search refine query (distillation few step consistency model rectified flow diffusion) against /search/assets. The engine's normal ranking did not place this paper in the interest's broad pool; it was promoted because the audit pass identified an under-represented theme this paper covers. Confidence 🟡 moderate (Remyx relevance 0.65); research interest: diffusers.

Why this paper fits the team. Clockwork shares the contribution shape of the repo's hooks/ caching gallery — each of FBC, FasterCache, MAG, and TaylorSeer is one paper wired in exactly as apply_<name>() + Config + stateful ModelHook, and that gallery is the team's active investment in inference-caching acceleration. As a cache hook it maximizes call-site coverage (attaches to every DiT/UNet pipeline) versus a single net-new pipeline, and it reuses the same StateManager cross-step cache mechanism.

License & code availability. 🟡 No code repository surfaced — a LICENSE could not be fetched to evaluate. No repository or model URL appeared in the paper, the recommendation envelope, or the arXiv abstract page; license reported as (none detected) (class: no-code-link, compat: 0.30). Worth confirming the paper has an open release before investing in adoption.

Suggested experiment. None.

[smellslikeml]

Closing to unblock a controlled cocoindex-code A/B on the same paper (REMYX-165 experiment). Diff preserved for baseline comparison against the re-run.