cosmos3: disable the bundled content guardrail#21
Merged
Conversation
Set guardrails=False on the OmniSetupOverrides so the Cosmos worker skips the bundled text (Blocklist + Qwen3Guard) and video guardrail runners entirely. The Qwen3Guard text guard is unfit to gate customer prompts: - one coarse "Sexual Content or Sexual Acts" category, no CSAM/minor distinction, so it cannot express "block illegal only, allow legal"; - demonstrable demographic bias -- "an african american woman being picked up by a robot" is blocked as sexual while the same prompt with "white" returns normally; - the prompt check runs after generation (inference.py Save-outputs loop), so every rejected prompt still burns a full ~60s clip and surfaces as an opaque 500 "Failed to fold videos stream". Content filtering will move to a platform-level video-gen guardrail applied uniformly across all models, rather than a per-model guard here. Covers both Cosmos3-Nano and Cosmos3-Super (same worker). NVIDIA's own latency/throughput inference configs already run guardrails: False. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Set
guardrails=Falseon the worker'sOmniSetupOverridesso the Cosmos worker skips the bundled text (Blocklist + Qwen3Guard) and video guardrail runners entirely. One line; no cosmos-framework patch. Covers both Cosmos3-Nano and Cosmos3-Super (same worker).Why
The bundled Qwen3Guard text guard is unfit to gate customer prompts:
"Sexual Content or Sexual Acts"category, no CSAM/minor distinction — it cannot express "block illegal only, allow legal adult."whitereturns normally. The only variable is the subject's race.inference.pySave-outputs loop, ~line 1538), so every rejected prompt burns a full ~60s clip and surfaces to the user as a 500"Failed to fold videos stream".Disabling it also makes video gen uniform — no other videogen model (LTX, FastWan…) has a worker-level guard. Real content filtering will be a platform-level video-gen guardrail applied across all models, as a separate future project — not a per-model guard here.
NVIDIA's own
latency.yaml/throughput.yamlinference configs already runguardrails: False, so this is a supported mode.Off = no content filtering on Cosmos in the interim, including no CSAM/illegal-content filter, until the platform guardrail exists. This resolves the previously-"pending legal" posture toward "off" for a live generative video model — flagged for legal/leadership awareness.
Follow-ups (not in this PR)
nvidia/Cosmos-Guardrail1prefetch from the Dockerfile — unused once guardrails are off (removes the build-time HF-token requirement).Rollout
Rebuild the Cosmos worker image → roll Cosmos3-Nano (Super later). Holding the prod roll until this is reviewed/approved.
🤖 Generated with Claude Code