feat(gpu-backend): unified base64 contract + parameterized deployer for named instances#38
Merged
Merged
Conversation
The Modal GPU backend returns inline base64 (video_b64/last_frame_b64) while a file-serving backend returns URLs — and Generator previously only handled URLs, so real generation against the Modal backend would KeyError. Add _fetch_asset that decodes base64 or downloads a URL (resolving relative URLs against the backend base), unifying the consumer contract. Tests cover both paths.
modal_server.py: serve a single FastAPI ASGI app (POST /generate/t2v|i2v|flf2v, GET /health) returning base64 instead of four per-label subdomain endpoints, so the base URL + path contract matches the flow.Generator consumer and the self-hosted server.py. Parameterize APP_NAME/GPU_TYPE/models/scaledown via env so multiple NAMED instances can be deployed into one account. server.py: return video_b64/last_frame_b64 (drop the unserved /files URLs) so self-hosted == Modal == what Generator consumes.
Adds flow/deploy: a Deployer interface + ModalDeployer (real — validates the modal CLI, passes the spec through as env, runs modal deploy, reports the *.modal.run base URL) and honest AWS/GCP scaffolds (manual_required with concrete steps, never fake success). New [deploy] config section supplies defaults; the "flow deploy PROVIDER --name ... --gpu ..." CLI overrides them, so users can stand up several NAMED instances. Tests cover registration, missing-CLI failure, and the honest AWS/GCP scaffolds.
README: add a 'Deploy the GPU Backend' section explaining that a token isn't enough — the open-source backend must be deployed (flow deploy modal) to yield the endpoint URL jobs route to; note named instances + self-hosted path. config.example.toml: document the [deploy] section + clarify gpu_backend.url is the deployed base URL.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
BYOC ("bring your own compute") only works if the open-source GPU backend is
actually deployed into the user's account — a token/API key alone can't
generate. And the backend contract was inconsistent across three places, so real
generation would break. This makes the backend deployable as named instances
and unifies the contract end-to-end.
What
1. One backend contract (base64, path-based)
modal_server.pynow serves a single FastAPI ASGI app (POST /generate/t2v|i2v|flf2v,GET /health) returning base64 — instead of four per-label subdomain endpoints. One base URL, path routing, matching the self-hostedserver.pyand theflow.Generatorconsumer.server.pyreturnsvideo_b64/last_frame_b64(dropped the unserved/filesURLs).Generator._fetch_assetaccepts base64 or URL (relative URLs resolve against the backend base) — previously it only handled URLs, so generation against the Modal backend (which returns base64) wouldKeyError.2. Parameterized for named instances
modal_server.pyreadsFLOW_GPU_APP_NAME/FLOW_GPU_TYPE/ model ids / scaledown from env, so several named instances can be deployed into one account (weights volume is namespaced per instance).3. A real deployer + CLI
flow/deploy/: aDeployerinterface +ModalDeployer(validates the modal CLI, passes the spec through as env, runsmodal deploy, reports the*.modal.runbase URL) and honest AWS/GCP scaffolds (manual_requiredwith concrete steps — never fake success).flow deploy <provider> --name … --gpu …, with defaults from a new[deploy]config section (CLI overrides config).4. Docs
config.example.toml[deploy]section.Tests
test_engine.py:_fetch_assetbase64 / absolute-url / relative-url / missing.test_deploy.py: provider registration, missing-CLI honest failure, AWS/GCP honest scaffolds, backend module is packaged/locatable.ruff checkclean on changed files.