Skip to content

Cryspia/comfyui_agent_query

BranchesTags

Repository files navigation

ComfyUI Local Queue & Agent Skill

A small single-instance queue server that sits between your agents and a shared ComfyUI install on the LAN, plus a bash agent skill agents can drop in. Faithful to the design in comfyui_queue_service_design.md: low concurrency, low frequency, simple and reliable.

             +----------------+              +--------------------+
 agent(s) -->| queue server   |--HTTP /prompt---> ComfyUI (LAN)   |
             | FastAPI+SQLite |              +--------------------+
             +----------------+                         |
                      ^                                 v
                      |                           output/ input/
                      +-------- shared mount <----------+

Layout

.
├── compose.yaml            # docker-compose (env-driven; no hardcoded IP / path)
├── .env.example            # copy to .env and fill in
├── init-skill.sh           # render skill-template/ -> ./skill/ with your env
├── comfyui-workflows/      # original GUI-format workflows (reference)
├── server/                 # FastAPI queue server (Python 3.12)
│   ├── Dockerfile
│   ├── requirements.txt
│   ├── app/
│   │   ├── main.py          config.py   db.py       schemas.py
│   │   ├── repository.py    scheduler.py   cleanup.py
│   │   ├── comfy_client.py  workflow_builder.py   utils.py
│   │   └── routers/tasks.py
│   └── workflows/          # pre-converted API-format JSON
├── scripts/
│   └── convert_workflow.py # GUI-format -> API-format converter
├── skill-template/         # bash skill template (placeholders unfilled)
│   ├── SKILL.md
│   ├── generate.sh
│   └── examples/run_t2i.sh  run_i2i.sh  run_i2v.sh
└── tests/
    ├── test_server.sh
    └── test_workflows_validate.py

init-skill.sh renders skill-template/ into ./skill/ (or a path you choose), substituting each __PLACEHOLDER__ for the env you supplied. The rendered ./skill/ is meant to be copied into the agent container.

Features

  • Single /submit, /status, /cancel, /healthz REST API.
  • Priority queue (priority_key-gated); one task runs at a time.
  • Scheduler loop (configurable, 60 s default) that fires the next queued task when ComfyUI is idle, polls the output directory for completion, and expires timed-out tasks.
  • Startup recovery: any task left in running after a restart is failed.
  • Per-agent quota (20 unfinished tasks by default).
  • Workflow JSON templates (ComfyUI API format) filled in on every request:
    • prompt text — identified by _meta.title containing "Positive".
    • input image filename(s) — LoadImage nodes in ascending id order.
    • output filename prefix — always {agent_name}/{task_id}, so the final file is predictable.
    • turbo toggle — PrimitiveBoolean node with "4 Steps" / "Lightning" in its title. turbo=true picks the 4-step lightning LoRA; turbo=false gives the full-step path (slower but higher fidelity).
    • seed — randomized per submission.
  • Skill-side helper precision_hint maps Chinese precision cues (精细, 高精, 精致, 精修, 细致, 精品, 高品质, 高质量) to turbo=false automatically, and speed cues (快速, 草稿, 速出, turbo, draft, quick) to turbo=true.

Supported workflows

mode workflow_name template file input imgs output
t2i qwen_generate_v1 qwen-generate-api.json 0 png
i2i qwen_edit_v1 qwen-edit-api.json 1 png
i2i qwen_merge_v1 qwen-merge-api.json 2 png
i2i qwen_merge3_v1 qwen-merge3-api.json 3 png
t2v wan22_t2v_v1 video_wan2_2_14B_t2v-api.json 0 mp4
i2v wan22_i2v_v1 video_wan2_2_14B_i2v_fp8-api.json 1 mp4

Templates live under server/workflows/. They are pre-converted from the original GUI-format JSON in comfyui-workflows/ via scripts/convert_workflow.py; if you need to add or re-export a workflow, edit it in ComfyUI and re-run the converter.

Quick start

  1. Clone this repo into a directory that can see the ComfyUI SMB share.

  2. Copy .env.example to .env and fill in COMFY_API_BASE, COMFY_INPUT_HOST, COMFY_OUTPUT_HOST, and the two keys:

    cp .env.example .env
$EDITOR .env   # set COMFY_API_BASE=http://<your-comfyui>:8188  etc.

  3. Build & run the queue server:

    docker compose --env-file .env up -d --build
curl http://127.0.0.1:18080/healthz   # -> {"status":"ok"}

  4. Render the agent skill for each agent that will use it:

    SERVER_BASE_URL=http://127.0.0.1:18080 \
AGENT_NAME=agent_alpha \
SUBMIT_KEY=$(grep ^SUBMIT_KEY .env | cut -d= -f2) \
PRIORITY_KEY=$(grep ^PRIORITY_KEY .env | cut -d= -f2) \
INPUT_ROOT=/mnt/comfy/input OUTPUT_ROOT=/mnt/comfy/output \
    ./init-skill.sh ./skill_agent_alpha

    Copy skill_agent_alpha/ into the agent container at whatever path the agent runtime expects its skills. The rendered skill has no placeholders and no Python / JSON-parser dependency — just bash + curl.

  5. Invoke from the agent:

    ./skill_agent_alpha/generate.sh generate t2i "a red cube on a white table"
# prints the absolute output file path on stdout, exit 0 on success.

Environment variables

Server (read from container env)

All defaults are enforced in server/app/config.py.

var required default purpose
COMFY_API_BASE yes — e.g. http://192.168.x.y:8188
SUBMIT_KEY yes — Shared secret for /submit
PRIORITY_KEY yes — Shared secret for priority jobs
SERVER_HOST no 0.0.0.0 Uvicorn bind
SERVER_PORT no 8080 Uvicorn port (inside container)
COMFY_INPUT_ROOT no /mnt/comfy/input Input dir inside container
COMFY_OUTPUT_ROOT no /mnt/comfy/output Output dir inside container
WORKFLOWS_DIR no /app/workflows API-format JSON templates
DB_PATH no /app/data/tasks.db SQLite file
MAX_PENDING_PER_AGENT no 20 per-agent quota
TASK_TTL_HOURS no 24 auto-fail after this
SCHEDULER_INTERVAL_SECONDS no 60 scheduler tick
CLEANUP_INTERVAL_SECONDS no 3600 cleanup tick
CLEANUP_RETAIN_HOURS no 24 drop terminal rows older than this

Compose host-side

var default purpose
HOST_BIND 127.0.0.1 which host iface to expose on
HOST_PORT 18080 host-side listening port
COMFY_INPUT_HOST ./mounts/comfy_input bind-mount source
COMFY_OUTPUT_HOST ./mounts/comfy_output bind-mount source

init-skill.sh

var required default purpose
SERVER_BASE_URL yes — URL of the queue server, e.g. http://127.0.0.1:18080
AGENT_NAME yes — [A-Za-z0-9_-]+; used as input/output subdir name and quota key
SUBMIT_KEY yes — matches server's SUBMIT_KEY
PRIORITY_KEY no — leave empty to disable priority for this agent
INPUT_ROOT no /mnt/comfy/input shared input mount (on the agent host)
OUTPUT_ROOT no /mnt/comfy/output shared output mount
TEMPLATE_DIR no ./skill-template where to read from
FORCE no 0 set to 1 to overwrite an existing output dir

CLI: init-skill.sh [OUTPUT_DIR] (defaults to ./skill).

REST API

POST /submit

{
  "submit_key": "YOUR_SUBMIT_KEY",
  "priority": false,
  "priority_key": null,
  "agent_name": "agent_alpha",
  "mode": "i2v",
  "workflow_name": "wan22_i2v_v1",
  "input_filename": "20260423T153000_local123_input.png",
  "prompt_params": {
    "prompt": "a cat walking in a cyberpunk street",
    "negative_prompt": "blurry",
    "turbo": true,
    "seed": 12345
  }
}

prompt_params.turbo defaults to true. Set priority: true with a valid priority_key to jump ahead of non-priority queued tasks.

For multi-input workflows (qwen_merge_v1 / qwen_merge3_v1), use input_filenames: [...] instead of input_filename.

GET /status/{task_id}

{
  "task_id": "20260423T153012_a8f3cd_i2v",
  "status": "running",
  "output_filename": "20260423T153012_a8f3cd_i2v_00001_.mp4",
  "error_message": null,
  "agent_name": "agent_alpha",
  "mode": "i2v",
  "workflow_name": "wan22_i2v_v1",
  "created_at": "2026-04-23T15:30:12Z",
  "updated_at": "2026-04-23T15:30:27Z",
  "finished_at": null
}

POST /cancel/{task_id}

Only tasks in queued can be cancelled. Running tasks return HTTP 409.

GET /healthz

{"status":"ok"}.

Agent skill (bash)

The skill ships as skill-template/generate.sh. After rendering with init-skill.sh, agents call it directly:

./skill/generate.sh generate t2i "a snow leopard on a misty ridge"
# stdout: /mnt/comfy/output/<agent>/20260423T153012_a8f3cd_t2i_00001_.png

./skill/generate.sh generate i2i "brighten the scene" /tmp/input.png
./skill/generate.sh generate i2i_merge2 "merge" /tmp/a.png /tmp/b.png
./skill/generate.sh generate t2v "a cat walking"
./skill/generate.sh generate i2v "gentle breeze moves through" /tmp/still.png

HINT_TEXT="请精细画一幅雪山" ./skill/generate.sh generate t2i "a snowy mountain"
# -> detects 精细 and submits with turbo=false (higher quality path).

PRIORITY=true ./skill/generate.sh generate t2i "urgent"
./skill/generate.sh status <task_id>
./skill/generate.sh cancel <task_id>
./skill/generate.sh health

Exit codes: 0 success, 10 bad input, 11 quota full, 12 bad submit (auth / workflow), 13 cancelled, 14 failed, 15 poll timeout, 16 success-without-file, 2 bad usage. See skill-template/SKILL.md for the full contract.

The skill is stdlib-only bash: no Python, no jq, no extra packages. JSON extraction uses a small grep-based helper. Agents only need bash, curl, and common GNU coreutils.

Testing

tests/test_server.sh ships a handful of end-to-end checks (health, submit, quota, priority, cancel, restart-recovery). Run them against a running stack after the container is up:

bash tests/test_server.sh health
bash tests/test_server.sh quota
bash tests/test_server.sh priority_order
bash tests/test_server.sh restart_recovery

tests/test_workflows_validate.py submits each workflow template directly to ComfyUI's /prompt endpoint to verify structural validity (and clears the queue between submissions so they don't actually run):

COMFY_API_BASE=http://<your-comfyui>:8188 AGENT_NAME=agent_test \
python3 tests/test_workflows_validate.py

Adding a workflow

  1. Export a new workflow from ComfyUI. You can either:

    • save the API-format JSON from the UI (preferred), or

    • save the GUI-format JSON and run the converter:

      python3 scripts/convert_workflow.py \
    comfyui-workflows/mynew.json \
    server/workflows/mynew-api.json
# converter talks to ComfyUI's /object_info once to learn widget ordering.

  2. Register it in server/app/workflow_builder.py:

    WORKFLOW_FILES["my_new_v1"] = "mynew-api.json"
WORKFLOW_INPUT_IMAGE_COUNT["my_new_v1"] = 1
WORKFLOW_OUTPUT_EXT["my_new_v1"] = "png"

  3. Restart the server: docker compose --env-file .env restart.

Positive/negative prompt nodes are identified by _meta.title substring match — make sure the CLIPTextEncode / TextEncodeQwenImageEditPlus node in your workflow has (Positive) / (Negative) in its title. The filename prefix, LoadImage nodes, turbo PrimitiveBoolean, and KSampler seed are auto-detected.

Troubleshooting

docker compose build fails with Temporary failure in name resolution

The build container needs outbound DNS for pip install. Ubuntu's default /etc/resolv.conf points at 127.0.0.53 (the systemd-resolved stub), which is unreachable from Docker bridge namespaces, so DNS silently fails during the build.

Fix your host so that /etc/resolv.conf lists a routable upstream DNS server (e.g. the one handed out by your router over DHCP), then rebuild. Verify from a fresh container before rebuilding:

docker run --rm python:3.12-slim python3 -c \
    "import socket;print(socket.getaddrinfo('pypi.org',443)[0][4])"

If the command above prints an IP address quickly, you're fine; if it hangs or errors with Temporary failure in name resolution, your host still needs adjustment.

As a one-off workaround that doesn't touch system DNS, run the build with docker build --network host ./server -t comfyui_agent_query-comfy-queue-server. That only affects the transient build container; the running service still uses the bridge network defined in compose.yaml.

Design non-goals

This project deliberately does not implement:

  • multi-server HA,
  • interrupting a running task to let a higher-priority task jump ahead,
  • recovering a task that was running when the server restarted,
  • websocket progress streams,
  • multi-tenant auth beyond a single shared key.

See section 2.2 of comfyui_queue_service_design.md for the complete non-goal list.

About

a skill driven comfyui query server

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages