[Feature]: About AI image generation, Can I use local deployed Comfyui API?

[sunisstar]

Problem / pain point

as title, gpt-image-2 is expensive.

Proposed solution

Could the locally hosted ComfyUI API be leveraged as the image generation endpoint, thereby mitigating operational expenditures?

Alternatives considered

No response

Scope

Provider / model

Hard-constraints check

• I have read the hard constraints in CLAUDE.md and this proposal does not conflict with any of them.

[github-actions]

Thank you for the suggestion! Using a local ComfyUI API as the image generation backend is an interesting idea that aligns well with Open CoDesign's local-first and BYOK principles.

Current image generation architecture

Image generation is handled through the gen_image tool, which delegates to the configured provider's image generation capability via pi-ai. The supported providers currently include OpenAI, Anthropic, Google, and others that pi-ai natively wraps — ComfyUI is not among them today.

Relevant code:

• packages/providers/ — adapter layer for pi-ai provider registration
• packages/core/ — tool orchestration, including gen_image dispatch
• AGENTS.md:111 — documents gen_image(prompt, path) as a design tool that "writes generated images to disk when capability and provider config allow it"

How it could work

Open CoDesign supports registering custom providers via pi.registerProvider() (per AGENTS.md). A ComfyUI provider adapter would need to:

1. Implement the pi-ai ImageModel interface (capabilities + generation)
2. Translate Open CoDesign's image prompt into ComfyUI workflow JSON (e.g., using a pre‑defined or user‑provided workflow)
3. Handle the ComfyUI API's async queue mechanism (upload prompt, poll queue, download result)
4. Return the generated image file path for use in the design workspace

This is not built-in today, but the custom provider registration mechanism makes it technically feasible. The biggest effort is the adapter itself, not the Open CoDesign integration.

Need more info:

• Do you have a specific ComfyUI workflow in mind (e.g., SDXL, Flux, SD3)? Certain model types may require different workflow formats.
• Is your ComfyUI instance accessible via a local HTTP endpoint (default http://127.0.0.1:8188)?
• Are there any authentication or CORS considerations for the Electron app to communicate with it?

A custom provider contribution would be welcome. If you'd like to prototype it, I can point you to the relevant interfaces in packages/providers/ and packages/shared/ for the model capability schema.

Open-CoDesign Bot