Workplane AI

This repository contains a working MVP for an AI-assisted CAD pipeline:

• frontend/: Next.js + TypeScript UI for prompt entry, image upload, STL preview, code editing, and STL download.
• backend/: FastAPI service that calls Gemini with vision input, extracts CadQuery code, executes it, retries on failure, and serves generated STL files.

The core loop is:

1. Upload an image and/or enter a text prompt.
2. POST /api/generate sends the request to Gemini with the CAD system prompt.
3. The backend extracts the <code> block, executes it in a constrained CadQuery runtime, and exports an STL.
4. If execution fails, the backend sends the error traceback back to Gemini and retries up to 3 times.
5. The frontend renders the STL with React Three Fiber and shows the generated CadQuery source in an editable Monaco editor.

Features

• Vision + text generation with Google Gemini
• CadQuery intermediate representation with server-side STL export
• Automatic retry loop with traceback-aware self-correction
• Editable code panel with POST /api/execute for fast iteration without another LLM call
• Follow-up refinement flow that reuses the original prompt, original image, and current code
• In-browser STL preview with orbit controls and grid reference
• Simple ephemeral file storage in /tmp/generated

Project Structure

.
├── backend/
│   ├── config.py
│   ├── executor.py
│   ├── llm.py
│   ├── main.py
│   ├── models.py
│   ├── requirements.txt
│   └── generated/
├── frontend/
│   ├── app/
│   ├── components/
│   ├── lib/
│   └── package.json
├── docker-compose.yml
├── README.md
└── .env.example

Prerequisites

• Node.js 24+
• Python 3.12+
• A Gemini API key
• CadQuery runtime support

Python 3.12 is the newest verified backend runtime for this repo. Python 3.13 and 3.14 are newer overall, but the current cadquery==2.5.2 pip install path depends on cadquery-ocp wheels that were not available for those interpreters during this upgrade.

CadQuery can be the hardest dependency. The backend requirements.txt includes cadquery, but if your platform has trouble with pip wheels, use conda instead:

conda install -c cadquery -c conda-forge cadquery
pip install -r backend/requirements.txt

Environment Setup

Copy the example file and fill in your Gemini key:

cp .env.example .env

Available variables:

• GEMINI_API_KEY: required for /api/generate
• GEMINI_MODEL: optional Gemini model override
• GEMINI_THINKING_LEVEL: optional Gemini thinking level, defaults to minimal
• GEMINI_MAX_OUTPUT_TOKENS: optional response token budget, defaults to 65536
• CADQUERY_MAX_RETRIES: retry count for automatic self-correction
• GENERATED_DIR: STL output directory, defaults to /tmp/generated
• BACKEND_CORS_ORIGINS: comma-separated frontend origins
• NEXT_PUBLIC_API_BASE_URL: frontend API base URL

Run the Backend Locally

cd backend
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --reload

The API will be available at http://localhost:8000.

Available endpoints:

• POST /api/generate
• POST /api/iterate
• POST /api/execute
• GET /api/stl/{filename}
• GET /api/health

Example request:

curl -X POST http://localhost:8000/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A phone stand with a 15-degree angle, 80mm wide, with a cable slot in the back"
  }'

Run the Frontend Locally

cd frontend
npm install
npm run dev

The app runs at http://localhost:3000.

If your backend is not running on http://localhost:8000, create frontend/.env.local:

NEXT_PUBLIC_API_BASE_URL=http://localhost:8000

Run with Docker Compose

cp .env.example .env
docker compose up --build

This starts:

• frontend: http://localhost:3000
• backend: http://localhost:8000

The generated STL directory is mounted from ./backend/generated into /tmp/generated inside the backend container.

API Notes

POST /api/generate

Request body:

{
  "prompt": "A desk cable organizer with three channels and chamfered edges",
  "image": "data:image/png;base64,..."
}

Response body:

{
  "stl_url": "http://localhost:8000/api/stl/abc123.stl",
  "code": "import cadquery as cq\n...",
  "analysis": "Parametric reasoning...",
  "filename": "abc123.stl",
  "attempts": 1
}

POST /api/execute

Request body:

{
  "code": "import cadquery as cq\n..."
}

Response body:

{
  "stl_url": "http://localhost:8000/api/stl/def456.stl",
  "filename": "def456.stl"
}

POST /api/iterate

Request body:

{
  "original_prompt": "A phone stand with a 15-degree angle, 80mm wide, with a cable slot in the back",
  "image": "data:image/png;base64,...",
  "current_code": "import cadquery as cq\n...",
  "follow_up_prompt": "Make it 15mm taller and add two countersunk wall mounting holes."
}

Response body:

{
  "stl_url": "http://localhost:8000/api/stl/ghi789.stl",
  "code": "import cadquery as cq\n...",
  "analysis": "Updated parametric reasoning...",
  "filename": "ghi789.stl",
  "attempts": 1
}

Security Notes

This MVP intentionally prioritizes a working loop over hardened isolation:

• CadQuery execution currently uses exec() with a restricted import/builtins surface.
• This is not a production sandbox.
• For production, move execution into an isolated container or job runner with CPU, memory, filesystem, and network limits.
• Treat /api/execute as trusted-user functionality only.

Known Limitations

• Status updates are client-side staged messages, not streamed server progress.
• The backend serves STL files ephemerally and does not clean old artifacts automatically.
• Complex or highly organic objects may still require several prompt iterations.

Gemini Model Note

Google’s official Gemini 3.1 Pro Preview page shows gemini-3.1-pro-preview as the model code, with the page last updated on March 18, 2026. This project now defaults to:

GEMINI_MODEL=gemini-3.1-pro-preview

Suggested Next Steps

• Add server-sent events or WebSockets for true retry/progress updates
• Persist generation history and artifacts
• Add conversation-based follow-up edits
• Support STEP and OBJ export