✦ airdraw

Paint with your hands. No tablet. No stylus. Just a camera and a finger in the air.

Live demo →

---

What it is

airdraw turns your webcam into a sketchpad. The app watches your hand at 30–60 fps, locks onto 21 landmark points per frame, and translates a handful of poses into drawing commands. Lift your index finger and a glowing stroke trails behind it. Pinch and the same motion erases. Throw up an open palm and the canvas clears.

Everything runs on-device. No frames leave the browser, no analytics on the strokes, no server-side inference — just MediaPipe's WebAssembly runtime, a 2D canvas, and a bit of math.

I built this because pointer-only interfaces feel boring, and because dragging a sticker on an iPad never quite scratched the "I am drawing in mid-air" itch. Turns out the camera in your laptop is enough.

Screenshot

Tip: the README hero is just a placeholder until you record your own session — replace docs/screenshot.png with a clean capture and the badge above falls into place.

Highlights

• 21-point hand tracking via Google's MediaPipe HandLandmarker (GPU delegate, falls back to CPU)
• Five gestures, all classified locally with simple geometric heuristics — no ML model in the gesture layer, easy to extend
• 20-step undo / redo using full ImageData snapshots so erases survive history
• One-tap PNG export — both via the dock button and the thumbs-up gesture
• Glow brush with shadow-driven bloom that scales with brush size
• Mirror toggle, an 8-colour palette, brush size 2 → 48 px
• Live HUD showing the current gesture, FPS, tracker state (calibrating / live / offline)
• Reduced-motion aware — animations disable themselves if you've asked for less motion at the OS level
• Designed dark-first with a custom Ethereal Glass visual language: OLED black, radial mesh gradients, double-bezel cards, subtle film grain

Gesture cheat sheet

Gesture	Hand pose	Result
✦ Draw	Index up, every other finger curled	Lays a stroke in your selected colour
◌ Erase	Pinch (thumb tip + index tip touching)	Rubs out pixels along the path
✌ Pen up	Peace sign (index + middle up)	Moves the cursor without drawing
✺ Clear	All five fingers extended (open palm)	Wipes the canvas — pushes a snapshot for undo
👍 Save	Thumbs up, all other fingers folded	Downloads the canvas as airdraw-<timestamp>.png

Clear and Save are debounced to 1.2 seconds so a held pose doesn't fire repeatedly.

How to use it

1. Open the live demo, or run it locally (see below).
2. Grant camera access when the browser prompts you. The footage stays on your device — no upload, no backend.
3. Wait a beat for the model to warm up. The nav badge swaps from Booting tracker to Tracking once it's ready.
4. Pick a colour from the dock and drag the brush slider to your taste.
5. Hold up your hand 30 – 60 cm from the lens with palm facing the camera. Strong, even light helps a lot.
6. Lift your index finger and start drawing. Pinch to erase. Open palm to wipe. Thumbs up to export.
7. The dock at the bottom mirrors every gesture as a button — you can drive the entire app with a mouse if you want, or mix both.

A few things that help:

• Light: avoid backlight (windows behind you). Front-on light at face level is gold.
• Background: the model handles cluttered backgrounds, but a plain wall buys you a few extra fps.
• Distance: too close and your wrist drops out of frame; too far and small finger movements get noisy.
• Mirror: on by default so your hand moves where you expect. Toggle it off if you're projecting onto a screen behind you.

Tech stack

• Next.js 16 (App Router, Turbopack) on React 19
• TypeScript with strict mode
• Tailwind CSS v4 with hand-rolled design tokens
• @mediapipe/tasks-vision for hand-landmark detection
• Framer Motion for entry choreography and micro-interactions
• HTML5 Canvas (2D context) for the drawing layer
• Geist Sans + Geist Mono via next/font

The MediaPipe WASM runtime and the HandLandmarker .task model are pulled from the public jsdelivr and storage.googleapis.com CDNs on first load and cached by the browser thereafter.

Run it locally

Prerequisites

• Node 20+
• pnpm 9+ (or swap in npm / yarn / bun — the lockfile is pnpm)
• A webcam
• A Chromium-based browser (Chrome / Edge / Brave / Arc) or recent Safari / Firefox

Setup

git clone git@github.com:tuttucodes/airdraw.git
cd airdraw
pnpm install
pnpm dev

Open http://localhost:3000 and grant camera access.

Available scripts

Command	What it does
pnpm dev	Dev server with Turbopack on port 3000
pnpm build	Production build
pnpm start	Serve the production build
pnpm lint	ESLint with the Next.js + TypeScript presets

Project layout

app/
  layout.tsx          Fonts, metadata, viewport
  page.tsx            Stage + dock + nav composition
  globals.css         Design tokens, glass utilities
components/
  StagePanel.tsx      Video element wrapped in a double-bezel
  DrawCanvas.tsx      Canvas + stroke engine + history
  HandOverlay.tsx     SVG skeleton on top of the video
  CursorDot.tsx       Glow cursor that tracks the index tip
  ControlDock.tsx     Floating glass dock at the bottom
  ColorPalette.tsx    Eight-swatch picker with shared layout id
  BrushSlider.tsx     Custom range input
  GestureHUD.tsx      Live gesture badge + cheat sheet
  IslandNav.tsx       Floating top pill (FPS, tracker state)
  PermissionGate.tsx  Camera-permission overlay
  DockButton.tsx      Reusable circular dock button
  icons.tsx           Hand-tuned ultralight SVG icons
hooks/
  useWebcam.ts        Camera lifecycle (request / play / stop)
  useHandTracker.ts   MediaPipe loader + per-frame detection loop
lib/
  landmarks.ts        Landmark index map + distance helper
  gestures.ts         Geometric gesture classifier
  palette.ts          Swatches + brush bounds
public/               Static assets
docs/                 README artwork

How the gesture classifier works

It's blunt on purpose. For each frame:

1. MediaPipe returns 21 normalised (x, y, z) landmarks plus the handedness label.
2. lib/gestures.ts measures whether each finger is up by comparing the tip's y to the PIP joint's y (image space, so smaller y is higher up).
3. The thumb gets two cues — extended along the x-axis relative to the IP joint, and a "thumb up" check against the wrist.
4. Pinch is the Euclidean distance between landmark 4 (thumb tip) and landmark 8 (index tip), thresholded at 0.055 in normalised space.
5. The combination of those booleans falls through a short if ladder to pick a gesture.

If you want to add a new pose — say, a finger-gun for spawning shapes — extend the Gesture union, add the boolean check, and wire the action into DrawCanvas. No retraining needed.

Performance notes

• Detection runs on requestAnimationFrame with a guard so we never call detectForVideo twice with the same timestamp (MediaPipe throws otherwise).
• The drawing canvas uses getImageData / putImageData for history — that's heavier than a stroke list but it survives erases cleanly. History is capped at 20 entries; tune HISTORY_LIMIT in DrawCanvas.tsx.
• All UI animation is transform + opacity only. backdrop-blur is restricted to fixed elements per the design tokens.

Roadmap

• Two-hand support for symmetrical / chord-style drawing
• Pressure curves driven by pinch-tightness
• Shareable session links via signed canvas snapshots
• WebGPU brush engine for textured strokes
• Gesture-trained shape primitives (circle / line / arrow)
• PWA install + offline model caching

Contributing

PRs and issues welcome. The project is small enough that you can read the whole thing in an afternoon.

Reporting a bug

Open an issue with:

• Browser + OS + camera (e.g. Chrome 132 / macOS 14 / built-in FaceTime HD)
• A screen recording or screenshot if visual
• The FPS badge value at the time
• What you expected vs. what happened

Submitting a change

# fork on GitHub, then:
git clone git@github.com:<your-handle>/airdraw.git
cd airdraw
pnpm install
git checkout -b feat/your-thing

# hack...
pnpm lint        # must pass
pnpm build       # must pass

git commit -m "feat: your thing"
git push origin feat/your-thing
# then open a PR against main

Coding conventions

• TypeScript strict — no any unless you justify it in the PR
• Conventional commits (feat:, fix:, chore:, refactor:, docs:, perf:)
• Components stay under ~200 lines; lift logic into hooks/ or lib/ if a file grows
• Animations use the project's cubic-bezier tokens (--ease-spring, --ease-out-expo) — please don't reach for ease-in-out
• transform and opacity only for animation properties

Good first issues

• Add a "rainbow" brush mode that cycles colour by stroke distance
• Build a Storybook (or Ladle) playground for the dock components
• Replace the JSDelivr WASM URL with a self-hosted copy under /public/wasm
• Write Vitest tests for classifyGesture covering every pose

FAQ

Does it work on phones? Mostly. The WASM runtime ships, MediaPipe initialises, and gestures register — but most laptops are easier to draw on because you can rest your hand. Android Chrome is fine, iOS Safari is hit-and-miss depending on the camera permission flow.

Can I run it fully offline? Not yet — the model file is fetched from a CDN on first load. Drop it under /public/models/ and update the path in useHandTracker.ts and you're set. PR welcome.

Is my video stored anywhere? No. There is no backend in this project, no analytics on hand frames, and no third-party tracker. The only network traffic is the one-time CDN fetch for the WASM + model.

Why MediaPipe and not TensorFlow.js / MoveNet? MediaPipe's HandLandmarker is the most accurate browser-side hand model I tested for this task and the GPU delegate is genuinely fast. MoveNet doesn't do hands. If you have a better recommendation I'd love to hear it.

Acknowledgements

Built on top of work by the MediaPipe team at Google, the Next.js team at Vercel, and the Framer Motion crew. Geist is Vercel's superb sans / mono pair.

License

MIT — do whatever you want, just don't sue me if your wrist gets tired.

---

_{Made by Rahul BK · @tuttucodes}