Turn ordinary 2D images into depth maps, compare neural depth models, benchmark optional ONNX Runtime acceleration, evaluate against ground truth, and export approximate 3D point clouds — all from a desktop app running on your own machine.
No cloud uploads. No API keys. No subscription.
Images are processed through a local Electron + FastAPI + PyTorch/ONNX Runtime pipeline.
Overview · Quick Start · Build Matrix · Docs · API
demo-short.mp4
- What DepthLens Pro does
- Why it is technically strong
- Architecture at a glance
- Feature tour
- Quick Start
- Choose your setup path
- Installation and build matrix
- ONNX acceleration note
- Docs and deep dives
- API quick reference
- Testing and CI
- Security and privacy
- Project structure
- Contributing
- License
- Acknowledgements
DepthLens Pro is a local-first desktop application for monocular depth estimation: it predicts a relative depth map from a single RGB image. The app combines an Electron desktop shell, a local FastAPI inference service, PyTorch MiDaS/DPT models, optional ONNX Runtime execution, and local export workflows.
DepthLens Pro supports:
- Depth-map generation for PNG, JPG, WEBP, and BMP images.
- Side-by-side model comparisons across MiDaS Small, DPT Hybrid, and DPT Large.
- Optional ONNX Runtime benchmarks and provider diagnostics.
- Ground truth evaluation with PNG/TIFF/NPY depth labels.
- Approximate colored point-cloud exports as PLY or OBJ.
- Local observability through Prometheus metrics and in-app telemetry.
Depth interpretation: DepthLens Pro predicts relative depth, not real-world metric distance. It is useful for visual depth understanding, model comparison, and approximate geometry; it is not a replacement for calibrated metric depth sensors.
To know more, read How Monocular Depth Estimation Works.
| Capability | Production-oriented behavior |
|---|---|
| Local-first desktop ML system | Runs the UI, API, inference runtime, cache fallback, and exports on the local machine. |
| FastAPI inference service | Provides typed routes for estimation, batch processing, benchmarking, reconstruction, diagnostics, health, and readiness. |
| Lazy model loading and device-aware execution | Loads models on demand, detects CPU/CUDA/MPS/XPU availability, and protects model forward passes with concurrency controls. |
| Optional ONNX Runtime acceleration | Keeps PyTorch as the standard path while supporting locally generated ONNX files and provider diagnostics. |
| Redis/in-memory cache fallback | Uses Redis when configured and falls back to a bounded in-process LRU cache when Redis is unavailable. |
| Ground truth evaluation | Supports GT resizing, masking, median-scale alignment, and standard depth-estimation metrics. |
| 3D point-cloud export | Converts relative depth into approximate PLY/OBJ point clouds using a pinhole projection model. |
| Observability and Prometheus metrics | Exposes runtime snapshots, metrics, traces, cache events, crash analytics, and benchmark history without logging raw images. |
| Secure Electron bridge and backend process lifecycle | Uses context isolation, a narrow preload API, localhost-only navigation policy, port checks, PID metadata, and packaged-resource verification. |
Read the full design rationale in System Design Decisions.
| Layer | Responsibility |
|---|---|
| Electron main process | Window lifecycle, backend spawn, port checks, PID metadata, settings persistence, packaged-resource validation. |
| Preload bridge | Narrow context-isolated API for backend URL, dialogs, and platform information. |
| Renderer UI | Workspace, webcam, compare, performance, experiments, 3D reconstruction, guide, charts, and exports. |
| FastAPI backend | Health/readiness routes, inference endpoints, diagnostics, metrics, JSON logging, async lifespan hooks. |
| Runtime services | PyTorch/ONNX dispatch, image preprocessing, depth normalization, GT metrics, reconstruction, observability. |
| Local cache/resources | Optional Redis, in-memory LRU fallback, locally generated ONNX files, packaged Python/backend/frontend resources. |
To know more, read System Architecture Deep Dive.
- Upload images, choose device/model/colormap, generate depth maps, and download results.
- Tracks processed images, latency, cache hits, errors, throughput, and cumulative inference time.
- Supports automatic
max_dimdownscaling without modifying the original file.
Read more in Models, Colormaps & Metrics.
- Processes one image and one depth-label file together.
- Supports PNG, TIFF, and safe NPY loading with invalid-pixel masking and unit scaling.
- Applies nearest-neighbor GT resizing and median-scale alignment before metrics.
Read more in Ground Truth Evaluation and Understanding Depth Metrics.
- Runs controlled local webcam inference at 1–5 FPS.
- Provides frame max-dimension limits and optional temporal smoothing.
- Shows backend latency, end-to-end latency, effective FPS, skipped frames, and active runtime settings.
Read more in Terminal-Only Development for lightweight runtime verification.
- Runs MiDaS Small, DPT Hybrid, and DPT Large on the same image.
- Displays side-by-side previews and latency badges.
- Charts latency, SSIM, SILog, PSNR, gradients, edge density, entropy, and dynamic range.
Read more in Models, Colormaps & Metrics.
- Benchmarks PyTorch and optional ONNX Runtime with a deterministic synthetic frame.
- Reports latency, speedup, throughput, RSS memory, provider status, and fallback state.
- Continues to run PyTorch benchmarks even when ONNX files are absent.
Read more in Setup and Build and Production Packaging.
- Records structured workspace results into named validation runs.
- Includes optional ground truth metrics when GT mode is enabled.
- Exports JSON or CSV without base64 previews.
Read more in Testing and CI.
- Converts RGB + predicted depth into approximate colored point clouds.
- Supports PLY and OBJ export with point budgets and preview downsampling.
- Exposes focal scale, depth scale, near/far percentile clipping, and coordinate-system controls.
Read more in Models, Colormaps & Metrics.
- Provides an offline accordion reference for workflows, metrics, models, 3D controls, and troubleshooting.
- Does not require the backend to be online.
- Complements the deeper Markdown documentation in
docs/.
Read more in Troubleshooting.
Run commands from the repository root unless a command explicitly says otherwise.
git clone https://github.com/AyushmanRaha/DepthLensPro.git
cd DepthLensPro
npm run setup
npm run backend:dev
npm run frontend:devVerify the local backend in another terminal:
curl http://127.0.0.1:8765/live
curl http://127.0.0.1:8765/readyExpected /live shape:
{
"status": "ok",
"busy": false,
"state": "idle",
"service": "DepthLens Pro API",
"version": "1.0.0"
}For terminal-only setup details, read Terminal-Only Development.
| Goal | Recommended path | Details |
|---|---|---|
| Run/edit locally | Terminal-only development | npm run setup:<platform>, then npm run backend:dev and npm run frontend:dev; see Terminal-Only Development. |
| Build a packaged desktop app | Native app build | Use the platform matrix below; full details in Setup and Build. |
| Use only the local API | Backend-only | npm run setup, npm run backend:dev, then call 127.0.0.1:8765; see API Reference. |
| Use standard PyTorch | Standard setup | ONNX files are not required; use setup:mac, setup:win, or setup:linux. |
| Use ONNX acceleration | ONNX setup/build | Generate and validate all required local ONNX files before packaging; see Setup and Build. |
| Run backend + Redis in containers | Docker Compose | docker compose up --build; see Production Packaging. |
Commands are run from the repository root. On Windows, run them in PowerShell.
| Platform | Standard native build | ONNX native build |
|---|---|---|
| macOS Apple Silicon only arm64 | npm run setup:mac → npm run verify:resources → npm run build:mac:arm64 → npm run launch:mac |
npm run setup:mac:onnx → npm run verify:onnx:required → npm run build:mac:arm64:onnx → npm run launch:mac |
| macOS x64 / universal | Not supported; build scripts fail with a clear unsupported-architecture message. | Not supported. |
| Windows ARM64 | npm run setup:win → npm run verify:resources → npm run build:win:arm64 → npm run launch:win |
npm run setup:win:onnx → npm run verify:onnx:required → npm run build:win:arm64:onnx → npm run launch:win |
| Windows x64 | npm run setup:win → npm run verify:resources → npm run build:win:x64 → npm run launch:win |
npm run setup:win:onnx → npm run verify:onnx:required → npm run build:win:x64:onnx → npm run launch:win |
| Linux ARM64 | npm run setup:linux → npm run verify:resources → npm run build:linux:arm64 → npm run launch:linux |
npm run setup:linux:onnx → npm run verify:onnx:required → npm run build:linux:arm64:onnx → npm run launch:linux |
| Linux x64 | npm run setup:linux → npm run verify:resources → npm run build:linux:x64 → npm run launch:linux |
npm run setup:linux:onnx → npm run verify:onnx:required → npm run build:linux:x64:onnx → npm run launch:linux |
| Backend-only/API | npm run setup → npm run backend:dev → curl http://127.0.0.1:8765/live |
ONNX validation optional unless using ONNX endpoints. |
| Docker | docker compose up --build → docker compose down |
Container usage follows the documented backend workflow. |
For full platform prerequisites, setup reports, diagnostics, resource verification, package-size preflight, and no-silent-download policy, read Setup and Build.
ONNX is optional for standard PyTorch usage. Standard builds do not require ONNX files. ONNX builds require local generation and validation of all requested ONNX model files before packaging.
ONNX export and benchmarking can use sustained CPU/GPU resources, high RAM, and large model files. Use a powerful, well-cooled machine with active cooling/fans, sufficient free disk space, and adequate memory. Avoid long export or benchmark runs on thermally constrained laptops or systems with limited cooling headroom.
| Document | What it contains |
|---|---|
| How Monocular Depth Estimation Works | Image preprocessing, PyTorch/ONNX inference, normalization, colorization, caching. |
| System Architecture Deep Dive | Mermaid diagram, layer responsibilities, request flow, concurrency model. |
| System Design Decisions | Caching, fallback, packaging, local-first tradeoffs, runtime policies. |
| Setup and Build | Platform setup, native builds, ONNX builds, diagnostics, resource verification. |
| Terminal-Only Development | Dev-mode verification without packaged apps. |
| Configuration | Environment variables and runtime settings. |
| API Reference | Full endpoint fields, curl examples, response shapes. |
| Models, Colormaps & Metrics | Model IDs, colormaps, metric modes, output interpretation. |
| Ground Truth Evaluation | GT formats, masking, scaling, median alignment. |
| Understanding Depth Metrics | Metric definitions and practical interpretation. |
| Testing and CI | Lightweight tests, stubs, CI policy, command matrix. |
| Production Packaging | Packaged startup readiness, native artifacts, Docker, ONNX variants. |
| Troubleshooting | Common setup/runtime failures and fixes. |
| Security | Local-first privacy, Electron isolation, process safeguards. |
| Project Structure | Full repository tree and module descriptions. |
| Debugging | Maintainer debugging notes. |
| Maintenance | Maintenance workflows. |
| Engineering Audit | Existing engineering audit notes. |
| Endpoint | Method | Purpose |
|---|---|---|
/ |
GET | Service name and API version. |
/live |
GET | Lightweight process liveness check used by packaged startup. |
/ready |
GET | Runtime readiness diagnostics; supports quick/deep checks. |
/health |
GET | Full backend health and dependency diagnostics. |
/estimate |
POST | Generate a depth map for one image with selected model/device/outputs. |
/batch |
POST | Process multiple images in one request. |
/compare |
POST | Run supported models on one image and compare outputs. |
/benchmark |
POST | Benchmark PyTorch against optional ONNX Runtime. |
/reconstruct |
POST | Generate approximate point-cloud data/export artifacts. |
/devices |
GET | Report detected compute devices. |
/onnx/status |
GET | Report ONNX file/provider readiness. |
/metrics |
GET | Prometheus metrics exposition. |
/api/observability, /observability |
GET | Local JSON telemetry snapshots for the UI. |
Read the full endpoint documentation in API Reference.
python -m py_compile backend/api/live.py backend/main.py
python -m pytest backend/tests/test_lightweight_live.py
npm --prefix electron-app test
npm run verify:resources
rg -n "4\\.0\\.0|3\\.1\\.0"The test strategy favors lightweight unit tests and stubs for Torch, ONNX Runtime, Redis, model downloads, and platform-specific packaging behavior. Read the full policy in Testing and CI.
| Area | Approach |
|---|---|
| Local-first processing | Images are sent only to the local backend on 127.0.0.1 unless the user intentionally configures otherwise. |
| Electron isolation | Context isolation and a narrow preload bridge limit renderer access to native APIs. |
| Navigation policy | The app allows local frontend files and localhost backend URLs only. |
| Backend lifecycle | Electron owns backend spawn, liveness polling, PID metadata, and controlled shutdown. |
| Telemetry hygiene | Observability avoids raw images, base64 payloads, filenames, local paths, image hashes, and cache keys. |
| GT/NPY safety | NPY ground-truth loading uses allow_pickle=False. |
Read the full security design in Security.
DepthLensPro/
├── backend/ # FastAPI app, routes, services, model registry, tests
├── electron-app/ # Electron main/preload code, packaging scripts, Electron tests
├── frontend/ # Browser UI, tabs, charts, 3D viewer, guide
├── docs/ # Deep-dive documentation, screenshots, architecture diagram
├── models/ # Local model/ONNX resource location; generated files are not committed
├── scripts/ # Setup, build, launch, and diagnostics scripts
└── docker-compose.yml
Read the full module map in Project Structure.
- Keep changes small and reviewable.
- Preserve existing API response shapes unless a breaking change is explicitly discussed.
- Prefer lightweight tests with mocks/stubs over real model downloads or GPU benchmarks.
- Update README and docs when setup, routes, runtime behavior, or security properties change.
- Run resource verification when touching Electron packaging scripts.
DepthLens Pro is licensed under the MIT License. See LICENSE for the full terms.
DepthLens Pro builds on excellent open-source projects:
| Project | Role |
|---|---|
| Intel ISL MiDaS | MiDaS/DPT monocular depth estimation models |
| PyTorch | Primary ML runtime and Torch Hub model loading |
| ONNX Runtime | Optional accelerated inference across CPU, CUDA, CoreML, OpenVINO |
| FastAPI | Local HTTP API with async support and automatic OpenAPI docs |
| Electron | Desktop application shell with context isolation |
| OpenCV | Image decoding, resizing, colourisation, and GT alignment |
| NumPy | Depth array arithmetic and GT metric computation |
| Pillow | PNG/TIFF/NPY GT file decoding |
| Redis | Optional distributed cache backend |
| Chart.js | Latency and benchmark charts in the frontend |
Made with care by Ayushman Raha
com.ayushmanraha.depthlens-pro