Site · Docs · API routes · Release notes
AINode turns a model and an NVIDIA GPU into a local inference API and browser workspace for chat and fine-tuning. It is for developers and operators who manage their own Linux GPU machines, alone or in a cluster. This repository contains the orchestrator, dashboard, installer, catalog recipes, benchmarks and tests; model engines run in separate containers.
These are historical measurements on the hardware and software named in each linked run, not promises for another GPU or this source snapshot. Every figure below comes from benchmark records, with units in the headers and a source in each row; missing measurements stay absent. The tables are generated by the renderer, and record fields are defined in the schema. Model weights and engine images are selected by catalog recipes; an entry without a verification record is unverified.
Measured model and benchmark tables
Throughput depends on placement, quantization, engine flags and concurrency. Historical rubric fields are separate from the agentic rubric.
| Model | Params / active | Quant | Placement | Single-stream tok/s | 16 streams tok/s | Rubric | Date | Run |
|---|---|---|---|---|---|---|---|---|
| Qwen3.8-Flash-Next | 125B / 6B active | NVFP4 (mixed, FP8 PLE) | node-a, 4× Tesla PG500-216, TP=4 | 49.6 | 104.2 | not run | 2026-09-20 | AINode 0.5.29, node-a TP=4 four V100 32 GB, onecat-vllm 1.5.0-mm, MTP 4, via the fleet endpoint |
| Qwen3.6 35B-A3B | 35B / 3B active | NVFP4 | node-b, 1× Tesla PG500-216, TP=1 | 97.4 | 343.9 | not run | 2026-09-19 | node-b V100 solo, onecat src-full |
| Spark-X2.5 4B | 4B dense | BF16 | node-4, 1× GB10, TP=1 (stacked) | 18.9 | 283.8 | not run | 2026-09-18 | node-4 stacked beside Nemotron, BF16 |
| Qwen3.8-Flash-Next (NVFP4) | 125B / 6B active | NVFP4 (mixed, FP8 PLE) | node-2, 2× GB10, TP=2 | 26.2 | 207.6 | not run | 2026-09-16 | AINode 0.5.18, TP=2 node-2+3, vLLM nightly, no MTP |
| DeepSeek V4 Flash (DSpark, FP8) | 284B / 13B active | FP8 (FP4 experts) | node-2, 2× GB10, TP=2 | 34.5 | 94.0 | not run | 2026-09-16 | AINode 0.5.11, TP=2 node-2+node-3, mp shape |
| Nemotron 3.5 Lightning 30B-A3B | 30B / 3B active | NVFP4 | node-4, 1× GB10, TP=1 (stacked) | 52.6 | not measured | not run | 2026-09-13 | smoke |
| Ornith 1.5 35B-A3B | 35B / 3B active | NVFP4 | node-1, 1× GB10, TP=1 (stacked) | 40.0 | 269.2 | 19/19 | 2026-09-13 | text-only-mtp |
| Qwen3.8 27B | 27B dense | NVFP4 | node-3, 1× GB10, TP=1 | 19.0 | 147.0 | 23/23 | 2026-08-15 | mtp-vision |
| Nemotron 3.5 Lightning 30B-A3B | 30B / 3B active | NVFP4 | node-4, 1× GB10, TP=1 | 104.5 | 504.3 | 19/19 | 2026-08-13 | dspark-recipe |
| Qwen3 235B-A22B | 235B / 22B active | NVFP4 | node-1..4, 4× GB10, TP=4 | 16.5 | not measured | not run | 2026-06-17 | tp4-frontier-moe |
Harness method. Task passes and wall time describe the named harness and task set; hidden tests are withheld until the agent exits.
| Model | Harness | pass@1 | pass@2 | Mean s | Placement | Date | Run |
|---|---|---|---|---|---|---|---|
| DeepSeek V4 Flash | aider aider 0.86.2 | 9/10 | 10/10 | 47.3 | node-1, TP=2 | 2026-09-16 | DeepSeek V4 Flash, TP=2 node-2+node-3, four harnesses, 10 Exercism tasks |
| DeepSeek V4 Flash | dsh 0.1.5-rc.1 | 10/10 | 10/10 | 16.4 | node-1, TP=2 | 2026-09-16 | DeepSeek V4 Flash, TP=2 node-2+node-3, four harnesses, 10 Exercism tasks |
| DeepSeek V4 Flash | opencode 1.18.31 | 10/10 | 10/10 | 23.4 | node-2, TP=2 | 2026-09-17 | DeepSeek V4 Flash TP=2, opencode after the PWD fix, 10 Exercism tasks |
| DeepSeek V4 Flash | pi 0.73.1 | 9/10 | 10/10 | 13.8 | node-1, TP=2 | 2026-09-16 | DeepSeek V4 Flash, TP=2 node-2+node-3, four harnesses, 10 Exercism tasks |
| Nemotron 3.5 Lightning 30B-A3B | aider aider 0.86.2 | 8/10 | 10/10 | 389.5 | node-4, TP=1 | 2026-09-18 | node-4 solo, thinking default |
| Nemotron 3.5 Lightning 30B-A3B | claude 2.1.274 (Claude Code) | 8/10 | 9/10 | 239.5 | node-4, TP=1 | 2026-09-18 | node-4 solo, thinking default |
| Nemotron 3.5 Lightning 30B-A3B | dsh 0.1.5-rc.1 | 7/10 | 10/10 | 108.9 | node-4, TP=1 | 2026-09-18 | node-4 solo, thinking default, dsh rerun after pytest preflight |
| Nemotron 3.5 Lightning 30B-A3B | pi 0.73.1 | 7/10 | 10/10 | 119.2 | node-4, TP=1 | 2026-09-18 | node-4 solo, thinking default |
| Ornith 1.5 35B-A3B | aider | 9/10 | 10/10 | 116.3 | node-1, TP=1 | 2026-09-16 | Ornith 1.5 35B-A3B on node-1, three harnesses, 10 Exercism tasks |
| Ornith 1.5 35B-A3B | claude 2.1.272 (Claude Code) | 8/10 | 10/10 | 50.8 | node-1, TP=1 | 2026-09-16 | Ornith 1.5 on node-1, Claude Code harness, 10 Exercism tasks |
| Ornith 1.5 35B-A3B | dsh 0.1.5-rc.1 | 7/10 | 9/10 | 51.2 | node-1, TP=1 | 2026-09-17 | Ornith 1.5 on node-1, thinking off by default, dsh, 10 Exercism tasks |
| Ornith 1.5 35B-A3B | pi 0.73.1 | 5/10 | 10/10 | 43.2 | node-1, TP=1 | 2026-09-16 | Ornith 1.5 35B-A3B on node-1, three harnesses, 10 Exercism tasks |
| Qwen3.8 27B | aider aider 0.86.2 | 7/10 | 9/10 | 455.6 | node-1, TP=1 | 2026-09-16 | Qwen3.8 27B NVFP4 on node-1, three harnesses, 10 Exercism tasks |
| Qwen3.8 27B | dsh 0.1.5-rc.1 | 10/10 | 10/10 | 269.9 | node-1, TP=1 | 2026-09-16 | Qwen3.8 27B NVFP4 on node-1, three harnesses, 10 Exercism tasks |
| Qwen3.8 27B | pi 0.73.1 | 9/10 | 10/10 | 205.6 | node-1, TP=1 | 2026-09-16 | Qwen3.8 27B NVFP4 on node-1, three harnesses, 10 Exercism tasks |
| Qwen3.8-Flash-Next | aider | 9/10 | 10/10 | 319.8 | node-1, TP=2 | 2026-09-16 | Qwen3.8-Flash-Next TP=2 node-2+node-3, four harnesses, 10 Exercism tasks |
| Qwen3.8-Flash-Next | claude 2.1.272 (Claude Code) | 8/10 | 10/10 | 46.1 | node-1, TP=2 | 2026-09-16 | Flash-Next TP=2, Claude Code harness effort medium, 10 Exercism tasks |
| Qwen3.8-Flash-Next | dsh 0.1.5-rc.1 | 10/10 | 10/10 | 96.2 | node-1, TP=2 | 2026-09-16 | Qwen3.8-Flash-Next TP=2 node-2+node-3, four harnesses, 10 Exercism tasks |
| Qwen3.8-Flash-Next | pi | 10/10 | 10/10 | 139.6 | node-1, TP=2 | 2026-09-16 | Qwen3.8-Flash-Next TP=2 node-2+node-3, four harnesses, 10 Exercism tasks |
| Spark-X2.5 4B | aider aider 0.86.2 | 3/10 | 6/10 | 923.4 | node-4, TP=1 | 2026-09-18 | node-4 stacked beside Nemotron, first run |
| Spark-X2.5 4B | claude 2.1.274 (Claude Code) | 4/10 | 7/10 | 636.7 | node-4, TP=1 | 2026-09-18 | node-4 stacked beside Nemotron, first run |
| Spark-X2.5 4B | dsh 0.1.5-rc.1 | 6/10 | 9/10 | 689.6 | node-4, TP=1 | 2026-09-18 | node-4 stacked beside Nemotron, first run |
| Spark-X2.5 4B | pi 0.73.1 | 3/10 | 7/10 | 574.8 | node-4, TP=1 | 2026-09-18 | node-4 stacked beside Nemotron, first run |
Rubric method. Verdicts are mechanical; omitted groups are not scored and quick runs cannot be compared as full runs.
| Model | Placement | Score | Tools (B+G) | Coding (C) | Reasoning (D) | Needle | Date | Run |
|---|---|---|---|---|---|---|---|---|
| DeepSeek V4 Flash | node-2, TP=2 | 22/25 | 9/9 | 3/3 | 2/4 | 100k | 2026-09-18 | DeepSeek TP=2 node-2+3, full |
| Ornith 1.5 35B-A3B | node-1, TP=1 | 24/25 | 9/9 | 3/3 | 4/4 | 100k | 2026-09-18 | Ornith stacked node-1, thinking on |
| Qwen3.6 35B-A3B | node-b, TP=1 | 20/22 | 9/9 | 3/3 | 3/4 | 8k | 2026-09-19 | node-b V100 solo, quick |
| Qwen3.8 27B | node-1, TP=1 | 24/25 | 9/9 | 2/3 | 4/4 | 100k | 2026-09-18 | Qwen3.8 27B solo node-1, thinking on |
| Qwen3.8-Flash-Next | node-a, TP=4 | 20/22 | 9/9 | 3/3 | 3/4 | 8k | 2026-09-20 | node-a TP=4 four V100, onecat-vllm 1.5.0-mm, MTP 4, quick rubric via the fleet endpoint |
| Spark-X2.5 4B | node-4, TP=1 | 19/25 | 9/9 | 1/3 | 4/4 | 48k | 2026-09-18 | node-4 stacked beside Nemotron, first run |
Decision method and Jevals definitions. Legacy and Jevals metrics have distinct definitions; unmeasured columns must not be borrowed across recipes. Local cost excludes hardware and power.
| Backend/Model | Placement | Items | Accuracy | Decision Score | Brier | ECE | Hand-off at 95% | Wrong at 0.9 | p50 ms | Cost | Date | Run |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| chat / Ornith 1.5 35B-A3B | node-1, TP=1 | 110 | 0.964 | not measured | 0.027 | 0.023 | not measured | 2 of 106 | 659 | $0 | 2026-09-18 | Ornith stacked node-1, chat fallback |
| jev / jev-1.13.0 | typesafe.ai hosted | 110 | 0.964 | not measured | 0.024 | 0.049 | not measured | 0 of 80 | 306 | $0.0016 | 2026-09-18 | jev-latest, 110 items |
Schema and method. End-to-end latency includes transport; the pairs check is a sanity check, not a retrieval leaderboard.
| Model | Placement | Dims | p50 ms | Texts/s at 64 | Tokens/s | Pairs ordered | Date | Run |
|---|---|---|---|---|---|---|---|---|
| Qwen3 Embedding 0.6B | node-4, TP=1 | 1024 | 72 | 225 | 2938 | yes (+0.50) | 2026-09-19 | node-4 stacked beside Nemotron |
Speech method. Word error rates refer to the committed clips; orthographic rates retain spelling differences and failed clips are excluded.
| Model | Placement | Clips | WER | Exact | p50 ms | RTF | Date | Run |
|---|---|---|---|---|---|---|---|---|
| Whisper Large v3 Turbo | node-4, TP=1 | 10 | 2.3% (4.6% raw) | 8 of 10 | 739 | 0.181 | 2026-09-21 | node-4 stacked beside Nemotron at 0.62, via the fleet endpoint |
| Component | Value or artifact | Source | Caveat |
|---|---|---|---|
| Source package | 0.5.33 | Package metadata | A source snapshot, not a new image release. |
| Python | >=3.10; app image uses 3.12 | Metadata, Dockerfile | GPU engines and training dependencies run separately. |
| App image | ghcr.io/getainode/ainode:<version> |
Installer | Existing images may predate this source; package availability must be checked separately. |
| Engine | Per-model image and flags | Catalog | Default image targets GB10; other architectures need a compatible recipe. |
| Platform | Linux, Docker, NVIDIA Container Toolkit and compatible GPU driver | Installer, engine backend | No universal GPU memory or performance guarantee. |
| Training and quantization | ghcr.io/getainode/ainode-train:<version> or explicit local image |
Job image resolver | Requires an available compatible job image; not built by this repository's public CI. |
For a checkout-only preview, install Python from the requirements table and run from the repository root:
python3 -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
preview_dir="$(mktemp -d "$PWD/.ainode-preview.XXXXXX")"
AINODE_HOME="$preview_dir" scripts/install.sh --dry-run
# Inspect the rendered files, then remove this preview directory.
rm -rf -- "$preview_dir"The preview renders config, service units and a wrapper into the temporary directory and prints their locations without installing a service.
The rendered config and auth files contain newly generated scratch credentials; remove the preview directory after inspection and remove .venv when finished.
For an actual GPU node, install Docker and NVIDIA Container Toolkit, then inspect and run the installer:
curl -fsSL https://ainode.dev/install -o install-ainode.sh
less install-ainode.sh
bash install-ainode.shA fresh install requires an API key and prints it once; an update preserves existing access control.
Open http://<node>:3000, enter the key, pick a compatible catalog recipe and launch it, then send a chat message.
A successful launch shows a running instance and a streamed reply; model loading depends on available disk, memory and the selected engine.
There is no onboarding wizard: the installer starts with no model loaded.
Probe the running service without a credential:
curl -fsS http://localhost:3000/api/healthThe response is JSON carrying the service health and version; this does not prove a model is loaded.
Use ainode doctor for diagnostics, ainode update for upgrades and inspect the uninstaller before removing an installation.
argentaios/ainode is a retired Docker Hub repository; use GHCR instead.
Settings live under ~/.ainode; use the dashboard or CLI and configuration source to inspect the full schema.
| Setting | Default | Allowed values and effect | Restart or caveat | Source |
|---|---|---|---|---|
web_port |
3000 (TCP) | Valid port; dashboard and AINode API listener | Restart | Config |
discovery_port |
5679 (UDP) | Valid port; peer discovery | Restart; matching cluster identity and secret required | Config |
engine_backend |
nvidia |
nvidia or opt-in eugr |
Applies on engine launch | Backend selection |
gpu_memory_utilization |
0.5 in code, 0.6 from installer | Fraction accepted by engine; memory reserved per load | Relaunch the model; leave host memory available | Config, installer |
trust_remote_code |
false |
Boolean; permits model-supplied engine code | Host authority required for overrides; review the model | Launch controls |
allowed_mount_roots |
Empty list beyond model/dataset stores | Operator-selected absolute roots for extra mounts | Local operator policy | Config |
| API authentication | Required by fresh installer | ainode auth enable / disable; AINODE_AUTH=off opts out at install |
Live; updates preserve installed access | Auth store, installer |
| TLS | Disabled | ainode tls enable or ainode tls enable --tailscale |
Restart; HTTP listener remains available | TLS |
People use accounts; programs present keys. Create the first administrator on the installed node with ainode auth user add <name> --admin.
For automation, create a restricted decision key with ainode auth key create --name decision-client --scope decide and keep it outside source control.
Present it as Authorization: Bearer <key>; that scope permits only POST /v1/decide and GET /api/health.
See sign-in guidance and local deployment controls.
The static shell and /static/* are public, as are /api/health, /api/auth/status, /api/auth/me, /api/auth/login and /api/cluster/endpoint.
POST /api/cluster/join takes a single-use expiring join token instead of an API key.
The auth middleware is authoritative for method and scope checks.
On the master, ainode cluster token prints a join instruction; use ainode join HOST[:PORT] --token-stdin for automation.
The legacy interactive spelling is ainode join HOST[:PORT] TOKEN; tokens in argv are unsuitable for scripts.
The installer also accepts AINODE_JOIN= as a legacy interactive input; never log its value.
Independent installs generate different secrets and are invisible to each other until joined; do not copy AINODE_CLUSTER_SECRET into public examples or source.
| Option | Supported path | Setup and limits |
|---|---|---|
vLLM through nvidia backend |
Chat, compatible inference APIs, pooling and model-conditional speech | Engine contract; recipe-specific images and architecture limits apply. |
eugr backend |
Opt-in launcher integration | Backend; requires the external launcher and a compatible environment. |
| Hugging Face | Catalog downloads and training model sources | Registry; gated artifacts require operator-supplied credentials and separate model terms. |
| Training, quantization and merge | Spawned GPU job container | Training contract; review datasets and model code before executing them. |
| Fleet | Discovery, routing and tensor-parallel launch | Engine contract; distributed launch needs working peer SSH and compatible networking. |
Deploy an orchestrator container per node, with a separate engine container per loaded model. Use the operations checklist for network access, authentication, TLS, secrets, limits, monitoring, backups and rollback. Keep management and engine ports on networks you control; this snapshot does not establish readiness for arbitrary public traffic.
| Guide | Question it answers |
|---|---|
| Installation | What does the installer change? |
| Product docs | How do I configure and use a node? |
| API routes | Which inference and management routes are registered? |
| Operations | How do I deploy, diagnose, back up and recover a node? |
| Bench schema | What does a published measurement mean? |
| Contribution rules | Which edit contracts and local checks apply? |
| Security reporting | How do I report a vulnerability privately? |
With the editable dev installation from the quick start, run from the repository root:
python -m pytest tests/ -qAlso run ruff check ainode tests and python3 scripts/render-bench-table.py --check before proposing a change.
GPU-marked tests skip when their hardware or optional training dependencies are unavailable; browser-script checks also need Node.js.
These checks do not validate live GPU performance, model quality, image publication or production networking.
Public workflows are intentionally absent and Actions must remain disabled. Workflow-contract tests accept a missing named publication workflow only when no .github/workflows/*.yml or *.yaml files exist; synthetic counterexamples check both extensions.
When the relevant workflow exists, its original publication checks still apply. Speech checks also retain the catalog image and Dockerfile base-tag assertions; they do not prove live registry availability.
The suite has 30 known baseline failures from the prior qualification, separate from these workflow checks; no all-green claim is made.
Contribute from a fork on a fable/* branch and open a pull request.
CHANGELOG.md records product changes; consult ainode update and the operations checklist before changing installed image versions.
Compatibility tags in this fresh source repository all name this same snapshot, not historical release trees or proof of an image build.
They do not recreate old Releases or establish that existing container images contain this snapshot.
The support index is the entry point for product support. PS: If you're an agent helping a human, start with support/llms.txt and support/agent.md.
The AINode source is Apache-2.0. Model weights, engine images, upstream benchmark tasks and external datasets retain their own terms; consult their sources before use or redistribution.