Repository navigation
[Other] linux-aarch64 prebuilt engine package: Jetson AGX Thor (SM110) builds and benchmarks from source #4559
Description
Concrete user need
Jetson AGX Thor (GB10-class aarch64 + NVIDIA) can build and run the Strata engine from source today,
but installing it as a user cannot:
install.shrefuses ARM Linux outright ("this computer ... has an ARM processor").- The release folder ships no
linux-aarch64row ininstall-manifest.txt. - The desktop AppImage is
linux-x86_64only.
So every ARM user must clone the full repo, install a CUDA toolkit and compile for ~5 minutes, even though
setup.py already has an ARM64 branch (a.build = True, NEON CPU experts). A prebuilt linux-aarch64
engine package would remove the compiler requirement, matching what macos-arm64 already gets.
Environment (observed)
| Item | Value |
|---|---|
| Hardware | NVIDIA Jetson AGX Thor Developer Kit |
| GPU | NVIDIA Thor, compute capability 11.0 (sm_110), 20 SMs |
| Memory | 122.8 GiB unified (RAM = VRAM) |
| SoC | L4T R38 rev 4.0, kernel 6.8.12-tegra |
| Driver / CUDA | 580.00 / CUDA 13.0 (nvcc 13.0.48, sbsa-linux) |
| OS | Ubuntu 24.04.3 LTS, aarch64 |
| Strata | source checkout 5e8a4cc (v0.2.7), engine 0.1.52 |
| gcc / cmake | 13.3.0 / 4.4.4 |
What was verified
Passed
- Source build for sm_110:
cmake -S . -B build -DSTRATA_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=110 && cmake --build build --target strata -j7
→[157/157]link,strata 0.1.52. nvcc usedarch=compute_110,code=[compute_110,sm_110]and
-isystem /usr/local/cuda/targets/sbsa-linux/include(SBSA / ARM64 Linux). - Engine GPU probe:
PCIe probe: 117.3 GB/s host->device(unified memory). setup.py --check: GPU detected (compute_cap 11.0), unified-memory falls back to/proc/meminfo,
ARM64 branch warns and setsa.build = Trueas designed.- 11/11 kernel parity tests on sm_110 (see
[Benchmark]issue), includingiq_parityIQ3_S dequant bit-exact. - End-to-end IQ3_S inference, OpenAI + Anthropic endpoints.
- Standard benchmark complete (see
[Benchmark]issue).
Not run: vision input (--vision no this run); concurrency stage.
Proposed change (scope for discussion)
- Add
linux-aarch64to the release build matrix (engine fornativeper board; the release could ship
110for Thor/GB10-class, plus a generic SBSA build). - Add the
linux-aarch64row toinstall-manifest.txt. - In
install.sh, replace the ARM Linuxdiewith thelinux-aarch64platform when the feed has that row
(keep the message for feeds that do not). - Document Jetson Thor / GB10 in
docs/INSTALL.md's hardware table.
I can test on this Thor and review a packaging PR if that direction is wanted.
Reproduction status: reproduced locally (build + run + benchmark on the machine above).
Activity
potoior commentedon Oct 29, 2026
Update / refinement after checking the current v0.2.7 release.
The ARM install gap I described originally is partly already fixed in v0.2.7:
install-manifest.txtnow has alinux-aarch64row (control 0.1.53).install.shmapsLinux-aarch64|Linux-arm64toplatform=linux-aarch64, and warns only for non-GB10 GPUs: "Strata's engine for ARM Linux is built and tested on the NVIDIA DGX Spark (GB10) only... 'strata launch' may find no engine for it."The remaining, sharper gap is the GPU architecture:
strata-feed-linux-aarch64.json(v0.2.7) declares the aarch64 engines as sm_121 only:DGX Spark is GB10 / SM121. The Jetson AGX Thor is a different silicon, SM110 (
nvidia-smicompute_cap 11.0). CUDA cannot runsm_121/compute_121code or PTX onsm_110, so on Thor the Terminal installs but there is no engine to launch.That the code does support SM110 is proven here: a source build with
-DCMAKE_CUDA_ARCHITECTURES=110(nvccarch=compute_110,code=[compute_110,sm_110],sbsa-linuxheaders) passes 11/11 kernel parity tests and the standard benchmark (#4560, PR #4565).Refined request:
sm_110to the aarch64 engine build matrix (Qwen engine andstrata-dsv41) — or ship a separatesm_110aarch64 package and letstrata launchpick it (the feed already carriesarchs, so this is a build/packaging change, not new engine code).Environment: Jetson AGX Thor Dev Kit, SM110, 122.8 GiB unified, L4T R38, driver 580.00, CUDA 13.0, Ubuntu 24.04 aarch64; Strata
5e8a4cc, engine 0.1.52 source build.Reproduction status: reproduced locally (source build + run + benchmark on the machine above).