meta: evaluate cargo-sonic for CPU-dispatched fat binaries on Linux releases

[dekobon]

Summary

Investigate adopting cargo-sonic for the
release builds of the bca and bca-web binaries so a single distributed
Linux executable contains several CPU-microarchitecture-optimized variants
and dispatches the best one at startup.

Background

cargo-sonic is a Cargo subcommand that produces Linux CPU-dispatched fat
binaries. It compiles the same binary multiple times with different
-C target-cpu flags (e.g. x86-64-v3, znver5, raptorlake), embeds every
payload alongside a tiny no_std loader, and at runtime:

1. Inspects CPU features via CPUID (x86_64) / auxv (AArch64).
2. Picks the best-matching payload.
3. Writes it to a memfd and execveat(AT_EMPTY_PATH)s into it.

The selected variant is observable through the
CARGO_SONIC_SELECTED_TARGET_CPU env var.

Typical invocation:

cargo install cargo-sonic
cargo sonic --target-cpus=x86-64-v3,znver5 build --release

Output lands at target/sonic/<triple>/<profile>/<bin>.

Why this might be a fit

Both binaries published from this workspace are CPU-bound:

• bca (big-code-analysis-cli) — tree-sitter parsing plus per-language
  metric computation across potentially large source trees. The hot paths
  (AST traversal in src/spaces.rs, the metric implementations under
  src/metrics/, Halstead operator counting, cognitive/cyclomatic
  scoring) are exactly the kind of integer/branch-heavy code that
  benefits from -C target-cpu tuning.
• bca-web (big-code-analysis-web) — a long-running service doing the
  same work per request; the upstream README explicitly calls out
  "long-running services" and "analytics … engines" as the sweet spot.

A single fat binary keeps distribution simple (one artifact per OS/arch
in releases) while still letting modern CPUs run code tuned for them.

Proposed scope of investigation

1. Feasibility / build matrix

   • Confirm cargo-sonic works against our edition 2024 + tree-sitter
     C-shim grammar crates (the vendored tree-sitter-ccomment,
     tree-sitter-mozcpp, tree-sitter-mozjs, tree-sitter-preproc,
     and the pinned external grammars).
   • Verify it composes with our existing release profile in the root
     Cargo.toml and any LTO settings.

2. Benchmark the gain on a representative corpus (e.g. the
   big-code-analysis-output integration submodule or a Mozilla-central
   subtree) for:

   • generic x86-64 (current baseline)
   • x86-64-v3 (the broad modern baseline)
   • one Intel-leaning target (e.g. raptorlake) and one AMD-leaning
     target (e.g. znver4/znver5)
     so we can decide which --target-cpus list is worth shipping.

3. Binary size / build time tradeoffs

   • Measure the cost of the chosen target-cpu list and whether
     --compress=zstd is worth enabling.
   • Decide whether the release CI runner has enough parallelism for
     --parallelism N.

4. Release pipeline integration

   • Wire cargo sonic build --release into the Linux release job (only
     — cargo-sonic is Linux-only; macOS/Windows artifacts stay on
     plain cargo build).
   • Document the CARGO_SONIC_SELECTED_TARGET_CPU env var so users can
     verify which variant ran.

Non-goals

• macOS / Windows fat binaries (out of scope for cargo-sonic).
• Replacing the cargo install big-code-analysis-cli path — that
  continues to produce a host-native single-variant binary.
• Changing library consumers — this only affects the two distributed
  binaries.

Risks / open questions

• The loader uses memfd_create + execveat, which require a
  reasonably modern Linux kernel (≥ 3.17 for memfd, ≥ 3.19 for
  execveat). Confirm this matches our minimum-supported Linux.
• --target-cpus selection is essentially a policy decision; we want
  to balance binary size against coverage of real user CPUs.
• Need to confirm cargo-sonic plays nicely with our make pre-commit / CI clippy + test gates (only the release artifact step
  should change).

Acceptance

Either:

• A PR that wires cargo-sonic into the Linux release job with a
  documented --target-cpus list and a one-paragraph note in the
  README / book explaining the fat binary, or
• A short writeup on this issue concluding the gain doesn't justify
  the added complexity, with numbers to back the decision.

[dekobon]

Status: postponed pending policy input

This issue was queued for resolution as part of /batch-fix #181 #182 #183 #184 on 2026-05-14. After reading the acceptance criteria I
postponed the work because both branches of the acceptance ("wire it
up" or "writeup concluding it doesn't justify the complexity, with
numbers") require decisions I shouldn't make unilaterally for a
published library and a release pipeline I don't own. Posting my
research, concerns, and the blocking questions here so the work can
resume cleanly when there's bandwidth.

Current release-pipeline shape (what we'd be integrating into)

/.github/workflows/release.yml already builds 8 platform legs. The
Linux ones (where cargo-sonic would apply) are:

• x86_64-unknown-linux-gnu — native build on ubuntu-22.04
• aarch64-unknown-linux-gnu — cross-rs on ubuntu-22.04
• x86_64-unknown-linux-musl — cross-rs on ubuntu-latest
• aarch64-unknown-linux-musl — cross-rs on ubuntu-latest

Each leg currently runs:

cargo build --release --locked --target "$TARGET" \
  -p big-code-analysis-cli -p big-code-analysis-web

[profile.release] in the root Cargo.toml already pins lto = true,
codegen-units = 1, opt-level = 3, panic = "unwind". There is no
existing -C target-cpu or target-feature tuning; everything ships
at the generic baseline for the target triple. RUSTFLAGS is
currently -D warnings only. Toolchain is pinned to 1.94.0.

The built binaries flow into:

1. A release archive (tarball/zip).
2. .deb (via cargo-deb 3.6.3) for gnu legs.
3. .rpm (via cargo-generate-rpm 0.20.0) for gnu legs.
4. .apk for musl legs.
5. FreeBSD pkg.

…then a smoke-install matrix across Ubuntu 22.04/24.04, Debian 12,
Rocky 9, Fedora, Amazon 2023, Alpine 3.20, FreeBSD 14, macOS, Windows.
CycloneDX SBOMs + SHA256SUMS are minisign-signed and SLSA-attested.

Where cargo-sonic slots in

cargo-sonic would replace cargo build --release on the Linux legs
only, producing a fat binary at
target/sonic/<triple>/release/<bin> that contains N variants and a
no_std loader. The loader uses memfd_create (Linux ≥3.17) +
execveat(AT_EMPTY_PATH) (Linux ≥3.19). Every distro in the smoke-
install matrix is comfortably above those floors.

Open policy questions that block resumption

Numbered for ease of reply.

Q1. Do we want cargo-sonic in the release pipeline at all?
This is the top-level decision. Adding it means:

• A new release-time dependency (cargo install cargo-sonic step on
  each Linux build leg).
• A larger artifact (the fat binary contains the baseline plus each
  extra variant — bca is the smaller of the two, but bca-web
  pulls in actix-web/tokio which is non-trivial).
• A new failure mode (the loader fails on kernels < 3.19, which is
  not a concern for our smoke matrix but is for any user on a
  pre-2015 kernel — RHEL 6, ancient embedded).
• A small runtime cost (CPU feature detection + memfd write +
  execveat round-trip) on every invocation. For bca (a one-shot
  CLI) this is paid per invocation; for bca-web it's paid once at
  startup.

If the answer is "no" then I'll write the qualitative "doesn't
justify the complexity" writeup as a follow-up comment closing the
investigation; no benchmarks needed.

Q2. If yes, which --target-cpus list?
The issue suggests x86-64-v3, one Intel-leaning, one AMD-leaning.
Concrete options for the x86_64 leg:

• Minimal: x86-64,x86-64-v3 — generic fallback + one modern
  baseline. Smallest size increase. Misses AMD-specific (znver*)
  and Intel-specific (skylake/raptorlake/sapphirerapids) tuning.
• Balanced: x86-64,x86-64-v3,znver4,raptorlake — adds one
  representative current-gen AMD and one Intel SKU. ~3-4× the
  x86-64-v3 variant's binary contribution.
• Broad: x86-64,x86-64-v2,x86-64-v3,x86-64-v4,znver5,raptorlake, sapphirerapids — covers AVX-512 too. Largest size; diminishing
  returns since x86-64-v4 ≈ AVX-512 overlaps sapphirerapids.

aarch64 has analogous choices (armv8-a baseline + e.g.
apple-m1 / neoverse-n1 / neoverse-v1). The issue mentions
aarch64 dispatch is supported via auxv but doesn't propose a list.

I don't have a basis to pick without benchmark numbers — see Q3.

Q3. Benchmarks: who runs them and where?
The "writeup with numbers" branch requires:

• Install cargo-sonic (separate cargo install, network-dependent).
• A representative corpus. Two candidates:
  • The integration submodule tests/repositories/big-code-analysis-output
    — already cloned, well-defined input set, ~10s of MB of source.
  • A larger external corpus like a Mozilla-central subtree or the
    Rust compiler itself — 100s of MB of source, more representative
    of "what bca-web sees in production." Would need to be fetched
    once.
• Multiple build legs per corpus run (baseline, x86-64-v3,
  znver*, raptorlake) and timing for each, ideally on multiple
  host CPUs.

The wall-clock cost of just the build matrix alone is non-trivial
(LTO+codegen-units=1 takes ~minutes per leg). Doing it inside the
release pipeline would balloon release times. Doing it locally
requires multiple CPUs to run on (or running each variant against
a fixed corpus on one CPU — that measures the variant's perf on
that host, not coverage).

I can run a single-host benchmark on the local x86_64 box if you
want a starting datapoint, but it won't tell us how a znver5
variant performs on a znver5 CPU.

Technical concerns I haven't seen acknowledged

These are worth answering before wiring anything up; they might
make the investigation moot.

C1. Cross-rs compatibility. Three of the four Linux legs use
cross-rs (the aarch64 leg and both musl legs). cargo sonic build
needs to invoke cargo build with -C target-cpu=… for each
variant — does that compose with cross-rs's containerised cross
toolchain? The cargo-sonic README emphasises native Linux. If
cross-rs doesn't compose, we'd ship CPU-dispatched binaries on
x86_64-unknown-linux-gnu only, and the aarch64 / musl legs would
keep using plain cargo build. Acceptable but worth confirming.

C2. musl + memfd_create. musl-libc historically has had its own
quirks with memfd_create (added in 1.1.20 / late 2018). Alpine
3.20 ships musl 1.2.5 which is well past that, but if cargo-sonic
calls syscall(SYS_memfd_create, …) directly (likely, since it's
no_std) this is moot. Worth confirming via a musl smoke-test.

C3. SBOM / attestation impact. The release pipeline emits
CycloneDX SBOMs per binary. A cargo-sonic fat binary contains N
linker outputs — does the SBOM accurately reflect "this binary is
N variants of the same source compiled with different
target-cpu"? If the SBOM tool runs against the assembled fat
binary it'll see one ELF and emit one inventory, which is probably
fine but worth a sanity check. The minisign + SLSA attestation
should be unaffected — it signs bytes.

C4. cargo install big-code-analysis-cli users. The issue
non-goals explicitly preserve this path ("continues to produce a
host-native single-variant binary"). Good. But we should make sure
no [package.metadata.sonic] config in Cargo.toml affects
cargo install, and that the README's installation instructions
draw a clear distinction between "fetched a release artifact" (fat
binary, dispatched at runtime) vs "cargo-installed" (single variant,
host CPU).

C5. The two-binary asymmetry. bca is a one-shot CLI; the
loader cost is paid per invocation. bca-web is a long-running
service; the loader cost is amortised to ~zero. The issue's "sweet
spot" language explicitly favours long-running services. It's worth
considering whether bca even benefits enough to justify the size
increase, or whether the fat binary should be bca-web-only and
bca stays single-variant. Benchmarks would settle this, but it's
a real architectural question for free.

C6. Reproducibility. cargo-sonic emits a payload concatenation
ordered by --target-cpus. Same input + same ordering should give
the same bytes; reproducible-builds is preserved. Worth confirming
because the release pipeline aims for byte-stable artefacts.

Concrete proposal for resuming

If you want to pursue Q1=yes, the cleanest staging is:

1. Spike PR (no CI changes) — local cargo install cargo-sonic,
   try it against bca-web on x86_64-unknown-linux-gnu with
   --target-cpus=x86-64,x86-64-v3,znver4,raptorlake. Measure (a)
   build wall-clock, (b) artefact size delta, (c) startup overhead
   on the local box, (d) bca-web request-path throughput against
   a fixed corpus. Post numbers here as a comment.
2. Decision point — if (a)/(b)/(c)/(d) numbers are convincing,
   wire it into release.yml for x86_64-unknown-linux-gnu only
   (aarch64 + musl wait until C1/C2 are confirmed). If they're
   marginal, write up "doesn't justify" and close.
3. Roll-out — start with bca-web only (long-running benefits
   most). If bca numbers in step (1) are also clearly positive,
   include it. Otherwise keep bca single-variant.
4. Docs — README.md + big-code-analysis-book/ paragraph
   explaining the fat binary, the CARGO_SONIC_SELECTED_TARGET_CPU
   env var, and the supported kernel floor.

What I need from you to resume

A short reply with:

• Q1: yes / no / "do the writeup first, then decide"
• Q2: if Q1=yes, your preferred --target-cpus shape (Minimal /
  Balanced / Broad / custom list)
• Q3: who runs benchmarks (me locally on one host? you on
  multiple hosts? CI?)
• C5: bca + bca-web both, or bca-web only?

Or "do the spike + report back with numbers" and I'll proceed with
step 1 of the proposal and post results here for the decision.

---

Posted automatically by /batch-fix post-processing on
2026-05-14 (commit 8576948 on branch fix/batch-2026-05-14).
No code changes have been made for this issue.

[dekobon]

Update — first known production CPU is znver3 (AMD EPYC 7763 / Milan), plus AWS instance-CPU coverage

Two concrete data points that should anchor the --target-cpus
selection in step 1 of the spike: the first user's hardware, and
the AWS-instance CPU families this fat binary would need to cover
if/when bca-web is deployed there.

Data point 1 — first user's host

The first user of the project will deploy on an AMD EPYC 7763
64-Core Processor (/proc/cpuinfo: cpu family : 25,
model : 1, stepping : 1). That maps to Zen 3 / Milan, i.e.
-C target-cpu=znver3 (LLVM's znver3 was added in LLVM 12 and is
available in every supported Rust toolchain).

Relevant flags from /proc/cpuinfo:

• avx, avx2, fma, f16c
• bmi1, bmi2, adx
• aes, vaes, pclmulqdq, vpclmulqdq, sha_ni
• rdseed, rdrand, rdpid
• clflushopt, clwb, xsaveopt, xsavec, xsaves, xgetbv1
• movbe, popcnt, lzcnt (via abm), fsrm

Notably absent: any avx512* flag. Milan does not have
AVX-512 — that arrived with Zen 4 (znver4). So this CPU is
solidly x86-64-v3-compatible (AVX2 + BMI2 + FMA + F16C +
LZCNT/MOVBE) but not x86-64-v4. A fat binary that ships only
x86-64 + x86-64-v4 would silently fall back to the generic
baseline for this host.

Kernel floor is fine: the host runs Linux 6.8.0-116-generic,
well above the cargo-sonic loader's memfd_create (≥ 3.17) and
execveat(AT_EMPTY_PATH) (≥ 3.19) requirements.

Data point 2 — AWS EC2 CPU families bca-web would likely run on

bca-web is the long-running variant most likely to be deployed
on AWS. The current production instance families and the
-C target-cpu value LLVM/AMD/Intel recommend for each:

Instance family	CPU	µarch	-C target-cpu	AVX-512
m6a/c6a/r6a	AMD EPYC 7R13	Zen 3 / Milan	znver3	no
m7a/c7a/r7a	AMD EPYC 9R14	Zen 4 / Genoa	znver4	yes
m8a (newest)	AMD EPYC (Turin)	Zen 5	znver5	yes
m6i/c6i/r6i	Intel Xeon (3rd-gen Scalable)	Ice Lake-SP	icelake-server	yes
m7i/c7i/r7i	Intel Xeon (4th-gen Scalable)	Sapphire Rapids	sapphirerapids	yes
m6g/c6g/r6g/t4g	Graviton2	Neoverse N1 (Arm v8.2)	neoverse-n1	n/a
m7g/c7g/r7g	Graviton3	Neoverse V1 (Arm v8.4)	neoverse-v1	n/a (SVE)
m8g/c8g/r8g	Graviton4	Neoverse V2 (Arm v9.0)	neoverse-v2	n/a (SVE2)

Key observations:

• The 6a family uses the same znver3 as the first user.
  Whatever znver3-vs-x86-64-v3 delta we measure on the user's
  local box transfers directly to that AWS family — that's the
  largest single AWS x86-AMD footprint right now.
• x86-64-v3 covers Ice Lake, Sapphire Rapids, Zen 3, Zen 4,
  and Zen 5 (AVX2 + BMI2 + FMA + F16C + LZCNT/MOVBE — present
  on all of them). It does not cover Skylake-era hosts that
  predate AVX2. If we want one Intel-tuned variant that beats
  x86-64-v3, icelake-server is the floor for 6i and
  sapphirerapids covers 7i/r7i.
• AVX-512 splits the AWS x86 fleet cleanly into haves
  (6i/7i/7a/8a) and have-nots (6a, plus the first
  user's host). Shipping an x86-64-v4 variant and a
  non-v4 variant is the only way to serve both halves with
  AVX-512-optimized code where available.
• Graviton (*g families) is aarch64 only. cargo-sonic
  supports aarch64 dispatch via auxv, so a separate aarch64 fat
  binary with neoverse-n1 / neoverse-v1 / neoverse-v2
  variants would be a clean second artifact — but it's a follow-up,
  not part of the initial spike. The original "Linux x86_64 only
  first" framing in the proposal still stands.

Revised --target-cpus shortlist

Given the above, the shortlist for the x86_64-unknown-linux-gnu
spike should be reframed:

--target-cpus=x86-64,x86-64-v3,znver3,znver4,sapphirerapids

• x86-64 — fallback for ancient hosts / unknown CPUs.
• x86-64-v3 — broad modern baseline; covers every AWS x86 family
  in the table, every modern laptop/desktop with AVX2.
• znver3 — the first user's host and AWS 6a family;
  highest-impact single addition.
• znver4 — AWS 7a/8a family and on-prem Zen 4 servers
  (adds AVX-512, BF16, VNNI).
• sapphirerapids — AWS 7i/r7i (adds AMX, AVX-512-VP2INTERSECT,
  CLDEMOTE). Probably wins over icelake-server for the spike
  because every Sapphire Rapids host already runs Ice Lake code
  acceptably; the inverse isn't true.

That's 5 variants, which is on the upper end of "reasonable" for
binary size. If the spike shows znver3 and znver4 don't beat
x86-64-v3 on bca-web request latency, we can drop to a
3-variant x86-64,x86-64-v3,sapphirerapids list.

Suggested step-1 measurement plan

When the spike PR runs cargo sonic build --release with the list
above, please post:

1. Build wall-clock and artifact size with and without
   --compress=zstd.
2. bca parsing throughput on the
   big-code-analysis-output integration submodule for each variant
   (force-select with CARGO_SONIC_FORCE_TARGET_CPU=… or equivalent).
3. bca-web request-path throughput on the same corpus, for
   the same variants.
4. A note on whether any transitive dep (twox-hash, the JSON/CBOR
   serializers, any string interning) opportunistically dispatches
   to SHA-NI / VAES / VPCLMULQDQ — those are the biggest
   feature-flag gaps between x86-64-v3 and znver3/znver4 and
   the most likely source of an out-of-proportion win.

If znver3-over-x86-64-v3 doesn't show a meaningful delta on
either binary on this corpus, that's the signal to ship the smaller
3-variant list. If it does, we have a concrete argument for the
full 5-variant list, anchored to a real first user and the
production AWS families they'd plausibly scale onto.

Sources used for the AWS family/microarch mapping:

• Amazon EC2 M6a — 3rd-gen AMD EPYC (Milan)
• Amazon EC2 C6a — 3rd-gen AMD EPYC (Milan), EPYC 7R13
• Phoronix C6a benchmark (confirms EPYC 7R13 / Zen 3)
• Amazon EC2 M7a — 4th-gen AMD EPYC (Genoa), -march=znver4
• Phoronix M8a benchmark (EPYC Turin / Zen 5)
• Amazon EC2 C7i — 4th-gen Intel Xeon (Sapphire Rapids)
• Amazon EC2 M7i — Sapphire Rapids, vs M6i / Ice Lake
• aws-graviton-getting-started — Rust target-cpu per Graviton gen