SuperSLM is a deterministic small-language-model inference runtime designed
specifically for real-time games. It loads a converted, quantized .sslm
artifact and runs it with integer-only arithmetic, no floating point on the
inference path: the same model, prompt, and decoding configuration produce
identical tokens every time, on every certified platform.
Being deterministic is what makes SuperSLM usable inside a frame budget. The engine can slice a decode step across a caller-chosen number of transformer layers per call — spreading one token's worth of work across several frames instead of spending it in one frame-time spike — and the exact same layer slicing produces the exact same output tokens as running the whole step at once. A game can therefore throttle inference to fit whatever GPU headroom a frame has left without changing what the model says.
Current release: 1.13.1, a GPU fix release. A GPU page pool is
now capped at the largest buffer D3D12 guarantees on the card (2 GiB on a card with 8 GB or more of video memory, a quarter of
its video memory below that), and a larger request is refused before anything is allocated; on the AMD Radeon RX 7900 XTX a
4 GiB pool used to make the driver drop the device. sslm_gpu_kv_pool_max_page_count reports the largest pool a card takes.
See the 1.13.1 release note.
In 1.13.0 the GPU backend got the same shared prompt prefixes
and budgeted sequences the CPU got in 1.12.0: a cast of characters that share a world and persona
prompt can keep that prompt in GPU memory once, and each character reserves only the memory its own
reply budget needs. Existing GPU code keeps working, with two changes to calls that were undefined
before (a second GPU submission while earlier work is unobserved, and two threads decoding one
sequence, now return SSLM_BUSY), and the shader set must be redeployed. On the CPU, large saves on x64
are now written with streaming stores, aimed at 1.12.0's save slowdown; the saved bytes are the same.
On the test machine a full-length save is faster than in 1.12.0 but still slower than in 1.11.0
(about 27% slower with the 0.5B model and 14% with the 1.5B model; single readings, not settled
results). Decoding through the new GPU memory pool stays within 5% of the old layout's speed on the
one card measured: about 1% faster with the 0.5B model and 3.7% slower with the 1.5B model at a
16,384-token context. It is not a speed-up. The GPU work is checked on the NVIDIA RTX 2080 SUPER
only so far, and some test runs are still open (the release note lists them). See
Shared prompt prefixes and budgeted sequences
and the 1.13.0 release note.
1.12.0: on the CPU, K/V memory is stored in pages. A sequence can declare a
token budget and reserve only what that budget needs, and sequences that start from the same prompt
prefix can share the prefix's memory instead of each holding a copy. Every output stays
bit-identical to 1.11.0 (read on Windows x64 with MSVC), and code written for 1.11.0 needs no
changes, but its saves are slower: sslm_seq_save of a no-budget sequence takes 1.14 to 2.00 times
as long as in 1.11.0 on the release code, at every length measured (24 of 24 readings across four
captures; readings, not resolved changes), and a later release is planned to fix that (the release
note lists the behaviour changes). See
Shared prompt prefixes and budgeted sequences
and the 1.12.0 release note.
1.11.0: a host's parallel-for hook can split decode, and one-token
prefill, across threads: set bit 1 (SSLM_PARALLEL_FOR_MATVEC) of sslm_parallel_for.reserved. It is
off unless the host asks for it. Every output stays bit-identical with and without it. On a Zen 2
desktop with the Qwen2.5-0.5B model and 4 tasks, decode runs about 1.5x faster and a one-token
prompt about 1.6x faster. No ABI struct, status or save-format change. See the
1.11.0 release note.
1.10.0, a CPU speed release: on the AVX2 and AVX-512 tiers, prompt reading runs a register-tiled matrix multiply, attention's per-row steps (probability-times-value, softmax, requantization and, on Qwen3 models, the Q31 score) run a row at a time, and the model's load-time integrity hash uses the CPU's SHA instructions where it has them. Every output is bit-identical to 1.9.0: the same tokens, save blobs and digest. No ABI, format or status change. See the 1.10.0 release note.
1.9.0: sslm_seq_save writes a new save format, SSB5, which carries the
four per-site saturation counts beside the saturation total they sum, so a restored sequence keeps
them; 1.8.1 restored the total with per-site counts of 0. These are internal diagnostic counters:
neither they nor the total is readable through the C ABI (sslm_stats_out has no saturation field),
so a caller sees the change only as the new save format. Blobs saved by earlier releases still
restore, with their total and per-site counts of 0. sslm_model_map now refuses an artifact whose
damped-greedy scale no decode can use, which 1.8.1 mapped and then refused at every damped-greedy
decode; no converter output has that shape. No token, status or ABI surface changes. See the
1.9.0 release note.
1.8.1: on the CPU, the four per-site saturation counts travel with the saturation total they sum through prefix adoption and prefill; 1.8.0 left them stale or unfilled on those two paths. See the 1.8.1 release note.
1.8.0: on the GPU, running out of memory is reported as running out of
memory: an allocation failure on a device that is not removed -- host memory, or GPU memory on any
heap -- returns SSLM_GPU_ALLOCATION_FAILED, leaves the process's command list closed, and leaves
the context, the device and every other handle usable, so the call can simply be retried. 1.7.1
reported many such failures as SSLM_DEVICE_LOST, could leave the next GPU call in the process to
fail in its place, and kept a failed first-time device setup for the life of the process. A GPU
failure that is not about memory is never reported as one. Several statuses change for the same
fault, so a host that maps GPU statuses should read the
1.8.0 release note before upgrading. No token, save format, or
SslmGpuStatus ordinal changes.
1.7.1: the engine library writes nothing to stdout: the # adapter: <name> line
SSLM_GPU_ADAPTER_INDEX prints now goes to stderr, like every other harness diagnostic, so a host
that pipes model output through stdout is no longer corrupted by it. See the
1.7.1 release note.
1.7.0: the token finish can take its logits off the calling thread, through a caller-supplied host
parallel-for hook on both backends or an opt-in device-resident head on the GPU, with tokens
identical to 1.6.0. On an RTX 2080 SUPER the device head took Qwen2.5-0.5B from 42.67 to
73.21 tok/s and Qwen2.5-1.5B from 27.85 to 55.49 tok/s, for 137,433,088 B and 234,627,072 B of
VRAM per mapped model. A host can also choose the directory GPU shaders load from, and a GPU
context stays usable after an out-of-memory failure during an upload. With no hook and no device
head, tokens are identical to 1.6.0 and the path is measurably slower in some cells (−1.98% tok/s
on the CPU at 0.5B). GpuContextConfig grew, so code compiled against an earlier gpu_1p0.h must
be recompiled. See the 1.7.0 release note for every figure with its
hardware and settings, and CHANGELOG.md.
The same model, prompt, and decoding configuration produce identical output tokens on every certified platform — see Certified platforms below for exactly which ones and what "certified" means there. Determinism is checked at the level of individual forward-pass outputs, not just final tokens: on a certified GPU, every intermediate layer's key/value state and every decoded token match a CPU reference run bit-for-bit.
Damped greedy is a deterministic anti-repetition decoder for calls where plain greedy falls into a loop. It scores the model's top candidates against a small anti-language-model built from the generated history. It is opt-in at both conversion and decode time; existing artifacts and callers continue to use greedy unchanged.
The ruled operating point is alpha=2, anti-LM order n=2, and top_k=6.
Across 192 paired real generations on Qwen2.5 0.5B and 1.5B, the three observed
0.5B greedy loop locks fell to zero and repeated-trigram rates dropped in all
four model/length cells. Many outputs read more coherently, but that is an
observation over this corpus rather than a general quality guarantee. The
decoder can penalize legitimate repetition too: one list-continuation case
changed list formatting. Prefer greedy when preserving the model's learned
format/style continuity matters, or use a compiled schema when validity is the
actual requirement. The full population, side-by-side text, and timing scope are in
the Phase E confirmation.
A LoRA specialization can be attached to and detached from a running model without unloading or duplicating the base weights. Measured on an NVIDIA RTX 2080 SUPER (Windows x64, a real 1.5B-parameter base model and a real LoRA adapter): switching adapters takes 0.128 s, against 7.52 s to fully reload the model with a different adapter merged in — about 58x faster. The base model's resident weights are untouched by the switch, and decode output remains bit-identical to the pre-switch determinism guarantee.
A caller sets a per-frame GPU budget, measured in transformer layers per decode call, and the engine spreads one token's worth of work across however many calls that budget requires — designed to keep inference off the frame-time spike path. This is proven slice-invariant: decoding the same prompt at any layer-budget granularity, down to one layer per call, produces the identical output tokens as decoding it in one call, on every certified platform. An in-game frame-time measurement (part of the SuperSLM-Unreal integration, which follows this engine's own 1.0 release) will turn "designed to avoid spikes" into a measured frame-time result; today the guarantee is the mechanism the design relies on, proven at the engine level.
1.1 makes prompt prefill — processing the tokens you send in before decoding begins — substantially faster on both paths, without changing a single output bit.
On the CPU, the bottleneck was the kernel itself: the integer matmul is now runtime-dispatched across SSE2, AVX2, and AVX-512 tiers, and batched prefill measures 1.68x-1.72x faster than 1.0's SSE2-only kernel on a real 1.5B model. Every tier produces bit-identical output to the scalar reference; the wider vectors change speed, not results.
On the GPU, the bottleneck was per-token submission: the 1.0 path paid a host round-trip fence wait per token per layer batch. Prefill now submits a whole chunk of tokens per command list, measuring 6.91x-7.19x faster on the certified NVIDIA GPU and 13.8x faster on the certified AMD GPU, where the round-trips cost more. Output is bit-identical to the per-token path at every chunk size and every chunk-boundary split, on both certified GPUs.
Both measurements, their exact hardware and span-length cells, and what remains open are in docs/platform-support.md and the roadmap.
Schema-constrained generation means the model cannot emit a token that breaks your schema — so the output parses whenever the decode completes within its token budget. A compiled schema is a table of valid-token masks, one per parser state; the engine indexes into that table by the sequence's own parse state before every decode step and masks out any token that would break the schema, so an invalid token is never a candidate in the first place — not a post-hoc filter on the model's raw output. Every token the engine does emit is legal at the moment it is emitted; format validity holds for whatever was emitted, not only for a decode that ran to completion.
An unbounded field (the free-text "type": "string" leaf, for example) has
no length bound the engine enforces, so its own close is up to the model —
if a caller's decode budget is reached first, the returned sequence is
legal so far but not closed, and will not parse as JSON on its own. A
caller detects this without guessing from the token count alone:
sslm_stats's schema_accepting field is 1 iff a schema is bound and the
sequence's current parse state is one where stopping is valid, and 0
otherwise. Because 0 also means no schema is bound, callers that need to
distinguish those cases use sslm_seq_schema_bound. A schema-bound decode
that stops (budget reached, or any other reason) while
schema_accepting == 0 returned an incomplete value; a caller that needs a
guaranteed-parseable result checks both facts before treating the output as
final. GPU callers use SslmGpuSeqSchemaBoundForG5Bridge and
SslmGpuSeqSchemaAcceptingForG5Bridge after draining and finishing the step.
Schema bind, rebind, and unbind are valid only on a newly created or reset sequence. A generation call that passes argument validation makes the sequence ineligible until reset, including a valid no-op call or CPU empty-prefix adoption. A call rejected for invalid arguments leaves the sequence untouched and still bindable; a GPU Submitted/busy refusal likewise preserves eligibility. A restored CPU or GPU sequence must be reset before binding.
The determinism guarantee above carries through even when the engine skips ahead through the parts of the output your schema already dictates (for example, a fixed key name or punctuation your schema forces regardless of what the model would otherwise produce) without running a real decode step for those tokens — a schema-constrained decode is exactly as reproducible, on the same certified platform, as an unconstrained one. Format — grammar validity for every token emitted, and parse validity for a decode that completes within its budget — is what this guarantees by construction; cross-field semantic validity (e.g. "this value must be less than that one") is reported, not enforced, since that is a property of your schema's meaning rather than its shape.
Schema-constrained decoding is proven bit-identical between the CPU and GPU paths on both certified GPUs — NVIDIA and AMD (see Certified platforms). The mechanism, the artifact format it relies on, and the full CPU consumer ABI it ships through are documented in docs/api.md and docs/sslm_format.md.
New in 1.12.0, CPU backend. Many characters in a game often start from the same text: the world description, then a persona, then their own conversation. SuperSLM already lets you run a shared prefix once and have every sequence adopt it instead of re-reading it. From 1.12.0 the CPU backend can also keep that prefix in memory once, however many sequences use it, and size each sequence by what it will actually generate rather than by the model's whole context length. Sharing and sizing by budget hold at a context cap of more than one page.
- Budgeted sequences. Create a sequence with
sslm_seq_create_budgetedand a budget: the number of tokens it may add itself. It reserves memory for that budget up front, so it never runs out of memory partway through a reply, and when it reaches its budget it stops withSSLM_KV_BUDGET_EXCEEDEDinstead of writing anything. Reset it and reuse it without going back to the pool. - Shared prefixes. Build a prefix once, freeze it, and adopt it into budgeted sequences: they
read the prefix's memory instead of copying it, at a context cap of more than one page (at a cap
that fits in one page a sequence is a single page, so adopt copies it).
sslm_prefix_begin_fromstacks prefixes, so a persona prefix can sit on a world prefix and a conversation on the persona. - Smaller saves. A budgeted sequence's save holds its live tokens, an adopted prefix's included,
so its size follows its live length, and
sslm_seq_restore_sharedreconnects a restored sequence to its live prefix, so a reloaded cast shares memory again. Pass the prefix handle when you restore; without it, each restored sequence holds a private copy of its prefix until it is reset. - Size your pool from what it holds.
sslm_kv_page_pool_createsizes a pool in pages, andsslm_kv_pool_statsreports how many are free. A prefix's memory is released when the prefix and every sequence sharing it have let go of it, so check the free count rather than counting handles.
As an example, at Qwen2.5-0.5B with a 4,096-token context, a cast of 50 characters sharing one 1,000-token world and 10 200-token personas, each with up to 512 tokens of its own, fits in a pool of 189.4 MiB, where whole-context sequences need 1,440 MiB (the pool figure is confirmed on the real 0.5B artifact). The cast and its numbers are an illustration, not a measured game workload.
Existing code needs no changes: sslm_seq_create, sslm_prefix_begin and the other 1.11.0 calls
behave as before in what they return and save in the same format, which older releases can read.
One call differs in speed: sslm_seq_save of a no-budget sequence takes 1.14 to 2.00 times as long
as in 1.11.0 on the release code, at every length measured (24 of 24 readings across four captures;
readings, not resolved changes), and 1.13.0 changes how large saves are written to address it. The savings apply only
to budgeted sequences, and the context cap is still the cap. Speed
is claimed for single-conversation decode on Windows x64 CPU only, at ordinary background load: paged decode is held to within 5%
of the contiguous cache of the same build, to the 0.5B model's 4,096-token cap and to width 18,682 at
1.5B (measured with whole-token calls and no host parallel-for hook, on the AVX2 tier), and reset
and adopt cost does not grow with the context cap (no more than 2.0 times higher at an eightfold
larger cap), measured with the page size held equal at the two caps (except that at head_dim 64 or below, a cap above 4,096 doubles the page and its one-page
tail copy). Batched decode works and stays bit-identical, with no speed claim. The release note has the
figures. The full contract is in
docs/api.md.
On the GPU, from 1.13.0. The same cast can live in GPU memory. Create a GPU page pool with
sslm_gpu_kv_pool_create, build the world and persona prefixes in it (sslm_gpu_prefix_begin,
sslm_gpu_prefix_begin_from, sslm_gpu_prefix_prefill, sslm_gpu_prefix_freeze), and give each
character a budgeted sequence (sslm_gpu_seq_create_budgeted) that adopts its persona
(sslm_gpu_seq_adopt_prefix). Decode, batch and finish them with the calls you already use.
- A budgeted GPU sequence stops with
SSLM_GPU_KV_BUDGET_EXCEEDEDat its budget (a prompt that runs past the budget keeps the tokens that fit and returns the same status), and a pool that cannot hold another reservation refuses the create withSSLM_GPU_KV_POOL_EXHAUSTED, so you find out at create time, not mid-reply. - Its save is a new format,
SLM6, sized by its live tokens. Restore it withsslm_gpu_seq_restore_into_pooland pass the persona's prefix handle so it shares again. Older releases cannot readSLM6. - Sequences made by
sslm_gpu_seq_createkeep their own buffer, sized by the context cap you pass, and keep savingSLM5, as before. - The GPU's page is smaller than the CPU's (16 positions for converted models), so the same cast takes a different number of pages on each backend: in the example above, 1,842 GPU pages (172.7 MiB). Fifty ordinary GPU sequences would take 1,200 MiB at the full 4,096-token context, or about 502 MiB if each is created with only the 1,712 positions it needs, so the pool's saving depends on how you would have sized them. The 1,842-page admission is confirmed on the real 0.5B artifact on the RTX 2080 SUPER.
What it costs: on the RTX 2080 SUPER, decoding one sequence through the page pool was measured
against a whole-context sequence of the same build, and stayed within the 5% bar this release holds
it to: a 0.5B cast member at 1,712 tokens read about 1% faster, and a 1.5B sequence at 16,384 tokens
about 3.7% slower. Neither is a promised speed-up or a promised cost beyond the bar. Nothing is
measured on the AMD card, at other context lengths or for batched decode yet. Two older GPU
behaviours now return
SSLM_BUSY instead of being undefined: a second GPU submission while earlier work has not been
seen to complete (call sslm_gpu_ready before the next submission, or use the batch call), and two
threads decoding the same sequence. The
1.13.0 release note has the figures and what was checked where, and
docs/api.md the summary of the calls.
Every capability above is built and measured on the platforms named. Determinism, sliceable inference, and schema-constrained generation are all measured on both certified GPUs — the schema-constrained-decoding GPU parity check, for example, passed bit-identical on the certified NVIDIA GPU, and passed bit-identical on the certified AMD GPU as well (measured 2026-08-17 on the Radeon RX 7900 XTX; see Certified platforms). Two capabilities are narrower: adapter switching is measured on the certified NVIDIA GPU only (see Runtime-switchable LoRA adapters above), and CPU-side prefill batching is proven on every certified platform on the CPU path, with the GPU path — new in 1.1 — measured and certified on both certified GPUs, NVIDIA and AMD. docs/platform-support.md carries every measured number with its device and cell.
"Certified" means: built, run, and measured on that exact hardware, with every determinism check passing bit-for-bit against the CPU reference on that same run.
| Platform | CPU inference | GPU inference |
|---|---|---|
| Windows x64 | Certified | Certified — NVIDIA Turing (measured on RTX 2080 SUPER) and AMD RDNA3 (measured on Radeon RX 7900 XTX) |
| Linux x64 | CI extent (see note below) — GCC and Clang | Not built (the GPU backend is D3D12, Windows-only) |
| macOS (Apple Silicon) | CI extent (see note below) | Not built |
"CI extent" means the platform is exercised by this project's own continuous
integration matrix, and no further hardware-specific measurement has been
made beyond what that gives. What that matrix has actually executed, and as
of when, is stated in exactly one place —
docs/platform-support.md § CI execution status
— rather than restated here. A consumer can reproduce the same verification
locally regardless of hosted-run status: the Building section's
cmake+ctest matrix on any platform, and build.bat on Windows. See
docs/platform-support.md for the full table,
every measured number, and where each one was measured.
GPU determinism is scoped to the certified adapters above: an AMD integrated GPU on the same RDNA3 test machine passes the direct-dispatch determinism check but diverges on the asynchronous decode path under investigation, and is explicitly not a certified target today.
A .sslm file is a versioned, integrity-checked container: a header, a
section table, and aligned sections (weights, scales, RoPE tables, biases,
tokenizer, schema masks, golden reference hashes). The offline converter
(build-time Python) reads a Hugging Face checkpoint and emits the artifact;
the runtime maps and validates it. The format is specified in
docs/sslm_format.md.
The loader is a trust boundary: it treats the whole file as hostile input and validates every field against declared bounds before reading a section byte. Any deviation — bad magic, unsupported version, an out-of-bounds or overlapping section, a hash mismatch — is a rejection with a versioned diagnostic, never a silent partial load.
docs/quickstart.md walks through converting your own
Hugging Face checkpoint into a .sslm model and decoding from it, including
a worked LoRA adapter conversion.
cmake -B build && cmake --build build && ctest --test-dir build
This builds the CPU-only product library (superslm), the public headers,
and the test suite. To build only the CPU product on Windows, with no DXC or
D3D12 test dependency, use cmake -B build -DBUILD_TESTING=OFF followed by
the ordinary default cmake --build build --config Release; no test target
is created. On Linux and macOS this is standard-library only, no
third-party runtime dependency, and works from CMake 3.16. On Windows, the
enabled test suite's own GPU-serial-port section calls the D3D12 GPU path
directly, so superslm_tests compiles and links src/gpu/superslm_gpu.cpp
and its compiled shaders when BUILD_TESTING=ON — this needs CMake 3.20+
and the DirectX Shader Compiler
(dxc.exe, from the Windows SDK), matching build.bat's own long-standing
requirement below; a Windows configure without dxc.exe fails at
cmake -B build with a one-line diagnostic naming what is missing, rather
than at link time. The public, installable GPU library is a separate,
opt-in CMake target on top of that (Windows/MSVC only):
cmake -B build -DSUPERSLM_BUILD_GPU=ON && cmake --build build --target superslm_gpu
This is for a consumer who wants the GPU acceleration library itself
installed and exported via find_package(superslm); it enables the same
shader compilation used by the Windows test build and installs the compiled
shaders with the package. By default the GPU library loads them from a
shaders directory beside the host executable, so a consuming CMake target
deploys them there:
find_package(superslm REQUIRED)
target_link_libraries(my_app PRIVATE superslm::superslm_gpu)
superslm_deploy_gpu_shaders(my_app)A host that does not own its executable's directory (an editor plugin, for
example) can skip the copy and point GpuContextConfig::shader_dir at the
installed superslm_GPU_SHADER_DIR instead. The shader directory is
process-wide; docs/api.md describes the rule.
On Windows with Visual Studio installed, build.bat is a one-shot MSVC
build that compiles the full local development suite, GPU included (it
requires the Windows SDK for dxc.exe, same as above). The test suite is
standard-library only and prints a checks / failures summary; a nonzero
exit means a failure. build.bat also runs a small number of optional
local checks (an ABI verb-count re-derivation, a couple of Python-based
CI-source checks) when bash/python happen to be on PATH; each is a
loud, named, non-fatal skip when its tool is absent — none is required to
build or to pass the suite.
| Path | What |
|---|---|
include/superslm/ |
public headers — the embeddable API |
src/ |
implementation |
tests/ |
the test suite (one harness, no third-party framework) |
tools/ |
build-time tooling — converters, calibration, verification |
docs/ |
format, API, and platform-support documentation |
Two public APIs are documented in docs/api.md, both shipped:
the C++-linkage GPU handle API (SslmGpu*) and the CPU-side C ABI (sslm_*)
for embedding SuperSLM directly in another process without the GPU handle types.
SuperSLM is licensed under Apache License 2.0. The permissive license and express patent grant are deliberate: they make adoption safe for consumers, and closed forks remain permitted.
Named follow-on work after 1.5:
- Paged GPU memory on the AMD card. 1.12.0 paged the CPU backend's K/V memory and 1.13.0 the GPU's (see Shared prompt prefixes and budgeted sequences), checked so far on the certified NVIDIA GPU only. Running the same checks on the certified AMD GPU is the next step for it.
- Async-wrapper overhead reduction. The API's convenience wrapper around the dispatch path measurably costs throughput versus the raw dispatch path — see the Direct dispatch vs. Async wrapper API rows in docs/platform-support.md; closing that gap is a named follow-on campaign.
- Launch-floor reduction. The fixed per-call GPU launch overhead has an identified, not-yet-built optimization path.
- AVX-512 on full-width silicon. The tier is proven and measured on Zen 4 (about 1.18x over AVX2, bounded by that microarchitecture's double-pumped 512-bit execution); measuring on full-width datapaths (Zen 5, server Intel) is open — see Known gaps.
- Flat-batch dispatch fusion and the remaining single-group tail sites. Two named, scoped opportunities to close the gap between batched and single-sequence throughput further.
- The integrated-GPU divergence. Root-causing why the async decode path diverges on an AMD integrated GPU where the direct-dispatch path does not (see Certified platforms), toward certifying integrated GPUs as a target.
- A second GPU generation. Extending cross-vendor certification to a newer NVIDIA generation (Blackwell) beyond the currently-certified Turing and RDNA3.
- macOS beyond CI extent. Measuring on real Apple Silicon hardware, beyond what GitHub's own macOS runners can exercise today.