v0.2.1

Full Changelog: v0.2.0...v0.2.1

v0.2

v0.2 is the architectural rework foreshadowed by the v0.1 series: every backend (Mojo native CPU, simdjson FFI, GPU) now emits the same packed Document tape, Value is a stable index into that tape rather than a separately-allocated tree, and the GPU pipeline is rewritten as a single lean kernel that's portable across NVIDIA, AMD, and Apple Metal. The public API (loads, dumps, load, dump) is unchanged for users on v0.1.x — most of v0.2 is internal — but the internals shifted enough to justify a minor bump.

Highlights

• Tape-backed Document shared by every backend. Mojo CPU, simdjson FFI, and GPU all write into the same packed 64-bit tape with side key_pool / int_pool / float_pool arenas. Value is now (doc, tape_idx) — iteration is a tape walk, not a re-parse, and nested mutation propagates through the parent (doc["a"]["b"].set(...) is observed by doc).
• Two-pass pure-Mojo CPU parser as the default. Stage 1 is a 64-byte branchless SIMD scan (PSHUFB-style nibble classifier + prefix-XOR escape mask) emitting structural offsets; stage 2 walks those offsets once and writes the tape. The simdjson FFI shim is now opt-in via target="cpu-simdjson".
• GPU rewritten as a lean pipeline, native on Apple Metal. A single fused kernel emits the structural bitmap; the CPU tape adapter applies the in-string escape state machine. Same code path on NVIDIA, AMD, and Apple Metal — Apple is no longer gated behind a fallback flag and now chunks at 64 MB so 800 MB inputs no longer crash the host.
• Copy-on-write mutation. Value lazily materializes only the tape regions that get mutated, so reads stay zero-copy and writes don't fork the whole document.
• Reflection serde extended. Dict[String, T], List[Optional[T]], Optional[List[T]], and List[List[T]] for scalar T are all supported now without hand-written to_json / from_json.
• json.prelude. One-line import for the everyday surface (loads, dumps, load, dump, Value, Null, ParserConfig, SerializerConfig, the reflection serde shortcuts).

Performance

GPU on twitter_large_record.json (804 MB)

Platform	Throughput	Pipeline
AMD MI355X	13 GB/s	lean (single-shot)
NVIDIA B200	6.5 GB/s	lean (single-shot, pinned wall-clock)
Apple M3 Pro	3.1 GB/s	lean Metal (chunked at 64 MB)

All three backends run the same lean pipeline — one fused kernel plus positions-only stream compaction. The CPU tape adapter consumes only gpu_result.structural and applies the in-string escape state machine on the host, so the popcount + hierarchical prefix-sum cascade and the pair_pos array from v0.1 are gone. GPU is only beneficial for files >100 MB on discrete cards.

CPU on x86 (Mojo native two-pass, vs v0.1)

File	Size	v0.1 parse_only	v0.2 parse_only	v0.2 parse_traverse
twitter.json	616 KB	~0.25 GB/s	1.06 GB/s	0.64 GB/s
citm_catalog.json	1.7 MB	~0.24 GB/s	1.56 GB/s	0.87 GB/s

parse_traverse only adds a small constant on top of parse_only because every Value is a stable tape index. Reproduce with pixi run -e dev bench-cpu <file> (3 warmup + 100 measured iterations, min-time-derived throughput). The remaining gap to native simdjson C++ on parse_only is algorithmic (no Eisel-Lemire float fast path, no AVX-512 64-byte chunks). Full breakdown in docs/performance.md.

What changed

Architecture (v0.2 phases A–F from design-0.2.mdc)

• Phase A — Document & Tape. Introduced json/document.mojo with packed 64-bit tape entries plus side key_pool, int_pool, float_pool arenas.
• Phase B — Copy-on-write mutation. value/owned.mojo lazily materializes on first write and propagates through the parent chain. Nested mutation (doc["a"]["b"].set(...)) is now observed by the root.
• Phase C — SIMD stage 1 + scalar oracle. Stage 1 (json/cpu/stage1.mojo) does the SIMD structural scan; stage1_scalar.mojo is a deliberate equivalence oracle wired into tests/test_stage1_equivalence.mojo. Stage 2 is the tape builder.
• Phase D — GPU adapter to tape. json/gpu/tape_adapter.mojo converts the GPU's structural bitmap straight into a Document. Apple Metal runs the real GPU pipeline now — the historic JSON_GPU_ALLOW_APPLE_FALLBACK opt-in is removed.
• Phase E — API hygiene + module split. MojoJSONParser, mojo_backend.mojo, and loads_with_config are deleted; value/ is split per concern; json/__init__.mojo is now 74 lines (cap was ≤200). Version bumped to 0.2.0.
• Phase F — Reflection extensions + docs. Dict[String, T], List[Optional[T]], Optional[List[T]], List[List[T]] for scalar T in json/reflection.mojo.

Perf (CPU)

• Single-pass stage 2 emit, dropped O(N×depth) close-bracket scan.
• PSHUFB-style 16-entry nibble classifier in stage 1 (replaces 8× .eq() per chunk with one _dynamic_shuffle lookup).
• Branchless 64-byte stage 1 with prefix_xor + find_escape_mask (CLMUL-style escape tracking, scalar fallback elsewhere).
• Zero-copy object keys via TAPE_TAG_KEY_INLINE (offset/length into input, not key_pool).
• SWAR 8-digit integer parser + SIMD backslash scan in stage 2.
• Bulk memcpy to flush headers_scratch into the tape on container close.
• SIMD whitespace skip in stage 2 with scalar prelude.
• Native simd_byte_width() — same source compiles to 16 B NEON, 32 B AVX2, 64 B AVX-512.
• Span-based string unescape, drop O(input) copy per string.
• Single-pass array/object emitters in stage 2.

Perf (GPU)

• Apple Metal goes from "raises by default + opt-in CPU fallback" to first-class native runtime.
• Lean pipeline: _parse_json_gpu_apple_lean drops the popcount + hierarchical prefix-sum stages because they only fed quote_prefix_in into fused_json_kernel, which discarded it. Cross-chunk and escaped-quote correctness moved to the CPU adapter, removing ~30% of GPU work and 5 device buffers per chunk.
• Apple inputs auto-chunk at 64 MB so launch overhead is amortized while the unified-memory / watchdog envelope is respected. Tunable via -D JSON_GPU_APPLE_CHUNK_MB=N.

API hygiene

• MojoJSONParser, mojo_backend.mojo, loads_with_config, dumps_with_config, and stale deprecation comments are removed.
• json.prelude added with curated re-exports for one-line imports.
• value/ directory split into value.mojo (tape-backed view), owned.mojo (COW), and raw_ops.mojo (internal scanners).

Tooling

• Five mozz fuzz harnesses (fuzz_loads, fuzz_simdjson_ffi, fuzz_value_access, fuzz_jsonpath, fuzz_ndjson) under a new fuzz pixi feature with a differential property that simdjson and the native parser must agree on canonical dumps output. 200,000 runs, 0 crashes / 0 rejects.
• ASan harness over the FFI and tape boundaries via tools/run_sanitizer_tests.sh and the tests-asan pixi task (Linux-only).
• bench_gpu_apple.mojo + bench-gpu-apple pixi task — Apple-specific sweep that runs loads[target='gpu'] over multiple file sizes for refreshing README throughput numbers.
• bench-cpu adds a full-traversal variant (parse_traverse) so the bench is honest about the cost of walking every Value, not just parsing.
• Examples reorganised into basic/ intermediate/ advanced/ tiers with one example per topic, all runnable via pixi run tests.

Bug fixes

• jsonpath: reject unclosed [?...] filter instead of triggering a slice underflow.
• Stage 1 SIMD escape carry — fix a corner case where the carry between 64-byte chunks could mis-classify the structural after a long backslash run.

Compatibility

• Mojo ==1.0.0b1.
• CI matrix: ubuntu-22.04 and macos-14.
• Public API (loads, dumps, load, dump) unchanged for users on v0.1.x.

Install

[dependencies]
json = { git = "https://github.com/ehsanmok/json.git", tag = "v0.2.0" }

Full changelog

v0.1.6...v0.2.0

v0.1.6

• Compatible with Mojo 1.0.0b1.
• CI: drop nightly cron schedule.

v0.1.5

Mojo nightly bump to 1.0.0b1.dev2026042717. Source-only conda
distribution unchanged from v0.1.4 (mojo package step removed,
ships json/ source plus libsimdjson_wrapper.so under
$PREFIX/lib/mojo/ for compile-against-consumer-Mojo import
semantics). No public API changes since v0.1.4.

Install:

json = { git = "https://github.com/ehsanmok/json.git", tag = "v0.1.5" }

v0.1.4

Source-only conda distribution: rattler-build recipe no longer calls mojo package, ships json/ source plus the libsimdjson_wrapper.so FFI artifact. Pin mojo == 0.26.3.0.dev2026042005. Internal: collapse stray bare comptime\nif/for lines in reflection.mojo. No public API changes since v0.1.3.

v0.1.3

Patch release that bumps the Mojo compiler pin to 0.26.3.0.dev2026042005 in both pixi.toml and recipe.yaml. The prior pin (dev2026041805) stopped resolving cleanly on the scheduled nightly CI run after upstream moved on.

No public API, feature, or perf changes since v0.1.2. Same GPU stream-compaction + bracket-match throughput, same surface area.

Install

```toml
[dependencies]
json = { git = "https://github.com/ehsanmok/json.git", tag = "v0.1.3" }
```

Full changelog

v0.1.2...v0.1.3

v0.1.2

Patch release that restores GPU parse throughput to its earlier published level, adds a leaner default pixi env, and bumps the Mojo compiler pin for reproducible installs.

Why this release exists

Between the initial GPU work (multi-GB/s on twitter_large_record.json) and v0.1.1, a regression crept into the pinned stream-compaction path that dropped parse_json_gpu_from_pinned to ~0.56 GB/s on the same hardware. v0.1.2 restores the expected throughput by parallelising the two prefix sums that had regressed to single-thread scans, and locks in the result with a stable Mojo pin.

GPU perf (NVIDIA B200, twitter_large_record.json, 803 MB)

row	v0.1.1 (regressed)	v0.1.2 (restored)
parse_json_gpu_from_pinned (pinned, wall-clock)	0.56 GB/s	~6 GB/s median, 7.3 GB/s best
parse_json_gpu_from_pinned (device-only, via iter_custom)	—	~6.6 GB/s median, 7.2 GB/s best
match_brackets_gpu (1 M elements)	55 ms	11.6 ms

These numbers are where the GPU path sat before the regression; v0.1.2 is "back on budget", not a new speedup.

Changes

Perf (regression fixes)

• Restore parallel GPU stream-compaction prefix sum using block.prefix_sum (the scan had been running single-threaded).
• Restore parallel GPU bracket-match depth prefix sum (match_brackets_gpu) the same way.
• Unroll the 32-byte scan in _quote_popcount_kernel with comptime for.
• Annotate every GPU kernel with MAX_THREADS_PER_BLOCK_METADATA so NVPTX can right-size the register budget.

Tooling / UX

• Replace the DEBUG_TIMING comptime flag with a runtime --debug-timing argv flag on the GPU benchmark binary; enable with pixi run -e dev bench-gpu -- --debug-timing <file>.
• Unify the GPU benchmark into a single std.benchmark.Bench report with four rows (from host bytes, pinned wall-clock, pinned device-only, loads[target='gpu']). The device-only row uses iter_custom + DeviceContext.execution_time for CUDA-event timing.
• Gate parse_json_gpu / parse_json_gpu_from_pinned on has_accelerator() with a friendly error when no GPU is detected.

Build / platform

• Pin sysroot_linux-64 = "2.34.*" so mojo build can resolve the glibc 2.34 symbols referenced by Mojo's shared libs, while staying below the sysroot 2.39 that destabilises Mojo's JIT on GitHub's ubuntu-latest image.
• Pin mojo / mojo-compiler to ==0.26.3.0.dev2026041805 in both pixi.toml and recipe.yaml for reproducible installs.

Dev env hygiene

• Make the default pixi env lean. mojodoc, pre-commit, gdown, git (and the mojo build-dependent build-bench-* / bench-* tasks) now live in the dev feature. End users (and the CI test matrix) no longer pull hundreds of MB of dev tooling. Run dev tasks with pixi run -e dev <task> or pixi shell -e dev.

Bug fixes

• Fix String indexing in tests/test_bracket_match.mojo after String.__getitem__ dropped the implicit codepoint accessor.

Docs

• Repair stale src/ -> json/ paths in docs/architecture.md, refresh the GPU benchmark metric tables in docs/performance.md and benchmark/README.md, and document the new --debug-timing flag and -e dev workflow.

Install

```toml
[dependencies]
json = { git = "https://github.com/ehsanmok/json.git", tag = "v0.1.2" }
```

Full changelog

v0.1.1...v0.1.2

v0.1.1

Patch release that pins the Mojo compiler version for reproducible installations.

Changes

• Pin mojo to ==0.26.3.0.dev2026041520 in pixi.toml and recipe.yaml.
• Untrack pixi.lock so pixi resolves a fresh lockfile on each install, avoiding platform-specific package-resolution issues on Linux x86_64.
• Minor internal fixes for the latest mojo nightly API changes (UnsafePointer.__getitem__ deprecation, init_pointee_copy for GPU tests).

Installation

[dependencies]
json = { git = "https://github.com/ehsanmok/json.git", tag = "v0.1.1" }

Notes

No public API changes. This release is fully compatible with v0.1.0 for users; the version bump exists to tag a known-good pairing of json source + mojo nightly for downstream consumers (e.g. flare v0.2.0).

v0.1.0

Initial release of json: high-performance JSON library for Mojo with GPU acceleration.

Supported features:

• Python-like API (loads, dumps, load, dump)
• Reflection-based struct serde (zero boilerplate)
• GPU-accelerated parsing (NVIDIA, AMD, Apple Silicon)
• Streaming and lazy parsing for large files
• JSONPath queries and JSON Schema validation
• JSON Patch, Merge Patch, JSON Pointer (RFC compliant)
• NDJSON support
• Custom serialization traits

API documentation: https://ehsanmok.github.io/json/