Skip to content

streaming_support.md

Latest commit

 

History

History
26 lines (15 loc) · 6.13 KB

streaming_support.md

File metadata and controls

26 lines (15 loc) · 6.13 KB

Streaming Support — All Libraries

Boost.JSON

Boost.JSON offers both event-driven and incremental DOM parsing. The basic_parser provides SAX-style events and accepts multi-segment input [EVID:EVID-20250824-StreamingSupport-BoostJSON-001]. The stream_parser ingests successive buffers and produces a value when completed [EVID:EVID-20250824-StreamingSupport-BoostJSON-002], so callers can feed chunks from files or sockets without loading the full document [EVID:EVID-20250824-StreamingSupport-BoostJSON-003]. UTF-8 sequences spanning buffers are handled by buffering incomplete multibyte code units until completion [EVID:EVID-20250824-StreamingSupport-BoostJSON-004]. Error offsets can be derived from consumed input when parsing incrementally, though there is no direct position API [EVID:EVID-20250824-StreamingSupport-BoostJSON-005]. For output, serializer supports incremental serialization via repeated reads [EVID:EVID-20250824-StreamingSupport-BoostJSON-006]. The default nesting depth limit is 32 (configurable via parse_options) [EVID:EVID-20250824-StreamingSupport-BoostJSON-007]. SAX handlers provide partial token callbacks (e.g., string/number parts) with a final event on completion [EVID:EVID-20250824-StreamingSupport-BoostJSON-008]. Parsers and serializer can be reset and reused to amortize allocations [EVID:EVID-20250824-StreamingSupport-BoostJSON-009]. Chunked parsing supports backpressure-friendly processing patterns [EVID:EVID-20250824-StreamingSupport-BoostJSON-010].

Glaze

Glaze targets contiguous, in-memory JSON. It does not support incremental/chunked JSON parsing; by default inputs are null-terminated for speed (configurable) [EVID:EVID-20250824-StreamingSupport-Glaze-001]. The author notes file-stream parsing is typically slower than reading the whole file first, though streaming can reduce RAM use on very large inputs [EVID:EVID-20250824-StreamingSupport-Glaze-002]. There is no coroutine/async JSON parser (async appears at transport level in HTTP) [EVID:EVID-20250824-StreamingSupport-Glaze-003]. A compile-time partial_read feature can skip unneeded parts of a document by reading only fields present in the target structure [EVID:EVID-20250824-StreamingSupport-Glaze-004]. Errors are reported with line/column formatting [EVID:EVID-20250824-StreamingSupport-Glaze-005]. Unknown keys error by default, with options/handlers to skip or capture them [EVID:EVID-20250824-StreamingSupport-Glaze-006]. The library reads/writes directly to C++ object memory without an intermediate DOM [EVID:EVID-20250824-StreamingSupport-Glaze-007].

simdjson

simdjson does not expose a SAX/event API; instead its On Demand API parses on-the-fly without building a full DOM [EVID:EVID-20250824-StreamingSupport-simdjson-002]. For multiple documents (NDJSON/JSON Lines), parse_many/iterate_many return a document_stream to iterate per-record [EVID:EVID-20250824-StreamingSupport-simdjson-001]. Parser instances can be reused to reduce allocations [EVID:EVID-20250824-StreamingSupport-simdjson-003]. Inputs require SIMDJSON_PADDING bytes (32) at the end for safe SIMD parsing [EVID:EVID-20250824-StreamingSupport-simdjson-004]. The default maximum nesting depth is 1024 [EVID:EVID-20250824-StreamingSupport-simdjson-005]. UTF-8 is fully validated; the README reports validation up to ~13 GB/s on Skylake [EVID:EVID-20250824-StreamingSupport-simdjson-006].

nlohmann/json

The library provides a SAX event-driven parser via json::sax_parse for strings and streams [EVID:EVID-20250824-StreamingSupport-nlohmann-001]. It does not offer incremental/resumable parsing; truncated JSON cannot be resumed [EVID:EVID-20250824-StreamingSupport-nlohmann-002]. JSON Lines/NDJSON is not directly supported and inputs are expected to contain a single JSON value [EVID:EVID-20250824-StreamingSupport-nlohmann-003]. There is no pull/on-demand iterator parser (DOM + SAX only) [EVID:EVID-20250824-StreamingSupport-nlohmann-004], and streaming serialization is not available (single-shot) [EVID:EVID-20250824-StreamingSupport-nlohmann-005]. Object key order can be preserved using ordered_json; default is lexicographic order [EVID:EVID-20250824-StreamingSupport-nlohmann-006]. Output to std::ostream is supported [EVID:EVID-20250824-StreamingSupport-nlohmann-007]. DOM parse errors throw exceptions with codes [EVID:EVID-20250824-StreamingSupport-nlohmann-008], while SAX mode reports errors via callback and returns false [EVID:EVID-20250824-StreamingSupport-nlohmann-009]. Inputs include std::istream and iterator ranges (UTF-8/16/32) [EVID:EVID-20250824-StreamingSupport-nlohmann-010].

Cross-Library Synthesis

  • Boost.JSON offers both SAX and incremental DOM parsing, plus incremental serialization, with explicit limits and partial-token events [EVID:EVID-20250824-StreamingSupport-BoostJSON-001][EVID:EVID-20250824-StreamingSupport-BoostJSON-002][EVID:EVID-20250824-StreamingSupport-BoostJSON-006][EVID:EVID-20250824-StreamingSupport-BoostJSON-007][EVID:EVID-20250824-StreamingSupport-BoostJSON-008].
  • simdjson emphasizes On Demand parsing and NDJSON streams, with strict input/padding rules and deep default nesting limit [EVID:EVID-20250824-StreamingSupport-simdjson-001][EVID:EVID-20250824-StreamingSupport-simdjson-002][EVID:EVID-20250824-StreamingSupport-simdjson-004][EVID:EVID-20250824-StreamingSupport-simdjson-005].
  • nlohmann/json provides SAX but lacks incremental/resumable parse and NDJSON support; serialization is single-shot [EVID:EVID-20250824-StreamingSupport-nlohmann-001][EVID:EVID-20250824-StreamingSupport-nlohmann-002][EVID:EVID-20250824-StreamingSupport-nlohmann-003][EVID:EVID-20250824-StreamingSupport-nlohmann-005].
  • Glaze focuses on contiguous, in-memory parsing/serialization, with partial_read for selective fields and strong unknown-key controls; no incremental or async JSON parser [EVID:EVID-20250824-StreamingSupport-Glaze-001][EVID:EVID-20250824-StreamingSupport-Glaze-003][EVID:EVID-20250824-StreamingSupport-Glaze-004][EVID:EVID-20250824-StreamingSupport-Glaze-006].

Evidence Gaps/Notes: simdjson README performance notes are platform-specific; NDJSON batch_size and truncated_bytes() behavior may warrant explicit citation in v2+/latest docs before finalization.