Skip to content

Releases: Query-farm/vgi-rpc-python

v0.25.0

Choose a tag to compare

@rustyconover rustyconover released this 17 Jul 01:30

perf(utils): cache the per-class serialization plan on the message hot path

ArrowSerializableDataclass._to_row_dict and deserialize_from_batch ran get_type_hints(include_extras=True) and re-derived the transient / required-field / ArrowType(pa.binary()) facts on every RPC message — profiling a downstream integration suite (the VGI DuckDB extension) showed typing introspection (~22%) and isinstance dispatch chains (~33%, including the runtime-checkable _BytesSerializable Protocol check) dominating the Python-side cost per message.

  • _serialization_plan(cls) resolves type hints once per class and caches a _SerializationPlan (per-field resolved/unwrapped annotation, transient flag, binary-override flag, defaults, required-field list) on the class itself — same cls.__dict__ idiom as _ArrowSchemaDescriptor, so subclasses never inherit a parent's cached plan.
  • Serialize and deserialize consume the plan instead of re-resolving annotations per call.
  • _convert_value_for_serialization gains an exact-type fast path for plain str/int/float/bool/bytes values (exact checks, so Enum members — str/int subclasses — still take the full dispatch).

Measured (8-field message, Python 3.13): _serialize 65 → 26 µs (−60%), serialize_to_bytes 78 → 38 µs (−51%), deserialize_from_batch 48 → 27 µs (−43%).

No wire-format or API changes. Verified with the full test suite (3659 passed), strict mypy/ty/ruff/pydoclint, and the VGI DuckDB extension's 285-test integration suite run against this build.

v0.24.0

Choose a tag to compare

@rustyconover rustyconover released this 09 Jul 21:35

feat(http): surface init-request metadata to a producer's first process()

Over HTTP a producer stream's first turn runs inside the /init request, so unlike the pipe transport the worker never saw a real first tick. _run_http_producer_init now delivers the init request's Arrow metadata to the producer's first process() call.

This enables the VGI extension's HTTP result-cache conditional revalidation (304 / not_modified): the vgi.cache.if_none_match validator now reaches the worker before it produces. Also fixes the general gap where HTTP producers couldn't observe any first-exchange metadata (e.g. vgi_pushdown_filters).

Full suite green (1198 passed), mypy clean.

v0.23.0

Choose a tag to compare

@rustyconover rustyconover released this 07 Jul 22:25

Set a JS-readable _vgi_identity cookie derived from the OIDC id_token at the OAuth callback, read by the shared VGI landing page so it shows the signed-in email/name even when the bearer is an access token. Clear it on logout.

v0.22.0

Choose a tag to compare

@rustyconover rustyconover released this 30 Jun 19:29

Add Windows shared-memory transport — page-file-backed named CreateFileMapping + MapViewOfFile, mirroring the POSIX shm_open/mmap path and wire-compatible with the C++/Python/Go/Rust/Java peers.

Add size-gated pipe-vs-shm batch routing: batches below a byte threshold use the faster pipe, larger ones use shared memory. Threshold via VGI_RPC_SHM_MIN_BATCH_BYTES; default 128 KiB on POSIX, 1 MiB on Windows.

v0.21.0

Choose a tag to compare

@rustyconover rustyconover released this 26 Jun 00:04

Highlights

Raw-TCP socket transport

New TcpTransport / serve_tcp / tcp_connect — the network analog of the existing Unix-socket transport, speaking the same raw Arrow-IPC framing without the HTTP envelope. Loopback-only by default (127.0.0.1); no auth/TLS, so use HTTP for untrusted networks.

  • run_server --tcp [HOST:]PORT worker flag, emitting a TCP:<host>:<port> discovery line.
  • TransportKind.TCP; workers see ctx.kind == TCP.
  • Cross-language conformance: vgi-rpc-test --tcp and vgi-rpc-conformance --tcp.

Wire/HTTP intermediary helpers

  • Public framing helpers (read_request / write_request / build_error_stream).
  • find_state_token to extract the stream-state continuation token.
  • Intermediary helpers for protocol-version, unary-result, and content-encoding; public __upload_url__ wire contract.
  • HEAD /health so capability discovery doesn't degrade.

Full changelog: v0.20.6...v0.21.0

v0.20.6

Choose a tag to compare

@rustyconover rustyconover released this 22 Jun 18:23

Ships the aiohttp security bump unblocked by the test-mock migration. No library code changes — the package surface is unchanged from 0.20.5.

Security

  • The external extra now requires aiohttp>=3.14.1, picking up a batch of client-side fixes: CVE-2026-34993, -47265, and the -502xx / -542xx series (11 advisories).

Internal

  • Test suite migrated from aioresponses (unmaintained; broke on aiohttp 3.14's required ClientResponse.stream_writer kwarg) to aiointercept, which is what made lifting the <3.14 cap possible (#22).

v0.20.5

Choose a tag to compare

@rustyconover rustyconover released this 22 Jun 17:46

Maintenance release rolling up the post-0.20.4 merges. No library code changes — the package surface is unchanged from 0.20.4.

Dependencies

  • cryptography 48.0.0 → 49.0.0 (#17)
  • pywin32 311 → 312 (#18)
  • github-actions group: actions/checkout v7, astral-sh/setup-uv v8.2, codecov/codecov-action v7 (#15)
  • Hold aiohttp <3.14 (aioresponses 0.7.8 is incompatible with 3.14's new ClientResponse.stream_writer kwarg) and add a Dependabot ignore so it isn't re-proposed (#20)

Build / CI

  • Run pydoclint isolated via uvx so its docstring-parser-fork dependency no longer collides with the canonical docstring-parser in the project venv. The collision made the lint gate non-deterministic across unrelated dependency bumps; it is now stable (#21)

v0.20.4

Choose a tag to compare

@rustyconover rustyconover released this 18 Jun 16:06

Fix: shared-memory request-batch resolution

Fixes a cross-language shm transport bug where a client (the C++ vgi extension) routes a large single-row request batch through the shm side channel: the worker previously only resolved external-location pointers, so such requests tripped Expected 1 row in request batch, got 0 (seen on accumulate/* and table_buffering/* over shm).

  • _read_request now resolves shm-pointer request batches (static segment, connection-cached segment, or one attached from the request's own metadata).
  • New per-connection _ConnectionShm cache: the client names its segment once and reuses it for later offset-only batches; the worker attaches on first sight and holds it for the connection lifetime.
  • Responses route through shm only for exchanges the client signalled as shm (the C++ client resolves shm responses only for table_buffering_/aggregate methods), so inline control responses (bind, catalog_, transaction_*) are no longer wrongly shm-routed.

Includes a cross-language regression test.

🤖 Generated with Claude Code

v0.20.3

Choose a tag to compare

@rustyconover rustyconover released this 16 Jun 15:21

Hypothesis property-based tests (test-only)

Adds property-based testing infrastructure (dev-only hypothesis dependency). No library code changes — the published wheel is functionally identical to v0.20.2.

  • Round-trip coverage (tests/test_property_roundtrip.py): auto-discovers every ArrowSerializableDataclass subclass (51 today) and asserts deserialize(serialize(x)) == x, building a strategy per class from its annotations + resolved ARROW_SCHEMA so integer widths, decimal scale, and temporal units are respected. New dataclasses are covered automatically; unmodelled field shapes skip with a reason.
  • shm allocator stateful machine (tests/test_property_shm.py): a RuleBasedStateMachine drives random allocate/free sequences against the first-fit ShmAllocator plus a Python model, asserting the safety invariants every step — no overlap, in-bounds, sorted, header count consistent, and "allocate returns None only when no gap fits."

Both run in ~2.5s at committed settings and pass mypy strict + ruff.

Full diff: v0.20.2...v0.20.3

v0.20.2

Choose a tag to compare

@rustyconover rustyconover released this 16 Jun 14:41

Zero-suppression pydoclint gate

Every previously-frozen docstring violation is now fixed, so the pydoclint gate runs fully clean — no baseline, no per-code suppressions. New violations fail CI immediately.

Docstrings completed

  • DOC101/103Args: added to 17 private helpers, dunders, and exception __init__s.
  • DOC201Returns: added to 16 value-returning functions/methods, including the Stream client-side stubs and conformance Protocol producers.
  • DOC602/603/606Message ClassVar constants documented in the class docstring, data fields moved to __init__ Args:; HttpTransientError.__init__ documented.
  • DOC304WorkerPool constructor docs moved from the class docstring to __init__.

Config

  • Enabled allow_init_docstring (validate __init__ Args: on the method), matching vgi-python's current config.
  • Removed the baseline and the non-functional ignore key (pydoclint 0.8.x has no per-code ignore).

No public API changes — docstrings, annotations, and lint config only.

Full diff: v0.20.1...v0.20.2