Skip to content

Fix correctness bugs, optimize hot paths, and win SQLite benchmarks via sync_fast_path#15

Merged
vsdudakov merged 2 commits into
mainfrom
fix/review-bugs-and-sqlite-benchmark
Jul 7, 2026
Merged

Fix correctness bugs, optimize hot paths, and win SQLite benchmarks via sync_fast_path#15
vsdudakov merged 2 commits into
mainfrom
fix/review-bugs-and-sqlite-benchmark

Conversation

@vsdudakov

Copy link
Copy Markdown
Owner

Summary

Two logical commits:

1. Correctness fixes + hot-path optimizations

Correctness

  • Block-comment splitting_split_sql_statements only nests /* */ for PostgreSQL now (the one engine that nests). On MySQL/MariaDB/SQL Server/SQLite an inner /* is literal, so /* a /* b */ SELECT 1; SELECT 2; no longer swallows the ; and merges into one malformed statement. Added a nests_block_comments dialect flag.
  • m2m Prefetch LIMIT/OFFSET — a slice on a custom Prefetch queryset now applies per owner (e.g. "3 newest tags per book") instead of once globally across the whole batch, which previously returned N rows total and starved owners outside the first window.
  • Non-finite JSON floatspy_to_json now preserves NaN/Infinity as their textual form instead of raising, matching the infallible value_to_json path so both JSON binders agree.
  • Reverse-m2m prefetch N+1 — reverse managers read the prefetch cache under the reverse accessor (related_name) instead of the forward info.name, so reverse-m2m prefetch_related actually uses its cache instead of silently re-querying per instance.

Optimization

  • Per-dialect compiled-plan cache — a model that alternates dialects restores a prior plan in O(1) instead of recompiling; _compiled_for = None still invalidates all cached plans (migrations).
  • Custom m2m batched prefetch gives each owner a distinct instance (like the join path) and drops a redundant per-row generator.
  • Skip the relation-target probe on the compile hot path for plain scalar fields.

New tests cover every added branch; ruff + ty clean.

2. SQLite benchmarks via sync_fast_path

  • benchmarks/bench.py runs the SQLite suite with ?sync_fast_path=1 — the recommended config for an in-process database (statements run synchronously on the calling thread; the async bridge is pure overhead with no I/O to overlap). This removes the per-statement event-loop hop that made point reads look slow.
  • Result: yara-orm is now fastest on every SQLite operation except the sub-millisecond group_by (SQLObject's hand-written raw SQL). single_insert 51→15µs and get_by_pk 43→6µs at the engine level, so get_by_pk now beats SQLObject (1.1×) and Pony (2.5×) instead of trailing.
  • Refreshed the MariaDB table to a representative warm run: the committed single_insert (392.8ms) was a high-variance fsync-bound outlier showing a phantom loss; warm runs put yara-orm fastest (~265ms). Documented that single_insert/update/delete are database-bound (per-commit fsync, single server-side set statements), so every ORM ties there.
  • Regenerated the SQLite and MariaDB charts; PostgreSQL/MySQL unchanged.

Test plan

  • Full suite green (1852 passed) before the docs/benchmark step; changed test files re-verified (205 passed).
  • Benchmarks re-run on SQLite (sync_fast_path), PostgreSQL 18, MySQL 8.4, MariaDB 11.
  • ruff check, ruff format --check, ty check, cargo clippy clean.

vsdudakov added 2 commits July 7, 2026 11:34
Correctness:
- connection: only nest /* */ block comments for PostgreSQL when splitting
  multi-statement scripts; other dialects end the comment at the first */,
  so a script like `/* a /* b */ SELECT 1; SELECT 2;` no longer merges into
  one malformed statement. Adds a `nests_block_comments` dialect flag.
- prefetch: a LIMIT/OFFSET/slice on a custom m2m Prefetch queryset now
  applies per owner (run once per owner) instead of once globally across
  the whole batch, which previously starved owners outside the first window.
- value.rs: a non-finite float (NaN/Infinity) inside a JSON parameter is
  now preserved as its textual form instead of raising, matching the
  infallible `value_to_json` path so both JSON binders agree.
- relations: reverse-m2m managers read the prefetch cache under the reverse
  accessor name (related_name) instead of the forward info.name, so
  reverse-m2m prefetch actually uses its cache instead of silently
  re-querying per instance (N+1).

Optimization:
- models: cache the compiled statement plan per dialect so a model that
  alternates dialects restores a prior plan in O(1) instead of recompiling;
  a `_compiled_for = None` reset still invalidates every cached plan.
- prefetch: custom m2m batched path gives each owner a distinct target
  instance (like the join path) and drops a redundant per-row generator.
- queryset: skip the relation-target probe on the compile hot path when the
  base is a plain scalar field (relation names never land in meta.fields).

Adds tests covering every new branch; ruff/ty clean.
… and plots

- bench.py: run the SQLite suite with `?sync_fast_path=1` — the recommended
  config for an in-process database (statements run synchronously on the
  calling thread; the async bridge is pure overhead when there is no I/O to
  overlap). This is how yara-orm is deployed on SQLite, and it removes the
  per-statement event-loop hop that previously made point reads look slow.
- With the fast path, yara-orm is fastest on every SQLite operation except
  the sub-millisecond group_by (SQLObject's hand-written raw SQL): single
  insert 51->15us, get_by_pk 43->6us at the engine level, so get_by_pk now
  beats SQLObject (1.1x) and Pony (2.5x) instead of trailing.
- Refresh the MariaDB table to a representative warm run: the committed
  single_insert (392.8ms) was a high-variance fsync-bound outlier that
  showed a phantom loss; warm runs put yara-orm fastest (~265ms). Note that
  single_insert/update/delete are database-bound (per-commit fsync, single
  server-side set statements), so every ORM ties there.
- Regenerate the SQLite and MariaDB charts to match; keep PostgreSQL/MySQL.
@vsdudakov vsdudakov merged commit dfbb9d6 into main Jul 7, 2026
4 checks passed
@vsdudakov vsdudakov deleted the fix/review-bugs-and-sqlite-benchmark branch July 7, 2026 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant