Skip to content

feat: add Convex collection with automatic pagination and durable offline writes - #2230

Open
tannerlinsley wants to merge 4 commits into
mainfrom
taren/convex-db-collection
Open

tannerlinsley wants to merge 4 commits into
mainfrom
taren/convex-db-collection

Conversation

@tannerlinsley

@tannerlinsley tannerlinsley commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

🎯 Changes

Add @tanstack/convex-db-collection so TanStack DB can consume reactive Convex queries through either the JavaScript or React client. Includes typed query arguments, full-row synchronization, optimistic mutation handlers, and automatic pagination using public cursor boundaries and server split hints.

Add durable offline mutation replay through the existing offline-transactions engine. An opt-in codec preserves Convex int64, special floats, binary values, and removed fields without changing the default outbox format. Applications supply an atomic, authenticated idempotency-key receipt in their server mutation.

Independent reviews found and fixed shared concrete-options ownership and repeated copying of incomplete pagination results. Direct collection construction now rejects reused options, separate DbClient materializations remain independent, and initial pagination copies rows only when the full result is ready.

🔬 Oracle Coverage

Owners and limits are recorded in docs/contributing/oracle-coverage.md:

  • Snapshot, ownership, pagination, offline bridge, native-value receiving owners, plus the shared transaction serializer owner.
  • Ownership: original admission fails seed 20261022/path 2:0:0, and a two-owner mutation settles stale data. Fixed/random owner histories and the focused regression pass after claims are enforced.
  • Pagination: omission of continuation or required splits fails public row comparison. Initial row-copy work fails the original at seed 20261022/path 0:0:0:0 and passes after deferred flattening. Cache inspection still grows quadratically with page count during sequential acquisition.
  • Serialization: original BigInt admission fails seed 20261022/path 0:0:0:0; repaired native-value round trips and incompatible-codec byte-retention checks pass. Legacy default formats retain their behavior.
  • Actual isolated local Convex backend: both SDK clients, real server splits, and 36 generated SIGKILL histories across insert/update/delete, including server acceptance before lost durable acknowledgement. Observations compare restored collection rows, final server rows, and exactly one receipt.

Validation: 43 adapter checks, 302 offline-engine tests, seven real-backend tests, build, typecheck, lint, publint, and sherif. Production size against current main: new adapter 2,633 gzip / 2,381 brotli bytes with dependencies external; shared outbox +192 gzip / +174 brotli. Existing tree-shaken app scenarios are unchanged. The codec hook and ownership claim are required for native values and independent collection state; shared snapshot and outbox machinery avoid duplicate implementations.

Limits: bounded finite queries, one pending mutation per real-backend crash history, insert-only native payload variation in that backend witness. These tests do not establish cloud availability, filesystem power-loss atomicity, cross-collection publication, or backend multi-tab/batch replay.

✅ Checklist

  • Tested affected packages locally, including their complete test suites and the real backend suite.

🚀 Release Impact

  • Published code changes have a changeset for the new adapter and offline-transactions codec API.

Summary by CodeRabbit

  • New Features
    • Added a Convex collection adapter for live query subscriptions, automatic pagination, and optimistic updates.
    • Added durable offline mutations with server-side idempotency and optional codecs for preserving Convex values across restarts.
  • Documentation
    • Added setup guides and API references for Convex collections, pagination, offline mutations, and mutation value codecs.
  • Tests
    • Added automated coverage for synchronization, pagination, offline recovery, and value preservation.

@changeset-bot

changeset-bot Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 4f97ab4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@tanstack/convex-db-collection Minor
@tanstack/offline-transactions Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5b3d1f8b-b5c2-404c-9a93-02bff85ce1c8
📥 Commits

Reviewing files that changed from the base of the PR and between dc45f5c and 4f97ab4.

⛔ Files ignored due to path filters: 1
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing: 59
  • .changeset/convex-collection.md
  • .github/workflows/e2e-tests.yml
  • .prettierignore
  • docs/collections/convex-collection.md
  • docs/config.json
  • docs/contributing/oracle-coverage.md
  • docs/guides/offline-transactions.md
  • docs/reference/convex-db-collection/functions/convexCollectionOptions.md
  • docs/reference/convex-db-collection/functions/convexPaginatedCollectionOptions.md
  • docs/reference/convex-db-collection/index.md
  • docs/reference/convex-db-collection/interfaces/ConvexCollectionUtils.md
  • docs/reference/convex-db-collection/offline/functions/convexOfflineMutation.md
  • docs/reference/convex-db-collection/offline/index.md
  • docs/reference/convex-db-collection/offline/variables/convexMutationCodec.md
  • docs/reference/convex-db-collection/type-aliases/ConvexCollectionClient.md
  • docs/reference/convex-db-collection/type-aliases/ConvexCollectionConfig.md
  • docs/reference/convex-db-collection/type-aliases/ConvexPaginatedCollectionConfig.md
  • eslint.config.js
  • packages/convex-db-collection/README.md
  • packages/convex-db-collection/e2e/backend-oracle.test.ts
  • packages/convex-db-collection/e2e/backend/.gitignore
  • packages/convex-db-collection/e2e/backend/convex.json
  • packages/convex-db-collection/e2e/backend/convex/schema.ts
  • packages/convex-db-collection/e2e/backend/convex/tasks.ts
  • packages/convex-db-collection/e2e/backend/convex/tsconfig.json
  • packages/convex-db-collection/e2e/backend/package.json
  • packages/convex-db-collection/e2e/offline-child.ts
  • packages/convex-db-collection/e2e/setup.ts
  • packages/convex-db-collection/package.json
  • packages/convex-db-collection/src/index.ts
  • packages/convex-db-collection/src/offline.ts
  • packages/convex-db-collection/src/options.ts
  • packages/convex-db-collection/src/paginated-options.ts
  • packages/convex-db-collection/src/pagination.ts
  • packages/convex-db-collection/src/subscription.ts
  • packages/convex-db-collection/tests/collection.test.ts
  • packages/convex-db-collection/tests/helpers.ts
  • packages/convex-db-collection/tests/offline-oracle.property.test.ts
  • packages/convex-db-collection/tests/offline.test.ts
  • packages/convex-db-collection/tests/ownership-oracle.property.test.ts
  • packages/convex-db-collection/tests/pagination-oracle.property.test.ts
  • packages/convex-db-collection/tests/sdk.test.ts
  • packages/convex-db-collection/tests/snapshot-oracle.property.test.ts
  • packages/convex-db-collection/tests/types.test-d.ts
  • packages/convex-db-collection/tests/values-oracle.property.test.ts
  • packages/convex-db-collection/tsconfig.docs.json
  • packages/convex-db-collection/tsconfig.json
  • packages/convex-db-collection/vite.config.ts
  • packages/convex-db-collection/vitest.e2e.config.ts
  • packages/offline-transactions/src/OfflineExecutor.ts
  • packages/offline-transactions/src/index.ts
  • packages/offline-transactions/src/outbox/OutboxManager.ts
  • packages/offline-transactions/src/outbox/TransactionSerializer.ts
  • packages/offline-transactions/src/react-native/index.ts
  • packages/offline-transactions/src/types.ts
  • packages/offline-transactions/tests/transaction-serializer-oracle.property.test.ts
  • scripts/ci-tests.mjs
  • scripts/generate-docs.ts
  • scripts/size-compare.mjs

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

Adds a Convex collection package with query subscriptions, full-row synchronization, reactive pagination, and offline mutation support. The offline transaction package gains optional codecs for durable mutation values. The pull request also adds package tests, a local Convex backend fixture, documentation, and CI integration.

Changes: Convex collection and durable offline writes

Layer / File(s) Summary
Mutation codec support in the offline outbox
packages/offline-transactions/src/*, packages/offline-transactions/tests/*, packages/convex-db-collection/src/offline.ts, packages/convex-db-collection/tests/values-oracle.property.test.ts, docs/guides/offline-transactions.md, docs/contributing/oracle-coverage.md
Adds an optional mutation codec contract and version 4 transaction serialization. The executor passes configured codecs to the outbox, which reports codec read failures with transaction IDs. The Convex codec encodes and decodes tagged values.
Convex query subscriptions and full-row synchronization
packages/convex-db-collection/src/{index,options,subscription}.ts, packages/convex-db-collection/tests/{collection,helpers,sdk,snapshot,ownership,types}*, docs/collections/convex-collection.md, docs/reference/convex-db-collection/*
Adds typed query subscriptions and collection options. The adapter validates query results and applies complete keyed snapshots. It exposes synchronization utilities and wraps mutation handlers to refresh from the subscription.
Reactive pagination
packages/convex-db-collection/src/{paginated-options,pagination}.ts, packages/convex-db-collection/tests/pagination-oracle.property.test.ts, packages/convex-db-collection/e2e/backend-oracle.test.ts, docs/collections/convex-collection.md, docs/reference/convex-db-collection/type-aliases/ConvexPaginatedCollectionConfig.md
Adds paginated collection options and page subscriptions. Split hints can replace page ranges, and rows are published when the active ranges have values.
Convex offline mutations and durable replay
packages/convex-db-collection/src/offline.ts, packages/convex-db-collection/tests/{offline,offline-oracle.property}.test.ts, packages/convex-db-collection/e2e/*, packages/convex-db-collection/e2e/backend/*, docs/collections/convex-collection.md, docs/reference/convex-db-collection/offline/*
Adds a Convex mutation bridge that passes the durable idempotency key and waits for configured collections to synchronize. The local backend tests paginated queries and insert, update, and delete replay histories after process termination.
Package, documentation, and test integration
packages/convex-db-collection/{package.json,README.md,tsconfig*,vite.config.ts,vitest.e2e.config.ts}, .github/workflows/e2e-tests.yml, scripts/*, docs/config.json, .changeset/convex-collection.md
Adds package build and test configuration, CI and documentation generation entries, navigation, release notes, and package documentation. The size comparison handles package directories that are absent from one compared tree.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant OfflineExecutor
  participant OutboxManager
  participant convexOfflineMutation
  participant ConvexClient
  participant Collection
  OfflineExecutor->>OutboxManager: persist offline transaction
  OutboxManager->>convexOfflineMutation: replay transaction with idempotency key
  convexOfflineMutation->>ConvexClient: call mutation
  ConvexClient-->>convexOfflineMutation: return mutation result
  convexOfflineMutation->>Collection: await collection synchronization
Loading

Suggested reviewers: kyleamathews

Merge Risk: ⚪ Minimal · up to 4f97a

This PR adds a Convex collection adapter and opt-in durable mutation codecs. No concrete merge-blocking defect was found in the supplied changes. Per the PR description, the author ran the adapter, offline-engine and real-backend tests. This review did not rerun them. The listed limits are documented in the PR and do not block merge.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 34 files. (25 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly summarizes the primary changes: the Convex collection, automatic pagination, and durable offline writes.
Description check Passed The description includes the required Changes, Oracle Coverage, Checklist, and Release Impact sections. It provides implementation scope, validation evidence, known limits, and confirms the changeset.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 34 files. (25 skipped: 25 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@socket-security

socket-security Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​convex@​1.47.0991008898100

View full report

@github-actions

Copy link
Copy Markdown
Contributor

Size Change: 0 B

Total Size: 220 kB

ℹ️ View Unchanged
Filename Size
packages/db/dist/esm/client.js 4.22 kB
packages/db/dist/esm/collection-options-brand.js 219 B
packages/db/dist/esm/collection-options.js 258 B
packages/db/dist/esm/collection/change-events.js 1.76 kB
packages/db/dist/esm/collection/changes.js 2.96 kB
packages/db/dist/esm/collection/cleanup-queue.js 901 B
packages/db/dist/esm/collection/config-errors.js 547 B
packages/db/dist/esm/collection/events.js 481 B
packages/db/dist/esm/collection/index.js 5.27 kB
packages/db/dist/esm/collection/indexes.js 2.3 kB
packages/db/dist/esm/collection/lifecycle.js 2.94 kB
packages/db/dist/esm/collection/mutations.js 2.99 kB
packages/db/dist/esm/collection/state.js 7.36 kB
packages/db/dist/esm/collection/subscription.js 8.89 kB
packages/db/dist/esm/collection/sync.js 5.4 kB
packages/db/dist/esm/collection/validate-config.js 1.17 kB
packages/db/dist/esm/deferred.js 207 B
packages/db/dist/esm/error-message.js 642 B
packages/db/dist/esm/errors.js 6.54 kB
packages/db/dist/esm/event-emitter.js 1.06 kB
packages/db/dist/esm/index.js 4.52 kB
packages/db/dist/esm/indexed-db-wrapper.js 1.91 kB
packages/db/dist/esm/indexed-db.js 3.79 kB
packages/db/dist/esm/indexes/auto-index.js 924 B
packages/db/dist/esm/indexes/base-index.js 1.92 kB
packages/db/dist/esm/indexes/basic-index.js 1.93 kB
packages/db/dist/esm/indexes/btree-index.js 1.62 kB
packages/db/dist/esm/indexes/index-registry.js 916 B
packages/db/dist/esm/indexes/reverse-index.js 1.01 kB
packages/db/dist/esm/live-query-adapter.js 338 B
packages/db/dist/esm/live-query-observer.js 4.8 kB
packages/db/dist/esm/live-query-options.js 1.41 kB
packages/db/dist/esm/live-query-window-controller.js 4.27 kB
packages/db/dist/esm/local-only.js 1.08 kB
packages/db/dist/esm/local-storage.js 4.58 kB
packages/db/dist/esm/optimistic-action.js 359 B
packages/db/dist/esm/paced-mutations.js 1.11 kB
packages/db/dist/esm/persisted-readiness.js 195 B
packages/db/dist/esm/proxy.js 3.51 kB
packages/db/dist/esm/query/builder/clone-query.js 849 B
packages/db/dist/esm/query/builder/functions.js 1.56 kB
packages/db/dist/esm/query/builder/index.js 7.48 kB
packages/db/dist/esm/query/builder/query-ir.js 312 B
packages/db/dist/esm/query/builder/ref-proxy-identity.js 292 B
packages/db/dist/esm/query/builder/ref-proxy.js 1.67 kB
packages/db/dist/esm/query/builder/wrapper-identity.js 221 B
packages/db/dist/esm/query/compiler/evaluators.js 2.51 kB
packages/db/dist/esm/query/compiler/expressions.js 603 B
packages/db/dist/esm/query/compiler/group-by.js 4.48 kB
packages/db/dist/esm/query/compiler/index.js 10.3 kB
packages/db/dist/esm/query/compiler/joins.js 3.42 kB
packages/db/dist/esm/query/compiler/lazy-targets.js 1.11 kB
packages/db/dist/esm/query/compiler/order-by.js 2.8 kB
packages/db/dist/esm/query/compiler/parent-routes.js 319 B
packages/db/dist/esm/query/compiler/query-equivalence.js 568 B
packages/db/dist/esm/query/compiler/route-metadata.js 1.24 kB
packages/db/dist/esm/query/compiler/select.js 1.5 kB
packages/db/dist/esm/query/effect.js 5.36 kB
packages/db/dist/esm/query/equality-conjunct.js 486 B
packages/db/dist/esm/query/equality-value-identity.js 735 B
packages/db/dist/esm/query/expression-helpers.js 1.61 kB
packages/db/dist/esm/query/ir-stable-identity.js 4.2 kB
packages/db/dist/esm/query/ir.js 2.2 kB
packages/db/dist/esm/query/join-conditions.js 257 B
packages/db/dist/esm/query/live-query-collection.js 391 B
packages/db/dist/esm/query/live/bucket-facade-adapter.js 3.17 kB
packages/db/dist/esm/query/live/collection-config-builder.js 6.9 kB
packages/db/dist/esm/query/live/collection-registry.js 264 B
packages/db/dist/esm/query/live/collection-subscriber.js 2.3 kB
packages/db/dist/esm/query/live/graph-scheduler.js 305 B
packages/db/dist/esm/query/live/internal.js 145 B
packages/db/dist/esm/query/live/materialized-pipeline.js 2.39 kB
packages/db/dist/esm/query/live/ordered-source-loader.js 6.46 kB
packages/db/dist/esm/query/live/subset-demand-controller.js 1.76 kB
packages/db/dist/esm/query/live/utils.js 1.35 kB
packages/db/dist/esm/query/optimizer.js 2.95 kB
packages/db/dist/esm/query/pooled-live-query.js 4.13 kB
packages/db/dist/esm/query/query-once.js 359 B
packages/db/dist/esm/query/runtime-reference-identity.js 630 B
packages/db/dist/esm/query/subset-dedupe.js 493 B
packages/db/dist/esm/scheduler.js 1.23 kB
packages/db/dist/esm/SortedMap.js 1.6 kB
packages/db/dist/esm/strategies/commit-completion.js 156 B
packages/db/dist/esm/strategies/debounceStrategy.js 658 B
packages/db/dist/esm/strategies/queueStrategy.js 523 B
packages/db/dist/esm/strategies/serial-pacer.js 486 B
packages/db/dist/esm/strategies/throttleStrategy.js 738 B
packages/db/dist/esm/sync-persistence.js 626 B
packages/db/dist/esm/sync-receipt.js 211 B
packages/db/dist/esm/transaction-commit-work.js 351 B
packages/db/dist/esm/transactions.js 4.61 kB
packages/db/dist/esm/utils.js 1.67 kB
packages/db/dist/esm/utils/array-utils.js 270 B
packages/db/dist/esm/utils/browser-polyfills.js 304 B
packages/db/dist/esm/utils/btree.js 3.02 kB
packages/db/dist/esm/utils/callbacks.js 219 B
packages/db/dist/esm/utils/comparison.js 1.94 kB
packages/db/dist/esm/utils/cursor.js 753 B
packages/db/dist/esm/utils/error.js 262 B
packages/db/dist/esm/utils/get-or-create.js 155 B
packages/db/dist/esm/utils/index-optimization.js 2.58 kB
packages/db/dist/esm/utils/source-record.js 140 B
packages/db/dist/esm/utils/type-guards.js 230 B
packages/db/dist/esm/utils/uuid.js 530 B
packages/db/dist/esm/virtual-props.js 413 B
packages/react-db/dist/esm/DbProvider.js 317 B
packages/react-db/dist/esm/development.js 190 B
packages/react-db/dist/esm/HydrationBoundary.js 263 B
packages/react-db/dist/esm/index.js 330 B
packages/react-db/dist/esm/live-query-internals.js 282 B
packages/react-db/dist/esm/source-id-bindings.js 746 B
packages/react-db/dist/esm/useLiveInfiniteQuery.js 2.15 kB
packages/react-db/dist/esm/useLiveQuery.js 3.38 kB
packages/react-db/dist/esm/useLiveQueryEffect.js 504 B
packages/react-db/dist/esm/useLiveSuspenseQuery.js 1.33 kB
packages/react-db/dist/esm/usePacedMutations.js 429 B

compressed-size-action::db-and-react-package-size

@pkg-pr-new

pkg-pr-new Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
More templates

@tanstack/angular-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/angular-db@2230

@tanstack/browser-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/browser-db-sqlite-persistence@2230

@tanstack/capacitor-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/capacitor-db-sqlite-persistence@2230

@tanstack/cloudflare-durable-objects-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@2230

@tanstack/convex-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/convex-db-collection@2230

@tanstack/db

npm i https://pkg.pr.new/TanStack/db/@tanstack/db@2230

@tanstack/db-ivm

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-ivm@2230

@tanstack/db-sqlite-persistence-core

npm i https://pkg.pr.new/TanStack/db/@tanstack/db-sqlite-persistence-core@2230

@tanstack/electric-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/electric-db-collection@2230

@tanstack/electron-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/electron-db-sqlite-persistence@2230

@tanstack/expo-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/expo-db-sqlite-persistence@2230

@tanstack/node-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/node-db-sqlite-persistence@2230

@tanstack/offline-transactions

npm i https://pkg.pr.new/TanStack/db/@tanstack/offline-transactions@2230

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/powersync-db-collection@2230

@tanstack/query-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/query-db-collection@2230

@tanstack/react-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/react-db@2230

@tanstack/react-native-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/react-native-db-sqlite-persistence@2230

@tanstack/react-router-with-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/react-router-with-db@2230

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/rxdb-db-collection@2230

@tanstack/solid-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/solid-db@2230

@tanstack/svelte-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/svelte-db@2230

@tanstack/tauri-db-sqlite-persistence

npm i https://pkg.pr.new/TanStack/db/@tanstack/tauri-db-sqlite-persistence@2230

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/TanStack/db/@tanstack/trailbase-db-collection@2230

@tanstack/vue-db

npm i https://pkg.pr.new/TanStack/db/@tanstack/vue-db@2230

commit: 4f97ab4

@KyleAMathews

Copy link
Copy Markdown
Collaborator

I reviewed head 4f97ab4 against the oracle rubric and ran the 43 adapter checks, 51 serializer-oracle tests, and seven real-backend tests. They pass, but I found one reproducible contract failure and several gaps in what the oracles establish.

Source acceptance is the blocking issue. awaitSync() in packages/convex-db-collection/src/options.ts:154–168,211–213 can resolve while the Collection is idle, waiting for its first result, or in error after subscription startup fails. I called convexOfflineMutation with a real idle Collection and a resolving mutation; the bridge returned while the Collection remained idle with zero subscriptions. This contradicts the guide’s promise that the bridge waits for each configured Collection to accept its source snapshot. The backend fixture preloads before starting the executor, while the offline oracle stubs awaitSync, so both miss cold replay. Please add a real-Collection bridge history for idle, pending-first-result, and failed-startup states, then make successful settlement require actual source acceptance.

The pagination API has a separate scalability question. convexPaginatedCollectionOptions follows every cursor, retains the page subscriptions, and materializes the entire result before preload() completes. pageSize controls request chunks; it does not bound client memory or initial load time. The coverage map also records quadratic cache scans as page count grows. I understand why a complete mirror supports arbitrary DB-local filters and joins, and why fixed cursor ranges help reactive correctness. I do not see a reason that this should be the only pagination mode.

I suggest describing the current API explicitly as a complete eager mirror and offering an on-demand option through DB’s existing loadSubset contract. Core already supplies cursor-aware subset requests (#960); Query Collection’s opaque-cursor pager in #1824 is a useful design and oracle reference. Its Query cache implementation would not transfer directly to Convex subscriptions, but it shows how to fetch only the pages needed for a requested window, reuse a loaded prefix, and distinguish a backend cursor from DB’s query cursor hints. The Convex option would need an application-supplied translation for filters and ordering, and should reject requests its server function cannot fulfill. An oracle should compare requested windows, network work, active subscriptions, and release behavior as demand grows and shrinks.

The pagination oracle also needs stronger evidence:

  • tests/pagination-oracle.property.test.ts:85–106 can generate a split cursor outside the returned page’s cursor range. For eight rows and pageSize: 1, it can return continueCursor: "0" with splitCursor: "3". Those are not legal Convex split responses.
  • Its two properties share CONVEX_ORACLE_SEED/PATH. Replaying a recorded row-history path passes that property but fails the unrelated work property with Unable to replay, got wrong path. Each property needs a direct replay selector.
  • It compares only after all ranges arrive and sorts actual and expected rows by ID. It therefore cannot catch partial publication while a new page is pending or a page-order error. Please add an intermediate public-row checkpoint and an order-sensitive observation.

The real-backend crash test reaches a valuable SIGKILL cut, but e2e/backend-oracle.test.ts:259–281 checks one receipt and final rows. The fixture’s insert, patch, and delete have the same visible result if applied twice, so those assertions do not prove one server application. The test also does not compare the receipt’s key with the durable key. In e2e/offline-child.ts:171–181, restored Collection rows are checked only by text; rank and native payload are checked on the server, not in the Collection. A non-idempotent application counter, key comparison, and full public-row comparison would make the stated claims distinguishable. Native payload variation currently reaches real-backend replay only for inserts.

Finally, the snapshot oracle exercises raw arrays but not a successful select projection, and the ownership oracle’s controlled client cannot establish live updates to two listeners after one sibling is cleaned up. These can remain stated limits, but the broader claims should wait for receiving witnesses. I also could not find the rubric’s exact-head review record for the new oracle owners.

I would address the source-acceptance failure before merge and make an explicit API decision about bounded eager use versus on-demand loading.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants