Skip to content

Fix issues #188–#193#197

Merged
farhan-syah merged 11 commits into
mainfrom
fix/issues-188-193
Jul 21, 2026
Merged

Fix issues #188–#193#197
farhan-syah merged 11 commits into
mainfrom
fix/issues-188-193

Conversation

@farhan-syah

@farhan-syah farhan-syah commented Jul 21, 2026

Copy link
Copy Markdown
Member

Summary

  • make descriptor replay and DDL preparation safe across nodes and restarts
  • clean tenant-owned objects during tenant teardown
  • correct graph statistics, rollback, replay, and temporal behavior
  • normalize and fold regclass expressions across nested plan shapes
  • strengthen aggregate cache identity
  • fix native transaction routing, response framing, Calvin database scope, and CDC publication

Correctness hardening

  • Graph edge undo: record the PutEdge/DeleteEdge compensation only after the edge-store write is durable and before the fallible CSR mutation, so a rejected dangling-edge insert (or a failed write) no longer leaves a phantom undo entry that corrupts bitemporal history on rollback.
  • Redo payloads: replace the positional edge put/delete tuples with shared, map-encoded EdgePutRedo / EdgeDeleteRedo structs used at every encode/decode site, so a field addition (e.g. system_from) can no longer silently drift arity across sites. Requires zerompk 0.6.
  • Calvin completion: a failed local flush/apply of an already-committed verdict now forces infra-abort completion (releasing locks and advancing the epoch watermark) instead of silently parking the transaction forever.
  • Calvin submit: wait through a bounded backoff for the sequencer-group leader election before a cross-shard submit, so the first write on a freshly-ready node no longer races the election.
  • Session commit: re-verify local vShard ownership immediately before the durable transaction-redo WAL append, so a mid-commit leadership handoff cannot leave an orphaned redo record while the client is told the commit aborted.
  • descriptor_stamp: a regressed carried version paired with a strictly-newer HLC is now surfaced as a DescriptorVersionAnomaly rather than silently accepted as historical.
  • regclass: fold and existence-validate catalog expressions inside the vector / multi-vector / sparse / text / hybrid / spatial / recursive plan shapes, not just the base scan/DML shapes.
  • Native client: discard stale response frames left by a prior fan-out request instead of misattributing them to the current request.
  • DROP TENANT audit: record the count of purged tenant-owned objects instead of misreporting "no owned objects required reassignment".
  • Aggregate cache key: bundle the full query-shape inputs into a params struct (removing a clippy too_many_arguments allow).

Closes #188
Closes #189
Closes #190
Closes #191
Closes #192
Closes #193

@farhan-syah
farhan-syah marked this pull request as draft July 21, 2026 04:39
… redo payload structs

Undo entries for edge put/delete were pushed before the edge-store
write completed, so a rejected insert (e.g. to a soft-deleted node) or
a failed store write left a phantom compensation entry that would
soft-delete or re-insert an edge that never existed on rollback.
Compensation is now recorded inside execute_edge_put_with_undo /
execute_edge_delete_with_undo, immediately after the edge-store version
is durable and before the fallible CSR mutation.

Also replace the positional-tuple encoding of edge put/delete redo
payloads with named EdgePutRedo/EdgeDeleteRedo structs (map-encoded via
zerompk 0.6) shared by every encode/decode site, removing the
hand-maintained fallback-tuple decode that could silently drift out of
sync with the encoder.
A fan-out query for a preceding request could leave stale trailing
frames on the wire after that request's terminal frame was already
returned to its caller. Check the response sequence number against
the request that was sent and drop any frame that doesn't match,
instead of misattributing another request's rows or status as this
request's response.
VectorSearch, TextSearch, SpatialScan, RecursiveScan, MultiVectorSearch,
SparseSearch, HybridSearch, and HybridSearchTriple plans fell through
catalog_fold's and catalog_plan_validate's wildcard arm, so their
filters and projections never had catalog identifiers resolved or
validated. Add explicit arms that fold/validate each plan's filters and
projection.
…e replay

A carried version lower than the persisted prior was always treated as
an already-applied stale replay, even when its HLC clock was strictly
newer than the persisted record's. Only mark it stale when the clock is
not ahead of the persisted one; a regressed version paired with a newer
HLC is a genuine anomaly and must still be rejected.
A failed flush/drop response or executor apply left the transaction
parked with its locks held instead of completing, wedging every txn
queued behind those keys and freezing the vShard's epoch watermark
(which anchors cross-shard BEGIN snapshots) with nothing left to
re-drive the entry. Since the outcome is already globally durable (the
verdict is committed and the resolved redo already WAL-appended), a
local failure is an infrastructure event, not an outcome change:
surface the infra abort and force completion, matching the
forward-progress contract the dispatch-failure path already follows.
…-shard submit

submit_calvin_routed failed immediately when no sequencer-group leader
was elected yet, which could fail a client's very first write during
the brief post-startup election window or a re-election gap. Since a
leader_id of 0 also guarantees nothing has been submitted, poll with a
bounded backoff schedule and re-read the leader before giving up; only
a genuinely leaderless cluster exhausts the schedule and surfaces the
typed error.
…trail

DROP USER during tenant teardown purged the user's owned objects
outright but recorded the same "no owned objects required reassignment"
audit detail as a user with nothing owned, misreporting a destructive
purge as a no-op. purge_owned_for_tenant_teardown now returns the
purged count, and the audit detail distinguishes reassignment, no
ownership, and teardown purge outcomes.
… append

run_commit resolves a vShard as Local, but a leadership handoff can
land during the ResolveTxn await that follows. Without a re-check right
before the durable append, the transaction redo could be written to a
WAL this node no longer owns, and the later batch dispatch (which
re-resolves leadership) would then reject the now-non-local commit,
leaving an orphaned durable redo record while the client is told the
commit aborted. Abort before any durable write instead, so the failure
stays side-effect-free and the client's retry can route through
Calvin's replicated barrier.
Replace aggregate_cache_key's nine positional arguments (and its
too_many_arguments allow) with an AggregateCacheKeyInputs struct,
naming the complete query shape that constitutes the cache identity.
# Conflicts:
#	nodedb/src/control/server/native/dispatch/transaction.rs
@farhan-syah
farhan-syah marked this pull request as ready for review July 21, 2026 07:55
@farhan-syah
farhan-syah merged commit f679209 into main Jul 21, 2026
2 checks passed
@farhan-syah
farhan-syah deleted the fix/issues-188-193 branch July 21, 2026 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment