Database research#253
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
mudigal
left a comment
mudigal
left a comment
There was a problem hiding this comment.
This is a great start and a genuinely useful foundation. A real, runnable harness over four engines, both architectures measured rather than assumed, results committed alongside, and unusually honest caveats (the tmpfs / "relative-not-absolute" framing especially). This is exactly the kind of evidence base the DB decision needs.
Sharing some things I'd love to see taken care of as this evolves — not blockers, just what I think has to be in scope before the recommendation is treated as final, since the storage-provider design is moving under it.
The main one: fold in the CDC + mutable, content-addressed design (PR #209) as an input. Right now the storage-provider workload models a sequential, position-keyed, append-only log, but the real store is content-hash-keyed, deduplicated, and mutable. That gap touches the recommendation in three places worth working through:
- Deletion: Sharded wins largely on "delete = unlink, 100% reclaimed." That holds for whole-bucket expiry, but the common event in a mutable design is version churn — edits orphan old content-addressed nodes inside a live bucket, which needs ref-counted GC over random hash keys (the same scattered in-place-delete pattern that counts against shared). Worth measuring that path explicitly.
- Engine choice. The real write path does a read-before-write dedup check on random 32-byte hashes, and reads walk random hashes — the state-trie pattern, where this very report found ParityDB 14× faster cold. Random-key reads weren't measured on the storage side, so it'd be great to see the engines compared under that profile too.
- Dedup scope. Sharded is clean only if dedup is per-bucket; cross-bucket dedup would need a shared store. So it'd help to pin that design question down, since it feeds directly into sharded-vs-shared.
A few smaller things in the same spirit (all point the same way — the harness models a sequential append log, and the real component is a random-hash, deduplicated, mutable store):
- An update/overwrite scenario — every write currently uses a fresh key, so the update path (where RocksDB's compaction and ParityDB's value-table rewrite diverge most) isn't exercised.
- Content-hash keys rather than sequential positions, to match the real access locality.
- A real MMR proof read (O(log n) node walk) — proof_read is currently a single point get.
- A variable chunk-size distribution (64 KiB–1 MiB) for the amplification numbers, since CDC values aren't fixed 256 KiB.
- The omni-node A/B the report already flags as follow-up — that's the piece that confirms magnitude.
One scoping thought, given the Asset Hub direction: if the on-chain logic is heading to Asset Hub (via pallet_revive contracts), the blockchain state-DB half — RocksDB vs ParityDB — is likely not a decision we own: we wouldn't operate the node, and Asset Hub already runs ParityDB by default. On a shared chain the on-chain cost question shifts to how much state we store and its deposit/PoV cost, not engine choice. So it might be worth scoping this PR toward the storage-provider evaluation (which stays fully ours regardless of where the chain logic lives) and marking the blockchain-node section as background/informational. Happy to be wrong if we're keeping our own parachain or running our own RPC/indexer nodes.
And to be clear about what's already solid: the relative ranking for the operations tested, the leaf-count scenarios (avg size is preserved by design), and the entire blockchain-node / ParityDB evaluation — CDC is storage-provider-only, so that half stands on its own.
Really nice groundwork — happy to help wire up any of the storage-side scenarios above.
| @@ -0,0 +1,267 @@ | |||
| //! Storage Provider workloads — model the post-Issue-#100 per-bucket database: | |||
| //! each bucket is one small, independent store keyed by MMR leaf position. | |||
There was a problem hiding this comment.
Nice modeling. One thing to fold in later: the real store keys nodes/chunks by random H256 content hash (storage/mod.rs), not sequential position — so keying these scenarios by hash would better match real access locality.
| let value = value_of(&mut rng, value_size); | ||
| bytes += (value.len() + 8) as u64; | ||
| batch.push((position_key(position), value)); | ||
| position += 1; |
There was a problem hiding this comment.
Append-only here — would be great to add an overwrite/update variant eventually, since the mutable/CDC design churns and orphans nodes, and that's the write+GC behavior that most differentiates the engines.
| for _ in 0..reads.min(5_000) { | ||
| let position = rng.next_u64() % leaf_count as u64; | ||
| let started = Instant::now(); | ||
| let got = store.get(&position_key(position)); |
There was a problem hiding this comment.
Worth noting proof_read is a single point lookup; a real MMR proof reads O(log n) sibling/peak nodes, so this is a floor on proof cost. A tree-walk variant would make it representative.
| for _ in 0..blocks { | ||
| let mut batch = Vec::with_capacity(writes_per_block); | ||
| for _ in 0..writes_per_block { | ||
| let key = hash_key(&mut rng); |
There was a problem hiding this comment.
Fresh key per write, so no in-place updates — an overwrite scenario would exercise the compaction/rewrite paths where RocksDB and ParityDB differ most. Good candidate for a follow-up.
2 participants
Addresses #101 and #100
Benchmarked different database options according to #101 and wrote some results to come to a decision.