Skip to content

feat: add --cache-index-filter-blocks flag#201

Merged
philippem merged 1 commit intoBlockstream:new-indexfrom
EddieHouston:pr/cache-index-filter-blocks
Mar 31, 2026
Merged

feat: add --cache-index-filter-blocks flag#201
philippem merged 1 commit intoBlockstream:new-indexfrom
EddieHouston:pr/cache-index-filter-blocks

Conversation

@EddieHouston
Copy link
Copy Markdown
Collaborator

@EddieHouston EddieHouston commented Mar 31, 2026

Summary

  • Makes cache_index_and_filter_blocks opt-in via --cache-index-filter-blocks CLI flag
  • By default (flag not set), index and filter blocks live on the heap where they may never be evicted — possibly better read performance compared to needing to go to disk, at the cost of ~18 MB per SST file of unbounded memory
  • When flag is set, index and filter blocks are stored in the block cache, bounding memory but allowing eviction under cache pressure

Background

With bloom filters enabled, each SST file carries ~18 MB of index + filter blocks in memory. On a mainnet node with ~1900 SST files, that's ~35 GB of table reader memory.

When these blocks live in the block cache (cache_index_and_filter_blocks=true), they compete with data blocks for cache space and can be evicted under pressure.

The flag exists for memory-constrained deployments where bounding memory is more important than peak read performance.

Test plan

  • cargo check / cargo build with default features
  • cargo build --features liquid
  • Verify default behavior: index/filter blocks on heap (table readers grow with SST count)
  • Verify --cache-index-filter-blocks: index/filter blocks bounded by --db-block-cache-mb
  • No functional changes — only controls where index/filter blocks are stored

  By default, index and filter blocks live on the heap where they may
  never be evicted (~18 MB per SST file). Pass --cache-index-filter-blocks
  to store them in the block cache instead, bounding memory at the cost
  of possible eviction under cache pressure.
@EddieHouston EddieHouston force-pushed the pr/cache-index-filter-blocks branch from c0a7f92 to 3ff1592 Compare March 31, 2026 18:21
@EddieHouston EddieHouston requested a review from philippem March 31, 2026 18:44
@philippem philippem merged commit dfcb4fe into Blockstream:new-index Mar 31, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants