Skip to content

Add byte-aware and per-use-case local-cache capacity controls #40

Description

@lan17

Priority

P1 — High — resolve before broad production adoption or large-scale use.

Problem

The local layer is one global LRU per DialCache instance, and every entry has weight 1 regardless of key/value size. Large object graphs can make memory effectively unbounded, while one high-cardinality use case can evict unrelated hot data.

Evidence:

The review's heap probe estimated roughly 259–270 bytes of LRU metadata per entry before key and value memory.

Open decisions

Choose an estimated-weight callback or another explicit model; decide whether per-use-case limits, reservations, or separate pools are supported. Do not claim exact JavaScript heap measurement.

Acceptance criteria

  • Define behavior for an item larger than its budget.
  • Add entry-count, estimated-weight, eviction-reason, and rejected-item telemetry.
  • Make localMaxSize: 0 behave and report as disabled rather than as a permanent miss layer.
  • Test large values, noisy-neighbor eviction, TTL expiry, and global/per-use-case interactions.
  • Benchmark accounting overhead.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions