feat: add configurable ttl on stampede protected cache, eviction telemetry by codingsnoop · Pull Request #454 · microsoft/oxidizer

codingsnoop · 2026-05-26T20:09:07Z

This pull request introduces version 0.6.0 of the cachet crate, adding major new features for per-entry TTL control and improved eviction telemetry. The most significant changes include new get_or_insert_with and try_get_or_insert_with APIs for flexible cache entry creation, support for emitting cache.eviction telemetry events, and updates to documentation and internal structures to reflect these enhancements.

New APIs for Per-Entry TTL Control:

Added get_or_insert_with and try_get_or_insert_with methods to the Cache type, allowing users to provide closures that return CacheEntry<V>, which enables setting custom TTLs per entry on cache miss. These methods support both infallible and fallible computation functions, and integrate with stampede protection and concurrency mechanisms. [1] [2]

Eviction Telemetry Enhancements:

Introduced a new max_capacity builder method on CacheBuilder to specify a capacity threshold for emitting cache.eviction telemetry events when an insert causes an eviction. This does not enforce storage limits but enables monitoring cache evictions.
Added a new cache.eviction event to the telemetry system, including updates to documentation, event constants, and the CacheTelemetry interface to record these events. [1] [2] [3] [4]

Documentation and Versioning Updates:

Updated the crate version to 0.6.0 in Cargo.toml and all documentation links, and added release notes to the CHANGELOG.md describing the new features. [1] [2] [3]
Expanded documentation and examples for the new APIs and telemetry event, including in the main crate docs and README. [1] [2] [3] [4]

Internal and Test Infrastructure:

Extended internal merger structures and test utilities to support the new APIs and the max_capacity parameter, ensuring comprehensive coverage and correct behavior. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

These changes provide more flexibility and observability for users of the cachet crate, especially for advanced caching scenarios requiring per-entry expiration and eviction monitoring.

Add methods where the closure returns CacheEntry<V> instead of raw V, enabling per-entry TTL control on cache-miss computations. Both methods include full stampede protection via dedicated Mergers.

Copilot

Pull request overview

This PR releases cachet v0.6.0, adding new cache-miss APIs that allow callers to return a full CacheEntry<V> (enabling per-entry TTL) and introducing eviction-related telemetry driven by an opt-in max_capacity threshold.

Changes:

Add Cache::{get_or_insert_with, try_get_or_insert_with} to compute-and-cache full CacheEntry<V> values (per-entry TTL support).
Add CacheBuilder::max_capacity and cache.eviction telemetry emission when inserts occur at/above the configured capacity threshold.
Bump cachet to 0.6.0 and update docs/tests accordingly.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
crates/cachet/src/cache.rs	Adds the new `*_with` APIs and stampede-protection merger wiring.
crates/cachet/tests/cache.rs	Adds integration tests covering the new APIs and stampede-protection behavior.
crates/cachet/src/builder/cache.rs	Introduces `max_capacity` builder knob for eviction telemetry.
crates/cachet/src/builder/buildable.rs	Plumbs `max_capacity` into constructed tiers (`CacheWrapper`).
crates/cachet/src/wrapper.rs	Stores `max_capacity` and emits the new eviction telemetry on inserts.
crates/cachet/src/telemetry/cache.rs	Adds `CacheTelemetry::cache_eviction` helper and updates tests.
crates/cachet/src/telemetry/attributes.rs	Adds `EVENT_EVICTION` constant and uniqueness test coverage.
crates/cachet/src/lib.rs	Documents the new `cache.eviction` INFO-level telemetry event.
crates/cachet/src/fallback.rs	Updates `CacheWrapper::new` call sites for the new parameter.
crates/cachet/src/refresh.rs	Updates `CacheWrapper::new` call sites for the new parameter.
crates/cachet/README.md	Updates documented telemetry event list and docs.rs links for 0.6.0.
crates/cachet/CHANGELOG.md	Adds 0.6.0 release notes for the new APIs.
crates/cachet/Cargo.toml	Bumps crate version to 0.6.0.
Cargo.lock	Updates lockfile to reflect the new `cachet` version.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-05-26T20:17:01Z

Codecov Report

❌ Patch coverage is 92.89617% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.9%. Comparing base (43d51a5) to head (732c4ad).

Files with missing lines	Patch %	Lines
crates/cachet_memory/src/builder.rs	70.4%	13 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #454     +/-   ##
========================================
- Coverage   100.0%   99.9%   -0.1%     
========================================
  Files         305     307      +2     
  Lines       23698   23879    +181     
========================================
+ Hits        23698   23866    +168     
- Misses          0      13     +13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 6 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 5 comments.

schgoo · 2026-05-26T20:43:24Z

I think the low-hanging fruit implementation for this is to implement a Moka eviction listener, and then use that to emit the event you want

…b.com/microsoft/oxidizer into u/emengesha/cachet-get-or-insert-with

Copilot

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 3 comments.

…et_memory builder

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…b.com/microsoft/oxidizer into u/emengesha/cachet-get-or-insert-with

Copilot

Pull request overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 5 comments.

 [package]
 name = "cachet"
 description = "A composable, customizable multi-tier caching library with rich feature support."
-version = "0.5.0"
+version = "0.5.1"
 readme = "README.md"


+ [__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjJhdIQbLiTyV0MU86EbZU15e0PmecoboQ9jo59bnAEbyDXw04U13GlhYvRhcoQbg_hDqE88LP4bMh0J5Y4y4Osb0zDJ1kwqOsoblCGrm49Rx2thZIiCaGJ5dGVzYnVmZTAuNS4wgmZjYWNoZXRlMC42LjCCbWNhY2hldF9tZW1vcnllMC4yLjCCbmNhY2hldF9zZXJ2aWNlZTAuMS4wgmtjYWNoZXRfdGllcmUwLjEuMIJkdGlja2UwLjMuMIJndHJhY2luZ2YwLjEuNDSCaXVuaWZsaWdodGUwLjIuMA
+ [__link0]: https://docs.rs/cachet/0.6.0/cachet/?search=TimeToRefresh
 [__link1]: https://crates.io/crates/uniflight/0.2.0
 [__link10]: https://docs.rs/cachet_tier/0.1.0/cachet_tier/?search=CacheTier
- [__link11]: https://docs.rs/cachet/0.5.0/cachet/?search=InsertPolicy
- [__link12]: https://docs.rs/cachet/0.5.0/cachet/?search=TimeToRefresh
+ [__link11]: https://docs.rs/cachet/0.6.0/cachet/?search=InsertPolicy
+ [__link12]: https://docs.rs/cachet/0.6.0/cachet/?search=TimeToRefresh


+        if let Some(entry) = self.storage.get(key).await? {
+            return Ok(entry);
+        }
+        let entry = f().await;


+        if let Some(entry) = self.storage.get(key).await? {
+            return Ok(entry);
+        }
+        let entry = f().await.map_err(Error::from_source)?;


+    // Drive enough churn to force size-based evictions. Moka's housekeeping
+    // runs periodically (and as a side effect of cache operations), so we keep
+    // exercising the cache while waiting for an eviction event to surface.
+    let deadline = std::time::Instant::now() + Duration::from_secs(10);
+    let mut i: i32 = 0;


Copilot

Pull request overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 1 comment.

 name = "cachet"
 description = "A composable, customizable multi-tier caching library with rich feature support."
-version = "0.5.0"
+version = "0.5.1"
 readme = "README.md"


codingsnoop added 4 commits May 21, 2026 10:58

feat(cachet): add get_or_insert_with and try_get_or_insert_with

a3913f1

Add methods where the closure returns CacheEntry<V> instead of raw V, enabling per-entry TTL control on cache-miss computations. Both methods include full stampede protection via dedicated Mergers.

get or insert with

bdd8c2c

add eviction telemetry

bb99884

docs

7203747

Copilot AI review requested due to automatic review settings May 26, 2026 20:09

Copilot started reviewing on behalf of codingsnoop May 26, 2026 20:09 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

Comment thread crates/cachet/src/telemetry/attributes.rs Outdated

Comment thread crates/cachet/Cargo.toml

Comment thread crates/cachet/src/wrapper.rs Outdated

Comment thread crates/cachet/src/builder/cache.rs Outdated

codingsnoop added 2 commits May 26, 2026 13:22

spellcheck fix

3ca6969

clippy

e39727a

Copilot AI review requested due to automatic review settings May 26, 2026 20:24

Copilot started reviewing on behalf of codingsnoop May 26, 2026 20:24 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

codingsnoop and others added 2 commits May 26, 2026 13:36

bump workspace version

e981439

add nuance to docs

52422ad

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 26, 2026 20:37

Copilot started reviewing on behalf of codingsnoop May 26, 2026 20:37 View session

schgoo requested changes May 26, 2026

View reviewed changes

Comment thread crates/cachet/src/builder/cache.rs Outdated

Comment thread crates/cachet/src/cache.rs

Comment thread crates/cachet/src/wrapper.rs Outdated

Comment thread crates/cachet/CHANGELOG.md Outdated

Copilot AI reviewed May 26, 2026

View reviewed changes

Comment thread crates/cachet/src/cache.rs

Comment thread crates/cachet/src/cache.rs

Comment thread crates/cachet/src/wrapper.rs Outdated

Comment thread crates/cachet/src/telemetry/cache.rs Outdated

Comment thread crates/cachet/CHANGELOG.md

codingsnoop added 3 commits May 27, 2026 08:07

implement eviction listener threading

5e0d6e3

Merge branch 'u/emengesha/cachet-get-or-insert-with' of https://githu…

60be0d2

…b.com/microsoft/oxidizer into u/emengesha/cachet-get-or-insert-with

add impl unwindsafe and refunwindsafe

1301827

Copilot AI review requested due to automatic review settings May 27, 2026 15:35

Copilot started reviewing on behalf of codingsnoop May 27, 2026 15:35 View session

Copilot AI reviewed May 27, 2026

View reviewed changes

Comment thread crates/cachet/src/telemetry/cache.rs Outdated

Comment thread crates/cachet/src/telemetry/attributes.rs Outdated

Comment thread crates/cachet/CHANGELOG.md

Comment thread crates/cachet/src/builder/cache.rs Outdated

sort cargo.toml

61de5b6

schgoo requested changes May 27, 2026

View reviewed changes

codingsnoop added 2 commits May 27, 2026 12:23

docs, version fixes

b3c73f1

spelling

edcd67f

codingsnoop added 2 commits May 27, 2026 12:51

pare docs

a486e05

move telemetry out of cache and into inmemorycache

b2fe698

Copilot AI review requested due to automatic review settings May 27, 2026 20:32

Copilot started reviewing on behalf of codingsnoop May 27, 2026 20:32 View session

doc to specify eviction telemetry details in memory api

9bb9341

Copilot AI reviewed May 27, 2026

View reviewed changes

Comment thread crates/cachet/README.md Outdated

Comment thread crates/cachet/src/telemetry/attributes.rs Outdated

Comment thread crates/cachet_memory/src/builder.rs

Merge branch 'main' into u/emengesha/cachet-get-or-insert-with

476d5e4

Copilot started work on behalf of codingsnoop May 27, 2026 21:04 View session

fix: replace broken intra-doc link with plain code formatting in cach…

b8e56fc

…et_memory builder

Copilot AI review requested due to automatic review settings May 27, 2026 21:10

codingsnoop review requested due to automatic review settings May 27, 2026 21:10

Potential fix for pull request finding

f692df2

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings May 27, 2026 21:10

Copilot finished work on behalf of codingsnoop May 27, 2026 21:10

Copilot started reviewing on behalf of codingsnoop May 27, 2026 21:11 View session

codingsnoop added 2 commits May 27, 2026 14:13

update readme

e9f2d22

Merge branch 'u/emengesha/cachet-get-or-insert-with' of https://githu…

0ee4f33

…b.com/microsoft/oxidizer into u/emengesha/cachet-get-or-insert-with

Copilot AI reviewed May 27, 2026

View reviewed changes

codingsnoop added 2 commits May 27, 2026 14:25

add tests to in memory builder

57fa37a

nit

732c4ad

Copilot AI review requested due to automatic review settings May 27, 2026 21:26

Copilot started reviewing on behalf of codingsnoop May 27, 2026 21:26 View session

Copilot AI reviewed May 27, 2026

View reviewed changes

Comment thread crates/cachet/Cargo.toml

Comment on lines 5 to 8

name = "cachet"

description = "A composable, customizable multi-tier caching library with rich feature support."

version = "0.5.0"

version = "0.5.1"

readme = "README.md"

Conversation

codingsnoop commented May 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

schgoo commented May 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented May 26, 2026 •

edited

Loading