feat(mcp): revalidate cached local-path indexes on query#78
Conversation
Port semble#211: an MCP session-cache entry for a local path is now rechecked against its live source fingerprint on each `get`, so an entry is rebuilt once its files change instead of serving stale results for the lifetime of the server. Git URLs (URL+ref keyed) are never revalidated. - index.rs: extract `source_fingerprint(source, content)` (the same content-hash oracle load_or_build_index already used) as a pub helper; load_or_build_index now calls it too (no behavior change). - mcp.rs: IndexCache entries carry the build-time fingerprint plus a `revalidate_after` cooldown (build duration x MIN_REVALIDATE_FACTOR=3), so a slow-to-build repo isn't re-walked on every query. On get, past the cooldown, a changed fingerprint evicts the entry -> rebuild. The LoadOrBuild seam gains a `fingerprint` method (None for git URLs). - Tests: cache_revalidates_stale_local_path (cooldown skip + stale rebuild + fresh-match reuse) and cache_git_url_not_revalidated. Refs #74
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 9 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements a cache revalidation mechanism for local source trees using content-hash fingerprints and a build-time-scaled cooldown period to prevent frequent disk I/O. The review feedback correctly identifies a performance issue: when a revalidation check succeeds (the fingerprint matches), the cooldown timer is not reset, causing subsequent queries to repeatedly perform fingerprint checks. The reviewer suggests adding a revalidate_cooldown field to CacheEntry and updating the revalidate_after timestamp upon successful revalidation.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Architecture diagram
sequenceDiagram
participant Client as MCP Client
participant Server as MCP Server
participant Cache as IndexCache
participant Seam as LoadOrBuild (DiskLoadOrBuild)
participant FP as source_fingerprint()
participant FS as File System
Client->>Server: search/find_related(source, query)
Server->>Cache: get(source, git_ref)
Cache->>Cache: computeKey(source, git_ref)
Cache->>Cache: look up entry in index map
alt entry exists AND local path (fingerprint is Some) AND cooldown elapsed
Cache->>Seam: fingerprint(source, content)
Seam->>FP: source_fingerprint(source, content)
FP->>FS: collect_source_files() + compute_content_hash()
FS-->>FP: hash value
FP-->>Seam: Some(hash)
Seam-->>Cache: fingerprint value
Cache->>Cache: compare with stored fingerprint
alt mismatch → stale
Cache->>Cache: shift_remove entry (evict)
else match
Note over Cache: entry stays valid
end
else
Note over Cache: skip revalidation (cooldown active or git URL)
end
Cache->>Cache: shift_remove key (if not already evicted)
alt entry still present
Cache->>Cache: re-insert for LRU touch
Cache-->>Server: Arc<CspIndex>
else entry evicted or missing
Note over Cache: LRU eviction if at capacity
Cache->>Seam: load_or_build(source, content, git_ref)
Seam->>Seam: build index (walk filesystem / clone repo)
Seam-->>Cache: CspIndex
Note over Cache: record build_duration = now - start
Cache->>Seam: fingerprint(source, content) again
Seam-->>Cache: live fingerprint (None for git)
Cache->>Cache: create CacheEntry(fingerprint, revalidate_after = now + build_duration × 3)
Cache->>Cache: insert into cache
Cache-->>Server: Arc<CspIndex>
end
Server-->>Client: search results
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR adds content-hash-based revalidation to the MCP session index cache (
Confidence Score: 5/5Safe to merge; the revalidation logic is well-structured and the previous cooldown-reset gap was addressed. The core revalidation logic is correct: stale entries are evicted, fresh entries have their cooldown advanced, git URLs are never revalidated, and the LRU invariant is preserved. The only noteworthy issue is a redundant filesystem walk after each rebuild, which is bounded by MIN_REVALIDATE_COOLDOWN and does not affect correctness. No files require special attention; mcp.rs carries the most logic but its test coverage is thorough. Important Files Changed
Reviews (3): Last reviewed commit: "perf(mcp): enforce minimum revalidation ..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to revalidate cached local source indexes against their live content-hash fingerprints in IndexCache::get after a build-time-scaled cooldown period, preventing stale cache hits when local files change. Git URLs are excluded from this revalidation. The review feedback suggests optimizing IndexCache::get to reduce redundant map lookups and duplicate logic when retrieving, validating, and updating cache entries.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a local source tree fingerprinting and revalidation mechanism for the index cache to detect staleness on query, using a build-time-scaled cooldown to prevent frequent directory re-walking. Git URLs are excluded from this revalidation. The feedback highlights two performance improvements: establishing a minimum cooldown threshold to prevent excessive revalidation when builds are extremely fast, and eliminating redundant fingerprint calculations that occur immediately after a cache miss or rebuild.
|
/gemini review |
There was a problem hiding this comment.
Code Review
본 풀 리퀘스트는 로컬 소스 트리의 변경 사항을 감지하여 캐시된 인덱스를 재검증하는 source_fingerprint 기능 및 쿨다운 메커니즘을 도입합니다. 리뷰에서는 디스크 캐시 히트 시 build_elapsed가 매우 짧아져 쿨다운이 최소치(2초)로 고정됨에 따라 대규모 저장소에서 빈번한 디렉토리 순회 및 해싱 병목이 발생할 수 있는 문제와, load_or_build 호출 직후 fingerprint를 중복 계산하는 성능 비효율성을 지적하고 개선 방향을 제시했습니다.
|
1 participant
Closes #74. Ports upstream semble#211 (
41c36f7).Problem
The MCP session cache (
IndexCache) served a local path's index for the lifetime of the server without ever rechecking it. If the repo's files changed mid-session,search/find_relatedreturned stale results.Change
index.rs— extractsource_fingerprint(source, content)(the same content-hash oracleload_or_build_indexalready used to validate the on-disk cache) as apubhelper.load_or_build_indexnow calls it too — no behavior change, just deduplication.mcp.rs— eachIndexCacheentry now carries the fingerprint captured at build time plus arevalidate_aftercooldown (build_duration × MIN_REVALIDATE_FACTOR, factor = 3, mirroring upstream's_MIN_REVALIDATE_FACTOR). Onget, once the cooldown has elapsed, the live fingerprint is recomputed; a mismatch evicts the entry so it rebuilds. Git URLs reportNoneand are never revalidated. The cooldown keeps a slow-to-build repo from being re-walked on every query.LoadOrBuildseam gains afingerprint()method.Why the fingerprint (not upstream's mtime+file-set)
Upstream
get_validated_cachere-stats every file against a stored write-time and compares file sets. csp already uses a content hash of the source tree as its cache-validity oracle (stronger and simpler), so revalidation reuses that same signal rather than introducing an mtime path.Verification
cargo fmt --all✅ ·cargo clippy --all-targets --all-features -- -D warnings✅ (clean)cargo test --workspace✅ — 259 lib + 20 CLI tests pass, incl. new:cache_revalidates_stale_local_path— within-cooldown reuse despite drift, then stale rebuild past the cooldown, then fresh-match reusecache_git_url_not_revalidated— git URLs served from cache even when the (local-only) fingerprint changescsp searchstill returns results after theload_or_build_indexrefactor.Notes
tokio::sync::Mutex; the fingerprint recompute is blocking like the existing build path already is (offloading to a worker is a pre-existing, separate concern noted in CLAUDE.md).Summary by cubic
Revalidates cached local-path MCP indexes on query using a content-hash fingerprint with a build-time-scaled cooldown (min 2s), so results refresh after file changes without re-walking on every query. Git URLs are never revalidated; the cooldown resets after a successful check, and cache-hit lookup is streamlined.
New Features
IndexCachestores a build-time fingerprint andrevalidate_after; past the cooldown, a changed live fingerprint evicts and rebuilds; a match resets the cooldown.MIN_REVALIDATE_FACTOR(3), with a 2s minimum to avoid near-every-query checks on fast builds.LoadOrBuildaddsfingerprint(); git URLs returnNoneand are not revalidated.Refactors
source_fingerprint(source, content)and reused it inload_or_build_index.IndexCache::getto reduce map operations.Written for commit 904f046. Summary will update on new commits.