fix(cache): key semantic cache on the extraction prompt (#1939)

[SinghAman21]

fix(cache): key the semantic cache on the extraction prompt

Fixes #1939

What was the issue

The semantic cache keyed entries on sha256(file content + "\x00" + relative path) and nothing else. Nothing in the key described which extraction prompt produced the entry.

So when a release changed the prompt, every unchanged file was still a cache hit and replayed whatever the older prompt had extracted. Nothing flagged it: the run exited 0, cost.json looked cheap, and the resulting graph.json quietly carried two prompt generations side by side — nodes shaped by an old prompt sitting next to nodes from the current one.

The reporter hit this through the bundled skill's prompt: their graphify-out/cache/semantic/ held 880 entries across two vintages, and a rebuild they expected to be cold re-extracted only 6 of 512 docs — the other 506 replayed the old prompt. Deleting the entire cache was the only workaround they found, which forfeits the AST cache too.

This was a gap, not an oversight. #1252 versioned the AST cache and deliberately left the semantic cache unversioned: its entries are LLM output, and invalidating them on every release would re-bill extraction for unchanged files. That reasoning is sound, and any fix has to preserve it. #1894 later added mode namespacing (semantic vs semantic-deep), but still no prompt dimension.

Approach

Fingerprint the extraction prompt and namespace entries by it, rather than by package version.

This keeps both properties the issue asks for: entries survive releases that don't touch the prompt (no re-bill on a patch release — #1252's constraint holds), and invalidate only when the prompt actually changed.

Entries live under cache/semantic/p{fingerprint}/, mirroring the AST cache's v{version}/ layout, so the two caches now read as one idea with two different validity keys — version for code-produced entries, prompt for LLM-produced ones.

Two decisions worth reviewer attention:

Pre-existing entries are served, not dropped. They predate fingerprinting, so their vintage is unknowable. Ignoring them would be strictly correct but would re-bill a whole corpus on upgrade (506 docs for the reporter). Instead they still hit, and check_semantic_cache warns with the count — turning the "no signal at all" the issue describes into a visible one. --force re-extracts them. The honest limitation: such an entry keeps serving its unknown vintage until its content changes or someone forces it. The transition is gradual, not immediate.

Old-fingerprint entries are pruned by liveness only, never swept wholesale the way _cleanup_stale_ast_entries sweeps old AST versions. Two hosts with different prompts (the verbose vs compact extraction-spec.md) can share one graphify-out/; a wholesale sweep would have each run delete the other's entries and re-bill on every alternation. Liveness keeps the total bounded by live docs × prompts seen.

Changes

graphify/cache.py — the core.

• prompt_fingerprint(prompt) accepts prompt text or a Path to a prompt file. Normalizes line endings and trailing whitespace before hashing, so a CRLF checkout isn't mistaken for a prompt change (which would re-bill every Windows run).
• cache_dir(..., prompt_fp=) adds the p{fingerprint}/ subdir for semantic kinds. prompt_fp=None yields the historical flat layout, so existing callers are byte-identical.
• load_cached / save_cached / check_semantic_cache / save_semantic_cache take prompt= (text) and prompt_file= (path). These are separate parameters on purpose: the skill callers are markdown snippets an agent copies with a path substituted in, and passing that path as prompt would hash the path string — a fingerprint that is stable, plausible, and tracks nothing. A silent wrong fingerprint is the exact failure class being fixed.
• merge_existing reads with allow_legacy=False: fusing a pre-fingerprint entry into a current-vintage write would mix two prompts inside one entry and then attest the result to a prompt that produced half of it.
• prune_semantic_cache / clear_cache / cached_files glob recursively, so fingerprinted entries can't become unprunable orphans (the Semantic cache is never pruned — orphan entries accumulate unbounded (AST cache is pruned, semantic isn't) #1527 failure mode).
• Prompt-file fingerprints are memoized on a (path, size, mtime_ns) signature — check_semantic_cache resolves the prompt once per file, so a 500-doc run would otherwise re-read and re-hash the same spec 500 times.

graphify/llm.py — the per-chunk checkpoint stamps entries with _extraction_system(deep=deep_mode), the prompt every backend actually sends.

graphify/cli.py — extract passes the same prompt to both the cache read and the write (a mismatch would strand writes where the next read won't look). cache-check gains --prompt-file.

tools/skillgen/fragments/core/core.md + 14 rendered skills — Step B0/B3 pass prompt_file='SPEC_PATH', the references/extraction-spec.md the subagents are handed, with instructions for substituting it. Regenerated via python -m tools.skillgen; --check is clean. The two monolith skills (aider, devin) inline their prompt rather than shipping a spec sidecar and stay on the unfingerprinted path — called out as a follow-up.

Tests — 12 new, covering: prompt change invalidates; unchanged prompt still hits; CRLF insensitivity; prompt × mode compose independently; legacy entries served with the count warning; fingerprinted entry beats legacy; merge_existing won't fuse vintages; prune/clear reach the subdirs; unreadable prompt_file warns and falls back; an edited spec defeats the memo; cache-check --prompt-file; and a skillgen guard asserting every split host passes prompt_file to both calls (they live ~80 lines apart in the rendered body, so adding it to one and not the other would silently disable the cache rather than fail loudly).

A few existing tests asserted the flat layout (glob("*.json"), unprompted load_cached) and were updated to the nested layout — behavior change, not a weakened assertion.

Why

The alternative fixes each break something the issue explicitly wants kept:

• Version the semantic cache like the AST cache — re-bills LLM extraction on every patch release. This is exactly what AST cache survives graphify upgrades: content-only keys keep serving stale extractor output #1252 rejected, and the issue agrees that reasoning is sound.
• Warn only — the issue offers this as a fallback ("even a warning ... would help a lot"), but it leaves the graph mixed. Warning is the right treatment for entries we can't attribute; it's not a fix for the ones we can.
• Wipe graphify-out/ on upgrade — the reporter's current workaround. Forfeits the AST cache, and is easy to forget.

Fingerprinting is the only option that invalidates precisely when correctness demands and never otherwise.

Verification

• Full suite: 3277 passed. The only failures are 4 pre-existing ones in tests/test_ollama_retry_cap.py, confirmed failing identically on a clean tree (openai optional extra absent from a stock dev venv — unrelated, documented separately).
• python -m tools.skillgen --check: clean, 134 artifacts match.
• End-to-end through the real CLI, reproducing the reporter's scenario:
  • unchanged prompt → Cache: 1 hit, 0 miss (no re-bill)
  • prompt changed by an "upgrade" → Cache: 0 hit, 1 miss (re-extract, no stale replay)
  • pre-fingerprint cache → Cache: 3 hit, 0 miss plus 3 semantic cache entries predate extraction-prompt fingerprinting..., confirmed visible under default warning filters
  • layout on disk: cache/semantic/pa55707ed205d/{hash}.json

Conclusion

An upgrade that changes the extraction prompt now re-extracts affected files instead of silently mixing vintages, and an upgrade that doesn't touch the prompt still costs nothing — the property #1252 protected. Users upgrading with an existing cache keep their entries and get a count of how many are of unknown vintage, so the silent case the issue reports is now either correct or visible.

Known follow-ups, deliberately not in this PR:

• The monolith skills (aider, devin) inline their prompt and stay unfingerprinted.
• Separately, no bundled skill passes mode either, so a skill-driven --mode deep still writes into the plain namespace — the gap extract: semantic cache key ignores extraction mode, and extract has no --force, so --mode deep over a warm cache is a silent no-op #1894's changelog already anticipates as skill-side follow-up.
• A stale installed SKILL.md (older than the package) will fingerprint against the spec it actually loaded, which is correct; the existing .graphify_version stale-skill warning remains the signal for that.

[safishamsi]

Landed on v8 in 0d018b4 (ships in 0.9.18), cherry-picked so your authorship is preserved in the history. Thanks @SinghAman21 — this keys the semantic cache on a fingerprint of the extraction prompt, so an upgrade that changes the prompt invalidates only the entries it should, while releases that leave the prompt untouched keep their cache. Verified end to end: same prompt hits, a changed prompt re-extracts instead of replaying stale output, and the fingerprint is CRLF-insensitive. Credited in the changelog alongside @HunterMcGrew for the report.