Phase E: cross-session memory + recall (WAL → lazy digest → recall)

[azalio]

Phase E — Cross-Session Memory + Recall

Gives the MAP Framework durable cross-session memory: cheap per-turn capture into a scratch WAL, lazy LLM "digest" finalization on the next session start, and recall injection of relevant past digests — so a later session knows why an approach was chosen.

Architecture: write-ahead-log → lazy checkpoint (NOT flush-on-SessionEnd). Stop is the only reliable durable-capture point; SessionStart carries finalize + recall. Works with zero SessionEnd dependency (HC-2) and survives compaction (which PreCompact does not reliably fire on, esp. 1M-token contexts).

Built via /map-efficient (8 subtasks, sequential RESEARCH → ACTOR → MONITOR, per-subtask commits). Final-verifier: PASS.

What's in it

• src/mapify_cli/memory/ (new pure-runtime package):
  • digest_schema.py — single-source field constants + redaction (sk-/gh_/base64/AKIA) + secret-path globs + control-char sanitizer (INV-7, Contract-First).
  • capture.py — LLM-free per-turn scratch WAL append (subprocess-free branch resolve via .git/HEAD), on_session_end best-effort marker.
  • finalize.py — finalize-if-dirty: claude -p (argv form, MAP_INVOKED_BY=memory-finalize recursion guard, hard timeout), atomic .md.tmp → os.replace → .finalized, per-branch flock, truncation-tolerant, empty→no-digest, memory-cost.log.
  • recall.py — current-branch keyword+recency ranking, MAP_MEMORY_RECALL_CAP (default 4000 chars) with whole-digest drops → recall-drop.log, sanitized additionalContext.
• 4 REQUIRE_GUARD hook shims (map-memory-{capture,finalize,recall,endmark}.py) authored as .jinja, registered in settings.json (Stop / SessionStart finalize→recall / SessionEnd / UserPromptSubmit), lint-hooks.py, and both hook doc tables.
• map-memory-now skill — on-demand finalize / --finalize-all, requires-cmd: [claude, git], host-gate pruned when claude absent (EC-4).
• .gitignore — ignores the scratch WAL; documents the MAP_MEMORY_COMMIT_DIGESTS=0 opt-out.
• Hook executable-bit fix — hooks now ship +x (the harness execs them via shebang); renderer force-sets +x for hook .py/.sh, plus CI guards that exec hooks the way the harness does (shebang, not python3 <path>).

Tests / gates

• ~150 new unit tests across the memory modules + an end-to-end integration smoke (capture×2 → finalize → recall with a fake claude on PATH; asserts the digest surfaces in additionalContext and memory-cost.log is written).
• make check (ruff + mypy + pyright + full pytest + check-render) green; full suite 2037 passed.

Notes

• Token accounting into token_accounting.json is intentionally deferred (Decision 9); finalize writes memory-cost.log instead.
• Single-source render invariant respected throughout (templates_src/**/*.jinja → make render-templates; check-render enforced).

🤖 Generated with Claude Code

[azalio]

Code-review fixes pushed (commit 901d991)

Resolved all 10 findings from a high-effort review of the memory subsystem:

Correctness

1. finalize: slug disambiguation now reserves suffix room before the 32-char truncation → os.replace can no longer clobber another session's digest.
2. finalize: slug derived from the title key; _parse_claude_output strips ```json fences so decisions/findings survive.
3. finalize: per-field redaction before YAML escaping; identifier fields (session_id/branch/date/slug) excluded → long session_id no longer becomes «redacted», owner-line dedup preserved.
4. recall: cap is rank-monotonic — break on first overflow so a lower-ranked smaller digest never jumps a dropped higher-ranked one.
   6/7. digest_schema: redact fine-grained github_pat_ tokens; stop over-redacting pure-hex git SHAs (mixed-case secrets still caught).
5. capture: advance the transcript <sid>.offset only AFTER the record write (crash-safety).
6. capture: fallback session id from the transcript stem instead of a shared unknown.jsonl.

Behavior/perf
2. settings: restored the UserPromptSubmit recall registration so prompt-relevance ranking actually runs.
10. capture: memoised _resolve_branch + O(1) tail-read turn count; finalize hook subprocess timeout 60→50 (below the 60s harness timeout so cleanup runs).

Adds tests/test_memory_review_fixes.py (11 regression tests). Full gate green: mypy, pyright src/ 0/0/0, lint-hooks, check-render, 2082 tests passed.

🤖 Generated with Claude Code