fix(knowledge): audit retention caveat — Lance versions are retention-policy-gated, not by-construction-immutable (codex P1 on #465)

[AdaWorldAPI]

Summary

Codex P1 follow-up correction on merged PR #465. The original doc §2.1 + §5.1 overclaimed Lance versions as immutable audit by construction; Lance 7.0+ exposes Dataset::cleanup_old_versions + lance.auto_cleanup.* settings that can remove old versions. The version log is retention-policy-gated, not by-construction-immutable — and following the original "introduce no separate audit store" guidance could make historical audit reads disappear after cleanup.

Codex P1 finding (verbatim from #465 review)

Preserve audit retention outside prunable Lance versions

For deployments that run Lance version cleanup, this premise is unsafe: Lance 7.0.0 exposes Dataset::cleanup_old_versions and lance.auto_cleanup.* settings that can remove old versions, so the version log is not guaranteed to be an immutable audit trail unless consumers explicitly disable cleanup/tag retained versions. Because this doc later tells substrate-b consumers not to introduce separate audit storage, following it can make historical audit reads disappear after cleanup.

Corrections applied

§2.1 third primitive — audit reframed

• Renamed from "Immutable audit" → "Audit (retention-policy-gated)"
• Old text: "append-only by construction — versions never disappear; the log IS the audit trail"
• New text: append-only at write time, but Lance supports version cleanup. The version log is not guaranteed immutable without explicit retention policy. For audit-class workloads, consumers MUST configure retention — disable auto-cleanup OR tag versions OR route audit-class events to a separate append-only sink. For regulatory-grade "cannot be deleted, cannot be manipulated" guarantees, the external signed sink is mandatory — Lance versions alone are NOT a substitute.

§5.1 — collapse pattern reframed

• Renamed from "Three OLD components collapse to one" → "Two-and-a-half OLD components collapse to one"
• Old guidance: "consumers should NOT introduce separate stores for these three capabilities"
• New guidance:
  • Non-regulatory audit (operational logging, compliance-as-best-effort): Lance versions serve IF retention is configured (auto-cleanup disabled, versions tagged, or cleanup_old_versions not invoked on the audit dataset)
  • Regulatory-grade audit (immutable-by-mandate compliance): Lance versions alone are NOT a substitute; a separate signed write-once sink remains a separate concern; substrate-b doesn't claim to replace it
• Honest framing: substrate-b collapses Historisation + TSDB into one primitive; the audit case is conditional on retention policy + workload class

What survives unchanged

The three-primitives codification (E-SUBSTRATE-B-CAPABILITY-ROADMAP from #465) is still load-bearing. The multi-purpose-Lance-versions claim (point-in-time + time-series) is unchanged; only the audit guarantee + the consumer-default guidance change. §2.2 (per-element auth) and §2.3 (Rubicon state machine) are untouched.

Board hygiene (per CLAUDE.md mandatory rule)

Same PR includes:

• .claude/board/EPIPHANIES.md PREPEND: E-AUDIT-RETENTION-CAVEAT — records the correction with full provenance.
• .claude/board/AGENT_LOG.md PREPEND: D-SUBSTRATE-B-CONSUMER-DOC-FIX.

Both PREPENDs follow the append-only discipline (no edit to past entries; new dated correction entry citing the original).

Severity

P1 — preventive correction. The original doc's audit guarantee was unsafe for any deployment that runs Lance cleanup; following its guidance could lose historical audit reads. This PR brings the guidance back in line with Lance's actual durability semantics.

[coderabbitai]

Warning

Review limit reached

@AdaWorldAPI, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 55 minutes and 20 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 75a0c239-2f46-42ac-ae67-2bd6f34c3b67

📥 Commits

Reviewing files that changed from the base of the PR and between 6c93d48 and cd850fb.

📒 Files selected for processing (3)

• .claude/board/AGENT_LOG.md
• .claude/board/EPIPHANIES.md
• .claude/knowledge/old-stack-capability-parity.md

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch doc/knowledge-old-stack-capability-parity-fix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}