|
| 1 | +--- |
| 2 | +date: 2026-04-21 |
| 3 | +topic: mainline-foundation-integration-and-post-checkpoint-direction |
| 4 | +--- |
| 5 | + |
| 6 | +# Mainline Foundation Integration and Post-Checkpoint Direction Requirements |
| 7 | + |
| 8 | +## Problem Frame |
| 9 | + |
| 10 | +`main` no longer needs another "did backend independence land?" checkpoint. |
| 11 | +That question is already answered by code and verification. |
| 12 | + |
| 13 | +Current drift is elsewhere: |
| 14 | + |
| 15 | +- the 2026-04-20 checkpoint document still describes foundation promotion as `in_progress` / `no-go`, |
| 16 | +- top-level EN/ZH dashboard sections were corrected, but the checkpoint source-of-truth chain still needs a superseding record, |
| 17 | +- the next lane question is no longer "can we land bounded graph/vector independence?" but "is the current local sqlite + ANN baseline enough, or is a heavier backend justified?" |
| 18 | + |
| 19 | +This document supersedes the 2026-04-20 checkpoint as the current source-of-truth item for post-M8 direction. |
| 20 | + |
| 21 | +## Verified Mainline Reality (2026-04-21) |
| 22 | + |
| 23 | +- `main` already carries the full bounded notification-threshold governance closure through `M7.29`. |
| 24 | +- `main` already carries the bounded foundation integration closure that earlier checkpoint documents treated as future work: |
| 25 | + - embedded sqlite graph persistence, |
| 26 | + - dedicated query backend boundary, |
| 27 | + - deterministic local embedding generation, |
| 28 | + - ANN-style bucket pruning, |
| 29 | + - readiness route/report/runtime parity, |
| 30 | + - promotion criteria matrix and derived ids. |
| 31 | +- current foundation status on `main` is: |
| 32 | + - `status = integrated` |
| 33 | + - `decision = go` |
| 34 | + - `promotionCriteriaPassed = 7/7` |
| 35 | + - `promotionBlockers = []` |
| 36 | + - `graphBackendIndependent = true` |
| 37 | + - `vectorAdapterIndependent = true` |
| 38 | +- current bounded backend semantics on `main` are: |
| 39 | + - `storeType = sqlite` |
| 40 | + - `graphBackendSignalKind = embedded_graphdb` |
| 41 | + - `queryBackendScoreSignals = [keyword_matches, title_match_bonus, vector_ann_similarity_bonus, relation_bonus]` |
| 42 | + - `vectorAdapterSignalKind = embedding_ann` |
| 43 | + - query/runtime traces surface `mode = local_ann`, `status = independent` |
| 44 | +- local verification already confirms that this is not doc-only drift: |
| 45 | + - `npm run verify:foundation:readiness` |
| 46 | + - `npm run test:migration` |
| 47 | + - `npm run test:agent-workspace:contracts` |
| 48 | + - `npm run docs:diataxis:check` |
| 49 | + - `npm run docs:site:build` |
| 50 | +- remote CI for the latest `main` push should still be watched normally, but failures must be interpreted against this integrated baseline rather than against older `no-go` assumptions. |
| 51 | + |
| 52 | +## Deep Comparison: Prior Direction vs Current Code |
| 53 | + |
| 54 | +| Prior source | Earlier direction | Current code reality | Judgment | Consequence | |
| 55 | +|---|---|---|---|---| |
| 56 | +| `docs/brainstorms/2026-04-20-mainline-checkpoint-ci-hardening-and-post-m8-direction-requirements.md` | checkpoint closure while foundation still remains `in_progress` / `no-go` | sqlite graph persistence and bounded local ANN retrieval are already integrated on `main` | now stale as current source | preserve as historical checkpoint, but stop using it as current truth | |
| 57 | +| `docs/diataxis/en/explanation/development-progress-dashboard.md` + ZH counterpart | top-level summary corrected to integrated/go | current-facing summary now matches code reality | correct | treat dashboard top sections as current product-facing truth | |
| 58 | +| `docs/diataxis/*/explanation/foundation-reentry-readiness-checklist.md` | anti-overclaim contract for pre-integration state | checklist now correctly records integrated/go bounded readiness | correct | keep using it as the operator contract page | |
| 59 | +| earlier post-M8 direction framing | backend independence should be the next execution target by default | bounded backend independence already landed locally; the remaining question is whether external backends are warranted | premise changed | next plan must evaluate sufficiency/ROI, not replay already-landed work | |
| 60 | + |
| 61 | +## Critical Findings |
| 62 | + |
| 63 | +1. Treating backend independence as still "not landed" is now factually wrong for the bounded local-first contract on `main`. |
| 64 | + |
| 65 | +2. The architecture question changed shape. |
| 66 | + The relevant decision is no longer whether `main` can escape file snapshots and lexical prefiltering. |
| 67 | + It already did. |
| 68 | + The relevant decision is whether local sqlite + deterministic ANN is sufficient for current product goals. |
| 69 | + |
| 70 | +3. External graph/vector backends are no longer default next steps. |
| 71 | + They now require explicit justification such as: |
| 72 | + - corpus size beyond comfortable local indexing bounds, |
| 73 | + - multi-process or multi-user write coordination, |
| 74 | + - remote/shared retrieval requirements, |
| 75 | + - latency/SLA targets local baseline cannot hit, |
| 76 | + - observability or recovery needs local embedded storage cannot satisfy. |
| 77 | + |
| 78 | +4. Markdown-reader and other product-surface work should not be blocked by a fictional "backend independence still missing" narrative. |
| 79 | + They should only remain secondary if higher-value user/runtime work is chosen deliberately. |
| 80 | + |
| 81 | +## Requirements |
| 82 | + |
| 83 | +**Truth Refresh** |
| 84 | +- R1. Add a fresh 2026-04-21 direction record instead of silently rewriting the 2026-04-20 checkpoint document. |
| 85 | +- R2. EN/ZH dashboard execution anchors must point to this 2026-04-21 document first. |
| 86 | +- R3. EN/ZH top-level delivery-focus text must describe post-checkpoint direction and CI interpretation, not pre-integration backend uncertainty. |
| 87 | +- R4. No current-facing summary section may regress to `file-backed`, `prefilter_only`, or `in_progress` / `no-go` wording for the present repository baseline. |
| 88 | + |
| 89 | +**Next-Lane Governance** |
| 90 | +- R5. The next planning round must answer one concrete question: is the current local backend baseline enough for the next product phase? |
| 91 | +- R6. If the answer is "no", the plan must state which deficiency justifies a heavier backend and why sqlite + local ANN is insufficient. |
| 92 | +- R7. If the answer is "yes", backend work should pause and the next user-facing lane should be chosen explicitly between markdown-reader and broader agent/product improvements. |
| 93 | + |
| 94 | +**CI Interpretation** |
| 95 | +- R8. Remote CI failures, if any, must be triaged against the integrated baseline now on `main`, not against older readiness assumptions. |
| 96 | +- R9. Long-running or queued jobs must not be misreported as architecture regressions without concrete failing evidence. |
| 97 | + |
| 98 | +## Success Criteria |
| 99 | + |
| 100 | +- a new 2026-04-21 checkpoint/direction document exists and becomes the top current item in `docs/brainstorms/index.md`, |
| 101 | +- EN/ZH progress dashboards point to it from their execution anchors and current-priority framing, |
| 102 | +- docs verification passes locally, |
| 103 | +- the work is pushed to `main`, |
| 104 | +- the worktree remains clean after push. |
| 105 | + |
| 106 | +## Scope Boundaries |
| 107 | + |
| 108 | +- This document does not reopen notification-threshold expansion work. |
| 109 | +- This document does not claim an external graph or vector service is required. |
| 110 | +- This document does not start markdown-reader execution by itself. |
| 111 | +- This document does not rewrite historical archive sections that are clearly chronological. |
| 112 | + |
| 113 | +## Key Decisions |
| 114 | + |
| 115 | +- **Preserve history by superseding, not mutating**: the 2026-04-20 checkpoint stays as an auditable snapshot of an earlier assumption set. |
| 116 | +- **Treat local sqlite + ANN as a real baseline, not a placeholder**: code and tests now support that claim directly. |
| 117 | +- **Raise the bar for heavier backends**: future graph/vector escalation must be justified by measurable need, not by roadmap inertia. |
| 118 | +- **Keep next-lane selection honest**: user-facing work should compete against backend escalation on product value, not lose to outdated architecture assumptions. |
| 119 | + |
| 120 | +## Outstanding Questions |
| 121 | + |
| 122 | +### Deferred to Planning |
| 123 | +- Which measurable thresholds would justify moving beyond local sqlite + deterministic ANN? |
| 124 | +- If the local backend baseline is sufficient, which next lane has higher product value: markdown-reader improvements or broader agent/product-surface work? |
| 125 | + |
| 126 | +## Next Steps |
| 127 | + |
| 128 | +-> /ce:plan for `local backend baseline sufficiency vs external backend escalation`, then choose the next user-facing lane explicitly. |
0 commit comments