Skip to content

Commit e4136bf

Browse files
josealekhinejosealekhine
committed
docs: full architecture refresh with principal analysis and GitNexus enrichment
Rewrite all architecture artifacts after the v0.8.0 restructuring (527 packages, 34 commands, MCP server, cmd/core taxonomy). Principal mode analysis with agent-agnosticism as strategic lens. GitNexus enrichment adds verified blast radius data to danger zones. New artifacts: - ARCHITECTURE.md rewritten with layered architecture (6 layers) - DETAILED_DESIGN split into 5 domain files (foundation, domain, mcp, cli, output) - CHEAT-SHEETS.md with 6 lifecycle flows + execution flow index - ARCHITECTURE-PRINCIPAL.md with strategic analysis, silent choices, clustering comparison, intervention points - DANGER-ZONES.md with 25 zones (4 CRITICAL after enrichment) - EXTENSION-POINTS.md with 14 patterns from call graph - CONVERGENCE-REPORT.md with per-module confidence scores Package READMEs for high-navigation packages: - internal/config/README.md: organizational guide + decision tree - internal/mcp/README.md: package map + how-to-add recipes - internal/cli/system/README.md: hook protocol + throttle patterns Also: 9 glossary terms, 2 decisions, 1 learning persisted. Updated config/doc.go to reflect the 60+ sub-package structure. Spec: specs/architecture-refresh-2026-04.md Signed-off-by: Jose Alekhinne <jose@ctx.ist>
1 parent bf42b1f commit e4136bf

23 files changed

Lines changed: 3618 additions & 1542 deletions

‎.context/ARCHITECTURE-PRINCIPAL.md‎

Lines changed: 507 additions & 0 deletions
Large diffs are not rendered by default.

‎.context/ARCHITECTURE.md‎

Lines changed: 289 additions & 237 deletions
Large diffs are not rendered by default.

‎.context/CHEAT-SHEETS.md‎

Lines changed: 243 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,243 @@
1+
# Cheat Sheets
2+
3+
Quick mental models for key lifecycles and flows in ctx.
4+
5+
## CLI Command Dispatch
6+
7+
Steps:
8+
1. `cmd/ctx/main.go` calls `bootstrap.RootCmd()`
9+
2. RootCmd creates Cobra root with global flags
10+
3. `Initialize()` registers 34 commands in 8 groups
11+
4. PersistentPreRunE fires: boundary check, init check
12+
5. Cobra routes to matched command's Run() handler
13+
6. Handler calls core/ logic, domain packages
14+
7. Output via write/* package, errors via err/* package
15+
16+
Key invariants:
17+
- PersistentPreRunE runs for ALL subcommands
18+
- AnnotationSkipInit bypasses init check (doctor, guide, setup)
19+
- All flag descriptions from YAML (enforced by audit)
20+
21+
Common failure modes:
22+
- Missing .context/ without SkipInit -> error before Run()
23+
- Flag name collision with global flag -> silent shadowing
24+
25+
```
26+
main.go --> RootCmd() --> PersistentPreRunE --> Run()
27+
| | |
28+
Initialize() boundary + core/ logic
29+
34 commands init guards --> write/*
30+
```
31+
32+
---
33+
34+
## Context Loading (ctx agent)
35+
36+
Steps:
37+
1. Agent calls `ctx agent --budget N`
38+
2. rc.TokenBudget() resolves budget (flag > env > .ctxrc > 8000)
39+
3. context/load.Do() reads all .md files from .context/
40+
4. Files sorted by config.FileReadOrder priority
41+
5. Each file's tokens estimated (4 chars/token)
42+
6. Files added in priority order until budget exhausted
43+
7. Overflow files listed as "Also Noted" summaries
44+
8. Markdown packet returned to stdout
45+
46+
Key invariants:
47+
- CONSTITUTION always loaded first
48+
- Symlinks rejected (M-2 defense)
49+
- Budget is conservative overestimate (never under-counts)
50+
51+
Common failure modes:
52+
- Very large TASKS.md consumes most of budget -> low-priority
53+
files (GLOSSARY, PLAYBOOK) never seen by agent
54+
- Empty file detection via EffectivelyEmpty() -> skipped with note
55+
56+
```
57+
Agent --> rc.TokenBudget() --> load.Do()
58+
| |
59+
resolve priority read + estimate tokens
60+
| |
61+
sort by order fit to budget
62+
| |
63+
format packet "Also Noted" overflow
64+
```
65+
66+
---
67+
68+
## MCP Request Lifecycle
69+
70+
Steps:
71+
1. Client sends JSON-RPC line to stdin
72+
2. Server.Serve() reads line from scanner
73+
3. parse.Request() unmarshals JSON
74+
4. dispatch.Do() routes by method name
75+
5. Handler executes domain logic
76+
6. session.CheckGovernance() appends warnings
77+
7. out.*Response() wraps result
78+
8. io.Writer.WriteJSON() writes to stdout
79+
80+
Key invariants:
81+
- Main loop is single-threaded (sequential processing)
82+
- Governance is advisory only (never blocks)
83+
- Notifications (no ID) produce no response
84+
- Poller runs independently on 5s interval
85+
86+
Common failure modes:
87+
- Slow handler blocks all subsequent requests
88+
- Parse error -> error response, loop continues
89+
- Scanner overflow -> truncated JSON -> parse error
90+
91+
```
92+
Client Server Handler
93+
|--JSON-RPC line------>| |
94+
| |--parse() |
95+
| |--dispatch()------>|
96+
| | |--domain logic
97+
| | |--governance check
98+
| |<--result----------|
99+
|<--JSON-RPC response--| |
100+
```
101+
102+
---
103+
104+
## Journal Import Pipeline
105+
106+
Steps:
107+
1. User runs `ctx journal source --all`
108+
2. parser.FindSessionsForCWD() scans ~/.claude/projects/
109+
3. Auto-detects format (JSONL, Copilot, Copilot CLI, Markdown)
110+
4. Matches sessions by git remote URL and CWD
111+
5. Loads journal state from .state.json
112+
6. Plans each session: new, regen, skip, or locked
113+
7. Formats matched sessions as Markdown
114+
8. Writes to .context/journal/
115+
9. Marks imported in state, saves state
116+
117+
Key invariants:
118+
- Locked entries never regenerated
119+
- State tracks 5 stages: exported -> enriched -> normalized
120+
-> fences_verified -> locked
121+
- Atomic state writes (temp + rename)
122+
123+
Common failure modes:
124+
- Changed JSONL format -> silent parse failures, empty sessions
125+
- Same project in multiple paths -> duplicate imports
126+
- 1MB buffer limit -> truncated large tool results
127+
128+
```
129+
[Scan] --> [Detect Format] --> [Match CWD]
130+
| | |
131+
4 parsers auto-detect git remote
132+
| | |
133+
[Plan] ----> [Format MD] ----> [Write]
134+
| |
135+
state check mark imported
136+
```
137+
138+
---
139+
140+
## Hook Lifecycle (UserPromptSubmit)
141+
142+
Steps:
143+
1. Claude Code fires UserPromptSubmit hook
144+
2. hooks.json routes to `ctx system check-*` commands
145+
3. system/input.go reads hook JSON from stdin (2s timeout)
146+
4. Each check runs independently, writes result to stdout
147+
5. Checks: context-size, ceremonies, persistence, journal,
148+
reminders, version, resources, knowledge, map-staleness,
149+
memory-drift, freshness, heartbeat
150+
6. Advisory output returned to Claude Code
151+
7. All hooks exit 0 (never block initialization)
152+
153+
Key invariants:
154+
- 2-second stdin read timeout (prevents hanging)
155+
- Daily throttle via marker file date comparison
156+
- Adaptive prompt counter: silent 1-15, periodic 16+
157+
- All hooks exit 0 (never block)
158+
159+
Common failure modes:
160+
- Missing stdin JSON -> timeout, graceful empty response
161+
- Throttle marker file corruption -> check runs every prompt
162+
163+
```
164+
Claude Code hooks.json ctx system check-*
165+
|--hook fire------->| |
166+
| |--route--------->|
167+
| | (12 checks) |--read stdin (2s)
168+
| | |--check logic
169+
| | |--throttle gate
170+
|<--advisory--------|<--result (0)----|
171+
```
172+
173+
---
174+
175+
## Entry Write Flow
176+
177+
Steps:
178+
1. Caller provides EntryParams (type, content, opts)
179+
2. entry.Validate() checks required fields per type
180+
- Decision: context, rationale, consequence
181+
- Learning: context, lesson, application
182+
- Task/Convention: content only
183+
3. entry.Write() reads existing file
184+
4. Formats entry per type template (from tpl/)
185+
5. Inserts at correct position (tasks: before first unchecked)
186+
6. Writes file back via io.SafeWriteFile()
187+
7. Updates index for decisions/learnings (not tasks/conventions)
188+
189+
Key invariants:
190+
- Entry headers are timestamped: `## [YYYY-MM-DD-HHMMSS] Title`
191+
- Index updated between INDEX:START/END markers
192+
- Three callers: CLI add, MCP handler, watch command
193+
194+
Common failure modes:
195+
- Concurrent writes -> last writer wins (no locking)
196+
- Index update fails after write -> stale index
197+
- Missing required field -> validation error before write
198+
199+
```
200+
Caller --> Validate() --> Write()
201+
| |
202+
check fields read-modify-write
203+
| |
204+
type-specific format + insert
205+
| |
206+
error early update index
207+
```
208+
209+
---
210+
211+
## Execution Flow Index (enriched 2026-04-03 via GitNexus)
212+
213+
_Auto-detected from the call graph. Complements the manually
214+
written cheat sheets above._
215+
216+
Top cross-community flows (spanning multiple domains):
217+
218+
| Flow | Steps | Entry Point | Key Symbols |
219+
|------|-------|-------------|-------------|
220+
| Deploy -> ContextDir | 10 | initialize | DeployTemplates, Do, ContextDir |
221+
| Deploy -> Symlinks | 10 | initialize | DeployTemplates, Do, Symlinks |
222+
| Write -> Init | 10 | MCP server | Serve, Init, SafeWriteFile |
223+
| Write -> Server | 10 | MCP server | Serve, New, Do |
224+
| Write -> TokenBudget | 10 | MCP server | Serve, TokenBudget, Do |
225+
| Run -> Init | 10 | CLI commands | Run, Do, Init |
226+
| Sync -> ContextDir | 10 | sync cmd | Run, Do, ContextDir |
227+
| Run -> Text | 9 | CLI commands | Run, Do, desc.Text |
228+
| Run -> URI | 9 | CLI commands | Run, catalog, URI |
229+
| Run -> NotFoundError | 8 | CLI commands | Run, Do, NotFound |
230+
231+
### Multi-Flow Hotspots
232+
233+
Symbols participating in 3+ flows (high-impact modification points):
234+
235+
| Symbol | Flows | Location |
236+
|--------|-------|----------|
237+
| desc.Text | 53 | internal/assets/read/desc/desc.go:75 |
238+
| load.Do | 100+ | internal/context/load/loader.go:34 |
239+
| SafeWriteFile | 69 callers | internal/io/security.go |
240+
| rc.ContextDir | 20+ | internal/rc/rc.go |
241+
| validate.Symlinks | 10+ | internal/validate/path.go |
242+
| err/context.NotFound | 30+ | internal/err/context/context.go |
243+
| rc.RC | 15+ | internal/rc/rc.go |

‎.context/CONVERGENCE-REPORT.md‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# Convergence Report
2+
3+
_Generated 2026-04-03 by /ctx-architecture principal_
4+
5+
## By Module
6+
7+
| Module | Confidence | Status | Blocker |
8+
|--------|------------|--------|---------|
9+
| internal/config/* | 0.85 | 🟡 Solid | Pattern understood; not all 60 sub-packages individually read |
10+
| internal/assets | 0.85 | 🟡 Solid | Embed and read/* pattern understood; tpl/ migration pending |
11+
| internal/io | 0.80 | 🟡 Solid | API understood; edge cases not all traced |
12+
| internal/format | 0.80 | 🟡 Solid | All functions cataloged |
13+
| internal/parse | 0.80 | 🟡 Solid | Small package, fully understood |
14+
| internal/sanitize | 0.80 | 🟡 Solid | Small package, fully understood |
15+
| internal/validate | 0.70 | 🟡 Solid | API inferred; source not directly read this run |
16+
| internal/inspect | 0.75 | 🟡 Solid | API cataloged from survey |
17+
| internal/flagbind | 0.85 | 🟡 Solid | Pattern and all variants documented |
18+
| internal/exec/* | 0.80 | 🟡 Solid | All 5 wrappers surveyed |
19+
| internal/log/* | 0.80 | 🟡 Solid | Event + warn split understood |
20+
| internal/crypto | 0.90 | ✅ Converged | Converged |
21+
| internal/sysinfo | 0.85 | 🟡 Solid | Platform build tags understood |
22+
| internal/rc | 0.90 | ✅ Converged | Converged |
23+
| internal/entity | 0.85 | 🟡 Solid | All types cataloged; some methods not fully traced |
24+
| internal/entry | 0.85 | 🟡 Solid | Flow understood; not all callers traced |
25+
| internal/context/* | 0.80 | 🟡 Solid | 6 sub-packages surveyed |
26+
| internal/drift | 0.85 | 🟡 Solid | 7 checks documented |
27+
| internal/index | 0.85 | 🟡 Solid | Converged |
28+
| internal/task | 0.85 | 🟡 Solid | Converged |
29+
| internal/tidy | 0.80 | 🟡 Solid | Block parsing understood |
30+
| internal/trace | 0.80 | 🟡 Solid | Flow documented; resolve helpers not all read |
31+
| internal/journal/parser | 0.80 | 🟡 Solid | 4 parsers documented; Copilot parsers shallow |
32+
| internal/journal/state | 0.85 | 🟡 Solid | Pipeline stages documented |
33+
| internal/memory | 0.80 | 🟡 Solid | Discovery + sync flow understood |
34+
| internal/notify | 0.85 | 🟡 Solid | Converged |
35+
| internal/claude | 0.80 | 🟡 Solid | Thin wrapper understood |
36+
| internal/mcp/* | 0.90 | ✅ Converged | All 15 sub-packages deeply read |
37+
| internal/cli/* (34 cmds) | 0.75 | 🟡 Solid | Taxonomy understood; not all core/ packages deeply read |
38+
| internal/bootstrap | 0.85 | 🟡 Solid | Registration and guards documented |
39+
| internal/write/* | 0.80 | 🟡 Solid | Pattern understood; not all 46 packages individually read |
40+
| internal/err/* | 0.80 | 🟡 Solid | Pattern understood; not all 35 packages individually read |
41+
| internal/audit | 0.75 | 🟡 Solid | Purpose understood; individual test files not read |
42+
| internal/compliance | 0.70 | 🟡 Solid | Purpose understood; tests not read |
43+
44+
## By Domain
45+
46+
| Domain | Modules | Converged | Avg Confidence |
47+
|--------|---------|-----------|----------------|
48+
| Foundation (config, assets, infra) | 12 | 2/12 | 0.82 |
49+
| Domain (entity through claude) | 13 | 0/13 | 0.82 |
50+
| MCP | 1 (15 sub-pkgs) | 1/1 | 0.90 |
51+
| CLI | 2 (34 commands) | 0/2 | 0.78 |
52+
| Output (write, err) | 2 | 0/2 | 0.80 |
53+
| Quality (audit, compliance) | 2 | 0/2 | 0.73 |
54+
55+
## Overall
56+
57+
- Total module groups: 32
58+
- Converged (>= 0.9): 3 ✅ (crypto, rc, mcp)
59+
- Solid (0.7-0.89): 28 🟡
60+
- Shallow (0.4-0.69): 0 🔶
61+
- Stubbed (< 0.4): 0 🔴
62+
63+
## What Would Help Next
64+
65+
🟡 internal/cli/* (0.75) - Solid
66+
→ Deep-read core/ packages for journal, agent, system, add
67+
(the largest/most complex commands)
68+
→ Read test files to understand edge case behavior
69+
→ Ask: "walk me through what happens when ctx journal site runs"
70+
71+
🟡 internal/audit (0.75) - Solid
72+
→ Read individual audit test files to catalog all 40+ checks
73+
→ Understand quarantine/allowlist management patterns
74+
75+
🟡 internal/compliance (0.70) - Solid
76+
→ Read compliance test files to understand file-level checks
77+
→ Understand relationship to Makefile lint targets
78+
79+
🟡 internal/journal/parser (0.80) - Solid
80+
→ Deep-read Copilot and Copilot CLI parsers for parity assessment
81+
→ Read query.go for session lookup capabilities
82+
83+
🟡 internal/trace (0.80) - Solid
84+
→ Read all resolve_* helper files for ref resolution logic
85+
→ Trace full commit lifecycle from collect to trailer
86+
87+
## Convergence Verdict
88+
89+
🟡 **MOSTLY CONVERGED** - All modules at 0.70+. Core modules (MCP,
90+
config pattern, domain flow) well understood. Diminishing returns on
91+
full re-run; use focus areas for journal, system hooks, and audit
92+
tests to reach full convergence.
93+
94+
---
95+
96+
## Search Prompts
97+
98+
The right keyword changes everything. Based on what I found in
99+
the codebase, here are targeted searches worth running — in your
100+
internal docs, Confluence, Notion, Slack, or publicly:
101+
102+
### Fill the gaps (ranked by how much they'd help)
103+
104+
🟡 internal/cli/system (0.75) - 34 hook subcommands
105+
Try searching:
106+
- "ctx system check-persistence" behavior or "adaptive prompt counter"
107+
- "Claude Code hook lifecycle" or "UserPromptSubmit hook protocol"
108+
- "ctx hook throttle" or "daily throttle marker"
109+
110+
🟡 internal/audit (0.75) - 40+ AST audit checks
111+
Try searching:
112+
- "TestNoDeadExports quarantine" or "grandfathered map audit"
113+
- "Go AST audit pattern" or "go/packages test convention"
114+
115+
### Concepts worth understanding deeply
116+
117+
- "MCP protocol 2024-11-05 spec" — the protocol ctx implements;
118+
understanding the full spec reveals capability gaps
119+
- "Claude Code JSONL session format" — reverse-engineered; the
120+
actual schema would replace guesswork in journal/parser
121+
- "Copilot instructions.md specification" — understanding what
122+
Copilot expects would inform agent-agnostic setup
123+
- "go:embed performance large binary" — ctx embeds skills, hooks,
124+
templates, YAML; understanding embed overhead at scale matters
125+
126+
### Architecture decision records
127+
128+
- "ctx context priority order rationale" or "FileReadOrder design"
129+
— why CONSTITUTION loads first is a design decision worth ADR
130+
- "ctx YAML externalization decision" or "i18n readiness Go CLI"
131+
— the 879-key text externalization is a major architectural bet
132+
- "ctx MCP vs CLI parity decision" — whether the capability gap
133+
between 34 CLI commands and 11 MCP tools is intentional
134+
135+
---
136+
137+
Note: I did not run these searches — you may have internal docs
138+
where these are more useful than public results, and you know
139+
which sources to trust.
140+
141+
---
142+
143+
## Enrichment Summary
144+
145+
_Last enrichment: 2026-04-03 via GitNexus (index: bf42b1f6)_
146+
147+
| Phase | Items Processed | Key Findings |
148+
|-------|----------------|--------------|
149+
| Danger zones | 25 entries | 4 upgraded to CRITICAL (desc.Text, SafeWriteFile, DescKey-YAML, DiscoverPath); 1 new (load.Do) |
150+
| Extension points | 14 patterns | Session parser (4 registrations), CLI commands (34), MCP tools (11), MCP prompts (5), agent setup (5) |
151+
| Execution flows | 257 total, 10 indexed | 7 multi-flow hotspots identified (desc.Text at 53 flows is #1) |
152+
| Clustering | 94 clusters vs 5 manual domains | Journal (47%) and Initialize (50%) are low-cohesion; write/err are leaves, not communities |
153+
| Shallow modules | 0 enriched | All modules already >= 0.70; no confidence bumps warranted |

0 commit comments

Comments
 (0)