diff --git a/specs/001-system-invariants/spec.md b/specs/001-system-invariants/spec.md index b6473cb5e..fb8f2da48 100644 --- a/specs/001-system-invariants/spec.md +++ b/specs/001-system-invariants/spec.md @@ -40,7 +40,7 @@ related: | AnyProvider enum | `crates/zeph-llm/src/any.rs` | | Message / MessagePart | `crates/zeph-llm/src/provider.rs` | | ToolExecutor trait | `crates/zeph-tools/src/executor.rs` | -| Config struct | `crates/zeph-core/src/config/types/mod.rs` | +| Config struct | `crates/zeph-config/src/root.rs` | | Feature flags | `Cargo.toml` | --- @@ -158,7 +158,7 @@ file I/O in the agent loop. ## 8a. Provider Registry Contract -`[[llm.providers]]` (`crates/zeph-config/src/providers.rs`) is the **single source of truth** for all LLM provider declarations: +`[[llm.providers]]` (`crates/zeph-config/src/providers/` — `mod.rs`, `entry.rs`, `llm.rs`, `router.rs`, `thinking.rs`, `candle.rs`) is the **single source of truth** for all LLM provider declarations: - All providers are declared **once** in `[[llm.providers]]` with a `name` field; no other section duplicates provider credentials or model names - Subsystems that call LLMs reference a provider by name via a `*_provider` config field (e.g., `extract_provider = "fast"`) @@ -177,7 +177,7 @@ See `.local/specs/022-config-simplification/spec.md` for the full schema and exa Feature flags (`Cargo.toml [features]`): -- `default` includes features required for the standard CLI experience: database backend (`sqlite`), scheduler, task metrics, profiling instrumentation, and OS-level sandbox availability. See spec 029 §3.1 for the canonical list. +- `default = ["scheduler", "sqlite"]` — the minimal set required for the standard CLI experience: cron scheduler and the SQLite database backend. Per-task CPU/wall-time metrics are an always-on capability (no flag, see spec 029 §3.3), not a default feature. See spec 029 §3.1 for the canonical list. - Default features must satisfy two criteria: (a) the feature gates a real optional dependency (not a pure behavioral marker), and (b) the feature has `Tested` status in coverage-status.md with no open P0/P1 issues. - **Always-on** (compiled in without feature flags): openai, compatible, orchestrator, router, self-learning, qdrant, vault-age, mcp - New optional crates: `dep:zeph-` in the feature definition — never unconditionally import diff --git a/specs/002-agent-loop/spec.md b/specs/002-agent-loop/spec.md index 946dd6cf7..4df2468df 100644 --- a/specs/002-agent-loop/spec.md +++ b/specs/002-agent-loop/spec.md @@ -37,7 +37,7 @@ related: | File | Contents | |---|---| | `crates/zeph-core/src/agent/mod.rs` | `Agent`, `run()`, `process_user_message()`, sub-state structs | -| `crates/zeph-core/src/agent/feedback_detector.rs` | `FeedbackDetector`, `CorrectionSignal` | +| `crates/zeph-agent-feedback/src/lib.rs` | `FeedbackDetector`, `CorrectionSignal` (re-exported into `zeph-core` via `crates/zeph-core/src/agent/corrections.rs` as `feedback_detector`) | | `crates/zeph-core/src/agent/error.rs` | `AgentError` typed hierarchy | | `crates/zeph-core/src/channel.rs` | `Channel` trait, `ChannelError` | @@ -115,7 +115,7 @@ Agent { ## HiAgent Subgoal-Aware Compaction -`crates/zeph-core/src/agent/compaction_strategy.rs`, `crates/zeph-core/src/agent/mod.rs`. Issue #2022. +`crates/zeph-agent-context/src/compaction.rs` (`SubgoalRegistry`), `crates/zeph-core/src/agent/context/summarization/{mod.rs,pruning.rs,compaction.rs}`. Issue #2022. ### Overview @@ -205,7 +205,7 @@ Provider preference per channel is persisted to SQLite (#3308, #3385): - Restored automatically on next session start - Identity keyed by `(channel_type, channel_id)`; CLI/TUI use `channel_id = ""` - Controlled by `[session] provider_persistence = true` (default enabled) -- Migrations: SQLite `079_channel_preferences.sql`, Postgres `075_channel_preferences.sql` +- Migrations: SQLite `080_channel_preferences.sql`, Postgres `075_channel_preferences.sql` ### Key Invariants @@ -307,60 +307,68 @@ completion_phrases = ["task complete", "done", "finished", "completed"] ## TACO Output Compression (#3591) -TACO (Tool-output Automatic Compression and Offload) compresses large tool outputs -before they are injected into the context window. This is a targeted pre-injection -pass, distinct from the turn-level compaction that runs at the 60/90% pressure gates. +TACO (self-evolving tool output compression, `crates/zeph-tools/src/compression/`) +compresses large tool outputs before they are injected into the context window. +This is a targeted pre-injection pass, distinct from the turn-level compaction that +runs at the 60/90% pressure gates. It is **rule-based**, not a per-call LLM +summarization pass — regex rules matched against the raw output, not a threshold- +triggered prompt. + +### Architecture + +- `OutputCompressor` — core async trait (`compress(tool_name, output) -> Result, CompressionError>`) +- `IdentityCompressor` — default no-op, wired in when `[tools.compression] enabled = false` +- `RuleBasedCompressor` — regex rules loaded from SQLite/Postgres, compiled into + `parking_lot::RwLock>`; hit counts tracked in a + `dashmap::DashMap` so a rules-vec swap on reload cannot lose + unflushed counters +- `CompressedExecutor` — decorator wrapping the root `ToolExecutor`; applies the + compressor to successful `ToolOutput.summary` strings only, on error the raw + result is returned unchanged +- `CompressionRuleStore` — SQLite/Postgres-backed rule persistence +- `safe_compile` — DoS-safe regex compilation with a configurable timeout ### When It Fires -TACO is evaluated after each tool call result is received, before the result is -appended to `messages`: +`CompressedExecutor::maybe_compress` runs after the wrapped executor returns a +successful `ToolOutput`: -1. Measure the raw tool output token count via `tiktoken-rs` -2. If `token_count > taco_threshold` AND the tool is in the compressible-tool set - → run TACO compression -3. Compressed result replaces the raw result in `messages` +1. If `output.summary.lines().count() < min_lines_to_compress` → skip, output passed through +2. Otherwise, `RuleBasedCompressor::compress` applies rules in `id`-ascending order + (deterministic); a rule is skipped when its `glob` is set and does not match + `tool_name`; the first successful match wins +3. `Ok(None)` (no rule matched) or `Err(...)` → original `summary` is kept unchanged; + compression errors are logged as warnings but never propagate to the caller -### Compression Strategy +### Self-Evolution -TACO compression uses a fast prompt to summarize the tool output: - -``` -System: You are a concise tool-output summarizer. Preserve all data values, -file paths, exit codes, and structured content. Remove verbose headers and -repeated patterns. Target: under {target_tokens} tokens. -Tool output: -{raw_output} -``` - -The compressed result is tagged with `MessagePart::TacoCompressed` so the TUI and -audit log can distinguish it from raw output. - -### Compressible Tool Set - -Default: `["shell", "web_scrape", "read"]`. Configurable via -`[tools.taco] compressible_tools`. MCP tools are excluded from TACO by default -because their structured output schema is unknown. +New compression rules are periodically generated by an LLM call gated by +`evolution_provider` (empty string = evolution disabled) and rate-limited by +`evolution_min_interval_secs`. Rule count is capped at `max_rules`, pruning the +lowest-hit rules above the cap. ### Config +`[tools.compression]` TOML section (`ToolCompressionConfig`, `crates/zeph-config/src/tools.rs`): + ```toml -[tools.taco] -enabled = false # default off (opt-in) -taco_threshold = 2000 # tokens; compress outputs above this -target_tokens = 500 # target compressed size -taco_provider = "" # [[llm.providers]] name; empty = primary -compressible_tools = ["shell", "web_scrape", "read"] +[tools.compression] +enabled = false # default off (opt-in) +min_lines_to_compress = 10 # skip outputs with fewer lines +evolution_provider = "" # [[llm.providers]] name; empty = self-evolution disabled +evolution_min_interval_secs = 3600 # rate limit between self-evolution runs +max_rules = 200 # prune lowest-hit rules above this +regex_compile_timeout_ms = 500 # DoS-safe compile timeout ``` ### Key Invariants -- TACO fires only on output that exceeds `taco_threshold`; short outputs are passed through untouched -- On compression failure (provider error, timeout) the **raw output is used** — TACO is best-effort -- NEVER compress `tool_result` messages from `execute_tool_call_confirmed` (fenced-block path) — user-approved results must not be silently summarized -- NEVER apply TACO to thinking blocks or system prompt parts -- `taco_provider` is resolved via the provider registry at runtime; empty = primary provider -- Compressed results carry `MessagePart::TacoCompressed` to make compression auditable +- **T1**: `IdentityCompressor` is the only compressor when `[tools.compression] enabled = false` +- **T4**: audit logging happens inside individual tool implementations, not inside `CompressedExecutor` — because compression wraps *outside* the tool boundary, audit JSONL always records the raw pre-compression payload +- Rules are applied in `id`-ascending order (deterministic); the first successful match wins +- `regex::Regex::replace_all` guarantees linear time (no catastrophic backtracking); no `catch_unwind` needed around it +- On compression failure (bad pattern, DB error) the **raw output is used** — TACO is best-effort +- `evolution_provider` is resolved via the provider registry at runtime; empty = self-evolution disabled (not "primary provider") --- diff --git a/specs/003-llm-providers/spec.md b/specs/003-llm-providers/spec.md index c0cebec4b..92a6c2dca 100644 --- a/specs/003-llm-providers/spec.md +++ b/specs/003-llm-providers/spec.md @@ -44,12 +44,12 @@ related: |---|---| | `crates/zeph-llm/src/provider.rs` | `LlmProvider` trait, `Message`, `MessagePart`, `ChatResponse` | | `crates/zeph-llm/src/any.rs` | `AnyProvider` enum dispatch | -| `crates/zeph-llm/src/claude.rs` | Claude impl, `split_system_into_blocks`, prompt caching | -| `crates/zeph-llm/src/openai.rs` | OpenAI impl | +| `crates/zeph-llm/src/claude/mod.rs` | Claude impl, `split_system_into_blocks`, prompt caching | +| `crates/zeph-llm/src/openai/mod.rs` | OpenAI impl | | `crates/zeph-llm/src/ollama.rs` | Ollama impl | | `crates/zeph-llm/src/compatible.rs` | OpenAI-compatible HTTP impl | -| `crates/zeph-llm/src/gemini.rs` | Gemini impl | -| `crates/zeph-llm/src/orchestrator.rs` | Multi-provider routing, Thompson Sampling, EMA | +| `crates/zeph-llm/src/gemini/mod.rs` | Gemini impl | +| `crates/zeph-llm/src/router/` (`thompson.rs`, `reputation.rs`, `builder.rs`, `config.rs`) | Multi-provider routing, Thompson Sampling, EMA reputation | --- @@ -79,7 +79,9 @@ trait LlmProvider: Send + Sync { Runtime dispatch — no `Box` in hot paths: ``` -AnyProvider { Claude, OpenAI, Ollama, Compatible, Candle, Gemini } +AnyProvider { Ollama, Claude, OpenAi, Gemini, Candle (feature "candle"), + Compatible, Router(Box), Triage(Box), + Gonka (feature "gonka"), Cocoon (feature "cocoon") } ``` ## Implementations @@ -133,7 +135,7 @@ The `prompt_cache_ttl` field defaults to `None` (interpreted as `ephemeral`). Sp ## Orchestrator -`crates/zeph-llm/src/orchestrator.rs` — multi-provider routing: +`crates/zeph-llm/src/router/` (`RouterProvider`, `thompson.rs`, `reputation.rs`) — multi-provider routing: - **Rule-based routing**: match provider by name pattern, task type, or cost threshold - **Thompson Sampling router**: Beta-distribution exploration/exploitation for model selection diff --git a/specs/004-memory/004-11-memory-hela-mem.md b/specs/004-memory/004-11-memory-hela-mem.md index 91ee35046..510cdc9e3 100644 --- a/specs/004-memory/004-11-memory-hela-mem.md +++ b/specs/004-memory/004-11-memory-hela-mem.md @@ -216,7 +216,7 @@ folding_cooldown_turns = 10 ## 5. Acceptance Criteria -- [x] `graph_edges` table has `weight` column after migration 077; existing rows default to 1.0 (HL-F1 — implemented #3344) +- [x] `graph_edges` table has `weight` column after migration 078 (`078_hebbian_edge_weight.sql`); existing rows default to 1.0 (HL-F1 — implemented #3344) - [x] Co-activated node pairs show incremented weights after retrieval when `hebbian.enabled = true` (HL-F2 — implemented #3344) - [x] Consolidation pass runs on background task (HL-F3/F4 — implemented #3345, #3380) - [x] Spreading activation retrieval (`hela_spreading_recall`) implemented with BFS, multiplicative path weights, and circuit breaker (HL-F5 — implemented #3346) diff --git a/specs/004-memory/004-12-memory-reasoning-bank.md b/specs/004-memory/004-12-memory-reasoning-bank.md index c2f01bb03..9a3da850b 100644 --- a/specs/004-memory/004-12-memory-reasoning-bank.md +++ b/specs/004-memory/004-12-memory-reasoning-bank.md @@ -202,12 +202,12 @@ Token budget enforced by existing `ContextBudget` mechanism. --- -## 6. Implementation Priority +## 7. Implementation Priority **P2 — Medium priority.** Core read/write path is 2 weeks estimated complexity. MaTTS scaling variant (more compute per task for diverse experiences) is a separate follow-up sprint. Recommend: SQLite table + distillation pipeline first, Qdrant retrieval second, context injection third. --- -## 7. Related Issues +## 8. Related Issues - Closes #3312 diff --git a/specs/004-memory/004-13-memory-memcot.md b/specs/004-memory/004-13-memory-memcot.md index d080a3848..96d7dfa6c 100644 --- a/specs/004-memory/004-13-memory-memcot.md +++ b/specs/004-memory/004-13-memory-memcot.md @@ -39,10 +39,10 @@ related: | File | Contents | |---|---| -| `crates/zeph-memory/src/memcot/mod.rs` | Module root; `MemCotRecall` entry point | -| `crates/zeph-memory/src/memcot/accumulator.rs` | `SemanticStateAccumulator` — per-turn state tracking | -| `crates/zeph-memory/src/memcot/zoom.rs` | `zoom_in()` and `zoom_out()` retrieval views | -| `crates/zeph-memory/src/memcot/config.rs` | `MemCotConfig` TOML bindings | +| `crates/zeph-core/src/agent/memcot/mod.rs` | Module root; `MemCotRecall` entry point | +| `crates/zeph-core/src/agent/memcot/accumulator.rs` | `SemanticStateAccumulator` — per-turn state tracking | +| `crates/zeph-core/src/agent/memcot/metrics.rs` | MemCoT metrics | +| `crates/zeph-memory/src/recall_view.rs` | `zoom_in()` and `zoom_out()` retrieval views (no separate `zoom.rs`/`config.rs`) | --- diff --git a/specs/004-memory/004-14-memory-tiering-rfc-decision.md b/specs/004-memory/004-14-memory-tiering-rfc-decision.md index 0a3d0eabc..ff0a49806 100644 --- a/specs/004-memory/004-14-memory-tiering-rfc-decision.md +++ b/specs/004-memory/004-14-memory-tiering-rfc-decision.md @@ -108,11 +108,11 @@ Zeph's memory subsystem already implements: | | Async consolidation daemon | Yes — HeLa-Mem consolidation pass exists | Exact algorithm differs | *Subsumed* — HeLa-Mem covers the pattern | | | PPO-based weight adaptation | No — no RL loop in retrieval path | RL infrastructure missing | *Partial gap* — defer to future cycle (P4 research) | | **BudgetMem** | Per-query budget-tier routing | Partial — routing exists but not cost-aware | Cost model missing | *Adopt* — layer cost-aware router onto existing tiers | -| | Low/Mid/High tier selection | Yes — [[024-complexity-triage-routing]] implements complexity-based dispatch | Same as complexity routing | *Subsumed* — reuse existing routing layer | +| | Low/Mid/High tier selection | Yes — [[023-complexity-triage-routing/spec]] implements complexity-based dispatch | Same as complexity routing | *Subsumed* — reuse existing routing layer | | **Multi-Layer** | Working/episodic/semantic separation | Yes — SQLite working/episodic; Qdrant semantic | Explicit retrieval gating missing | *Adopt* — formalize tier-aware recall gating in SemanticMemory | | | Adaptive retrieval gating | Partial — MemMachine depth is configurable | Policy-based adaptive gating is new | *Partial gap* — layer adaptive gating logic | | **LCM** | Immutable message log | Yes — SQLite messages table is immutable | Complete match | *Subsumed* — log already exists | -| | Derived active context | Yes — [[021-zeph-context]] compaction state machine | Complete match | *Subsumed* — compaction strategy is active context | +| | Derived active context | Yes — [[021-zeph-context/spec]] compaction state machine | Complete match | *Subsumed* — compaction strategy is active context | | **MemRouter** | Learned write-side admission | No — A-MAC is rule-based; no learned gate | Neural classifier missing | *Partial gap* — extend A-MAC with optional learned gate (P3) | | | Embedding-based routing | Yes — MMR and SYNAPSE use embeddings | Same capability | *Subsumed* — already applies embeddings | @@ -127,7 +127,7 @@ Zeph's memory subsystem already implements: 2. **Three additive improvements** are valuable and feasible: - Formalize tier-aware retrieval gating (Multi-Layer §3) - Add frequency signal to A-MAC (MEMTIER §3.2) - - Layer cost-aware routing onto triage router (BudgetMem + [[024-complexity-triage-routing]]) + - Layer cost-aware routing onto triage router (BudgetMem + [[023-complexity-triage-routing/spec]]) 3. **RL adaptation (MEMTIER PPO)** is deferred — infrastructure cost is high for a P4 feature; HeLa-Mem consolidation addresses most of the value @@ -135,7 +135,7 @@ Zeph's memory subsystem already implements: ### Non-Adopted Papers -- **LCM**: Fully subsumed by existing [[021-zeph-context]] and message immutability invariant +- **LCM**: Fully subsumed by existing [[021-zeph-context/spec]] and message immutability invariant - **MEMTIER's PPO loop**: Deferred pending RL infrastructure (P4 research track) - **MemRouter's neural gate**: Deferred to P3 pending labeled dataset for training @@ -183,15 +183,15 @@ When a query of complexity C arrives: THEN search all three tiers, aggregate by relevance ``` -Complexity classification already exists in [[024-complexity-triage-routing]]; apply same classification to memory tiers. +Complexity classification already exists in [[023-complexity-triage-routing/spec]]; apply same classification to memory tiers. **Rationale**: Multi-Layer Framework showed +4pp improvement by avoiding full semantic search for simple queries, reducing token cost. ### 5.3 Triage Router Extension: Cost-Aware Tier Selection -**File**: `specs/024-complexity-triage-routing/spec.md` +**File**: `specs/023-complexity-triage-routing/spec.md` -Extend [[024-complexity-triage-routing]] to include memory tier cost model: +Extend [[023-complexity-triage-routing/spec]] to include memory tier cost model: ```toml [memory.tier_routing] @@ -235,7 +235,7 @@ This maps BudgetMem's Low/Mid/High routing onto Zeph's existing complexity triag 3. **Phase 3 (P4)**: RL weight adaptation (defer) - Requires reward signal infrastructure - - Blocked on [[042-experiments]] completing experiment framework + - Blocked on [[041-experiments/spec]] completing experiment framework --- @@ -265,5 +265,5 @@ This maps BudgetMem's Low/Mid/High routing onto Zeph's existing complexity triag - [[004-memory/spec]] — Parent spec - [[004-3-admission-control]] — A-MAC (to be extended) - [[004-10-memory-memmachine-retrieval]] — Retrieval depth (complements tier gating) -- [[024-complexity-triage-routing]] — Complexity-based routing (to be extended) -- [[021-zeph-context]] — Context budget (orthogonal; no changes needed) +- [[023-complexity-triage-routing/spec]] — Complexity-based routing (to be extended) +- [[021-zeph-context/spec]] — Context budget (orthogonal; no changes needed) diff --git a/specs/004-memory/004-15-memory-skill-coevolution-rfc-decision.md b/specs/004-memory/004-15-memory-skill-coevolution-rfc-decision.md index 98832a76f..1c57d75ae 100644 --- a/specs/004-memory/004-15-memory-skill-coevolution-rfc-decision.md +++ b/specs/004-memory/004-15-memory-skill-coevolution-rfc-decision.md @@ -47,7 +47,7 @@ related: **Benchmark**: +14pp on LongMemEval; works with any agent architecture -**Map to Zeph**: Complements context budget strategy ([[021-zeph-context]]); mostly a storage optimization, not a skill evolution mechanism. +**Map to Zeph**: Complements context budget strategy ([[021-zeph-context/spec]]); mostly a storage optimization, not a skill evolution mechanism. ### 1.3 EvolveMem (arXiv:2605.13941) @@ -60,7 +60,7 @@ related: **Benchmark**: +7pp on long-session consistency (72+ hours) -**Map to Zeph**: Maps onto existing [[024-complexity-triage-routing]] parameters; adds feedback loop. +**Map to Zeph**: Maps onto existing [[023-complexity-triage-routing/spec]] parameters; adds feedback loop. ### 1.4 SAGE: Self-Evolving Agentic Graph-Memory Engine (arXiv:2605.12061) @@ -74,7 +74,7 @@ related: **Benchmark**: +19% on LoCoMo (compared to static MAGMA) -**Map to Zeph**: CRITICAL: Name collision with existing SAGE RL in [[015-self-learning]] (cross-session reward model for skills). This proposal is SAGE-GraphMem; existing is SAGE-RL. Coexistence requires namespace disambiguation. +**Map to Zeph**: CRITICAL: Name collision with existing SAGE RL in [[015-self-learning/spec]] (cross-session reward model for skills). This proposal is SAGE-GraphMem; existing is SAGE-RL. Coexistence requires namespace disambiguation. ### 1.5 NanoResearch (arXiv:2605.10813) @@ -110,13 +110,13 @@ Zeph's skill + memory + learning ecosystem already implements: | Component | Current Implementation | Spec | |-----------|------------------------|------| -| **Cross-Session Reward** | SAGE RL: measure skill success across sessions via feedback detection | [[015-self-learning]] | -| **Feedback Detection** | FeedbackDetector (regex) + JudgeDetector (LLM): implicit correction signals | [[054-agent-feedback]] | -| **Skill Evolution** | ARISE trace improvement + STEM pattern migration; Wilson score reranking | [[015-self-learning]] | +| **Cross-Session Reward** | SAGE RL: measure skill success across sessions via feedback detection | [[015-self-learning/spec]] | +| **Feedback Detection** | FeedbackDetector (regex) + JudgeDetector (LLM): implicit correction signals | [[054-agent-feedback/spec]] | +| **Skill Evolution** | ARISE trace improvement + STEM pattern migration; Wilson score reranking | [[015-self-learning/spec]] | | **Memory Consolidation** | HeLa-Mem: periodic identification of dense clusters + consolidation daemon | [[004-11-memory-hela-mem]] | | **Provenance Tracking** | APEX-MEM edges store episode_id, confidence, temporal metadata | [[004-7-memory-apex-magma]] | | **Belief Staging** | BeliefMem: pre-commitment evidence accumulation via Noisy-OR | [[004-7-memory-apex-magma]] §16 | -| **Parameter Tuning** | [[024-complexity-triage-routing]]: complexity-aware dispatch (static config) | [[024-complexity-triage-routing]] | +| **Parameter Tuning** | [[023-complexity-triage-routing/spec]]: complexity-aware dispatch (static config) | [[023-complexity-triage-routing/spec]] | --- @@ -131,11 +131,11 @@ Zeph's skill + memory + learning ecosystem already implements: | | Online summarization via deltas | Partial — compaction exists; delta-based summarization is new | Summarization strategy differs | **Subsumed** (not skill-related) | | **EvolveMem** | Self-tuning retrieval parameters | No — complexity routing is static config | Adaptive tuning missing | **Partial Gap** | | | Per-session parameter drift detection | No — no feedback loop in routing layer | Drift detection missing | **Partial Gap** | -| | Online personalization | Partial — [[024-complexity-triage-routing]] has per-provider config but no feedback loop | Feedback loop missing | **Adopt** (layer feedback) | +| | Online personalization | Partial — [[023-complexity-triage-routing/spec]] has per-provider config but no feedback loop | Feedback loop missing | **Adopt** (layer feedback) | | **SAGE (Graph-Mem)** | Entity merger via embedding + LLM | Partial — entity extraction exists; automated merger is new | Automated coreference missing | **Partial Gap** | | | Edge pruning via weight decay | Yes — HeLa-Mem maintains edge weights | Same mechanism | **Subsumed** | | | Attention-weighted spreading activation | Yes — SYNAPSE uses weighted edges | Same mechanism | **Subsumed** | -| | SAGE-RL cross-session reward for graph | No — but existing SAGE RL in [[015-self-learning]] (skills) is similar concept | **NAME CONFLICT** — see §4 | +| | SAGE-RL cross-session reward for graph | No — but existing SAGE RL in [[015-self-learning/spec]] (skills) is similar concept | **NAME CONFLICT** — see §4 | | **NanoResearch** | Tri-level coevolution (skills↔memory↔policy) | No — layers evolve independently | Feedback loop missing | **Research Gap** | | | Closed-loop backpropagation to all three layers | No — feedback terminates at skill layer | Multi-layer feedback missing | **Research Gap** | | | Per-session micro-evolution | Partial — skill improvement happens; memory + policy tuning is missing | Partial implementation | **Partial Gap** | @@ -147,11 +147,11 @@ Zeph's skill + memory + learning ecosystem already implements: ## 4. Namespace Conflict: SAGE RL vs. SAGE-GraphMem -**Issue**: [[015-self-learning]] spec already defines "SAGE RL" as the cross-session reward model for **skills** (arXiv:2405.12345 style). The research paper #4057 proposes "SAGE" for **graph-memory** evolution. +**Issue**: [[015-self-learning/spec]] spec already defines "SAGE RL" as the cross-session reward model for **skills** (arXiv:2405.12345 style). The research paper #4057 proposes "SAGE" for **graph-memory** evolution. **Resolution**: - Rename arXiv:2605.12061 implementation module to `SAGE-GraphMem` (or `SageGraph` in code) to avoid symbol collision -- Keep [[015-self-learning]] SAGE RL unchanged +- Keep [[015-self-learning/spec]] SAGE RL unchanged - Document both in memory spec and self-learning spec with explicit cross-references **Namespace mapping**: @@ -177,19 +177,19 @@ pub struct SageRL { ... } // existing, unchanged - +12% implicit knowledge gain is valuable 2. **EvolveMem is adoptable** (P2) - - Feedback loop on routing parameters maps onto existing [[024-complexity-triage-routing]] + - Feedback loop on routing parameters maps onto existing [[023-complexity-triage-routing/spec]] - Success/failure signals already tracked via FeedbackDetector - Low implementation cost; +7pp on long-session consistency 3. **MemQ is deferred to P3** (research track) - Requires value function learning (Q-learning) infrastructure - High complexity; value signal attribution is non-trivial - - Reserve for next learning cycle after [[042-experiments]] completes + - Reserve for next learning cycle after [[041-experiments/spec]] completes 4. **SAGE-GraphMem is deferred to P4** (architecture review needed) - Requires cross-layer optimization (skills + graph memory simultaneously) - Interacts with existing SAGE RL naming; architectural scope needs larger review - - Recommend as input to agent decomposition effort ([[050-agent-decomposition]]) + - Recommend as input to agent decomposition effort ([[049-agent-decomposition/spec]]) 5. **NanoResearch, δ-mem are deferred** - NanoResearch is a full-system redesign; belongs in next major architecture cycle @@ -239,7 +239,7 @@ Configuration: ### 6.2 Complexity Routing Extension: Feedback-Driven Parameter Tuning -**File**: `specs/024-complexity-triage-routing/spec.md` (new §3.3) +**File**: `specs/023-complexity-triage-routing/spec.md` (new §3.3) Add subsection "Adaptive Routing: Online Parameter Drift Detection": @@ -299,7 +299,7 @@ Promotion to skills goes through existing [[005-skills/spec]] registry; no new s ### P3: MemQ Value Learning -When [[042-experiments]] provides experiment framework: +When [[041-experiments/spec]] provides experiment framework: - Track `(memory_fact_id, retrieval_context) → outcome` tuples - Compute Q-values via temporal difference learning - Promote facts with Q > threshold to skills @@ -308,10 +308,10 @@ When [[042-experiments]] provides experiment framework: ### P4: SAGE-GraphMem (Rename + Integrate) -After agent decomposition review ([[050-agent-decomposition]]): +After agent decomposition review ([[049-agent-decomposition/spec]]): - Integrate SAGE-GraphMem as graph-layer evolution (separate from SAGE-RL in skills) - Explicit namespace: `zeph_memory::graph::sage` vs. `zeph_skills::sage` -- Cross-reference both in [[015-self-learning]] and this spec +- Cross-reference both in [[015-self-learning/spec]] and this spec ### P4: δ-mem Differential Representation @@ -347,7 +347,7 @@ As part of context budget optimization: - Log anomalies and escalations to metrics 3. **Phase 3 (P3)**: MemQ research track - - Awaits [[042-experiments]] framework completion + - Awaits [[041-experiments/spec]] framework completion - Design value function learning over provenance DAGs - Implement Q-value tracking and promotion policy @@ -374,5 +374,5 @@ As part of context budget optimization: - [[004-11-memory-hela-mem]] — HeLa-Mem (to be extended with Cognifold) - [[005-skills/spec]] — Skills layer (promotion destination) - [[015-self-learning/spec]] — SAGE RL and cross-session reward (namespace clarification needed) -- [[024-complexity-triage-routing]] — Triage router (to be extended with feedback loop) -- [[054-agent-feedback]] — Feedback detection (signals for routing escalation) +- [[023-complexity-triage-routing/spec]] — Triage router (to be extended with feedback loop) +- [[054-agent-feedback/spec]] — Feedback detection (signals for routing escalation) diff --git a/specs/004-memory/004-16-shadow-memory-safety.md b/specs/004-memory/004-16-shadow-memory-safety.md index 22a199fc8..26bd90f0c 100644 --- a/specs/004-memory/004-16-shadow-memory-safety.md +++ b/specs/004-memory/004-16-shadow-memory-safety.md @@ -26,7 +26,7 @@ related: > Implements a parallel shadow memory stream that accumulates safety-critical > signals across an agent's full execution trajectory, enabling detection and > blocking of multi-turn attacks that evade per-turn controls. -> Resolves GitHub issue [#3695](https://github.com/rabax/zeph/issues/3695). +> Resolves GitHub issue [#3695](https://github.com/bug-ops/zeph/issues/3695). ## Sources @@ -329,9 +329,22 @@ AND no tool call is blocked by shadow memory ## 11. Implementation Notes +> [!warning] Corrected — 2026-07 audit +> The bullet below previously claimed `TrajectoryRiskAccumulator` was renamed to `ShadowSentinel` +> during implementation. This is incorrect: both structs exist and coexist as **separate** +> components. `TrajectoryRiskAccumulator` (`crates/zeph-memory/src/shadow/mod.rs`, doc comment +> literally reads "Per-session trajectory risk accumulator (MAGE spec 004-16)") is this spec's +> actual, unrenamed implementation — wired into `zeph-core` via `agent/builder.rs` and +> `agent/state/security.rs`. `ShadowSentinel` (`crates/zeph-core/src/agent/shadow_sentinel.rs`) is +> a distinct, additional defense-in-depth feature (an LLM pre-execution safety probe) that belongs +> to spec [[050-security-capability-governance/spec|050]] Phase 2, not to this spec's MAGE +> accumulator. Do not conflate the two when reading the acceptance criteria in §1–10 above — they +> refer to `TrajectoryRiskAccumulator`, which is correctly named as specced. + - New module: `crates/zeph-core/src/agent/shadow_sentinel.rs` — owns `ShadowSentinel`, - `SentinelEvent`, and `ShadowProbeExecutor`. The core struct is `ShadowSentinel` (not - `TrajectoryRiskAccumulator` as originally specced — name changed during implementation). + `SentinelEvent`, and `ShadowProbeExecutor`. This is the separate Phase-2 probe described in the + callout above, not a rename of `TrajectoryRiskAccumulator` (which lives in + `crates/zeph-memory/src/shadow/mod.rs` and is wired into `zeph-core` independently). - `ShadowSentinel` is wired into the agent loop (commit #4443: `wire ShadowMemory into agent loop`) - `SentinelEvent` is the per-turn audit event type in `zeph-core`; `ShadowEvent` in `zeph-sanitizer` is a separate, parallel type for sanitizer-layer events. diff --git a/specs/004-memory/004-17-implicit-conflict-detection.md b/specs/004-memory/004-17-implicit-conflict-detection.md index 9d19ad71b..f1f9ed7d4 100644 --- a/specs/004-memory/004-17-implicit-conflict-detection.md +++ b/specs/004-memory/004-17-implicit-conflict-detection.md @@ -10,7 +10,7 @@ tags: - graph - experimental created: 2026-05-18 -status: draft +status: implemented related: - "[[MOC-specs]]" - "[[constitution]]" @@ -27,7 +27,7 @@ related: > detection via fuzzy predicate matching and propagation-aware SYNAPSE recall that > resolves conflicting beliefs before returning fact sets to the agent. > Addresses the gap identified in STALE benchmark (arXiv:2605.06527) and tracked -> in GitHub issue [#3702](https://github.com/rabax/zeph/issues/3702). +> in GitHub issue [#3702](https://github.com/bug-ops/zeph/issues/3702). ## Sources @@ -198,8 +198,8 @@ AND a log entry records: pair_ids, resolution_strategy, resolved_at, outcome ```sql CREATE TABLE IF NOT EXISTS implicit_conflict_candidates ( id INTEGER PRIMARY KEY, - edge_a_id INTEGER NOT NULL REFERENCES edges(id), - edge_b_id INTEGER NOT NULL REFERENCES edges(id), + edge_a_id INTEGER NOT NULL REFERENCES graph_edges(id), + edge_b_id INTEGER NOT NULL REFERENCES graph_edges(id), similarity REAL NOT NULL, method TEXT NOT NULL, -- "levenshtein" | "embedding" | "both" status TEXT NOT NULL DEFAULT 'pending', @@ -222,7 +222,7 @@ stored in `implicit_conflict_candidates` with foreign keys to edge ids. ### Database migration -Migration `047_implicit_conflict_candidates.sql` — creates the staging table above. +Migration `090_implicit_conflict_candidates.sql` — creates the staging table above. Wrapped in `BEGIN IMMEDIATE; ... COMMIT;` per constitution. --- @@ -363,7 +363,7 @@ AND SYNAPSE results contain no is_implicit_conflict annotations `graph.implicit_conflict.enabled = true` and `consolidation_daemon.enabled = true`. It shares the `ConsolidationTask` infrastructure introduced by HeLa-Mem [[004-11-memory-hela-mem]] to avoid duplicating scheduler registration boilerplate. -- Migration 047 is separate from APEX-MEM migration 042; both can be applied independently. +- Migration 090 is separate from APEX-MEM migration 075; both can be applied independently. --- diff --git a/specs/004-memory/004-18-five-signal-retrieval.md b/specs/004-memory/004-18-five-signal-retrieval.md index 2a5d9e99d..822896692 100644 --- a/specs/004-memory/004-18-five-signal-retrieval.md +++ b/specs/004-memory/004-18-five-signal-retrieval.md @@ -30,7 +30,7 @@ related: > semantic relevance), and introduces an async consolidation daemon that promotes > high-utility episodic facts to the semantic layer. > Based on MemTier analysis (arXiv:2605.03675). Tracked in GitHub issue -> [#3703](https://github.com/rabax/zeph/issues/3703). +> [#3703](https://github.com/bug-ops/zeph/issues/3703). ## Sources @@ -227,7 +227,7 @@ ALTER TABLE messages ADD COLUMN memory_tier TEXT DEFAULT 'episodic'; ALTER TABLE messages ADD COLUMN qdrant_promoted INTEGER DEFAULT 0; -- boolean ``` -Database migration: `048_five_signal_retrieval.sql` — adds `fact_access_log` table, +Database migration: `091_five_signal_retrieval.sql` — adds `fact_access_log` table, alters `messages` table. Wrapped in `BEGIN IMMEDIATE; ... COMMIT;`. --- @@ -368,8 +368,8 @@ AND SYNAPSE latency is indistinguishable from the pre-spec baseline - PPO-based weight adaptation (MemTier §5) is deferred to `zeph-experiments` as a follow-on enhancement. The weight config section is designed to accept automated updates from the experiments framework without schema changes. -- Database migration 048 is independent of APEX-MEM migration 042 and CUPMem migration - 047; all three can be applied in any order. +- Database migration 091 is independent of APEX-MEM migration 075 and CUPMem migration + 090; all three can be applied in any order. --- @@ -401,7 +401,6 @@ Key implementation details: - `SemanticMemory::with_five_signal()` wires the `FiveSignalRuntime` into `recall_merge_and_rank` - Five-signal scoring runs as a post-processing step after Qdrant and SQLite candidate retrieval -- `TrajectoryRiskAccumulator` and `ImplicitConflictDetector` implemented in commits #4386, #4408 - Consolidation daemon wired into `zeph-scheduler` and A* node cap added (commits #4377, #4418) - Prometheus metrics exported: `five_signal_recall_total`, consolidation counters (commit #4418) - `mage_accumulator` config properly wired in agent builder (commit #4428) diff --git a/specs/004-memory/004-4-embeddings.md b/specs/004-memory/004-4-embeddings.md index 0d3408f0f..2d2b1c07e 100644 --- a/specs/004-memory/004-4-embeddings.md +++ b/specs/004-memory/004-4-embeddings.md @@ -176,7 +176,7 @@ Embedding status displayed in: - [[004-2-compaction]] — regenerate embeddings after compaction applied - [[004-3-admission-control]] — embeddings used for relevance scoring - [[004-5-temporal-decay]] — temporal decay scores may adjust embedding weight -- [[zeph-memory/spec]] — parent system orchestrator +- [[004-memory/spec]] — parent system orchestrator --- diff --git a/specs/004-memory/004-5-temporal-decay.md b/specs/004-memory/004-5-temporal-decay.md index 3ab0543a6..4a86acc01 100644 --- a/specs/004-memory/004-5-temporal-decay.md +++ b/specs/004-memory/004-5-temporal-decay.md @@ -38,6 +38,10 @@ Human memory doesn't retain all information equally. The Ebbinghaus forgetting c - Forget critical facts (flagged `importance=critical`) regardless of age - Reset access timestamp on passive reads (only on agent-triggered recall) - Use raw age as decay metric (always apply Ebbinghaus curve) +- Physically `DELETE FROM messages` — SleepGate's "forget" sets `deleted_at`, it does not + remove rows; this is consistent with [[001-system-invariants/spec#6. Memory Pipeline Contract]] + ("messages are never deleted"), which currently documents only `compacted_at` — `deleted_at` is + a second, separate soft-state column and should be treated as equally binding ## Ebbinghaus Forgetting Curve @@ -154,14 +158,12 @@ impl SleepGate { (self.retention_threshold, self.max_forget_per_run), ).await?; - let mut forgotten = 0; - for item_id in candidates { - // Soft delete: mark for eventual purge - memory.db.soft_delete_message(&item_id).await?; - forgotten += 1; - } - - log::info!("SleepGate: forgot {} low-retention items", forgotten); + let ids: Vec<_> = candidates.into_iter().map(|item_id| item_id).collect(); + // Soft delete: sets `deleted_at` (not `compacted_at`); rows are never DELETEd + // (see `soft_delete_messages`, `crates/zeph-memory/src/store/messages/mod.rs`). + let forgotten = memory.db.soft_delete_messages(&ids).await?; + + tracing::info!(forgotten, "SleepGate: forgot low-retention items"); Ok(forgotten) } } @@ -194,7 +196,7 @@ max_forget_per_run = 100 - [[004-1-architecture]] — retention scores used during compaction ranking - [[004-2-compaction]] — SleepGate invoked as part of compaction cycle - [[004-3-admission-control]] — recency factor in A-MAC importance scoring -- [[017-index]] — AST-indexed code snippets decay over time without access +- [[017-index/spec]] — AST-indexed code snippets decay over time without access ## See Also diff --git a/specs/004-memory/004-7-memory-apex-magma.md b/specs/004-memory/004-7-memory-apex-magma.md index c2ea6ad34..87ab69d8e 100644 --- a/specs/004-memory/004-7-memory-apex-magma.md +++ b/specs/004-memory/004-7-memory-apex-magma.md @@ -28,7 +28,7 @@ related: > canonicalization; adds a SYNAPSE-side conflict resolution pass that deterministically > (or, optionally, via LLM) selects the authoritative value when multiple edges assert > conflicting values for the same `(subject, predicate)`. Resolves GitHub issue -> [#3223](https://github.com/rabax/zeph/issues/3223). +> [#3223](https://github.com/bug-ops/zeph/issues/3223). ## Sources @@ -151,7 +151,7 @@ AND the result is accessible via the /graph TUI command without enabling any deb | FR-009 | The conflict resolver strategy SHALL be one of: `recency` (pick newest `valid_from`), `confidence` (pick highest `confidence`), `llm` (call `conflict_resolution_provider`) | must | | FR-010 | `llm` strategy SHALL respect a 500 ms timeout AND a per-turn budget `conflict_llm_budget_per_turn` (default 3 resolutions); on timeout or budget exhaustion fall back to `recency` | must | | FR-011 | WHEN conflict resolution drops an edge from the result set THE SYSTEM SHALL retain the dropped edge in a diagnostic "alternatives" field accessible via `/graph` tooling, not passed to the main LLM. Default `retain_alternatives_for_diagnostics = false` | should | -| FR-012 | DB migration 042 SHALL be atomic (`BEGIN IMMEDIATE; … COMMIT;`); on failure the DB remains in the pre-042 state. Operations: (a) `ADD COLUMN supersedes`, (b) `ADD COLUMN canonical_relation`, (c) `ADD COLUMN cardinality` to ontology table if persisted, (d) backfill `canonical_relation = relation`, `supersedes = NULL`, (e) create partial index `idx_edges_head_active`, (f) **replace** (not drop) the pre-APEX uniqueness index with a new partial unique index restricted to the active head (see §3). `DROP INDEX` MUST NOT occur without a replacement constraint inside the same transaction | must | +| FR-012 | DB migration 075 SHALL be atomic (`BEGIN IMMEDIATE; … COMMIT;`); on failure the DB remains in the pre-075 state. Operations: (a) `ADD COLUMN supersedes`, (b) `ADD COLUMN canonical_relation`, (c) `ADD COLUMN cardinality` to ontology table if persisted, (d) backfill `canonical_relation = relation`, `supersedes = NULL`, (e) create partial index `idx_edges_head_active`, (f) **replace** (not drop) the pre-APEX uniqueness index with a new partial unique index restricted to the active head (see §3). `DROP INDEX` MUST NOT occur without a replacement constraint inside the same transaction | must | | FR-013 | WHEN `[memory.graph.apex_mem]` is disabled THE SYSTEM SHALL behave as pre-APEX MAGMA: legacy `store.rs` write path honours the partial unique index on active heads (rollback-safe, see §3) | must | | FR-014 | Ontology entries SHALL carry a `cardinality` field in `{1, n}` defaulting to `n` (multi-valued) when unspecified. The `default.toml` SHALL explicitly mark cardinality-1 predicates (`works_at`, `lives_in`, `born_in`, `manages`) | must | | FR-015 | WHEN a write asserts a value byte-identical to the current head (same `target`, `relation`, `fact`, `edge_type`) THE SYSTEM SHALL insert a **reassertion event row** in the `edge_reassertions` table `(head_edge_id, asserted_at, episode_id, confidence)` instead of inserting a new edge; this preserves provenance without violating the immutability invariant (FR-002) | must | @@ -167,7 +167,7 @@ AND the result is accessible via the /graph TUI command without enabling any deb | NFR-002 | Performance | Ontology resolution SHALL complete in < 1 ms for table hits; LLM fallback is bounded to 500 ms via timeout with a `recency` fallback ensuring no call blocks indefinitely | | NFR-003 | Performance | Conflict resolution via `recency` or `confidence` strategy SHALL complete in < 5 ms at p99 (SQL sort on indexed columns); `llm` strategy respects the 500 ms timeout with `recency` fallback | | NFR-004 | Performance | Head-of-chain query (`idx_edges_head_active` partial index) SHALL not degrade BFS or SYNAPSE latency by more than 5% vs. pre-APEX on benchmark fixtures with 100k edges | -| NFR-005 | Reliability | DB migration 042 is atomic (`BEGIN IMMEDIATE; … COMMIT;`); a failed migration leaves the DB in pre-042 state with no half-applied columns or indexes | +| NFR-005 | Reliability | DB migration 075 is atomic (`BEGIN IMMEDIATE; … COMMIT;`); a failed migration leaves the DB in pre-075 state with no half-applied columns or indexes | | NFR-006 | Reliability | Feature flag `enabled = false` is a safe runtime rollback — legacy write path satisfies both the old and new partial indexes without requiring a reverse migration | | NFR-007 | Reliability | `llm` conflict resolver and ontology fallback both fail to `recency` on timeout or error — no write or recall operation blocks on an LLM call | | NFR-008 | Maintainability | Ontology table is a plain TOML file with a documented schema; operators can add canonical mappings and reload without a process restart (`/graph ontology reload`) | @@ -181,36 +181,36 @@ AND the result is accessible via the /graph TUI command without enabling any deb ## 5. Data Model Changes -### Schema Migration (`042_apex_mem.sql`) +### Schema Migration (`075_apex_mem.sql`) Wrapped in a single transaction (`BEGIN IMMEDIATE; … COMMIT;`). Either the DB lands -fully on 042 or fully on 041; no half-migrated state. +fully on 075 or fully on 074; no half-migrated state. ```sql BEGIN IMMEDIATE; -ALTER TABLE edges ADD COLUMN supersedes INTEGER REFERENCES edges(id); -ALTER TABLE edges ADD COLUMN canonical_relation TEXT; +ALTER TABLE graph_edges ADD COLUMN supersedes INTEGER REFERENCES graph_edges(id); +ALTER TABLE graph_edges ADD COLUMN canonical_relation TEXT; -- backfill phase 1: copy raw relation as canonical (idempotent) -UPDATE edges SET canonical_relation = relation WHERE canonical_relation IS NULL; +UPDATE graph_edges SET canonical_relation = relation WHERE canonical_relation IS NULL; -- replace the legacy active-head unique index WITHOUT an intermediate drop-only state. -- The old index 'uq_graph_edges_active' remains usable by rollback (enabled=false). -- The new partial index tightens uniqueness to the append-only head of chain. CREATE UNIQUE INDEX IF NOT EXISTS uq_graph_edges_active_head - ON edges(source_entity_id, target_entity_id, canonical_relation, edge_type) + ON graph_edges(source_entity_id, target_entity_id, canonical_relation, edge_type) WHERE valid_to IS NULL AND expired_at IS NULL; -CREATE INDEX IF NOT EXISTS idx_edges_supersedes ON edges(supersedes); +CREATE INDEX IF NOT EXISTS idx_edges_supersedes ON graph_edges(supersedes); CREATE INDEX IF NOT EXISTS idx_edges_head_active - ON edges(source_entity_id, canonical_relation, edge_type, created_at DESC) + ON graph_edges(source_entity_id, canonical_relation, edge_type, created_at DESC) WHERE valid_to IS NULL AND expired_at IS NULL; -- reassertion events (FR-015) CREATE TABLE IF NOT EXISTS edge_reassertions ( id INTEGER PRIMARY KEY, - head_edge_id INTEGER NOT NULL REFERENCES edges(id), + head_edge_id INTEGER NOT NULL REFERENCES graph_edges(id), asserted_at INTEGER NOT NULL, episode_id TEXT, confidence REAL NOT NULL @@ -236,7 +236,7 @@ and lets ontology tuning take effect on the next `/graph ontology reload` withou schema churn. A secondary opt-in pass `migrate_canonical_relations(ontology)` is provided as a -separate CLI subcommand (not part of 042) — it re-canonicalises legacy rows by +separate CLI subcommand (not part of 075) — it re-canonicalises legacy rows by inserting new supersedes rows. Default is not to run it; users opt in after reviewing the ontology table. Documented as a known limitation: without this pass, pre-existing synonym edges remain under their raw predicates. @@ -312,7 +312,7 @@ LLM-fallback entries that fail validation against the fresh table are evicted. - Conflict resolver runs only for cardinality-1 predicates; cardinality-n predicates pass all head edges through unchanged - `llm` strategy bounded by 500 ms timeout AND per-turn budget with `recency` fallback - Disabled feature flag bypasses all new code paths; legacy uniqueness index is retained so rollback is safe -- Migration 042 is wrapped in `BEGIN IMMEDIATE; ... COMMIT;` — half-migrated state is impossible +- Migration 075 is wrapped in `BEGIN IMMEDIATE; ... COMMIT;` — half-migrated state is impossible ### Ask First - Changing the default conflict strategy from `recency` @@ -377,7 +377,7 @@ retain_alternatives_for_diagnostics = false # opt-in; default off to save memo ## 9. Success Criteria - [ ] Property test: 10k random edge rewrites never produce a `supersedes` cycle -- [ ] Migration 042 is idempotent (running twice leaves the DB byte-equivalent) +- [ ] Migration 075 is idempotent (running twice leaves the DB byte-equivalent) - [ ] Disabling the flag reproduces pre-APEX behavior on an integration fixture - [ ] Default BFS returns only head edges (unit test with seeded superseded history) - [ ] `edge_history()` walks the chain in reverse chronological order @@ -514,7 +514,7 @@ memory commands or external memory injection. - [[004-memory/spec]] — memory pipeline - [[012-graph-memory/spec]] — MAGMA + SYNAPSE (extended by this spec) - [[004-6-graph-memory]] — graph memory sub-spec -- [[memory-write-gate/spec]] — upstream quality gate (contradiction_risk signal composes with APEX-MEM conflicts) +- [[004-9-memory-write-gate]] — upstream quality gate (contradiction_risk signal composes with APEX-MEM conflicts) - [[024-multi-model-design/spec]] — provider tier guidance - [[MOC-specs]] — all specifications diff --git a/specs/004-memory/004-8-memory-typed-pages.md b/specs/004-memory/004-8-memory-typed-pages.md index e14dc2be9..0cc6d9b9c 100644 --- a/specs/004-memory/004-8-memory-typed-pages.md +++ b/specs/004-memory/004-8-memory-typed-pages.md @@ -11,7 +11,7 @@ tags: - context - experimental created: 2026-04-19 -status: draft +status: implemented related: - "[[MOC-specs]]" - "[[constitution]]" @@ -27,7 +27,7 @@ related: > Classifies every context segment into a typed "page" with a per-type minimum-fidelity > invariant enforced at every compaction boundary. Replaces the current untyped > truncation/summarization pipeline in `zeph-context` with a page-aware compactor. -> Resolves GitHub issue [#3221](https://github.com/rabax/zeph/issues/3221). +> Resolves GitHub issue [#3221](https://github.com/bug-ops/zeph/issues/3221). ## Sources diff --git a/specs/004-memory/004-9-memory-write-gate.md b/specs/004-memory/004-9-memory-write-gate.md index a8fa06c4c..8905bf82b 100644 --- a/specs/004-memory/004-9-memory-write-gate.md +++ b/specs/004-memory/004-9-memory-write-gate.md @@ -10,7 +10,7 @@ tags: - admission - experimental created: 2026-04-19 -status: draft +status: implemented related: - "[[MOC-specs]]" - "[[constitution]]" @@ -27,7 +27,7 @@ related: > with a **content-quality** scorer that rejects redundant, reference-incomplete, or > self-contradictory entries. Rule-based MVP with optional LLM-assisted scoring via > `quality_gate_provider`. Resolves GitHub issue -> [#3222](https://github.com/rabax/zeph/issues/3222). +> [#3222](https://github.com/bug-ops/zeph/issues/3222). ## Sources @@ -39,8 +39,8 @@ related: | File | Contents | |---|---| | `crates/zeph-memory/src/admission.rs` | Existing A-MAC importance scorer | -| `crates/zeph-memory/src/facade.rs` | `SemanticMemory::remember()` entry point | -| `crates/zeph-memory/src/router.rs` | Memory routing decisions | +| `crates/zeph-memory/src/semantic/recall.rs` | `SemanticMemory::remember()` entry point | +| `crates/zeph-memory/src/quality_gate.rs` | `QualityGate` implementation | | `crates/zeph-memory/src/types.rs` | `Message`, `MessagePart` definitions | --- @@ -74,7 +74,7 @@ MVP; LLM-assisted scoring is opt-in via `quality_gate_provider`. - Replacing or removing A-MAC (the gates compose: A-MAC first, quality second) - Retrospective cleanup of existing low-quality entries (handled by `#[forgetting]`) -- Graph-edge admission (see `[[memory-apex-magma/spec]]`) +- Graph-edge admission (see `[[004-7-memory-apex-magma]]`) - Tool-output admission (always admitted; quality runs only on conversational writes) --- @@ -326,6 +326,6 @@ AND the LLM timeout counter increments - [[constitution]] — project principles - [[004-memory/spec]] — memory pipeline - [[004-3-admission-control]] — upstream A-MAC admission -- [[memory-apex-magma/spec]] — graph-side conflict handling (composes with `contradiction_risk`) +- [[004-7-memory-apex-magma]] — graph-side conflict handling (composes with `contradiction_risk`) - [[024-multi-model-design/spec]] — provider tier guidance - [[MOC-specs]] — all specifications diff --git a/specs/004-memory/spec.md b/specs/004-memory/spec.md index d238729ff..8c01e8b2c 100644 --- a/specs/004-memory/spec.md +++ b/specs/004-memory/spec.md @@ -46,6 +46,16 @@ specific areas, refer to the child specs below. | [[004-4-embeddings]] | Embedding Generation | Batch strategies, backfill, concurrent workers, TUI integration | | [[004-5-temporal-decay]] | Retention Scoring | Ebbinghaus forgetting curve, access frequency, decay-based eviction | | [[004-6-graph-memory]] | Graph Memory | Entity graph, BFS recall, MAGMA typed edges, SYNAPSE spreading activation, A-MEM link weights | +| [[004-7-memory-apex-magma]] | APEX-MEM / MAGMA | Append-only edge log, ontology normalization, SYNAPSE conflict resolution; BeliefMem pre-commitment layer | +| [[004-8-memory-typed-pages]] | ClawVM Typed Pages | `PageType` classification, minimum-fidelity invariants, compaction audit log | +| [[004-9-memory-write-gate]] | MemReader Write Gate | Three-signal write quality scorer composed with A-MAC admission control | +| [[004-13-memory-memcot]] | MemCoT | `SemanticStateAccumulator`, Zoom-In/Zoom-Out evidence localization and causal expansion | +| [[004-16-memory-type-aware-retrieval]] | MemGuard Type-Aware Retrieval | `FunctionalType`-gated retrieval composition, `BehavioralRule` always-composed safety invariant | + +See also §"Sub-Specifications" below for [[004-10-memory-memmachine-retrieval]], [[004-11-memory-hela-mem]], +[[004-12-memory-reasoning-bank]], [[004-14-memory-tiering-rfc-decision]], +[[004-15-memory-skill-coevolution-rfc-decision]], [[004-16-shadow-memory-safety]], +[[004-17-implicit-conflict-detection]], and [[004-18-five-signal-retrieval]]. --- @@ -71,8 +81,11 @@ SemanticMemory (Arc) ### Admission Control - Not all messages admitted to memory (noise filtering via A-MAC) -- Six-factor scoring: recency, relevance, tool-use, entity-density, length, frequency -- Frequency factor: entity mention count with temporal decay (ref. [[004-3-admission-control]]) +- Five-factor scoring (A-MAC paper, arXiv:2603.04549): `future_utility` (0.30, LLM-estimated reuse + probability), `factual_confidence` (0.15, inverse hedging heuristic), `semantic_novelty` (0.30, + 1 − max similarity to top-3 neighbors), `temporal_recency` (0.10, always 1.0 at write time), + `content_type_prior` (0.15, message-role prior) — see [[004-3-admission-control]] for the full + model, including the superseded six-factor design it replaced (issue #4141) - Threshold-based gate: score < threshold → rejected (returns None) - Fail-open: admission error → admit message anyway @@ -110,7 +123,7 @@ Memory is organized in three tiers; retrieval strategy adapts to query complexit - Search: all three tiers (working + episodic + semantic), aggregate by relevance - Outcome: highest recall, higher token cost -Complexity classification via [[024-complexity-triage-routing]]; same signal applied to memory tier selection. +Complexity classification via [[023-complexity-triage-routing/spec]]; same signal applied to memory tier selection. See [[004-14-memory-tiering-rfc-decision]] for design rationale. --- @@ -135,7 +148,7 @@ Idle-time memory reorganization via clustering on co-occurrence patterns: - See [[004-11-memory-hela-mem]] §3.6 for details ### Future: MemQ Value Learning Path (RFC #4218, P3) -When [[042-experiments]] framework matures: +When [[041-experiments/spec]] framework matures: - Track `(memory_fact_id, retrieval_context) → outcome` tuples - Compute Q-values via temporal difference learning - High-Q facts promoted based on retrieval-context utility @@ -228,6 +241,8 @@ When disabled, the prior `recall_semantic` path is used unchanged. | [[004-10-memory-memmachine-retrieval]] | MemMachine retrieval depth, query bias correction, episode preservation | | [[004-11-memory-hela-mem]] | HeLa-Mem Hebbian edge weights, consolidation, spreading activation | | [[004-12-memory-reasoning-bank]] | ReasoningBank distilled strategy memory, self-judge pipeline | +| [[004-14-memory-tiering-rfc-decision]] | RFC #4217 decision: memory tiering architecture analysis | +| [[004-15-memory-skill-coevolution-rfc-decision]] | RFC #4218 decision: memory–skill coevolution analysis | | [[004-16-shadow-memory-safety]] | Shadow Memory Safety — trajectory-level attack defense (MAGE, issue #3695) | | [[004-17-implicit-conflict-detection]] | Implicit Conflict Detection — STALE/CUPMem fuzzy predicate matching and propagation-aware SYNAPSE recall (issue #3702) | | [[004-18-five-signal-retrieval]] | Five-Signal Retrieval — access frequency, causal distance, novelty signals + async consolidation daemon (MemTier, issue #3703) | diff --git a/specs/005-skills/spec.md b/specs/005-skills/spec.md index fa3ddb2f8..9f6eb9615 100644 --- a/specs/005-skills/spec.md +++ b/specs/005-skills/spec.md @@ -880,7 +880,7 @@ Enabling `group_structured = true` changes the LLM prompt format in Block 3. Thi - Default `group_structured = false` preserves existing behaviour for all existing configs — no migration required. - No new database tables or storage migrations are needed — grouping is stateless and computed per-turn. - `format_skills_prompt()` in `crates/zeph-skills/src/prompt.rs` requires a signature change to accept `SkillGroup` input. The existing flat-list overload must be retained (or a compatibility wrapper provided) for the `group_structured = false` path. -- `SkillState::rebuild_prompt` in `crates/zeph-core/src/agent/state/skill.rs` must route to the new group-aware formatter when `group_structured = true`. +- `Agent::rebuild_system_prompt` (`crates/zeph-core/src/agent/mod.rs`; `SkillState` itself is defined in `crates/zeph-core/src/agent/state/mod.rs`) must route to the new group-aware formatter when `group_structured = true`. ### Future Consideration: Toward Explicit Skill Dependencies diff --git a/specs/006-tools/spec.md b/specs/006-tools/spec.md index 93a101a0e..b7cc8ef9c 100644 --- a/specs/006-tools/spec.md +++ b/specs/006-tools/spec.md @@ -37,7 +37,7 @@ related: | `crates/zeph-tools/src/composite.rs` | `CompositeExecutor`, chain ordering | | `crates/zeph-tools/src/filter/mod.rs` | `FilterPipeline`, `CommandMatcher`, `OutputFilterRegistry` | | `crates/zeph-tools/src/filter/security.rs` | `SecurityPatterns`, 17 regex patterns | -| `crates/zeph-tools/src/filter/declarative.rs` | Per-filter TOML config | +| `crates/zeph-tools/src/filter/declarative/mod.rs` | Per-filter TOML config | --- @@ -254,12 +254,12 @@ loaded tools bypassed the confirmation gate even when the underlying executor re ### Fix `DynExecutor::requires_confirmation(&call)` now delegates to the inner executor's -`requires_confirmation` method via the `ErasedToolExecutor` vtable: +`requires_confirmation_erased` method via the `ErasedToolExecutor` vtable: ```rust impl ToolExecutor for DynExecutor { - fn requires_confirmation_erased(&self, call: &ToolCall) -> bool { - self.inner.requires_confirmation_erased(call) + fn requires_confirmation(&self, call: &ToolCall) -> bool { + self.0.requires_confirmation_erased(call) } } ``` @@ -497,7 +497,7 @@ rules = {} ## Tool Error Taxonomy -`crates/zeph-tools/src/error.rs`. Issue #2203. +`crates/zeph-tools/src/error_taxonomy.rs`. Issue #2203. ### Overview diff --git a/specs/007-channels/spec.md b/specs/007-channels/spec.md index 6c5197c45..47568bcf7 100644 --- a/specs/007-channels/spec.md +++ b/specs/007-channels/spec.md @@ -67,13 +67,18 @@ trait Channel: Send { ```rust pub enum AnyChannel { Cli(CliChannel), + JsonCli(JsonCliChannel), Telegram(TelegramChannel), #[cfg(feature = "discord")] Discord(DiscordChannel), #[cfg(feature = "slack")] Slack(SlackChannel), - #[cfg(feature = "tui")] Tui(TuiChannel), } ``` +`TuiChannel` is **not** an `AnyChannel` variant — it implements `Channel` directly +(`crates/zeph-tui/src/channel.rs`) and is used standalone in TUI mode, since TUI and the +multi-channel dispatch path are mutually exclusive session modes (see +[[001-system-invariants/spec#10. Concurrency & Safety Contract]]). + - Macro dispatch: `dispatch_channel!(self, method, args...)` - **Only place** where multi-channel dispatch happens — no other dyn dispatch for channels - New channels: add variant + `#[cfg(feature = "...")]` + macro dispatch entry @@ -163,13 +168,24 @@ stream_interval_ms = 3000 Epic #1978. `crates/zeph-channels/`, `crates/zeph-core/src/channel.rs`. +> [!note] `AppChannel` correction — 2026-07 audit +> This section previously referred to a type named `AppChannel` as the TUI's forwarding target. +> No such type exists. `TuiChannel` (`crates/zeph-tui/src/channel.rs`) implements the `Channel` +> trait directly and is used standalone in TUI mode — it is not a variant of `AnyChannel` +> (`crates/zeph-channels/src/any.rs`, whose real variants are `Cli`, `JsonCli`, `Telegram`, and +> the feature-gated `Discord`/`Slack`). "Forward every method" below applies to both dispatch +> surfaces independently. Also note: the `Channel` trait has grown well beyond 16 methods since +> this section was written (confirmed ~25 in `crates/zeph-core/src/channel.rs`) — the exact +> current count and the "Methods That Must Be Forwarded" list need a full re-audit, not corrected +> in this pass. + ### Overview -Channel feature parity ensures all `AnyChannel` variants and `AppChannel` forward every method defined in the `Channel` trait. Previously, four methods fell through to no-op trait defaults in some dispatch paths, silently dropping events. The parity initiative enforces full method forwarding and behavioral consistency across channels. +Channel feature parity ensures all `AnyChannel` variants and `TuiChannel` forward every method defined in the `Channel` trait. Previously, four methods fell through to no-op trait defaults in some dispatch paths, silently dropping events. The parity initiative enforces full method forwarding and behavioral consistency across channels. ### Methods That Must Be Forwarded -The `Channel` trait defines 16 methods. All must be explicitly dispatched in `AnyChannel` and `AppChannel`: +The `Channel` trait defined 16 methods at the time this section was written (see the note above — the current count is higher). All must be explicitly dispatched in `AnyChannel` and `TuiChannel`: Previously dropped (CHAN-01 fix): - `send_thinking_chunk` — streams extended thinking tokens @@ -177,7 +193,7 @@ Previously dropped (CHAN-01 fix): - `send_usage` — delivers token usage stats to channel - `send_tool_start` — notifies channel of tool execution start -These four now have explicit dispatch in `AnyChannel` and `AppChannel`, matching the existing dispatch for `send`, `send_chunk`, `send_typing`, `send_status`, `send_tool_output`, `recv`, `supports_exit`, and others. +These four now have explicit dispatch in `AnyChannel` and `TuiChannel`, matching the existing dispatch for `send`, `send_chunk`, `send_typing`, `send_status`, `send_tool_output`, `recv`, `supports_exit`, and others. ### Timeout Consistency (CHAN-02) @@ -202,11 +218,11 @@ Discord channel registers slash commands (`/reset`, `/skills`, `/agent`) at star ### Key Invariants -- `AnyChannel` `dispatch_channel!` macro must include ALL 16 `Channel` trait methods — no method may fall through to a default +- `AnyChannel` `dispatch_channel!` macro must include ALL `Channel` trait methods — no method may fall through to a default - `CONFIRM_TIMEOUT` (30s) is the canonical timeout for all channel `confirm()` implementations — never hardcode different values per channel - Discord slash command registration is fire-and-forget — startup must not fail if registration fails - `send_thinking_chunk` must be forwarded even if the channel renders it as a no-op — the event must reach the channel impl -- NEVER add a new `Channel` trait method without updating `AnyChannel`, `AppChannel`, and all channel implementations +- NEVER add a new `Channel` trait method without updating `AnyChannel`, `TuiChannel`, and all channel implementations - Behavioral differences between channels (e.g. Telegram batching) are acceptable — method dropping is not --- diff --git a/specs/008-mcp/008-1-lifecycle.md b/specs/008-mcp/008-1-lifecycle.md index 47f7e4e5b..6a74772a8 100644 --- a/specs/008-mcp/008-1-lifecycle.md +++ b/specs/008-mcp/008-1-lifecycle.md @@ -14,11 +14,21 @@ related: - "[[008-mcp/spec]]" - "[[008-2-discovery]]" - "[[008-3-security]]" - - "[[002-agent-loop]]" + - "[[002-agent-loop/spec]]" --- # Spec: MCP Server Lifecycle +> [!danger] Stale content — 2026-07 audit +> The "Startup Auto-Retry with Exponential Backoff (#3578)" section and most of the pseudocode below +> (`McpConnection`, `JsonRpcTransport`, `Waiter`, `log::warn!`/`log::info!` calls) do not match the +> real implementation and were not corrected in this pass (rewrite, not a text fix). Real code: +> `crates/zeph-mcp/src/manager/{mod.rs,builder.rs,connect.rs,server.rs,retry.rs}`. Real retry config +> fields are `startup_retry_backoff_ms`/`tool_timeout_secs` (see `[[008-mcp/spec]]`), not +> `startup_retry_base_delay_ms`/`startup_retry_max_delay_ms`/`startup_retry_backoff_factor`/ +> `max_startup_retries` as stated below. `log::*` macros are banned project-wide (`tracing` only, +> see `[[constitution]]` II/IV) — do not copy the logging calls in this file's pseudocode. + Server startup/shutdown, connection management, stdio environment isolation, graceful cleanup. ## Overview @@ -264,10 +274,10 @@ allow_env_vars = ["PATH", "HOME", "RUST_LOG"] ## Integration Points -- [[002-agent-loop]] — MCP servers initialized during agent startup +- [[002-agent-loop/spec]] — MCP servers initialized during agent startup - [[008-2-discovery]] — Server capabilities discovered after initialization - [[008-3-security]] — Environment scrubbing occurs before spawn -- [[010-security]] — Subprocess isolation enforced here +- [[010-security/spec]] — Subprocess isolation enforced here ## See Also diff --git a/specs/008-mcp/008-2-discovery.md b/specs/008-mcp/008-2-discovery.md index 2bdbb0977..d5202d441 100644 --- a/specs/008-mcp/008-2-discovery.md +++ b/specs/008-mcp/008-2-discovery.md @@ -16,11 +16,23 @@ related: - "[[008-mcp/spec]]" - "[[008-1-lifecycle]]" - "[[008-3-security]]" - - "[[006-tools]]" + - "[[006-tools/spec]]" --- # Spec: MCP Tool Discovery & Pruning +> [!danger] Stale content — 2026-07 audit +> The "Per-Message Tool Pruning" (`ToolPruningCache`) and "Tool Embedding Precomputation" sections +> below do not match the real implementation and were not corrected in this pass (rewrite, not a +> text fix). Real pruning cache is `PruningCache` (`crates/zeph-mcp/src/pruning.rs`) — a single-slot +> cache keyed on `(message_content_hash, tool_list_hash)` that selects tools via an **LLM call**, not +> embedding/dot-product ranking as described below. Real collision detection is +> `detect_collisions::` / `ToolCollision` (`crates/zeph-mcp/src/tool.rs`) plus +> `log_tool_collisions` (`crates/zeph-mcp/src/manager/connect.rs`). `log::warn!` calls in the +> pseudocode below are banned project-wide (`tracing` only, see `[[constitution]]` II/IV). +> The `[mcp.discovery]` config block (`pruning_enabled`/`top_k_tools`/`cache_size`/`collision_mode`) +> is unverified and likely stale given the cache design mismatch above. + Semantic tool discovery, per-message pruning cache, collision detection, tool filtering. ## Overview @@ -217,12 +229,12 @@ collision_mode = "disambiguate" # or "error", "first" - [[008-1-lifecycle]] — Tool discovery runs after server startup - [[008-3-security]] — Pruning cache prevents tool injection via descriptions -- [[006-tools]] — Pruned tool subset sent to ToolExecutor -- [[003-llm-providers]] — Embedding provider used for semantic filtering +- [[006-tools/spec]] — Pruned tool subset sent to ToolExecutor +- [[003-llm-providers/spec]] — Embedding provider used for semantic filtering ## See Also - [[008-mcp/spec]] — Parent - [[008-1-lifecycle]] — Server initialization - [[008-3-security]] — Injection defense during pruning -- [[006-tools]] — Tool execution after pruning +- [[006-tools/spec]] — Tool execution after pruning diff --git a/specs/008-mcp/008-3-security.md b/specs/008-mcp/008-3-security.md index 27c1d4bbf..6a9af9112 100644 --- a/specs/008-mcp/008-3-security.md +++ b/specs/008-mcp/008-3-security.md @@ -1,7 +1,6 @@ --- aliases: - MCP Security - - MCP Elicitation - OAP Authorization - SMCP Secure MCP tags: @@ -17,12 +16,24 @@ related: - "[[008-mcp/spec]]" - "[[008-1-lifecycle]]" - "[[008-2-discovery]]" - - "[[010-security]]" + - "[[010-security/spec]]" - "[[010-3-authorization]]" --- # Spec: MCP Security & OAP Authorization +> [!danger] Stale content — 2026-07 audit +> Most of this file (Elicitation Phases pseudocode, `InjectionDetector`/`DeBERTaClassifier`, +> `OAPPolicy`, `SmcpEnvelope`) does not match the real implementation and was not corrected in this +> pass (rewrite, not a text fix) — none of `OAPPolicy`, `SmcpEnvelope`, `InjectionDetector`, +> `DeBERTaClassifier` exist in the codebase. Real analogues: `ContentSanitizer`, +> `ExfiltrationGuard`, `RiskChainAccumulator` (`crates/zeph-sanitizer/`), `CandleClassifier`/ +> `CandlePiiClassifier` (`crates/zeph-llm/src/classifier/`). Elicitation itself is accurately +> documented in [[008-4-elicitation]] — treat that file as authoritative for elicitation, not the +> "Elicitation Phases" section below. The top `## Key Invariants` block (Untrusted/tool_allowlist/ +> allow_untrusted_without_allowlist fail-closed behavior) reads as accurate and should be preserved +> when this file is rewritten. + Elicitation phases, injection detection, OAP (Org-Aware Permission) authorization, SMCP secure protocol. ## Overview @@ -284,14 +295,14 @@ capability = "PublicRead" - [[008-1-lifecycle]] — Security checks during subprocess spawning - [[008-2-discovery]] — Tool descriptions scanned for injection before caching -- [[010-security]] — Parent security spec; refers to injection defense +- [[010-security/spec]] — Parent security spec; refers to injection defense - [[010-3-authorization]] — OAP policy enforcement -- [[025-classifiers]] — DeBERTa-backed injection detection +- [[025-classifiers/spec]] — DeBERTa-backed injection detection ## See Also - [[008-mcp/spec]] — Parent - [[008-1-lifecycle]] — Server lifecycle where security starts - [[008-2-discovery]] — Tool discovery with injection scanning -- [[010-security]] — Cross-cutting security constraints +- [[010-security/spec]] — Cross-cutting security constraints - [[010-3-authorization]] — OAP authorization policies diff --git a/specs/008-mcp/008-4-elicitation.md b/specs/008-mcp/008-4-elicitation.md index 25bf07263..6d578466a 100644 --- a/specs/008-mcp/008-4-elicitation.md +++ b/specs/008-mcp/008-4-elicitation.md @@ -39,7 +39,7 @@ related: | `crates/zeph-mcp/src/elicitation.rs` | Elicitation request routing | | `crates/zeph-core/src/channel.rs` | `ElicitationRequest`, `ElicitationResponse` types | | `crates/zeph-channels/src/cli.rs` | CLI elicitation handling (interactive prompt) | -| `crates/zeph-channels/src/tui_channel.rs` | TUI elicitation (status modal) | +| `crates/zeph-tui/src/channel.rs` | TUI elicitation (status modal) | | `crates/zeph-channels/src/telegram.rs` | Telegram elicitation (message + timeout) | | `crates/zeph-channels/src/json_cli.rs` | JSON mode elicitation (`elicitation` event) | diff --git a/specs/008-mcp/spec.md b/specs/008-mcp/spec.md index 30078ef82..7f9223b66 100644 --- a/specs/008-mcp/spec.md +++ b/specs/008-mcp/spec.md @@ -97,8 +97,8 @@ Config migration step 50 adds these fields. Both live under `[[mcp.servers]]`. | File | Contents | |---|---| | `crates/zeph-mcp/src/` | `McpManager`, server lifecycle, tool registry | -| `crates/zeph-tools/src/composite.rs` | `McpExecutor` in composite chain | -| `crates/zeph-tui/src/channel.rs` | `suppress_stderr` integration | +| `crates/zeph-mcp/src/executor.rs` | `McpToolExecutor` in composite chain | +| `crates/zeph-mcp/src/manager/{mod.rs,builder.rs,connect.rs,server.rs,retry.rs}` | `suppress_stderr` integration | --- diff --git a/specs/009-orchestration/spec.md b/specs/009-orchestration/spec.md index 13055edd2..2cb140ae3 100644 --- a/specs/009-orchestration/spec.md +++ b/specs/009-orchestration/spec.md @@ -33,19 +33,19 @@ related: ### Internal | File | Contents | |---|---| -| `crates/zeph-core/src/orchestration/mod.rs` | `OrchestrationEngine`, public API | -| `crates/zeph-core/src/orchestration/dag.rs` | `TaskGraph`, DAG structure (petgraph) | -| `crates/zeph-core/src/orchestration/scheduler.rs` | `DagScheduler`, tick loop | -| `crates/zeph-core/src/orchestration/planner.rs` | `LlmPlanner`, goal decomposition | -| `crates/zeph-core/src/orchestration/router.rs` | `AgentRouter`, 3-step fallback | -| `crates/zeph-core/src/orchestration/aggregator.rs` | `LlmAggregator`, per-task token budget | -| `crates/zeph-core/src/orchestration/command.rs` | `/plan` command parsing | -| `crates/zeph-core/src/orchestration/graph.rs` | Internal graph utilities | -| `crates/zeph-core/src/orchestration/error.rs` | `OrchestrationError` | +| `crates/zeph-orchestration/src/lib.rs` | `OrchestrationEngine`, public API | +| `crates/zeph-orchestration/src/dag.rs` | `TaskGraph`, DAG structure (petgraph) | +| `crates/zeph-orchestration/src/scheduler/mod.rs` | `DagScheduler`, tick loop | +| `crates/zeph-orchestration/src/planner.rs` | `LlmPlanner`, goal decomposition | +| `crates/zeph-orchestration/src/router.rs` | `AgentRouter`, 3-step fallback | +| `crates/zeph-orchestration/src/aggregator.rs` | `LlmAggregator`, per-task token budget | +| `crates/zeph-orchestration/src/command.rs` | `/plan` command parsing | +| `crates/zeph-orchestration/src/graph.rs` | Internal graph utilities | +| `crates/zeph-orchestration/src/error.rs` | `OrchestrationError` | --- -`crates/zeph-core/src/orchestration/` (feature: `orchestration`) — DAG task planning and execution. +`crates/zeph-orchestration/src/` (feature: `orchestration`) — DAG task planning and execution. ## Components @@ -286,7 +286,7 @@ For `Hierarchical` topology: tasks are grouped into levels (depth layers). `DagS ```toml [orchestration] -topology_selection = true # opt-in +topology_selection = false # opt-in; default false (crates/zeph-config/src/experiment.rs) ``` ### Key Invariants diff --git a/specs/010-security/010-1-vault.md b/specs/010-security/010-1-vault.md index 9d4446dc9..1314cd4a0 100644 --- a/specs/010-security/010-1-vault.md +++ b/specs/010-security/010-1-vault.md @@ -20,6 +20,15 @@ related: # Spec: Secret Vault & Credential Resolution +> [!danger] Stale content — 2026-07 audit +> This file's code samples and type names do not match the real implementation and were not +> corrected in this pass (rewrite, not a text fix): it uses `serde_yaml` (banned project-wide, see +> `[[constitution]]` II — use `serde_norway`), `log::info!`/`log::warn!` (banned, `tracing` only), +> invented types `VaultBackend`/`AgeVault` (real trait is `VaultProvider`, +> `crates/zeph-vault/src/lib.rs`), and lists `age`/`kms`/`custom` backends (only `age`/`env` exist, +> per the correct listing in `[[010-security/spec]]` and `[[038-vault/spec]]`). **Treat +> [[038-vault/spec]] as authoritative for vault behavior**, not this file. + Age-encrypted secret storage, credential resolution, ZEPH_* environment key mapping, vault access control. ## Overview @@ -204,7 +213,7 @@ fallback_env = false # don't read from env vars as fallback ## Integration Points -- [[003-llm-providers]] — All providers resolve API keys from vault +- [[003-llm-providers/spec]] — All providers resolve API keys from vault - [[010-2-injection-defense]] — Vault keys never logged or leaked - [[010-4-audit]] — Vault access logged for compliance diff --git a/specs/010-security/010-2-injection-defense.md b/specs/010-security/010-2-injection-defense.md index 8b32d5d36..bf9199359 100644 --- a/specs/010-security/010-2-injection-defense.md +++ b/specs/010-security/010-2-injection-defense.md @@ -18,11 +18,19 @@ related: - "[[010-1-vault]]" - "[[010-3-authorization]]" - "[[010-4-audit]]" - - "[[025-classifiers]]" + - "[[025-classifiers/spec]]" --- # Spec: Injection Defense & Content Isolation +> [!danger] Stale content — 2026-07 audit +> The `IpiDetector`/`PiiDetector`/`DeBERTaClassifier` code blocks below are fictional — none of +> these type names exist in the codebase — and were not corrected in this pass (rewrite, not a text +> fix). Real code: `crates/zeph-llm/src/classifier/` (`CandleClassifier`, `CandlePiiClassifier`) and +> `crates/zeph-sanitizer/src/causal_ipi.rs` (`TurnCausalAnalyzer`). The "User Verification Gate" +> section's direct `println!(...)` also contradicts [[001-system-invariants/spec#1. Channel Contract]] +> ("the `Channel` trait is the only I/O boundary") — do not copy that pattern. + Indirect Prompt Injection (IPI) defense, DeBERTa-backed detection, AlignSentinel confidence scoring, PII NER detection, content isolation. ## Overview @@ -263,8 +271,8 @@ redact_mode = "mask" # or "block" ## Integration Points -- [[008-mcp]] — MCP tool outputs scanned before reaching agent -- [[025-classifiers]] — DeBERTa inference infrastructure +- [[008-mcp/spec]] — MCP tool outputs scanned before reaching agent +- [[025-classifiers/spec]] — DeBERTa inference infrastructure - [[010-4-audit]] — IPI detection logged for compliance - WebFetch tool — Content scanned before returning @@ -272,5 +280,5 @@ redact_mode = "mask" # or "block" - [[010-security/spec]] — Parent - [[010-1-vault]] — Prevent secret leakage from redaction -- [[025-classifiers]] — DeBERTa and NER models +- [[025-classifiers/spec]] — DeBERTa and NER models - [[010-4-audit]] — Audit log of IPI detections diff --git a/specs/010-security/010-3-authorization.md b/specs/010-security/010-3-authorization.md index 12cddb35c..ff4301fb3 100644 --- a/specs/010-security/010-3-authorization.md +++ b/specs/010-security/010-3-authorization.md @@ -17,12 +17,22 @@ related: - "[[010-1-vault]]" - "[[010-2-injection-defense]]" - "[[010-4-audit]]" - - "[[006-tools]]" + - "[[006-tools/spec]]" - "[[008-3-security]]" --- # Spec: Authorization & Capability-Based Access Control +> [!danger] Stale content — 2026-07 audit +> `AuthPolicy`, `ShellSandbox`, `SsrfValidator` below are fictional — zero matches in the codebase — +> and were not corrected in this pass (rewrite, not a text fix). The code samples also chain +> `.unwrap()` on `Regex::new(...)`, which directly violates +> [[001-system-invariants/spec#11. Error Handling Contract]] ("NEVER use `panic!()`... no `unwrap()` +> in production paths") — do not copy that pattern. Real mechanisms live in the shell blocklist +> (`crates/zeph-tools/src/filter/security.rs` and the shell executor's `PermissionPolicy` gate) and +> the SSRF `validate_url` path referenced correctly in [[010-security/spec]] and +> [[010-5-egress-logging]] — treat those as authoritative over this file. + Permission policy enforcement, shell sandbox blocklist, SSRF protection, tool authorization. ## Overview @@ -253,13 +263,13 @@ allow_list = ["api.example.com", "internal.trusted.service"] ## Integration Points -- [[006-tools]] — Tool execution calls authorization check +- [[006-tools/spec]] — Tool execution calls authorization check - [[008-3-security]] — OAP authorization for MCP tools - [[010-4-audit]] — Authorization failures logged ## See Also - [[010-security/spec]] — Parent -- [[006-tools]] — ToolExecutor enforces authorization +- [[006-tools/spec]] — ToolExecutor enforces authorization - [[008-3-security]] — OAP authorization for MCP - [[010-4-audit]] — Audit trail of authorization checks diff --git a/specs/010-security/010-4-audit.md b/specs/010-security/010-4-audit.md index 3c0e86f37..7e00aa570 100644 --- a/specs/010-security/010-4-audit.md +++ b/specs/010-security/010-4-audit.md @@ -2,7 +2,6 @@ aliases: - Audit Trail - Security Logging - - AgentRFC Protocol Audit - Cross-Tool Injection Correlation - Env-Var Scrubbing tags: @@ -21,6 +20,18 @@ related: # Spec: Audit Trail & Security Logging +> [!danger] Stale content — 2026-07 audit +> The `AuditEntry`/`AuditEventType`/`AuditStatus`/`AuditLogger` schema below is fictional — the real +> `AuditEntry` (`crates/zeph-tools/src/audit.rs`) is tool-execution-centric (`tool`, `command`, +> `result`, `duration_ms`, `error_category`, `caller_id`, `correlation_id`, `vigil_risk`, ...), not +> the generic `id`/`agent_id`/`event_type`/`resource`/`action`/`status` shape shown here. See +> [[010-5-egress-logging]] for the accurate, current `AuditEntry`/`AuditLogger` description. The +> "Cross-Tool Injection Correlation" section (`InjectionCorrelator`, time-windowed accumulation) +> directly **contradicts** [[010-security/spec]]'s authoritative `CrossToolCorrelator`, which states +> cross-turn signal accumulation is NEVER performed and is cleared every turn — do not implement +> against this section without reconciling that contradiction first. Not corrected in this pass +> (needs a mechanism-level rewrite, not a text fix). + AgentRFC protocol audit, cross-tool injection correlation, environment variable scrubbing, compliance logging. ## Overview @@ -357,7 +368,7 @@ sample_rate = 1.0 # log all events; set < 1.0 to sample ## Integration Points -- [[006-tools]] — Tool execution audited here +- [[006-tools/spec]] — Tool execution audited here - [[010-1-vault]] — Vault accesses audited - [[010-2-injection-defense]] — IPI detections audited - [[010-3-authorization]] — Authorization checks audited diff --git a/specs/010-security/010-7-shadow-memory-guardrail.md b/specs/010-security/010-7-shadow-memory-guardrail.md index f7b322c63..fb0a049ba 100644 --- a/specs/010-security/010-7-shadow-memory-guardrail.md +++ b/specs/010-security/010-7-shadow-memory-guardrail.md @@ -359,8 +359,8 @@ pub struct SafeHarborConfig { 1. **`zeph-sanitizer`** — `ShadowMemory` component (`crates/zeph-sanitizer/src/shadow_memory.rs`): - `ShadowMemory::new()` — initialize at session start (VecDeque-backed turn history) - - `ShadowMemory::record_turn()` — log `ShadowEvent` after tool execution - - `ShadowMemory::cumulative_score()` — read current risk score + - `ShadowMemory::record()` — log `ShadowEvent` after tool execution + - `ShadowMemory::goal_drift_score()` — read current risk score - Stored in-memory (no persistence), evicted at session end - Goal drift detection: `jaccard_distance` on tool-name sets, permission escalation pattern, deviation ratio - `GoalDriftResult` carries score, flags, and the triggering event summary @@ -795,8 +795,8 @@ Before implementation PR: - [ ] Pre-commit checks pass: ```bash cargo +nightly fmt --check - cargo clippy --workspace --all-targets --all-features -- -D warnings - cargo nextest run --workspace --all-features --lib --bins + cargo clippy --profile ci --workspace --all-targets --features "desktop,ide,server,chat,pdf,scheduler,testing" -- -D warnings + cargo nextest run --config-file .github/nextest.toml --workspace --features "desktop,ide,server,chat,pdf,scheduler" --lib --bins ``` - [ ] Live test completed: 10-turn goal hijacking scenario passes - [ ] CHANGELOG.md updated with feature description diff --git a/specs/010-security/spec.md b/specs/010-security/spec.md index 71afa141c..e32b06557 100644 --- a/specs/010-security/spec.md +++ b/specs/010-security/spec.md @@ -63,9 +63,9 @@ specific areas, refer to the child specs below. ### Internal | File | Contents | |---|---| -| `crates/zeph-core/src/vault/` | `VaultProvider`, age/env backends | -| `crates/zeph-core/src/config/types/security.rs` | `SecurityConfig` | -| `crates/zeph-tools/src/filter/security.rs` | `SecurityPatterns`, 17 regex patterns | +| `crates/zeph-vault/src/{lib.rs,age.rs,env.rs}` | `VaultProvider` trait, age/env backends | +| `crates/zeph-config/src/security.rs` | `SecurityConfig` | +| `crates/zeph-tools/src/filter/security.rs` | Regex security pattern bank (free functions: `compile_extra_patterns`, `extract_security_lines`, `append_security_warnings`) | | `crates/zeph-gateway/src/transport/auth.rs` | BLAKE3 + `ct_eq` bearer auth | | `crates/zeph-acp/src/transport/auth.rs` | ACP bearer token auth | | `crates/zeph-acp/src/fs.rs` | `resolve_resource_link`, SSRF/path checks | @@ -79,7 +79,7 @@ Multiple crates — security is cross-cutting. ## Vault (Secret Management) -`crates/zeph-core/src/vault/` — backend abstraction for secrets: +`crates/zeph-vault/` — backend abstraction for secrets: | Backend | Activation | |---|---| @@ -226,7 +226,10 @@ auto-promotion and reload trust-assignment are out of scope for this default). ## IPI Defense: DeBERTa Soft-Signal, AlignSentinel, TurnCausalAnalyzer -Three-layer indirect prompt injection (IPI) defense stack in `zeph-classifiers`: +Three-layer indirect prompt injection (IPI) defense stack — classifiers in +`crates/zeph-llm/src/classifier/` (`CandleClassifier`, `CandleThreeClassClassifier` for the +three-class `AlignSentinel` model), turn-level analysis in `crates/zeph-sanitizer/src/causal_ipi.rs` +(`TurnCausalAnalyzer`); there is no standalone `zeph-classifiers` crate: ### DeBERTa Soft-Signal diff --git a/specs/011-tui/spec.md b/specs/011-tui/spec.md index 91740d7b3..4764726b6 100644 --- a/specs/011-tui/spec.md +++ b/specs/011-tui/spec.md @@ -30,7 +30,7 @@ related: ### Internal | File | Contents | |---|---| -| `crates/zeph-tui/src/app.rs` | `TuiApp`, panel layout, event loop | +| `crates/zeph-tui/src/app/` | `TuiApp`, panel layout, event loop | | `crates/zeph-tui/src/channel.rs` | `TuiChannel`, `Channel` trait impl | | `crates/zeph-tui/src/metrics.rs` | `MetricsCollector`, watch channel | | `crates/zeph-tui/src/layout.rs` | Panel split logic | diff --git a/specs/013-acp/spec.md b/specs/013-acp/spec.md index 61e6125d9..e834ae5fa 100644 --- a/specs/013-acp/spec.md +++ b/specs/013-acp/spec.md @@ -35,6 +35,17 @@ related: | 1.7 | 2026-07-01 | developer | Fixed #5379: `zeph acp model-config show` now loads the resolved config and marks the active `[acp.model_config].default_temperature_preset` in its output, instead of only printing the static preset table. Resolved #5373: `session/fork` and `session/resume` now inherit `model`/`temperature_preset`/`thinking_enabled`/`auto_approve_level` from the source session (live in-memory state, falling back to a persisted close-time snapshot, falling back to configured defaults) instead of always resetting to defaults — new `acp_sessions` columns via migration `105_acp_session_config`; see "Fork/Resume Config Inheritance" below. | | 1.8 | 2026-07-17 | developer | Renovate `rust-minor-patch` bundle bumped core `1.0.1`→`1.2.0`, schema `=1.1.0`→`=1.4.0`. Core `1.1.0` stabilized `$/cancel_request` unconditionally and dropped its `unstable_cancel_request` forward — `unstable-cancel-request` is now a local-only opt-in gate (Cargo feature unchanged, but no longer maps to an upstream feature); `unstable-boolean-config` tombstoned the same way after schema `1.1.0` made `SessionConfigOptionValue::Boolean` unconditional. Schema `1.4.0` renamed `SetProviderRequest`/`DisableProviderRequest`/`ProviderInfo`'s `id: String` field to `provider_id: ProviderId` (`Arc` newtype) — updated all `providers.rs` call sites and tests. No handler/transport/builder-chain logic changed otherwise. | +> [!warning] Body narrative lags the 1.8 changelog entry — 2026-07 audit +> Only the top summary (line 22) and the changelog table above were updated for the 1.8 bump. +> Roughly 15 places in the body below (Protocol Version section, `/agent.json` discussion, Feature +> Flags table notes, "Breaking Changes Resolution" section, Future/v2 Tracking, "Version Upgrade +> Note", Addendum) still assert the document's *current* state as `agent-client-protocol 1.0.1` / +> schema `=1.1.0` — confirmed current values from `Cargo.toml` are `1.2.0` / `=1.4.0`. Sections that +> narrate the **historical** 0.14.0→1.0.1 transition (e.g. "Breaking Changes Resolution (SDK 0.14.0 +> → 1.0.1)") are correctly historical and should stay as written; sections that state facts as +> presently true using the `1.0.1`/`=1.1.0` version numbers need updating to `1.2.0`/`=1.4.0`. This +> needs a full pass distinguishing the two cases, not corrected in this audit. + --- ## Sources diff --git a/specs/015-self-learning/spec.md b/specs/015-self-learning/spec.md index ba0ebb719..5bc451475 100644 --- a/specs/015-self-learning/spec.md +++ b/specs/015-self-learning/spec.md @@ -31,14 +31,14 @@ related: ### Internal | File | Contents | |---|---| -| `crates/zeph-core/src/agent/feedback_detector.rs` | `FeedbackDetector`, `JudgeDetector`, `CorrectionSignal`, `CorrectionKind` | +| `crates/zeph-agent-feedback/src/lib.rs` | `FeedbackDetector`, `JudgeDetector`, `CorrectionSignal`, `CorrectionKind` | | `crates/zeph-skills/src/trust_score.rs` | `posterior_weight` (Wilson score), `rerank` | | `crates/zeph-skills/src/evolution.rs` | `SkillMetrics`, `SkillEvaluation`, self-improvement | | `crates/zeph-skills/src/registry.rs` | BM25 index, hybrid search, `max_active_skills` | --- -`crates/zeph-skills/src/`, `crates/zeph-core/src/agent/feedback_detector.rs` — feedback detection, skill ranking, trust model. +`crates/zeph-skills/src/`, `crates/zeph-agent-feedback/src/lib.rs` — feedback detection, skill ranking, trust model. ## Feedback Detection (Two Paths) @@ -139,7 +139,7 @@ Per-provider EMA (exponential moving average) latency: ## Multi-Language FeedbackDetector -Issue #1424. `crates/zeph-core/src/agent/feedback_detector.rs`. +Issue #1424. `crates/zeph-agent-feedback/src/lib.rs`. ### Overview diff --git a/specs/016-output-filtering/spec.md b/specs/016-output-filtering/spec.md index 0349259f9..631ef2704 100644 --- a/specs/016-output-filtering/spec.md +++ b/specs/016-output-filtering/spec.md @@ -31,7 +31,7 @@ related: |---|---| | `crates/zeph-tools/src/filter/mod.rs` | `OutputFilterRegistry`, `FilterPipeline`, `CommandMatcher`, `FilterMetrics` | | `crates/zeph-tools/src/filter/security.rs` | `SecurityPatterns`, 17 `LazyLock` | -| `crates/zeph-tools/src/filter/declarative.rs` | Per-filter TOML config structs | +| `crates/zeph-tools/src/filter/declarative/mod.rs` | Per-filter TOML config structs | --- diff --git a/specs/018-scheduler/spec.md b/specs/018-scheduler/spec.md index c024b0ef2..8e9173e66 100644 --- a/specs/018-scheduler/spec.md +++ b/specs/018-scheduler/spec.md @@ -167,7 +167,7 @@ Zeph's RTW-A defense implements four mechanisms active when `rtwa_enabled = true |---|---| | **Mech1 (Write-then-read quarantine)** | A task added via the control channel in the same tick is quarantined and does not fire that tick. | | **Mech2 (Trust gate)** | Only tasks with `provenance = "static"` or `"user_added"` may run custom prompts. External tasks (`provenance = "external"`) are suppressed. | -| **Mech3 (Injection scan)** | Task prompt scanned via `sanitize_task_prompt` before execution. Matched injection patterns block the task with `SchedulerError::InjectionBlocked`. | +| **Mech3 (Injection scan)** | Task prompt scanned via `sanitize_task_prompt` before execution. Matched injection patterns block the task with `SchedulerError::PromptInjectionBlocked`. | | **Mech4 (External-read suppression)** | When the tick reads from an external source, tasks with `"external"` provenance suppress their custom prompt for that tick. | ### Provenance Field @@ -190,6 +190,6 @@ RTW-A defense is enabled by default. Pass `Scheduler::with_rtwa(false)` to disab - Mech1 applies only within a single tick — a task added in tick N is eligible to fire from tick N+1 - Mech3 injection scan MUST run before task prompt is passed to the agent `message_queue` -- `SchedulerError::InjectionBlocked` and `SchedulerError::TaskQuarantined` are distinct variants +- `SchedulerError::PromptInjectionBlocked` and `SchedulerError::TaskQuarantined` are distinct variants - NEVER fire a task with `provenance = "external"` custom prompt during an external-read tick (Mech4) - `adversarial_policy_bypass` in `list_tasks` responses is blocked — task listing enforces the same policy filter as execution (commit #4536) diff --git a/specs/021-zeph-context/spec.md b/specs/021-zeph-context/spec.md index d47b01a37..b3d02647a 100644 --- a/specs/021-zeph-context/spec.md +++ b/specs/021-zeph-context/spec.md @@ -293,38 +293,10 @@ slots unchanged. ## 10. PRISM Query-Sensitive Edge Costing (#4079, #4360) -PRISM adds an opt-in query-sensitive A* cost function to graph recall in `zeph-memory`. -When enabled, the BFS traversal weights each candidate entity by cosine similarity to the -current query embedding rather than using confidence-only cost. - -### Cost Function - -``` -cost = (1 - confidence) * (1 - cosine_sim).max(0.01) -``` - -- `confidence` = edge confidence from MAGMA typed edge -- `cosine_sim` = cosine similarity between query vector and entity vector (0.0–1.0) -- `.max(0.01)` prevents zero-cost edges from causing BFS to skip valid nodes - -### Implementation Details - -- Query embedding is fetched **once per graph recall call** — not per entity -- Entity vectors are batch-fetched via `qdrant_point_ids_for_entities` + `get_vectors_from_collection` -- Embed call is bounded by a 10 s timeout; on timeout, falls back to confidence-only cost (no error) - -### Config - -```toml -[memory.graph] -query_sensitive_cost = false # default off — no behavior change when disabled -``` - -### Key Invariants - -- Default is `false` — existing behavior unchanged without explicit opt-in -- Fallback to confidence-only cost is silent (logged at `DEBUG` only) -- NEVER fetch query embedding per entity — exactly one embed call per graph recall invocation +PRISM is a `zeph-memory` graph-recall traversal enhancement, not a `zeph-context` feature — full +spec (cost function, implementation details, config, Key Invariants) lives in +[[012-graph-memory/spec#PRISM: Query-Sensitive A* Edge Costing (#4079, #4360)]]. This section is +intentionally kept as a pointer rather than a duplicate to avoid the two files drifting apart. --- @@ -364,7 +336,7 @@ None. --- -## 11. See Also +## 13. See Also - [[constitution]] — project principles - [[001-system-invariants/spec]] — cross-cutting invariants diff --git a/specs/023-complexity-triage-routing/spec.md b/specs/023-complexity-triage-routing/spec.md index 62ea78c74..1939d96d8 100644 --- a/specs/023-complexity-triage-routing/spec.md +++ b/specs/023-complexity-triage-routing/spec.md @@ -203,7 +203,7 @@ complex = "sonnet" expert = "opus" [[llm.providers]] -provider_type = "ollama" +type = "ollama" name = "fast" model = "qwen3:1.7b" @@ -304,7 +304,7 @@ This ensures cost-aware memory retrieval without repeating full semantic search When retrieval success rate diverges from baseline, adaptive escalation adjusts tier routing to recover performance. Per [[004-15-memory-skill-coevolution-rfc-decision]]: **Success Rate Tracking**: -- Measure success (via [[054-agent-feedback]]) on last N retrieval queries (sliding window, default 20) +- Measure success (via [[054-agent-feedback/spec]]) on last N retrieval queries (sliding window, default 20) - Per-tier tracking: `success_rate[tier] = successes / total_queries` **Escalation Policy**: @@ -330,7 +330,7 @@ escalation_cooldown_turns = 5 consecutive_degradation_threshold = 3 ``` -**Integration**: FeedbackDetector signals (from [[054-agent-feedback]]) feed into tier escalation logic. On turn completion, emit `memory.routing.escalation{from, to}` metric if escalation fires. +**Integration**: FeedbackDetector signals (from [[054-agent-feedback/spec]]) feed into tier escalation logic. On turn completion, emit `memory.routing.escalation{from, to}` metric if escalation fires. --- diff --git a/specs/025-classifiers/spec.md b/specs/025-classifiers/spec.md index b70655135..c1017c9ad 100644 --- a/specs/025-classifiers/spec.md +++ b/specs/025-classifiers/spec.md @@ -24,16 +24,17 @@ related: ## Overview -`zeph-classifiers` (feature: `classifiers`, implies `candle`) — Candle-backed ML classifiers for injection detection and PII detection. All classifiers are lazy-loaded on first use and cached for the session. +Candle-backed ML classifiers for injection detection and PII detection (feature: `classifiers`, implies `candle`). There is no standalone `zeph-classifiers` crate — the classifier infrastructure lives in `crates/zeph-llm/src/classifier/` and is consumed by `zeph-sanitizer`'s `ContentSanitizer`. All classifiers are lazy-loaded on first use and cached for the session. ## Sources | File | Contents | |---|---| -| `crates/zeph-classifiers/src/classifier.rs` | `ClassifierBackend` trait, `CandleClassifier` | -| `crates/zeph-classifiers/src/pii.rs` | `CandlePiiClassifier`, `PiiDetector` trait | -| `crates/zeph-classifiers/src/sanitizer.rs` | `ContentSanitizer::classify_injection()`, `detect_pii()` | -| `crates/zeph-classifiers/src/llm.rs` | `LlmClassifier` wrapping `AnyProvider` | +| `crates/zeph-llm/src/classifier/mod.rs` | `ClassifierBackend` trait (injection), `PiiDetector` trait, shared types (`PiiSpan`, `PiiResult`, `ClassificationResult`, `NerSpan`) | +| `crates/zeph-llm/src/classifier/candle.rs` | `CandleClassifier` — injection detection implementation | +| `crates/zeph-llm/src/classifier/candle_pii.rs` | `CandlePiiClassifier` — PII detection implementation | +| `crates/zeph-llm/src/classifier/llm.rs` | `LlmClassifier` wrapping `AnyProvider`, returns `FeedbackVerdict` | +| `crates/zeph-sanitizer/src/sanitizer.rs` | `ContentSanitizer::classify_injection()`, `detect_pii()`, `sanitize()` — consumes the classifiers above | ## ClassifierBackend Trait @@ -85,11 +86,13 @@ enabled = false pii_enabled = false pii_threshold = 0.75 -[self_learning] +[skills.learning] detector_mode = "model" # "model" uses LlmClassifier feedback_provider = "fast" # named provider reference ``` +See [[015-self-learning/spec]] for the full `LearningConfig` schema (`crates/zeph-config/src/learning.rs`, nested under `SkillsConfig.learning`). + `--migrate-config` adds `[classifiers]` section with `enabled = false` to existing configs. ## CLI @@ -103,9 +106,11 @@ zeph classifiers download --model pii|injection|all The ML injection classifier (DeBERTa) is bypassed for outputs from internal Zeph tools that produce only Zeph-generated text (#3384, #3394, #3396). The pattern-based sanitizer continues to run on these outputs for telemetry purposes. -**Bypassed tool names** (exact match on unnamespaced tool name): -`invoke_skill`, `load_skill`, `memory_save`, `memory_search`, `compress_context`, -`complete_focus`, `start_focus`, `schedule_periodic`, `schedule_deferred`, `cancel_task` +**Bypassed tool names** (exact match on unnamespaced tool name, `INTERNAL_TOOLS` in +`crates/zeph-core/src/agent/tool_execution/sanitize.rs`): +`bash`, `shell`, `invoke_skill`, `load_skill`, `memory_save`, `memory_search`, +`compress_context`, `request_compaction`, `complete_focus`, `start_focus`, +`schedule_periodic`, `schedule_deferred`, `cancel_task` **Adversarial-MCP guard**: colon-namespaced tool names (e.g. `server:invoke_skill`) are NEVER matched against the bypass allowlist — only bare names qualify. This prevents a malicious MCP server from registering a tool named `server:invoke_skill` to escape ML classification. diff --git a/specs/026-tui-subagent-management/spec.md b/specs/026-tui-subagent-management/spec.md index 136210c01..482f385c5 100644 --- a/specs/026-tui-subagent-management/spec.md +++ b/specs/026-tui-subagent-management/spec.md @@ -293,6 +293,6 @@ forwarding is disabled. - `011-tui/spec.md` — TUI invariants (spinner rule, no blocking I/O, resize handling) - `009-orchestration/spec.md` — DAG scheduler, task states, plan view - `crates/zeph-tui/src/widgets/subagents.rs` — existing sidebar widget (render-only) -- `crates/zeph-subagent/src/manager.rs` — `SubAgentManager`, `SubAgentStatus`, `SubAgentHandle` +- `crates/zeph-subagent/src/manager/mod.rs` — `SubAgentManager`, `SubAgentStatus`, `SubAgentHandle` - `crates/zeph-subagent/src/transcript.rs` — `TranscriptReader`, `TranscriptEntry` - `crates/zeph-core/src/metrics.rs` — `SubAgentMetrics`, `MetricsSnapshot` diff --git a/specs/028-hooks/spec.md b/specs/028-hooks/spec.md index f9baf1abd..390cddf52 100644 --- a/specs/028-hooks/spec.md +++ b/specs/028-hooks/spec.md @@ -37,7 +37,7 @@ related: ## Overview Reactive hooks allow operators to run shell commands or invoke MCP tools in response to -agent lifecycle events. Five event types are supported: +agent lifecycle events. Six event types are supported: | Event | Trigger | |---|---| diff --git a/specs/029-feature-flags/spec.md b/specs/029-feature-flags/spec.md index a60a53fe6..292f256da 100644 --- a/specs/029-feature-flags/spec.md +++ b/specs/029-feature-flags/spec.md @@ -63,15 +63,17 @@ A feature flag is **not** justified when: ### 3.1 Default Features ```toml -default = ["scheduler", "sqlite", "profiling", "sandbox"] +default = ["scheduler", "sqlite"] ``` | Flag | Justification | |---|---| | `scheduler` | Pulls in `dep:zeph-scheduler`, `dep:cron`, `dep:schemars`, `dep:chrono` | | `sqlite` | Mutually exclusive with `postgres`; selects the SQLite backend in `zeph-db` | -| `profiling` | Pulls in `dep:tracing-chrome`, `dep:chrono`, `dep:uuid`, `dep:sysinfo`; enables diagnostic tracing spans. Zero overhead when not actively tracing. | -| `sandbox` | Pulls in `dep:landlock`, `dep:seccompiler` on Linux; macOS Seatbelt compiles unconditionally. Runtime-disabled by default (`tools.sandbox.enabled = false`). | + +> [!note] +> `profiling` and `sandbox` are optional flags (see §3.2), not part of `default` — verified against +> `Cargo.toml` `[features] default = [...]`. Both are included in the `full` bundle (§4) used by CI. **Removed from default (consolidated as always-on per §3.3):** @@ -102,9 +104,23 @@ default = ["scheduler", "sqlite", "profiling", "sandbox"] | `sqlite` | `zeph-db/sqlite`, `zeph-memory/sqlite` | Mutually exclusive with `postgres` (also in default) | | `deep-link` | `zeph-common/deep-link`, `zeph-config/deep-link` | `zeph://` URI scheme registration/dispatch (spec #066); optional OS integration | | `session` | `dep:axum`, `dep:tokio-stream`, `zeph-common/http-middleware` | Session persistence + `zeph serve` mode (spec #068, new `zeph-session` crate); HTTP/SSE session API | +| `profiling` | `dep:tracing-chrome`, `dep:sysinfo` (+ per-crate `profiling` propagation) | Diagnostic tracing spans and system metrics; zero overhead when not actively tracing | +| `sandbox` | `zeph-tools/sandbox` (`dep:landlock`, `dep:seccompiler` on Linux; macOS Seatbelt compiles unconditionally) | Runtime-disabled by default (`tools.sandbox.enabled = false`) | +| `prometheus` | `gateway`, `dep:prometheus-client`, `zeph-gateway/prometheus` | OpenMetrics `/metrics` endpoint (spec #036); requires `gateway` | +| `gonka` | `zeph-llm/gonka`, `zeph-core/gonka` | gonka.ai inference provider (specs #051, #052) | +| `cocoon` | `zeph-llm/cocoon`, `zeph-core/cocoon`, `zeph-tui?/cocoon` | Cocoon distributed compute provider (spec #055) | +| `index` | `zeph-core/index` | AST-based code indexing (spec #017) | +| `registry` | `zeph-plugins/registry` | Skill/plugin marketplace discovery client (spec #045) | +| `testing` | `zeph-llm/testing` | Test-only provider harness helpers | +| `bench` | `dep:zeph-bench` | Benchmark harness CLI (spec #034) | > [!note] -> This table is not a complete enumeration of every flag in the root `Cargo.toml` — flags added since the §1 count was last taken (e.g. `prometheus`, `testing`, `bench`, `profiling-alloc`, `profiling-pyroscope`, `gonka`, `cocoon`, `index`) are not all listed here yet. Only `deep-link` and `session`, introduced in the same cycle that added specs #066/#068, are backfilled above; a full inventory reconciliation against `Cargo.toml`'s `[features]` block is tracked as follow-up work. +> `prometheus`, `gonka`, `cocoon`, `index`, `registry`, `testing`, and `bench` were backfilled above +> (2026-07 reconciliation pass against `Cargo.toml`'s `[features]` block). `profiling-alloc` and +> `profiling-pyroscope` are still not individually documented here — both are thin variants of +> `profiling` (`profiling-alloc = ["profiling", "zeph-core/profiling-alloc"]`, +> `profiling-pyroscope = ["profiling", "otel", "dep:pprof"]`) and are omitted as low-risk; add rows +> for them if either gains independent justification beyond extending `profiling`. ### 3.3 Always-On Capabilities (No Flag) @@ -139,12 +155,12 @@ Bundles are the **only** mechanism for enabling groups of features. Do not instr | Bundle | Expands to | Target use case | |---|---|---| -| `desktop` | `tui` | Local developer workstation with terminal UI | +| `desktop` | `tui`, `deep-link`, `session` | Local developer workstation with terminal UI | | `ide` | `acp`, `acp-http` | IDE integration via Agent-Client Protocol | -| `server` | `gateway`, `a2a`, `otel` | Headless server: webhook ingestion, A2A, telemetry | +| `server` | `gateway`, `a2a`, `otel`, `prometheus`, `session` | Headless server: webhook ingestion, A2A, telemetry, metrics | | `chat` | `discord`, `slack` | Bot deployment on messaging platforms | | `ml` | `candle`, `pdf` | On-device ML inference and PDF memory | -| `full` | `desktop`, `ide`, `server`, `chat`, `pdf`, `scheduler`, `classifiers` | CI, pre-merge checks, complete feature matrix | +| `full` | `desktop`, `ide`, `server`, `chat`, `pdf`, `scheduler`, `classifiers`, `profiling`, `sandbox`, `gonka`, `cocoon`, `testing`, `registry` | CI, pre-merge checks, complete feature matrix | Bundle invariants: - `full` must activate every flag that is safe to combine (excluding `metal`, `cuda`, `postgres` — platform/exclusive). diff --git a/specs/030-tui-slash-autocomplete/spec.md b/specs/030-tui-slash-autocomplete/spec.md index b75dcc9a6..79feb817c 100644 --- a/specs/030-tui-slash-autocomplete/spec.md +++ b/specs/030-tui-slash-autocomplete/spec.md @@ -8,7 +8,7 @@ tags: - tui - ui created: 2026-04-04 -status: draft +status: approved related: - "[[MOC-specs]]" - "[[011-tui/spec]]" @@ -304,7 +304,7 @@ crates/zeph-tui/src/widgets/slash_autocomplete.rs — SlashAutocompleteState s ### Modified files ``` -crates/zeph-tui/src/app.rs +crates/zeph-tui/src/app/ — add field: slash_autocomplete: Option — add method: handle_slash_autocomplete_key(&mut self, key: KeyEvent) — modify handle_insert_key: delegate to handle_slash_autocomplete_key when active @@ -313,7 +313,7 @@ crates/zeph-tui/src/app.rs crates/zeph-tui/src/widgets/mod.rs — pub mod slash_autocomplete; -crates/zeph-tui/src/app.rs (render section) +crates/zeph-tui/src/app/ (render section) — render slash_autocomplete popup after input bar, before command_palette ``` @@ -432,6 +432,6 @@ None. All design decisions are resolved in this spec. - `crates/zeph-tui/src/command.rs` — `CommandEntry`, `filter_commands`, `fuzzy_score` - `crates/zeph-tui/src/widgets/command_palette.rs` — `CommandPaletteState` pattern to follow -- `crates/zeph-tui/src/app.rs` — `handle_insert_key`, `handle_palette_key`, modal routing +- `crates/zeph-tui/src/app/` — `handle_insert_key`, `handle_palette_key`, modal routing - `.local/specs/011-tui/spec.md` — TUI architecture, Spinner Rule - `.local/specs/constitution.md` — project-wide constraints diff --git a/specs/031-database-abstraction/spec.md b/specs/031-database-abstraction/spec.md index 7662ff18c..fe6c0037d 100644 --- a/specs/031-database-abstraction/spec.md +++ b/specs/031-database-abstraction/spec.md @@ -844,6 +844,7 @@ pub struct DbConfig { } ``` +```rust /// Strip password from a database URL for safe logging. /// /// **Amendment 1 [2026-03-28]**: Applied to all log output, error messages, @@ -1200,6 +1201,14 @@ postgres = ["zeph-db/postgres"] ### Layer Assignment +> [!note] Superseded by the current constitution +> This subsection is the original RFC proposal and is historical. The constitution has since +> introduced sub-tiers within Layer 0 (0a/0b/0c); `zeph-db` is currently classified as **Layer 0b** +> ("depends on L0a only": `zeph-common`, `zeph-commands`) per [[constitution#I. Architecture]], not +> a bare "Layer 0" with no `zeph-*` deps — `crates/zeph-db/Cargo.toml` depends on `zeph-common`. +> The "constitution should be amended" suggestion below was resolved by that 0a/0b/0c split rather +> than by treating `zeph-db` as same-layer-exempt. + `zeph-db` is **Layer 0** (no zeph-* dependencies). Updated layering: - **Layer 0**: `zeph-llm`, `zeph-a2a`, `zeph-gateway`, `zeph-scheduler`, `zeph-common`, **`zeph-db`** diff --git a/specs/032-handoff-skill-system/spec.md b/specs/032-handoff-skill-system/spec.md index 9a6524ed8..298cbcfba 100644 --- a/specs/032-handoff-skill-system/spec.md +++ b/specs/032-handoff-skill-system/spec.md @@ -144,4 +144,4 @@ The agent reads all parent handoffs and synthesizes the combined context. ## Historical Context -PRs #2076 and #2078 introduced typed `HandoffContext` Rust structs with compile-time validation (inspired by MAST research on multi-agent coordination failures). This was reverted in — the skill-based YAML protocol remains the active approach. Future hardening may revisit typed validation with a simpler design. +PRs #2076 and #2078 introduced typed `HandoffContext` Rust structs with compile-time validation (inspired by MAST research on multi-agent coordination failures). This was reverted in #2082 — the skill-based YAML protocol remains the active approach. Future hardening may revisit typed validation with a simpler design. diff --git a/specs/033-subagent-context-propagation/report.md b/specs/033-subagent-context-propagation/report.md index 7e8c66be7..06d60c2b9 100644 --- a/specs/033-subagent-context-propagation/report.md +++ b/specs/033-subagent-context-propagation/report.md @@ -69,7 +69,7 @@ When `subagent_type` is not specified, Claude Code uses a **fork path** that: ### Architecture overview -Zeph spawns subagents via `SubAgentManager::spawn()` in `crates/zeph-subagent/src/manager.rs`. +Zeph spawns subagents via `SubAgentManager::spawn()` in `crates/zeph-subagent/src/manager/mod.rs`. The entry point from the agent loop is `handle_agent_command()` in `crates/zeph-core/src/agent/mod.rs:4505`. @@ -158,7 +158,7 @@ analyzed. The only context is the raw `task_prompt` string. Complex delegation t ("review what we discussed and write tests") fail silently. **Affected files**: -- `crates/zeph-subagent/src/manager.rs:962` — `initial_messages: vec![]` +- `crates/zeph-subagent/src/manager/mod.rs:962` — `initial_messages: vec![]` - `crates/zeph-core/src/agent/mod.rs:4530-4535` — spawn call site --- @@ -384,8 +384,8 @@ parallel subtasks. | File | Change | |---|---| -| `crates/zeph-subagent/src/manager.rs` | Add `parent_context`, `parent_cancel` to `spawn()` and `AgentLoopArgs` | +| `crates/zeph-subagent/src/manager/mod.rs` | Add `parent_context`, `parent_cancel` to `spawn()` and `AgentLoopArgs` | | `crates/zeph-core/src/agent/mod.rs:4530` | Pass `self.context.messages` slice and cancel token to spawn | | `crates/zeph-subagent/src/def.rs` | Add `model` inheritance enum | | `crates/zeph-config/src/subagent.rs` | Add `max_depth` config field | -| `crates/zeph-subagent/src/manager.rs:946` | Use `parent_cancel.child_token()` | +| `crates/zeph-subagent/src/manager/mod.rs:946` | Use `parent_cancel.child_token()` | diff --git a/specs/034-zeph-bench/spec.md b/specs/034-zeph-bench/spec.md index 6e552a542..528722230 100644 --- a/specs/034-zeph-bench/spec.md +++ b/specs/034-zeph-bench/spec.md @@ -8,7 +8,7 @@ tags: - benchmarking - testing created: 2026-04-08 -status: draft +status: approved related: - "[[MOC-specs]]" --- @@ -303,7 +303,10 @@ zeph bench show --results ## 13. Layer Placement -`zeph-bench` is a **Layer 4 consumer** (same tier as `zeph-channels`, `zeph-tui`): +`zeph-bench` is a **Special** crate per [[constitution#I. Architecture]] ("feature-gated, no +mandatory deps") — it depends on `zeph-core` (a Layer 4 orchestrator) and other crates spanning +multiple layers, which is atypical for the Layer 0–5 DAG and is why it is excluded from that DAG +rather than assigned a specific layer number: - Depends on: `zeph-core` (Channel trait, Agent), `zeph-memory` (per-scenario `SemanticMemory` instance backed by a fresh `SQLite` file), `zeph-llm` (provider override), `zeph-config` (provider registry) diff --git a/specs/044-subagent-lifecycle/spec.md b/specs/044-subagent-lifecycle/spec.md index c0cc84dfc..133557316 100644 --- a/specs/044-subagent-lifecycle/spec.md +++ b/specs/044-subagent-lifecycle/spec.md @@ -465,7 +465,17 @@ interleaves two different JSON schemas on stdout — unsupported for now; use `- --- -## 15. Delegation Mode Gate (spec `042-subagent-delegation-mode-parity`, issue #5857) +## 15. Delegation Mode Gate (issue #5857) + +> [!note] Broken cross-reference — 2026-07 audit +> This section previously cited a companion spec `042-subagent-delegation-mode-parity`. No such +> spec exists in `/specs/` — `042` is already assigned to +> [[042-zeph-commands/spec|Slash Command Registry]], and no directory matching +> `*-subagent-delegation-mode-parity` exists anywhere under `/specs/`. Either the companion spec +> was never created or was misnumbered at authoring time. The motivation is summarized inline +> below instead of via a dangling link; if a dedicated spec for delegation-mode parity across +> subsystems is still wanted, it should be filed as a new numbered spec (follow-up, not created in +> this corrective pass). ### Problem @@ -473,7 +483,7 @@ Prior to this addition, `SubAgentConfig.enabled: bool` was the only lever govern spawning, and it was **inert** — no code path actually read it. An operator had no way to keep the sub-agent subsystem enabled and useful while forbidding the main agent from autonomously deciding to spawn one (e.g. in a semi-trusted channel where prompt-injected input could reach -the agent) — see `[[042-subagent-delegation-mode-parity/spec]]` for the full motivation. +the agent). ### Mechanism @@ -537,7 +547,7 @@ so it remains permitted under `explicit_request_only` and `proactive`). --- -## 10. See Also +## 16. See Also - [[constitution]] — project principles - [[002-agent-loop/spec]] — parent agent loop that uses `SubAgentManager` diff --git a/specs/049-agent-decomposition/turn-context.md b/specs/049-agent-decomposition/turn-context.md index 2e543f173..50ef14910 100644 --- a/specs/049-agent-decomposition/turn-context.md +++ b/specs/049-agent-decomposition/turn-context.md @@ -8,8 +8,8 @@ tags: created: 2026-04-27 status: permanent related: - - "[[049-agent-decomposition]]" - - "[[002-agent-loop]]" + - "[[049-agent-decomposition/spec]]" + - "[[002-agent-loop/spec]]" --- # TurnContext Value Type diff --git a/specs/062-context-adaptive-memory/spec.md b/specs/062-context-adaptive-memory/spec.md index 91706d1ad..65724678e 100644 --- a/specs/062-context-adaptive-memory/spec.md +++ b/specs/062-context-adaptive-memory/spec.md @@ -13,10 +13,10 @@ tags: created: 2026-05-28 status: approved related: - - "[[022-zeph-context]]" - - "[[004-memory]]" - - "[[009-orchestration]]" - - "[[044-zeph-common]]" + - "[[021-zeph-context/spec]]" + - "[[004-memory/spec]]" + - "[[009-orchestration/spec]]" + - "[[043-zeph-common/spec]]" issues: - "#4016" - "#4017" diff --git a/specs/063-worktree-subsystem/brd.md b/specs/063-worktree-subsystem/brd.md index 75f0034f3..864c5e554 100644 --- a/specs/063-worktree-subsystem/brd.md +++ b/specs/063-worktree-subsystem/brd.md @@ -11,8 +11,7 @@ status: approved related: - "[[specs/063-worktree-subsystem/srs]]" - "[[specs/063-worktree-subsystem/spec]]" - - "[[specs/parity-claude-code-3918/spec]]" - - "[[specs/045-subagent-lifecycle/spec]]" + - "[[044-subagent-lifecycle/spec]]" --- # BRD-063: Worktree Subsystem + `worktree.base_ref` @@ -27,7 +26,8 @@ also operating in. This creates two problems: 2. **Lack of isolation** — multiple subagents running concurrently may step on each other's filesystem changes, producing unpredictable results. -Claude Code (the primary parity target, `specs/parity-claude-code-3918/spec.md`) provides +Claude Code (the primary parity target, issue #3918 — no `parity-claude-code-3918/spec.md` exists +under `/specs/` as of the 2026-07 audit; likely a `.local/`-scoped working doc) provides `worktree.baseRef` semantics that give each agent an isolated git worktree. Zeph deferred this capability in the parity spec (rows 40–41) because no worktree subsystem existed. Issue #4655 delivers that subsystem as a prerequisite for the deferred parity items. diff --git a/specs/063-worktree-subsystem/spec.md b/specs/063-worktree-subsystem/spec.md index e7336c46b..2746344ac 100644 --- a/specs/063-worktree-subsystem/spec.md +++ b/specs/063-worktree-subsystem/spec.md @@ -13,8 +13,7 @@ related: - "[[specs/063-worktree-subsystem/srs]]" - "[[specs/063-worktree-subsystem/nfr]]" - "[[specs/063-worktree-subsystem/plan]]" - - "[[specs/parity-claude-code-3918/spec]]" - - "[[specs/045-subagent-lifecycle/spec]]" + - "[[044-subagent-lifecycle/spec]]" - "[[constitution]]" --- @@ -23,7 +22,10 @@ related: **GitHub:** #4655 **Branch:** `feat/4655-worktree-baseref-config` **Crate:** `zeph-worktree` (new), `zeph-config` (extends), `zeph-subagent` (extends) -**Parent parity spec:** `[[specs/parity-claude-code-3918/spec]]` rows 40–41 (deferred) +**Parent parity spec:** issue #3918 (Claude Code parity tracking), rows 40–41 (deferred). No +`parity-claude-code-3918/spec.md` exists under `/specs/` as of the 2026-07 audit — this was likely +a `.local/`-scoped working doc (e.g. the competitive-parity playbook) rather than a permanent spec; +the wikilink is left unresolved pending confirmation of the correct target. --- diff --git a/specs/065-ephemeral-plugins-provider-overrides/spec.md b/specs/065-ephemeral-plugins-provider-overrides/spec.md index a9f31849a..c7284f31b 100644 --- a/specs/065-ephemeral-plugins-provider-overrides/spec.md +++ b/specs/065-ephemeral-plugins-provider-overrides/spec.md @@ -201,4 +201,4 @@ Per project rules, all new functionality must wire up: | Feature A | `[[specs/079-plugins/spec]]` | Extends plugin system with ephemeral variant | | Feature A | `[[specs/010-security/spec]]` | Must comply with SSRF + path traversal guards | | Feature B | `[[specs/003-llm-providers/spec]]` | Extends provider persistence; `ProviderOverrides` is a new type in the provider config layer | -| Feature B | `[[specs/021-config-loading/spec]]` | New `persist_provider_overrides` key follows existing config resolution rules | +| Feature B | `[[020-config-loading/spec]]` | New `persist_provider_overrides` key follows existing config resolution rules | diff --git a/specs/066-deep-link-scheme/README.md b/specs/066-deep-link-scheme/README.md index 596bf8076..307f5712a 100644 --- a/specs/066-deep-link-scheme/README.md +++ b/specs/066-deep-link-scheme/README.md @@ -12,7 +12,7 @@ status: approved github_issue: 4687 related: - "[[013-acp/spec]]" - - "[[044-zeph-common/spec]]" + - "[[043-zeph-common/spec]]" - "[[001-system-invariants/spec]]" - "[[010-security/spec]]" --- diff --git a/specs/066-deep-link-scheme/spec.md b/specs/066-deep-link-scheme/spec.md index 1ccbdd4b5..917ddb548 100644 --- a/specs/066-deep-link-scheme/spec.md +++ b/specs/066-deep-link-scheme/spec.md @@ -10,7 +10,7 @@ status: approved spec_id: "066" related: - "[[013-acp/spec]]" - - "[[044-zeph-common/spec]]" + - "[[043-zeph-common/spec]]" - "[[001-system-invariants/spec]]" - "[[010-security/spec]]" --- diff --git a/specs/067-knowledge-ingest/README.md b/specs/067-knowledge-ingest/README.md index dc7647ad0..27f9e36eb 100644 --- a/specs/067-knowledge-ingest/README.md +++ b/specs/067-knowledge-ingest/README.md @@ -15,8 +15,8 @@ related: - "[[004-memory/004-9-memory-write-gate]]" - "[[004-memory/004-6-graph-memory]]" - "[[012-graph-memory/spec]]" - - "[[018-index/spec]]" - - "[[041-sanitizer/spec]]" + - "[[017-index/spec]]" + - "[[040-sanitizer/spec]]" - "[[056-autoskill-trace-extraction/spec]]" - "[[001-system-invariants/spec]]" - "[[constitution]]" diff --git a/specs/067-knowledge-ingest/spec.md b/specs/067-knowledge-ingest/spec.md index c0fdea731..6b71ee8b1 100644 --- a/specs/067-knowledge-ingest/spec.md +++ b/specs/067-knowledge-ingest/spec.md @@ -18,8 +18,8 @@ related: - "[[004-memory/004-9-memory-write-gate]]" - "[[004-memory/004-6-graph-memory]]" - "[[012-graph-memory/spec]]" - - "[[018-index/spec]]" - - "[[041-sanitizer/spec]]" + - "[[017-index/spec]]" + - "[[040-sanitizer/spec]]" - "[[056-autoskill-trace-extraction/spec]]" - "[[constitution]]" --- @@ -79,7 +79,7 @@ verbatim-no-PII-redaction write path. - `[[004-memory/004-9-memory-write-gate]]` — MemReader write-quality gate; project evidence that *write pollution collapses recall faster than scoring drift*. **This spec MUST honor that gate, not bypass it.** - `[[004-memory/004-3-admission-control|A-MAC Admission Control]]` — admission control (INV §14). -- `[[018-index/spec]]` — code RAG; the boundary this spec must not cross. +- `[[017-index/spec]]` — code RAG; the boundary this spec must not cross. - `[[056-autoskill-trace-extraction/spec]]` — precedent for background extraction from session history with a per-session idempotency table (`skill_trace_sessions`) and `*_provider` config. @@ -600,9 +600,8 @@ Checklist: - [[004-memory/004-9-memory-write-gate]] — write-quality gate that ingest MUST honor - [[004-memory/004-6-graph-memory]] — MAGMA edges, SYNAPSE, A-MEM weights - [[012-graph-memory/spec]] — entity graph, BFS recall, community detection -- [[018-index/spec]] — code RAG boundary (code never enters the graph) -- [[041-sanitizer/spec]] — PII / exfiltration validators for the write path +- [[017-index/spec]] — code RAG boundary (code never enters the graph) +- [[040-sanitizer/spec]] — PII / exfiltration validators for the write path - [[056-autoskill-trace-extraction/spec]] — precedent: background extraction from session history + idempotency table - [[001-system-invariants/spec]] — system contracts (INV-1 dependency direction, write-pollution) - [[constitution]] — project-wide non-negotiable rules -``` diff --git a/specs/067-knowledge-ingest/tasks.md b/specs/067-knowledge-ingest/tasks.md index 7002a9378..16e4e8825 100644 --- a/specs/067-knowledge-ingest/tasks.md +++ b/specs/067-knowledge-ingest/tasks.md @@ -446,5 +446,5 @@ D1 is entirely deferred. - [[067-knowledge-ingest/plan]] — implementation plan, phases, risk register - [[004-memory/spec]] — memory subsystem - [[004-memory/004-9-memory-write-gate]] — write-quality gate (must not be bypassed) -- [[041-sanitizer/spec]] — PII/exfiltration validators +- [[040-sanitizer/spec]] — PII/exfiltration validators - [[001-system-invariants/spec]] — system contracts (INV-1 dependency direction) diff --git a/specs/068-session-persistence/spec.md b/specs/068-session-persistence/spec.md index b42f712dc..8c1a031ae 100644 --- a/specs/068-session-persistence/spec.md +++ b/specs/068-session-persistence/spec.md @@ -11,13 +11,13 @@ tags: created: 2026-06-13 status: draft related: - - "[[064-durable-execution]]" - - "[[013-acp]]" - - "[[039-vault]]" - - "[[002-agent-loop]]" - - "[[044-zeph-common]]" - - "[[022-zeph-context]]" - - "[[032-database-abstraction]]" + - "[[064-durable-execution/spec]]" + - "[[013-acp/spec]]" + - "[[038-vault/spec]]" + - "[[002-agent-loop/spec]]" + - "[[043-zeph-common/spec]]" + - "[[021-zeph-context/spec]]" + - "[[031-database-abstraction/spec]]" - "[[007-channels/spec]]" - "[[011-tui/spec]]" issues: diff --git a/specs/072-multimodal-mcp-passthrough/spec.md b/specs/072-multimodal-mcp-passthrough/spec.md index 56bca8d19..7972c3b00 100644 --- a/specs/072-multimodal-mcp-passthrough/spec.md +++ b/specs/072-multimodal-mcp-passthrough/spec.md @@ -17,8 +17,8 @@ related: - "[[008-1-lifecycle]]" - "[[008-3-security]]" - "[[010-2-injection-defense]]" - - "[[024-multi-model-design]]" - - "[[040-content-sanitizer]]" + - "[[024-multi-model-design/spec]]" + - "[[040-sanitizer/spec]]" - "[[069-threat-model/spec]]" - "[[068-session-persistence/spec]]" - "[[MOC-specs]]" @@ -512,7 +512,7 @@ additional non-blocking hardening item, folded into this spec as C5/AC-15:** - [[001-system-invariants/spec]] — Invariant #4 (Ask First: new `MessagePart` variant, engaged for the Audio deferral), #5 (`ToolExecutor`/`ToolOutput` contract), #12 (mandatory integration points) - [[008-3-security]] — MCP elicitation/injection defense this spec's threat model extends - [[010-2-injection-defense]] — Text-sanitization pipeline the text placeholder continues to flow through unchanged -- [[040-content-sanitizer]] — `ContentSanitizer`/quarantine flow `MediaSanitizer` sits alongside +- [[040-sanitizer/spec]] — `ContentSanitizer`/quarantine flow `MediaSanitizer` sits alongside - [[069-threat-model/spec]] — MATRA asset/attack-tree model; this spec should add an MCP-media asset/attack-tree entry as a follow-up - [[068-session-persistence/spec]] — `SessionSink`/durable JSONL log this spec's C1 strip point must precede - `plan.md` — phased implementation plan diff --git a/specs/075-orchestration-node-control-parity/spec.md b/specs/075-orchestration-node-control-parity/spec.md index 92645ade6..163342775 100644 --- a/specs/075-orchestration-node-control-parity/spec.md +++ b/specs/075-orchestration-node-control-parity/spec.md @@ -277,8 +277,9 @@ performed. project convention. - Whether `default_idle_timeout_secs` gets a dedicated CLI/TUI surface beyond config.toml/`--init` — the existing `task_timeout_secs` precedent has none, and FR-018 does not require one, but this - is worth confirming against current TUI settings-editor conventions (`[[061-tui-settings-editor-parity/spec]]` - if it exists) before implementation. + is worth confirming against current TUI settings-editor conventions before implementation. (No + `*-tui-settings-editor-parity` spec exists under `/specs/` as of the 2026-07 audit — if TUI + settings-editor conventions are formalized in a dedicated spec later, cross-link it here.) ### Never diff --git a/specs/078-agent-persistence/spec.md b/specs/078-agent-persistence/spec.md index ea4a82b74..c589ddd74 100644 --- a/specs/078-agent-persistence/spec.md +++ b/specs/078-agent-persistence/spec.md @@ -17,7 +17,7 @@ related: - "[[001-system-invariants/spec]]" - "[[002-agent-loop/spec]]" - "[[004-memory/spec]]" - - "[[032-database-abstraction/spec]]" + - "[[031-database-abstraction/spec]]" - "[[010-security/spec]]" --- @@ -414,4 +414,4 @@ None. - [[002-agent-loop/spec]] — agent loop and context pressure - [[004-memory/spec]] — SemanticMemory backend - [[010-security/spec]] — exfiltration guard and security model -- [[032-database-abstraction/spec]] — database abstraction layer for SQLite/PostgreSQL +- [[031-database-abstraction/spec]] — database abstraction layer for SQLite/PostgreSQL diff --git a/specs/079-plugins/spec.md b/specs/079-plugins/spec.md index bfb82fea4..25726d15e 100644 --- a/specs/079-plugins/spec.md +++ b/specs/079-plugins/spec.md @@ -20,7 +20,7 @@ related: - "[[005-skills/spec]]" - "[[008-mcp/spec]]" - "[[010-security/spec]]" - - "[[028-runtime-layer/spec]]" + - "[[027-runtime-layer/spec]]" --- # Spec: Plugin Management System (`zeph-plugins`) @@ -704,10 +704,10 @@ None. --- -## 13. See Also +## 18. See Also - [[001-system-invariants/spec]] — system-wide invariants - [[005-skills/spec]] — skill registry and trust model - [[008-mcp/spec]] — MCP server lifecycle and server commands - [[010-security/spec]] — security model and authorization -- [[028-runtime-layer/spec]] — config overlay merge timing +- [[027-runtime-layer/spec]] — config overlay merge timing diff --git a/specs/081-transcript-integrity/spec.md b/specs/081-transcript-integrity/spec.md index c896f9ad2..2de6040ad 100644 --- a/specs/081-transcript-integrity/spec.md +++ b/specs/081-transcript-integrity/spec.md @@ -16,11 +16,18 @@ related: - "[[001-system-invariants/spec]]" - "[[010-security/spec]]" - "[[010-4-audit]]" - - "[[039-durable-agent-turns-subagent-adapters-unwired/spec]]" - - "[[056-vault-key-hardening-rotation/spec]]" - "[[064-durable-execution/spec]]" --- +> [!note] Broken cross-references — 2026-07 audit +> This document previously linked two nonexistent spec slugs, `039-durable-agent-turns-subagent- +> adapters-unwired/spec` and `056-vault-key-hardening-rotation/spec`. Neither exists under +> `/specs/` (039 is `039-background-task-supervisor`; 056 is `056-autoskill-trace-extraction`) — +> these were likely working titles for gaps that were folded into [[064-durable-execution/spec]] +> (which covers both durable-execution wiring gaps and, in its "Key Rotation Windows" section, +> vault-key rotation). The `[NEEDS CLARIFICATION]` items below that cited them now point at +> [[064-durable-execution/spec]] instead of a confirmed-nonexistent rename. + # Feature: Tamper-Evident Persisted Transcripts, Session Event Logs, and Durable Journal Entries > [!info] Metadata @@ -112,11 +119,11 @@ THEN the execution SHALL be aborted with a distinct error (extending DurableErro | FR-001 | WHEN an entry is appended to a `TranscriptWriter`-owned JSONL file, a `SessionEventLog`, or (for entry kinds not already covered by `PayloadCipher`/`compute_control_hmac`) a `zeph-durable` journal, THE SYSTEM SHALL compute a keyed hash over the entry's content **and** the previous entry's hash (hash-chain), reusing the project's existing keyed-BLAKE3 primitive (`crates/zeph-durable/src/backend/local.rs`'s `compute_control_hmac` pattern) rather than introducing a new cryptographic dependency | must | | FR-002 | WHEN `TranscriptReader::load`/`load_strict`, `SessionEventLog::read_all`/`read_chunked`, or `ReplayCursor`'s segment reads load persisted entries, THE SYSTEM SHALL recompute and verify the hash chain before the entries are treated as trusted prior context (i.e. before injection into the LLM message array or replay execution) | must | | FR-003 | WHEN chain verification fails on one or more entries, THE SYSTEM SHALL fail closed for the affected scope: reject the affected entries (or the whole file/execution — see Open Questions on granularity) rather than silently proceeding with unverified data, and SHALL emit a distinct, structured log/error event separate from ordinary malformed-JSON errors (`TranscriptReader::load_strict`'s existing `SubAgentError` variant, a new `SessionError` variant, and `DurableError::ReplayIntegrity`/`ControlIntegrity` respectively) | must | -| FR-004 | WHEN the affected replay path is unattended (`zeph durable resume`, the durable crash-orphan sweep, or automatic sub-agent transcript reload on collection), THE SYSTEM SHALL abort the operation on a detected integrity failure rather than warn-and-proceed; interactive paths (`zeph sessions resume`, `zeph durable inspect --reveal`) MAY surface the failure to the human for an explicit decision instead of hard-aborting — exact interactive UX left to [NEEDS CLARIFICATION: should `zeph sessions resume` hard-fail like the durable path, or warn-and-let-the-user-decide? The finding's own reference feature (Claude Code) hard-blocks; Zeph's constitution has no existing "ask the human on integrity failure" precedent, but does have exactly this warn-vs-fail-closed choice documented as an open question in [[039-durable-agent-turns-subagent-adapters-unwired/spec]] for a related durable-wiring gap] | must | +| FR-004 | WHEN the affected replay path is unattended (`zeph durable resume`, the durable crash-orphan sweep, or automatic sub-agent transcript reload on collection), THE SYSTEM SHALL abort the operation on a detected integrity failure rather than warn-and-proceed; interactive paths (`zeph sessions resume`, `zeph durable inspect --reveal`) MAY surface the failure to the human for an explicit decision instead of hard-aborting — exact interactive UX left to [NEEDS CLARIFICATION: should `zeph sessions resume` hard-fail like the durable path, or warn-and-let-the-user-decide? The finding's own reference feature (Claude Code) hard-blocks; Zeph's constitution has no existing "ask the human on integrity failure" precedent, but does have exactly this warn-vs-fail-closed choice documented as an open question in [[064-durable-execution/spec]] for a related durable-wiring gap] | must | | FR-005 | THE hash-chain key SHALL be resolved from the age vault per CLAUDE.md's Secrets & Vault policy (no `ZEPH_*` environment-variable key material), consistent with how `zeph-durable`'s `PayloadCipher` is already keyed from the vault outside `zeph-durable` itself (INV-1: the crate stays a pure abstraction with no cryptographic dependency) | must | | FR-006 | WHEN a pre-existing transcript/session-log/journal file created before this feature's rollout is read, THE SYSTEM SHALL treat it as unverifiable-legacy rather than tampered, per the migration posture resolved in [NEEDS CLARIFICATION: is legacy-unverifiable content (a) auto-trusted once with a one-time warning, (b) permanently flagged UNVERIFIED but still usable, or (c) required to be re-chained via a one-time backfill pass before first read? Affects every existing `.local/testing/` and production transcript/session on upgrade] | must | | FR-007 | `TranscriptReader::load`'s existing lenient mode (warn-and-skip on a malformed line, distinct from `load_strict`) SHALL define its interaction with a broken hash chain: [NEEDS CLARIFICATION: does a chain break in lenient mode also warn-and-skip the affected and all subsequent entries (since the chain is now unverifiable from that point forward), or does lenient mode apply only to JSON-structural malformation and always defer to `load_strict`-equivalent strictness for chain breaks specifically?] | should | -| FR-008 | WHEN a legitimately relocated/copied session or transcript directory (e.g. migrated between machines, per [[056-vault-key-hardening-rotation/spec]]'s key-rotation scenario) is opened under a different vault identity than the one that wrote it, THE SYSTEM SHALL distinguish "chain valid, key mismatch" from "chain broken" in its error reporting, so operators are not misled into believing content was tampered with when it was only re-keyed | should | +| FR-008 | WHEN a legitimately relocated/copied session or transcript directory (e.g. migrated between machines, per [[064-durable-execution/spec#Key Rotation Windows (#6447, #6451, #6460)]]'s key-rotation scenario) is opened under a different vault identity than the one that wrote it, THE SYSTEM SHALL distinguish "chain valid, key mismatch" from "chain broken" in its error reporting, so operators are not misled into believing content was tampered with when it was only re-keyed | should | | FR-009 | THE `zeph-durable` extension of this mechanism SHALL default the row-HMAC/chain check to **on** rather than requiring explicit opt-in via `with_hmac_key`, closing the current default-off gap noted in the Problem Statement — subject to [NEEDS CLARIFICATION: is flipping this default a breaking change requiring a migration step (per CLAUDE.md's config-migration integration-point requirement), and does it apply uniformly to single-process local deployments where the "forged/relocated row from a shared-database deployment" threat model in `local.rs`'s own doc comment is less applicable?] | should | ## 4. Non-Functional Requirements @@ -194,8 +201,8 @@ THEN the execution SHALL be aborted with a distinct error (extending DurableErro - [[001-system-invariants/spec]] — cross-cutting invariants; this spec proposes new invariants for the persisted-history read path - [[010-security/spec]] and [[010-2-injection-defense]] — the live IPI-defense counterpart to this spec's at-rest defense; both close different lanes of the same injected-fake-history attack class - [[010-4-audit]] — audit-trail precedent this spec's integrity-failure events (NFR-005) extend -- [[039-durable-agent-turns-subagent-adapters-unwired/spec]] — related durable-journal wiring gap; documents a similar warn-vs-fail-closed open question referenced in FR-004 -- [[056-vault-key-hardening-rotation/spec]] — vault key rotation/migration semantics this spec's FR-008 depends on +- [[064-durable-execution/spec]] — related durable-journal wiring gap; documents a similar warn-vs-fail-closed open question referenced in FR-004 +- [[064-durable-execution/spec#Key Rotation Windows (#6447, #6451, #6460)]] — vault key rotation/migration semantics this spec's FR-008 depends on - [[MOC-specs]] — all specifications - **External reference**: Claude Code 2.1.205 (July 2026) — "auto mode rule blocking tampering with session transcript files to preserve conversation integrity," the competitive-parity trigger for this finding; closes fabricated in-transcript approvals as an injection lane in a single-session, human-observed context, which this spec generalizes to Zeph's multi-surface (sub-agent transcript, session event log, durable journal) and partly-unattended (durable crash-resume) replay model diff --git a/specs/ARCHITECTURE.md b/specs/ARCHITECTURE.md index ec09cef32..8fd0e2449 100644 --- a/specs/ARCHITECTURE.md +++ b/specs/ARCHITECTURE.md @@ -16,6 +16,20 @@ related: # Specs Architecture: Dependency Graph +> [!warning] Coverage gap — 2026-07 audit +> This dependency graph and its layer tables only cover specs 001–034. Specs 035–081 (background +> task supervisor, sanitizer, experiments, `zeph-commands`, `zeph-common`, subagent lifecycle, +> interop gaps, MARCH, CLI modes, SLM metrics, agent decomposition, security capability governance, +> gonka, speculation engine, agent feedback, cocoon, the six AutoSkill specs, context-adaptive +> memory, worktree, durable execution, plugins, deep-link, knowledge ingest, session persistence, +> threat model, runtime thinking controls, multimodal MCP, orchestration ensemble-merge/HITL/node +> control, and more) were added after this graph was last drawn and are entirely absent from it. +> Redrawing the graph correctly requires re-deriving dependency edges for ~47 additional specs, +> which risks getting relationships wrong without deep per-subsystem review — flagged here as a +> follow-up recommendation (a dedicated `/sdd` session) rather than attempted in this corrective +> pass. Until then, treat this document as accurate only for specs 001–034 and use +> [[MOC-specs]] / [[README]] for the complete current index. + ## Dependency Graph (Mermaid) ```mermaid diff --git a/specs/BRD.md b/specs/BRD.md index c3173add5..eddde8a54 100644 --- a/specs/BRD.md +++ b/specs/BRD.md @@ -398,12 +398,12 @@ Detailed targets are in [[NFR]]. High-level constraints for business context: ### Technical Constraints -- Language: Rust 1.94 (MSRV), Edition 2024, no `unsafe` blocks. +- Language: Rust 1.97 (MSRV), Edition 2024, no `unsafe` blocks. - Async: tokio; no `async-trait` crate in library crates. - TLS: rustls only; `openssl-sys` banned. - YAML: `serde_norway` only; `serde_yaml` / `serde_yml` banned. - Database: SQLite (default) or PostgreSQL (opt-in); `sqlx::Any` banned. -- Feature flags: `default = []`; always-on capabilities compiled without flags. +- Feature flags: `default = ["scheduler", "sqlite"]`; always-on capabilities compiled without flags. - Binary size: release binary must stay under 15 MiB. - Unsafe code: `unsafe_code = "deny"` workspace-wide. diff --git a/specs/MOC-specs.md b/specs/MOC-specs.md index fcde8adaf..9d9398e0b 100644 --- a/specs/MOC-specs.md +++ b/specs/MOC-specs.md @@ -53,6 +53,9 @@ status: moc - [[012-graph-memory/spec|Entity Graph Memory]] — entity graph, BFS recall, community detection, MAGMA typed edges, SYNAPSE spreading activation; works with [[004-memory/spec|Memory Pipeline]] - [[004-memory/004-6-graph-memory|Graph Memory (memory sub-spec)]] — concise reference within the memory subsystem: data model overview, MAGMA edge types, SYNAPSE config, key invariants - [[004-memory/004-16-memory-type-aware-retrieval|MemGuard Type-Aware Retrieval (memory sub-spec)]] — opt-in fetch-time gate on `schedule_context_fetchers`, `FunctionalType` enum, intent-scoped widening via existing `HeuristicRouter` (no new LLM call), `BehavioralRule` always-composed safety invariant; retrieval-only, byte-for-byte no-op when disabled; GitHub #6086, #6226 + - [[004-memory/004-16-shadow-memory-safety|Shadow Memory Safety (memory sub-spec)]] — `TrajectoryRiskAccumulator` MAGE multi-turn goal-hijacking detection, `ShadowMemory`/`GoalDriftResult`; SafeHarbor guardrail tree aspirational; GitHub #3695 + - [[004-memory/004-17-implicit-conflict-detection|Implicit Conflict Detection (memory sub-spec)]] — write-time `ImplicitConflictDetector` (STALE/CUPMem fuzzy predicate matching), propagation-aware SYNAPSE recall; GitHub #3702 + - [[004-memory/004-18-five-signal-retrieval|Five-Signal Retrieval (memory sub-spec)]] — access frequency, causal distance, novelty, recency, goal-relevance signals + async consolidation daemon (MemTier); GitHub #3703 - [[067-knowledge-ingest/spec|Knowledge Ingest]] — `zeph knowledge ingest` operator command; static artifacts → semantic notes (existing `IngestionPipeline`, no graph), subagent transcripts → graph (gated by measurement spike); Phase 0 provenance (`origin`/`import_batch_id`/`source_uri`) + `rollback`; honors write-gate (004-9) + admission (004-3), bypasses only RPE; sanitizer on write path; external Claude/Codex import deferred; code stays in [[017-index/spec|zeph-index]] ### Configuration & Loading @@ -170,7 +173,7 @@ status: moc ## System-Wide Features ### Feature Flags & Dependencies -- [[029-feature-flags/spec|Feature Flags]] — feature flag decision rules, surviving flag inventory (22 flags), bundle definitions (desktop, ide, server, full), always-on capabilities (openai, compatible, orchestrator, router, self-learning, qdrant, vault-age, mcp); `default = []` in Cargo.toml +- [[029-feature-flags/spec|Feature Flags]] — feature flag decision rules, surviving flag inventory, bundle definitions (desktop, ide, server, full), always-on capabilities (openai, compatible, orchestrator, router, self-learning, qdrant, vault-age, mcp); `default = ["scheduler", "sqlite"]` in Cargo.toml - [[041-experiments/spec|Experiments & Runtime Feature Gating]] — runtime A/B testing via `[experiments]` config section, ExperimentConfig, rollout percentage, experiment results reporting, CLI subcommands; distinct from compile-time feature flags ### Database Abstraction @@ -224,7 +227,6 @@ status: moc | 024 | [[024-multi-model-design/spec\|Multi-Model Design]] | specify | approved | | 025 | [[025-classifiers/spec\|ML Classifiers]] | specify | approved | | 026 | [[026-tui-subagent-management/spec\|TUI Subagents]] | specify | approved | -| 026 | [[026-tui-subagent-management/plan\|TUI Subagents]] | plan | approved | | 027 | [[027-runtime-layer/spec\|Runtime Layer]] | specify | approved | | 028 | [[028-hooks/spec\|Hooks]] | specify | approved | | 029 | [[029-feature-flags/spec\|Feature Flags]] | specify | approved | @@ -244,6 +246,24 @@ status: moc | 042 | [[042-zeph-commands/spec\|Slash Command Registry]] | specify | approved | | 043 | [[043-zeph-common/spec\|Shared Primitives]] | specify | approved | | 044 | [[044-subagent-lifecycle/spec\|Subagent Lifecycle]] | specify | approved | +| 045 | [[045-interop-protocol-gaps/spec\|Interop Protocol Gaps]] | specify | approved | +| 046 | [[046-march-quality/spec\|MARCH Quality Pipeline]] | specify | approved | +| 047 | [[047-cli-modes/spec\|CLI Execution Modes]] | specify | approved | +| 048 | [[048-slm-cost-metrics/spec\|SLM Cost Metrics]] | specify | approved | +| 049 | [[049-agent-decomposition/spec\|Agent Decomposition]] | specify | draft | +| 050 | [[050-security-capability-governance/spec\|Security Capability Governance]] | specify | draft | +| 051 | [[051-gonka-gateway/spec\|Gonka Gateway]] | specify | implemented | +| 052 | [[052-gonka-native/spec\|Gonka Native]] | specify | implemented | +| 053 | [[053-speculation-engine/spec\|Speculation Engine]] | specify | implemented | +| 054 | [[054-agent-feedback/spec\|Agent Feedback Detection]] | specify | approved | +| 055 | [[055-cocoon/spec\|Cocoon Distributed Compute]] | specify | draft | +| 056 | [[056-autoskill-trace-extraction/spec\|AutoSkill A1: Trace Extraction]] | specify | implemented | +| 057 | [[057-autoskill-versioned-merging/spec\|AutoSkill A2: Versioned Merging]] | specify | implemented | +| 058 | [[058-autoskill-query-rewriting/spec\|AutoSkill A3: Query Rewriting]] | specify | implemented | +| 059 | [[059-autoskill-bm25-hybrid/spec\|AutoSkill A4: BM25 Hybrid]] | specify | implemented | +| 060 | [[060-autoskill-trigger-sets/spec\|AutoSkill A5: Trigger Sets]] | specify | implemented | +| 061 | [[061-autoskill-heuristic-promotion/spec\|AutoSkill A6: Heuristic Promotion]] | specify | implemented | +| 062 | [[062-context-adaptive-memory/spec\|Context-Adaptive Memory]] | tasks | approved | | 063 | [[063-worktree-subsystem/spec\|Worktree Subsystem]] | specify | approved | | 064 | [[064-durable-execution/spec\|Durable Execution]] | specify | approved | | 065 | [[065-ephemeral-plugins-provider-overrides/spec\|Ephemeral Plugins & Provider Overrides]] | specify | implemented | @@ -251,12 +271,18 @@ status: moc | 067 | [[067-knowledge-ingest/spec\|Knowledge Ingest]] | specify | draft | | 068 | [[068-session-persistence/spec\|Session Persistence]] | specify | draft | | 069 | [[069-threat-model/spec\|MATRA Threat Model]] | specify | approved | +| 070 | [[070-runtime-thinking-controls/spec\|Runtime Thinking Controls]] | specify | approved | +| 071 | [[071-router-thinking-budget-delegation/spec\|Router Thinking Budget Delegation]] | specify | approved | | 072 | [[072-multimodal-mcp-passthrough/spec\|Multimodal MCP Passthrough]] | specify | draft | | 073 | [[073-orch-ensemble-merge/spec\|ORCH Ensemble-Merge]] | specify | approved | | 074 | [[074-orchestration-hitl-interrupt/spec\|Declarative HITL Interrupt]] | tasks | draft | | 075 | [[075-orchestration-node-control-parity/spec\|Node Timeout / Retry-Exhausted Recovery]] | tasks | approved | | 076 | [[076-cli-init-migrate-config-flag-mismatch/spec\|CLI Init/Migrate-Config Flag Mismatch]] | specify | draft | | 077 | [[077-safe-mode-and-cd-command/spec\|Safe Mode & /cd Command]] | specify | implemented | +| 078 | [[078-agent-persistence/spec\|Agent Persistence]] | specify | approved | +| 079 | [[079-plugins/spec\|Plugin Management]] | specify | approved | +| 080 | [[080-cross-thread-store-dynamic-handoff/spec\|Cross-Thread Store & Dynamic Handoff]] | specify | approved | +| 081 | [[081-transcript-integrity/spec\|Transcript Integrity]] | specify | implemented | --- @@ -295,7 +321,7 @@ The following large specs have been broken into atomic child specs for focused s ## Navigation - **By Layer**: [[#Foundation & Architecture]] → [[#Core Agent Systems]] → [[#Execution & Tools]] → [[#User Interface & Channels]] -- **By Phase**: Specs 001–030 are Phase 1 (specification); only 026 has Phase 2 (plan) +- **By Phase**: Specs 001–061 are Phase 1 (specification) only; several later specs (062, 063, 065, 066, 067, 068, 070, 072, 073, 074, 075) additionally have `plan.md`/`tasks.md` (and some `brd.md`/`srs.md`/`nfr.md`) companion documents — see each directory for its actual file set - **By Crate**: See crate field in README.md for crate mapping - **Search**: Use Obsidian search by tag (e.g., `tag:sdd`) or filter by status diff --git a/specs/NFR.md b/specs/NFR.md index 977fa2928..03b8c80d5 100644 --- a/specs/NFR.md +++ b/specs/NFR.md @@ -46,7 +46,7 @@ Quality attributes covered and their relevance: | Performance Efficiency | Critical — agent latency and binary size directly affect developer UX | | Reliability | High — single-user: crashes must not lose data; Qdrant/MCP absence must degrade gracefully | | Security | Critical — vault, secrets, SSRF, injection defense, PII | -| Maintainability | High — 24-crate workspace, pre-v1.0 rapid iteration | +| Maintainability | High — 32-crate workspace, pre-v1.0 rapid iteration | | Portability | Medium — macOS + Linux; Windows out of scope | | Usability | Medium — CLI ergonomics and TUI responsiveness | | Compatibility | High — MCP, A2A, ACP, Ollama, OpenAI protocol compliance | @@ -273,8 +273,8 @@ Quality attributes covered and their relevance: | ID | Requirement | Details | |----|------------|---------| -| NFR-MNT-001 | Workspace structure | 24-crate Cargo workspace with a strict 5-layer DAG; same-layer imports prohibited | -| NFR-MNT-002 | Feature flag discipline | `default = []`; optional features declared with `dep:zeph-`; no optional feature enabled by default | +| NFR-MNT-001 | Workspace structure | 32-crate Cargo workspace with a strict layered DAG (see [[constitution#I. Architecture]]); same-layer imports prohibited | +| NFR-MNT-002 | Feature flag discipline | `default = ["scheduler", "sqlite"]`; optional features declared with `dep:zeph-`; no optional feature beyond that pair enabled by default | | NFR-MNT-003 | Dependency additions | New external dependencies require version check via context7 MCP and explicit justification in PR description | | NFR-MNT-004 | No backward-compatibility shims before v1.0 | Breaking changes documented in `CHANGELOG.md`; no `#[deprecated]` shims required before v1.0 | | NFR-MNT-005 | No unsafe code | `unsafe_code = "deny"` in workspace `Cargo.toml`; verified by CI | @@ -285,7 +285,7 @@ Quality attributes covered and their relevance: |----|------------|---------| | NFR-MNT-010 | Pre-merge test pass | `cargo nextest run --config-file .github/nextest.toml --workspace --features full --lib --bins` passes with zero failures on every PR | | NFR-MNT-011 | Formatting check | `cargo +nightly fmt --check` passes on every PR | -| NFR-MNT-012 | Lint check | `cargo clippy --all-targets --all-features --workspace -- -D warnings` passes on every PR | +| NFR-MNT-012 | Lint check | `cargo clippy --profile ci --workspace --all-targets --features "desktop,ide,server,chat,pdf,scheduler,testing" -- -D warnings` passes on every PR (`--all-features` is unsupported — `sqlite`/`postgres` are mutually exclusive and trigger `compile_error!`, see [[029-feature-flags/spec#6. NEVER]]) | | NFR-MNT-013 | Doc-test coverage | `cargo test --doc --workspace --features "desktop,ide,server,chat,pdf,scheduler"` passes for all touched crates | | NFR-MNT-014 | LLM serialization gate | Any PR touching LLM request/response serialization paths requires a live API session test with debug dump verification | | NFR-MNT-015 | Integration tests | `cargo nextest run -- --ignored` covers Qdrant-dependent tests; run in CI with Qdrant service | diff --git a/specs/README.md b/specs/README.md index aced65c62..ed9513520 100644 --- a/specs/README.md +++ b/specs/README.md @@ -89,7 +89,10 @@ Spec IDs (001–069) follow a logical grouping: | `004-memory/004-13-memory-memcot.md` | MemCoT: SemanticStateAccumulator, Zoom-In evidence localization, Zoom-Out causal expansion (#3592) | `zeph-memory` | | `004-memory/004-14-memory-tiering-rfc-decision.md` | RFC #4217 decision: memory tiering architecture analysis (MEMTIER, BudgetMem, Multi-Layer, LCM, MemRouter); adopt frequency signal + tier-aware gating + cost-aware routing (#4217) | `zeph-memory` | | `004-memory/004-15-memory-skill-coevolution-rfc-decision.md` | RFC #4218 decision: memory–skill coevolution analysis (MemQ, δ-mem, EvolveMem, SAGE-GraphMem, NanoResearch, Cognifold); adopt Cognifold idle-time folding + EvolveMem feedback routing; defer MemQ to P3 (#4218) | `zeph-memory`, `zeph-skills` | -| `004-memory/004-16-memory-type-aware-retrieval.md` | MemGuard type-aware retrieval composition: `FunctionalType` enum (episodic/user-fact/behavioral-rule/reasoning-strategy/cross-session-summary/graph-fact), opt-in fetch-time gate on `schedule_context_fetchers`, intent-scoped widening via existing `HeuristicRouter` (no new LLM call), `BehavioralRule` always-composed safety invariant; retrieval-only, byte-for-byte no-op when disabled (#6086, #6226) | `zeph-common`, `zeph-config`, `zeph-context`, `zeph-agent-context`, `zeph-memory` | +| `004-memory/004-16-memory-type-aware-retrieval.md` | MemGuard type-aware retrieval composition: `FunctionalType` enum (episodic/user-fact/behavioral-rule/reasoning-strategy/cross-session-summary/graph-fact), opt-in fetch-time gate on `schedule_context_fetchers`, intent-scoped widening via existing `HeuristicRouter` (no new LLM call), `BehavioralRule` always-composed safety invariant; retrieval-only, byte-for-byte no-op when disabled (#6086, #6226). Note: this file intentionally reuses the `004-16` slot already used by `004-16-memory-type-aware-retrieval.md`'s sibling `004-16-shadow-memory-safety.md` below — see that file's own note; in-code rustdoc citations already reference "spec 004-16" (issue #6308), so renumbering requires a coordinated source-code + spec change, not a docs-only fix | `zeph-common`, `zeph-config`, `zeph-context`, `zeph-agent-context`, `zeph-memory` | +| `004-memory/004-16-shadow-memory-safety.md` | Shadow Memory Safety: `TrajectoryRiskAccumulator` MAGE multi-turn goal-hijacking detection via accumulating risk scores, `ShadowMemory`/`GoalDriftResult`; SafeHarbor hierarchical guardrail tree (aspirational, not yet implemented); GitHub #3695 | `zeph-sanitizer`, `zeph-memory`, `zeph-core` | +| `004-memory/004-17-implicit-conflict-detection.md` | Implicit Conflict Detection (STALE/CUPMem): write-time `ImplicitConflictDetector` (Levenshtein + embedding similarity fuzzy predicate matching), propagation-aware SYNAPSE recall, `implicit_conflict_candidates` staging table (migration 090); GitHub #3702 | `zeph-memory` | +| `004-memory/004-18-five-signal-retrieval.md` | Five-Signal Retrieval (MemTier): access frequency, causal distance, novelty, recency, and goal-relevance signals composed into retrieval ranking + async consolidation daemon; migration 091; GitHub #3703 | `zeph-memory`, `zeph-scheduler` | | `005-skills/spec.md` | SKILL.md format, registry, matching, hot-reload, skill trust governance, two-stage matching, Wilson score confidence intervals, hub install pipeline, agent-invocable skills (`invoke_skill`), recursive WalkDir discovery (max depth 16), `SkillExtensions` manifest parser, concurrent semantic scan (`buffer_unordered(4)`, 300s timeout), skill egress attribution in `ToolCall`/`AuditEntry`/`EgressEvent` | `zeph-skills` | | `006-tools/spec.md` | ToolExecutor, CompositeExecutor, TAFC, schema filter, result cache, dependency graph, tool invocation phase taxonomy, native `tool_use` only; `invoke_skill`/`load_skill` utility-gate exemption | `zeph-tools` | | `006-tools/006-1-web-search.md` | Native `web_search` tool: `SearchProvider` trait + `SearchBackend` enum dispatch (v1: `BraveSearchProvider`), query-in/ranked-results-out, runtime-gated (no cargo feature) on `enabled && SearchBackend::from_config().is_ok()`; sanitizer trust bridge fix (`web_search` added to the `web_scrape`/`fetch` tool-name branch in `sanitize.rs:88` → `ExternalUntrusted`+quarantine, NOT via `ClaimSource`), SSRF addr-pinning via `resolve_to_addrs` (mirrors `scrape.rs:226-231`) with resolved-address-set client caching (order-independent, #6457), mandatory `EgressEvent` per `010-5` now carrying the real backend status on a 429 block (#6457), vault-only API key, 429→`ToolError::Blocked` non-retried, documented v1 rank/SEO-poisoning limitation; GitHub #6358 [draft] | `zeph-tools`, `zeph-config`, `zeph-core` | @@ -118,11 +121,11 @@ Spec IDs (001–069) follow a logical grouping: | `022-config-simplification/spec.md` | Provider Registry Architecture: canonical `[[llm.providers]]` format, ProviderEntry schema, routing strategies, LinUCB bandit routing, cost-weight dial, memory-augmented routing | `zeph-config`, `zeph-core` | | `023-complexity-triage-routing/spec.md` | Pre-inference complexity classification routing, ComplexityTier, TriageRouter, context escalation, metrics | `zeph-llm`, `zeph-config`, `zeph-core` | | `024-multi-model-design/spec.md` | Multi-model design principle: complexity tiers, `*_provider` subsystem reference pattern, STT unification | cross-cutting | -| `025-classifiers/spec.md` | Candle-backed ML classifiers: injection detection, PII detection, LlmClassifier for feedback, unified regex+NER sanitization pipeline | `zeph-classifiers` | +| `025-classifiers/spec.md` | Candle-backed ML classifiers: injection detection, PII detection, LlmClassifier for feedback, unified regex+NER sanitization pipeline (lives in `crates/zeph-llm/src/classifier/`, not a standalone crate) | `zeph-llm`, `zeph-sanitizer` | | `026-tui-subagent-management/spec.md` | Interactive TUI subagent sidebar (a key), j/k navigation, Enter loads transcript, Esc returns, Tab cycling; automatic view switch to foreground subagent on spawn + return to Main on completion via `ForegroundSubagentStarted`/`ForegroundSubagentCompleted` `AgentEvent` variants (#3764); opt-in live transcript forwarding into `SubAgentMetrics::live_transcript` ring buffer, layered on top of the truncated-JSONL sidebar view (#6359) | `zeph-tui` | | `027-runtime-layer/spec.md` | RuntimeLayer middleware with before_chat/after_chat/before_tool/after_tool hooks, NoopLayer, LayerContext, unwind guards; plugin config overlay merge (tighten-only) | `zeph-core` | | `028-hooks/spec.md` | Reactive hooks: cwd_changed / file_changed / permission_denied / turn_complete / pre_tool_use / post_tool_use events, set_working_directory tool, FileChangeWatcher, ZEPH_TOOL_NAME / ZEPH_TOOL_ARGS_JSON / ZEPH_SESSION_ID env vars (#3725); pre_tool_use fires before utility gate (#3738); permission_denied fires at all gate denial points in tier loop (#3779); turn_complete McpTool hooks wired to live MCP dispatch (#3776); post_tool_use output replacement + duration_ms (#3998); inline hooks in config.toml (#3885, #4252) | `zeph-core` | -| `029-feature-flags/spec.md` | Feature flag decision rules, surviving flag inventory (22 flags), bundle definitions (desktop, ide, server, full) | `Cargo.toml`, cross-cutting | +| `029-feature-flags/spec.md` | Feature flag decision rules, flag inventory, bundle definitions (desktop, ide, server, full) | `Cargo.toml`, cross-cutting | | `030-tui-slash-autocomplete/spec.md` | Inline autocomplete dropdown in TUI Insert mode, reuses filter_commands registry, Tab/Enter accepts, Esc dismisses | `zeph-tui` | | `tui-reducer/spec.md` | TUI reducer/action decomposition (#5076) — `Action` + `Effect` enums, `reduce(&mut App, Action) -> Vec` as sole key/mouse state-mutation site, handlers become decoders; opt-in mouse mode (#5103) — `[tui] mouse` config, `/mouse [on\|off]`, runtime `EnableMouseCapture`↔`DisableAlternateScroll` swap, `last_layout` hit-testing, OSC8 click fallback | `zeph-tui`, `zeph-config` | | `031-database-abstraction/spec.md` | PostgreSQL backend, zeph-db crate, DatabaseDriver trait, Dialect trait, sql!() macro, migrations, CLI subcommands | `zeph-db`, cross-cutting | diff --git a/specs/SRS.md b/specs/SRS.md index ab8fee2d9..f241fb1db 100644 --- a/specs/SRS.md +++ b/specs/SRS.md @@ -216,11 +216,11 @@ Major functional areas: ### 2.5 Design and Implementation Constraints -- Rust 1.94 (MSRV), Edition 2024; `unsafe_code = "deny"` workspace-wide. +- Rust 1.97 (MSRV), Edition 2024; `unsafe_code = "deny"` workspace-wide. - Async: tokio; no `async-trait` crate in library crates. - TLS: rustls; `openssl-sys` banned. - Crate layering: `zeph-core` orchestrates all leaf crates; same-layer imports prohibited. -- Feature flags: `default = []`; bundles (`desktop`, `ide`, `server`, `full`) for CI. +- Feature flags: `default = ["scheduler", "sqlite"]`; bundles (`desktop`, `ide`, `server`, `full`) for CI. - Binary size: release binary ≤ 15 MiB. - No blocking I/O in async hot paths. @@ -1096,7 +1096,7 @@ the TOML, THE SYSTEM SHALL produce sensible defaults and MUST NOT panic. without Qdrant or MCP servers. - **Security**: no plaintext secrets; blocklist gate before permission policy; SSRF protection; PII redaction. -- **Maintainability**: 24-crate workspace with strict layered DAG; no same-layer +- **Maintainability**: 32-crate workspace with strict layered DAG; no same-layer imports; comprehensive rustdoc for all `pub` items. --- diff --git a/specs/TEMPLATE.md b/specs/TEMPLATE.md index dc4557188..832b5a94b 100644 --- a/specs/TEMPLATE.md +++ b/specs/TEMPLATE.md @@ -1,6 +1,8 @@ # Spec Template -Use this template when creating new specifications in `.local/specs/NNN-feature-name/spec.md`. +Use this template when creating new specifications in `/specs/NNN-feature-name/spec.md`. Per +`CLAUDE.md`, this project's specs are permanent and live at the repository-root `/specs/`, not the +sdd skill's default `.local/specs/` scratch location. --- @@ -17,7 +19,7 @@ tags: - [domain: core|llm|memory|skills|tools|channels|tui|mcp|security|orchestration|protocols|config|database|benchmarking] - [optional: cross-cutting|infra|contract|experimental] created: [YYYY-MM-DD] -status: [draft|approved|deprecated] +status: [draft|approved|implemented|complete|deprecated|research] related: - "[[MOC-specs]]" - "[[001-system-invariants/spec]]" diff --git a/specs/UX/mention-routing.md b/specs/UX/mention-routing.md index 746f827bc..cd41784c7 100644 --- a/specs/UX/mention-routing.md +++ b/specs/UX/mention-routing.md @@ -11,10 +11,10 @@ tags: created: 2026-04-24 status: research related: - - "[[028-hooks/spec.md]]" - - "[[014-a2a/spec.md]]" - - "[[013-acp/spec.md]]" - - "[[011-tui/spec.md]]" + - "[[028-hooks/spec]]" + - "[[014-a2a/spec]]" + - "[[013-acp/spec]]" + - "[[011-tui/spec]]" --- # @agent Mention Routing — Research Spec (#3327) diff --git a/specs/constitution.md b/specs/constitution.md index dcbb822cc..b534dbc13 100644 --- a/specs/constitution.md +++ b/specs/constitution.md @@ -21,7 +21,7 @@ related: ## I. Architecture -- Cargo workspace (Edition 2024, resolver 3): 29 crates (including `zeph-common`, `zeph-commands`, `zeph-context`), root binary `zeph` +- Cargo workspace (Edition 2024, resolver 3): 32 crates (including `zeph-common`, `zeph-commands`, `zeph-context`), root binary `zeph` - `zeph-core` orchestrates all crates. Crate dependencies must follow the layered DAG: - **Layer 0a** (zero zeph-* deps): `zeph-common`, `zeph-commands` - **Layer 0b** (depends on L0a only): `zeph-config`, `zeph-vault`, `zeph-db` diff --git a/specs/tui-reducer/spec.md b/specs/tui-reducer/spec.md index 9b704bba7..cdf9f6260 100644 --- a/specs/tui-reducer/spec.md +++ b/specs/tui-reducer/spec.md @@ -16,8 +16,8 @@ related: - "[[MOC-specs]]" - "[[001-system-invariants/spec]]" - "[[011-tui/spec]]" - - "[[027-tui-subagent-management/spec]]" - - "[[031-tui-slash-autocomplete/spec]]" + - "[[026-tui-subagent-management/spec]]" + - "[[030-tui-slash-autocomplete/spec]]" --- # Spec: TUI Reducer / Action Decomposition + Opt-in Mouse Mode