From 5500aedf365804485fb735d5711f0e331c47c240 Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:26:56 +0200 Subject: [PATCH 01/10] docs(specs): rewrite 010-security & 008-mcp sub-specs with real types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace fictional API types with actual codebase implementations: - VaultProvider, AgeVaultProvider, EnvVaultProvider (010-1-vault) - ContentSanitizer, CrossToolCorrelator, TrajectorySentinel (010-2-injection-defense) - PolicyEnforcer, shell blocklist validation (010-3-authorization) - ServerEntry, McpTransport, McpManager (008-1-lifecycle) - ToolCollision, PruningCache, SemanticToolIndex (008-2-discovery) - SanitizeResult, DefaultMcpProber, trust scoring (008-3-security) - AuditEntry, AuditSignal, AuditSignalType, Severity (010-4-audit) Fix critical factual error in 010-4-audit: TrajectoryRiskAccumulator maintains cross-turn, exponentially-decaying risk scores and makes hard-blocking tool-execution decisions. Add warning callout flagging unresolved architectural gap (system not addressed by parent spec's NEVER rule on cross-turn accumulation). Remove [!danger] callouts from all files — content is now accurate. Closes #6631 Closes #6630 --- specs/008-mcp/008-1-lifecycle.md | 324 +++++--------- specs/008-mcp/008-2-discovery.md | 302 ++++++------- specs/008-mcp/008-3-security.md | 423 ++++++++---------- specs/010-security/010-1-vault.md | 225 ++++------ specs/010-security/010-2-injection-defense.md | 351 ++++++--------- specs/010-security/010-3-authorization.md | 311 +++++-------- specs/010-security/010-4-audit.md | 407 +++++------------ 7 files changed, 912 insertions(+), 1431 deletions(-) diff --git a/specs/008-mcp/008-1-lifecycle.md b/specs/008-mcp/008-1-lifecycle.md index 6a74772a8..7d8171159 100644 --- a/specs/008-mcp/008-1-lifecycle.md +++ b/specs/008-mcp/008-1-lifecycle.md @@ -19,268 +19,160 @@ related: # Spec: MCP Server Lifecycle -> [!danger] Stale content — 2026-07 audit -> The "Startup Auto-Retry with Exponential Backoff (#3578)" section and most of the pseudocode below -> (`McpConnection`, `JsonRpcTransport`, `Waiter`, `log::warn!`/`log::info!` calls) do not match the -> real implementation and were not corrected in this pass (rewrite, not a text fix). Real code: -> `crates/zeph-mcp/src/manager/{mod.rs,builder.rs,connect.rs,server.rs,retry.rs}`. Real retry config -> fields are `startup_retry_backoff_ms`/`tool_timeout_secs` (see `[[008-mcp/spec]]`), not -> `startup_retry_base_delay_ms`/`startup_retry_max_delay_ms`/`startup_retry_backoff_factor`/ -> `max_startup_retries` as stated below. `log::*` macros are banned project-wide (`tracing` only, -> see `[[constitution]]` II/IV) — do not copy the logging calls in this file's pseudocode. - Server startup/shutdown, connection management, stdio environment isolation, graceful cleanup. ## Overview -MCP servers are subprocess-based tool providers. Zeph manages their complete lifecycle: spawning with environment isolation, maintaining connections, detecting failures, and graceful shutdown. +MCP servers are subprocess-based or HTTP tool providers. Zeph manages their complete lifecycle: spawning with environment isolation, maintaining connections, detecting failures, and graceful shutdown. The `McpManager` (`crates/zeph-mcp/src/manager/`) orchestrates multi-server lifecycle. ## Key Invariants **Always:** -- Each MCP server runs in isolated subprocess with env vars scrubbed -- Server startup failures logged with full context (stderr, exit code, timeout) -- Connections are bidirectional: Zeph sends requests, servers send notifications -- Server shutdown waits for pending requests (configurable timeout: default 5s) +- Each stdio MCP server runs in isolated subprocess with `ZEPH_*` secrets scrubbed from environment +- Server startup failures logged with stderr/exit code +- Server connections are bidirectional: Zeph sends requests, servers send notifications (`tools/list_changed`, etc.) +- Server shutdown waits for pending requests (timeout: configurable via `shutdown_timeout_secs`) +- Env var scrubbing happens unconditionally before subprocess spawn **Never:** -- Pass secrets (API keys, tokens) to MCP server environment -- Leave zombie processes on exit (always await subprocess termination) -- Assume server is healthy without heartbeat validation +- Pass `ZEPH_*` secrets to MCP server environment — they are already scrubbed +- Leave zombie processes on shutdown — always collect process status +- Assume server is healthy without probing — initial `tools/list` serves as health check +- Send requests to a server while it is shutting down ## Startup Sequence ``` -1. Resolve server config (name, command, env, stdio mode) -2. Scrub environment: remove ZEPH_* secrets, keep only safe vars -3. Spawn subprocess with stdio isolation -4. Send initialize request, await response (timeout: 10s) -5. Register tool registry, store connection metadata -6. Mark server as "ready" in MCP client state +1. Validate transport config (stdio command, HTTP URL, or OAuth credentials) +2. Scrub environment: remove ZEPH_*, AWS_*, GITHUB_TOKEN, etc. +3. Spawn subprocess (stdio) or establish HTTP connection (http/oauth) +4. Send initialize request via rmcp crate, await response (timeout: 10s) +5. Fetch tools/list via DefaultMcpProber for pre-connect injection scan +6. Sanitize tool definitions (names, descriptions, input schemas) +7. Register tools in `McpToolRegistry` with trust scores +8. Mark server as "connected" in manager state ``` -Code sketch: +API: ```rust -async fn start_server(&self, config: &ServerConfig) -> Result { - // 1. Sanitize environment - let env = self.scrub_secrets(&config.env); - - // 2. Spawn process - let mut child = Command::new(&config.command) - .env_clear() - .envs(&env) - .stdin(Stdio::piped()) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .spawn() - .context("Failed to spawn MCP server")?; - - let stdin = child.stdin.take().unwrap(); - let stdout = child.stdout.take().unwrap(); - - // 3. Initialize protocol (exchange capabilities) - let conn = JsonRpcTransport::new(stdin, stdout); - let caps = conn.initialize( - &InitializeRequest { - protocol_version: "2024-11-05", - capabilities: /* client capabilities */, - }, - Duration::from_secs(10), - ).await?; - - // 4. Store connection - let server_id = self.store_connection(config.name.clone(), conn).await?; +pub struct ServerEntry { + pub id: String, // Unique server ID + pub transport: McpTransport, // Stdio/Http/OAuth variant + pub timeout: Duration, // Request timeout + pub trust_level: McpTrustLevel, // Trusted/Untrusted/Sandboxed + pub tool_allowlist: Option>, // Optional tool allowlist + pub allow_untrusted_without_allowlist: bool, // Secure default: false + pub expected_tools: Vec, // Attestation list + pub roots: Vec, // Root resources for file servers + pub tool_metadata: HashMap, // Pre-defined tool metadata + pub elicitation_enabled: bool, // Tool parameter probing + pub elicitation_timeout_secs: u64, // Probing timeout + pub env_isolation: bool, // Scrub env vars (default: true) + pub media_passthrough: bool, // Allow media types in responses +} + +pub enum McpTransport { + Stdio { + command: String, + args: Vec, + env: HashMap, + }, + Http { + url: String, + headers: HashMap, + }, + OAuth { + auth_uri: String, + token_uri: String, + client_id: String, + client_secret: Secret, + }, +} + +impl McpManager { + /// Connect all configured servers in parallel. + pub async fn connect_all(&self) -> (Vec, Vec); - Ok(server_id) + /// Call a tool on a specific server. + pub async fn call_tool( + &self, + server_id: &str, + tool_name: &str, + arguments: serde_json::Value, + ) -> Result; } ``` -## Connection Management +## Connection Maintenance -Maintain bidirectional messaging: +`McpClient` (via rmcp crate) maintains bidirectional messaging: -```rust -pub struct McpConnection { - id: String, - name: String, - transport: JsonRpcTransport, - pending_requests: Arc>>, - server_notifications: Arc>>, - health_check_interval: Duration, -} +- **Requests**: Zeph sends tool calls, resource reads, prompt queries +- **Notifications**: Server sends `tools/list_changed`, `resources/list_changed`, etc. +- **Responses**: Async correlation by request ID; timeout triggers error +- **Errors**: JSON-RPC errors forwarded as `McpError` -impl McpConnection { - async fn send_request(&self, method: &str, params: Value) -> Result { - let id = self.next_request_id(); - let waiter = Waiter::new(); - - self.pending_requests.lock().insert(id, waiter.clone()); - self.transport.send_request(id, method, params).await?; - - // Wait for response with timeout - waiter.wait(Duration::from_secs(30)).await - } - - async fn handle_notification(&self, notif: Notification) { - // Server can send unsolicited notifications (e.g., resource changed) - self.server_notifications.lock().push_back(notif); - } -} +Config: +```toml +[[mcp.servers]] +id = "filesystem" +transport = { type = "stdio", command = "npx", args = ["-y", "@modelcontextprotocol/server-filesystem"] } +timeout_secs = 30 +trust_level = "untrusted" +tool_allowlist = ["read_file", "list_directory"] ``` ## Failure Detection & Reconnection -Monitor server health: +Initial connection failures are logged with stderr; transient failures during tool calls trigger retry logic (via rmcp's error handling and `McpToolExecutor` retry). Long-lived servers are not actively health-checked (only re-validated if a notification arrives). -```rust -async fn health_check_loop(&self, conn: &McpConnection) { - let mut interval = tokio::time::interval( - conn.health_check_interval - ); - - loop { - interval.tick().await; - - match conn.send_request("ping", json!({})).await { - Ok(_) => { - conn.mark_healthy(); - } - Err(e) => { - log::warn!("Server {} health check failed: {}", conn.name, e); - self.mark_unhealthy(&conn.id).await; - - // Trigger reconnect logic - self.attempt_reconnect(&conn.id).await; - } - } - } -} -``` +Failed servers are marked `Error` in `ServerConnectOutcome` and their tools are unavailable for dispatch. ## Graceful Shutdown -Cleanup on agent termination: +Cleanup on agent termination (via `TaskSupervisor::shutdown_all`): ```rust -async fn shutdown_server(&self, server_id: &str) -> Result<()> { - let conn = self.get_connection(server_id)?; - - // 1. Reject new requests - conn.mark_shutdown(); - - // 2. Wait for pending requests (timeout: 5s) - let timeout = Duration::from_secs(5); - match timeout_at( - Instant::now() + timeout, - self.wait_pending_requests(server_id), - ).await { - Ok(Ok(())) => log::info!("Server {} graceful shutdown", server_id), - Ok(Err(e)) => log::warn!("Server {} shutdown error: {}", server_id, e), - Err(_) => log::warn!("Server {} shutdown timeout", server_id), +impl McpManager { + /// Shutdown all servers gracefully. + pub async fn shutdown_all(&self, timeout: Duration) -> Result<()> { + for server in &self.servers { + // Cancel pending requests + // Send shutdown signal if server supports it + // Close stdio handles or HTTP connections + // Wait for graceful close or force-terminate + } } - - // 3. Terminate subprocess - self.kill_subprocess(server_id).await?; - - // 4. Remove from registry - self.remove_connection(server_id).await; - - Ok(()) } ``` -## Startup Auto-Retry with Exponential Backoff (#3578) - -MCP server startup is unreliable in practice: a server process may crash before -completing the `initialize` handshake, or a network MCP server may be temporarily -unavailable at agent start time. Without retry, a single failed server blocks agent -startup or silently reduces the tool catalog. - -### Retry Contract - -`McpManager::start_with_retry(config)` wraps `start_server()` in an exponential -backoff loop: - -``` -attempt 1: immediate -attempt 2: base_delay_ms (default 200 ms) -attempt 3: base_delay_ms × backoff_factor (default 2.0) -... -attempt N: min(base_delay_ms × backoff_factor^(N-2), max_delay_ms) -``` - -On exhaustion (all `max_startup_retries` attempts failed): - -- **`critical = false` servers**: log `ERROR`, skip server, agent starts without it. - The missing server's tools are absent from the catalog until a `/mcp reconnect` command. -- **`critical = true` servers**: return `Err(McpError::CriticalServerStartFailed)`, - aborting agent startup. - -### Jitter +Timeout: configurable per agent (default 5s). If shutdown exceeds timeout, remaining servers are force-closed without waiting for pending requests. -Each backoff delay is jittered by `±25%` (uniform random) to prevent thundering herds -when multiple MCP servers restart simultaneously after a crash. +## Tool Registry Coordination -### Tracing +`McpToolRegistry` (`crates/zeph-mcp/src/registry.rs`) indexes tools by server. When a server notifies `tools/list_changed`, the registry re-fetches and updates its index: -Each retry attempt emits a `tracing::warn!` with attempt number, server name, and -error. The initial failure emits `tracing::info!` (not warn — first attempt failure is -expected in slow-start environments). - -### Config - -```toml -[[mcp.servers]] -name = "local-tools" -command = "python3 /path/to/server.py" -stdio = "pipe" # or "pty" for terminal emulation -timeout_init_s = 10 -timeout_request_s = 30 -healthcheck_interval_s = 60 -critical = false # if true, startup failure aborts the agent -max_startup_retries = 3 # total attempts (1 initial + N-1 retries); 0 = no retry -startup_retry_base_delay_ms = 200 # base delay before first retry -startup_retry_max_delay_ms = 5000 # cap on exponential backoff -startup_retry_backoff_factor = 2.0 # multiplier applied per attempt - -# Environment scrubbing: keep only these vars -allow_env_vars = ["PATH", "HOME", "RUST_LOG"] -``` - -### Key Invariants - -- Retry delay is bounded by `startup_retry_max_delay_ms` — backoff cannot grow unbounded -- `critical = true` servers abort startup on first failure (no retry is attempted before aborting) - — override: set `max_startup_retries > 0` to retry even critical servers before aborting -- NEVER silently swallow a critical server failure — `Err` must propagate to `McpManager::start_all` -- Jitter is applied on retries only, not on the initial attempt -- The TUI startup spinner shows per-server retry status when `max_startup_retries > 0` - -## Configuration (Legacy) - -```toml -[[mcp.servers]] -name = "local-tools" -command = "python3 /path/to/server.py" -stdio = "pipe" # or "pty" for terminal emulation -timeout_init_s = 10 -timeout_request_s = 30 -healthcheck_interval_s = 60 - -# Environment scrubbing: keep only these vars -allow_env_vars = ["PATH", "HOME", "RUST_LOG"] +```rust +impl ToolListChangedHandler { + pub async fn on_tools_list_changed(&self, server_id: &str) -> Result<()> { + // 1. Fetch updated tools/list from server + // 2. Sanitize tool definitions + // 3. Update registry index + // 4. Fire MCP client callback if attached + } +} ``` -## Integration Points +## Transport Security -- [[002-agent-loop/spec]] — MCP servers initialized during agent startup -- [[008-2-discovery]] — Server capabilities discovered after initialization -- [[008-3-security]] — Environment scrubbing occurs before spawn -- [[010-security/spec]] — Subprocess isolation enforced here +- **Stdio**: inherited from parent environment (scrubbed of secrets); runs with same UID as agent +- **HTTP**: URL resolved via `validate_url_ssrf()` — private IPs blocked unless allowlisted +- **OAuth**: authorization endpoints validated via `validate_oauth_metadata_urls()` against SSRF blocklist ## See Also - [[008-mcp/spec]] — Parent -- [[008-2-discovery]] — Tool discovery after lifecycle setup -- [[008-3-security]] — Security constraints on subprocess spawning +- [[008-2-discovery]] — Tool discovery and semantic indexing +- [[008-3-security]] — Tool sanitization and policy enforcement +- [[010-3-authorization]] — SSRF protection diff --git a/specs/008-mcp/008-2-discovery.md b/specs/008-mcp/008-2-discovery.md index d5202d441..0a136ba3e 100644 --- a/specs/008-mcp/008-2-discovery.md +++ b/specs/008-mcp/008-2-discovery.md @@ -21,220 +21,190 @@ related: # Spec: MCP Tool Discovery & Pruning -> [!danger] Stale content — 2026-07 audit -> The "Per-Message Tool Pruning" (`ToolPruningCache`) and "Tool Embedding Precomputation" sections -> below do not match the real implementation and were not corrected in this pass (rewrite, not a -> text fix). Real pruning cache is `PruningCache` (`crates/zeph-mcp/src/pruning.rs`) — a single-slot -> cache keyed on `(message_content_hash, tool_list_hash)` that selects tools via an **LLM call**, not -> embedding/dot-product ranking as described below. Real collision detection is -> `detect_collisions::` / `ToolCollision` (`crates/zeph-mcp/src/tool.rs`) plus -> `log_tool_collisions` (`crates/zeph-mcp/src/manager/connect.rs`). `log::warn!` calls in the -> pseudocode below are banned project-wide (`tracing` only, see `[[constitution]]` II/IV). -> The `[mcp.discovery]` config block (`pruning_enabled`/`top_k_tools`/`cache_size`/`collision_mode`) -> is unverified and likely stale given the cache design mismatch above. - Semantic tool discovery, per-message pruning cache, collision detection, tool filtering. ## Overview -MCP servers expose hundreds of tools across multiple categories. Zeph discovers these at startup and applies semantic filtering per-message to reduce token overhead and prevent tool confusion. +MCP servers expose hundreds of tools across multiple categories. Zeph discovers these at startup via `tools/list`, applies sanitization, and selects relevant tools per-message to reduce token overhead and prevent tool confusion. ## Key Invariants **Always:** - All server tools registered with full schema (name, description, input_schema) -- Tool collision detection triggers on registration: same name from multiple servers -- Per-message tool set pruned based on context relevance (embedding similarity) -- Tool discovery errors are non-fatal: server remains active even if tool listing fails +- Tool collision detection triggers on registration: log and disambiguate same names from multiple servers +- Per-message tool set pruned via LLM call to reduce token overhead +- Tool list cache invalidated on server reconnection +- Pruning cache is single-slot: keyed on `(message_hash, tool_list_hash)` **Never:** -- Include duplicate tool names (same name from multiple servers) without disambiguation -- Pass full tool registry to LLM (always apply semantic pruning) -- Cache tool descriptions without versioning (invalidate on server restart) +- Pass full tool registry to LLM without semantic pruning +- Serve tools from collided names without disambiguation (e.g., "tool_x" from two servers ambiguous) +- Cache tool definitions after server restart without re-fetching ## Tool Registration & Collision Detection -At server startup, fetch and register tools: +At server startup, fetch and register tools via `tools/list`: ```rust -async fn discover_tools(&self, server: &McpServer) -> Result> { - // 1. Request tool list from server - let tools = server.connection.list_tools().await?; - - // 2. Validate and enrich schemas - let mut discovered = Vec::new(); - for tool in tools { - match validate_json_schema(&tool.input_schema) { - Ok(schema) => { - let enriched = ToolDefinition { - id: format!("{}:{}", server.name, tool.name), - name: tool.name.clone(), - description: tool.description, - input_schema: schema, - server_id: server.id.clone(), - }; - discovered.push(enriched); - } - Err(e) => { - log::warn!("Invalid tool schema for {}.{}: {}", server.name, tool.name, e); - // Continue; don't fail entire discovery - } - } - } - - // 3. Detect collisions - self.detect_collisions(&discovered)?; - - Ok(discovered) +pub struct ToolCollision { + pub name: String, // Tool name + pub servers: Vec, // Server IDs that define it } -fn detect_collisions(&self, tools: &[ToolDefinition]) -> Result<()> { - let mut by_name: HashMap<&str, Vec<&ToolDefinition>> = HashMap::new(); - - for tool in tools { - by_name.entry(&tool.name) - .or_insert_with(Vec::new) - .push(tool); - } - - for (name, defs) in by_name { - if defs.len() > 1 { - log::warn!( - "Tool name collision: '{}' defined in {}", - name, - defs.iter() - .map(|d| d.server_id.as_str()) - .collect::>() - .join(", ") - ); - - // Auto-rename to disambiguate: "tool_name" → "server1:tool_name" - // (Handled at invocation time) +impl McpManager { + /// Register tools from a server; detect collisions. + pub async fn register_tools(&self, server_id: &str) -> Result> { + let tools = /* fetch via tools/list */; + + let collisions = detect_collisions::( + self.tool_registry.all_tools(), + &tools, + ); + + if !collisions.is_empty() { + tracing::warn!("Tool collisions detected: {:?}", collisions); } + + self.tool_registry.add_tools(&tools); + Ok(collisions) } - - Ok(()) +} + +/// Detect tool name collisions. +pub fn detect_collisions( + existing: &[McpTool], + new_tools: &[McpTool], +) -> Vec { + // Returns list of name collisions across existing + new tools } ``` -## Per-Message Tool Pruning +Collision handling: +- Collisions are logged but do NOT block registration +- Disambiguation happens at tool invocation time via server prefix (e.g., `filesystem:read_file` vs `http:read_file`) +- Attestation (if `expected_tools` list provided) detects unexpected schema drifts + +## Per-Message Tool Pruning Cache -Semantic filtering reduces LLM token overhead: +`PruningCache` (`crates/zeph-mcp/src/pruning.rs`) selects relevant tools per LLM turn: ```rust -pub struct ToolPruningCache { - // Query embedding → recommended tool IDs (cached) - cache: Arc>>>, - embedding_provider: Arc, - tool_embeddings: Arc>>, // cached at startup - top_k: usize, // default: 10 +pub struct PruningCache { /* ... */ } + +pub struct PruningParams { + pub message_content_hash: u64, // Hash of LLM message context + pub tool_list_hash: u64, // Hash of available tools + pub strategy: ToolDiscoveryStrategy, // LLM-based or semantic } -impl ToolPruningCache { - async fn prune_tools_for_context( - &self, - context: &str, // user query + recent history - all_tools: &[ToolDefinition], - ) -> Result> { - // 1. Generate query embedding - let query_embedding = self.embedding_provider - .embed_text(context) - .await?; - - // 2. Check cache - let cache_key = format!("{:?}", &query_embedding[..10]); // simple hash - if let Some(cached) = self.cache.lock().get(&cache_key) { - let pruned: Vec<_> = cached.iter() - .filter_map(|id| all_tools.iter().find(|t| &t.id == id)) - .collect(); - return Ok(pruned); - } - - // 3. Semantic similarity: dot product with tool embeddings - let mut scores: Vec<(usize, f32)> = all_tools - .iter() - .enumerate() - .map(|(i, tool)| { - let embedding = self.tool_embeddings.get(&tool.id) - .map(|e| dot_product(&query_embedding, e)) - .unwrap_or(0.0); - (i, embedding) - }) - .collect(); - - // 4. Select top-K by score - scores.sort_by(|a, b| b.1.partial_cmp(&a.1).unwrap()); - let selected: Vec = scores - .into_iter() - .take(self.top_k) - .map(|(i, _)| all_tools[i].id.clone()) - .collect(); - - // 5. Cache result - self.cache.lock().put(cache_key, selected.clone()); +impl PruningCache { + /// Get relevant tools for this turn (LLM-selected). + pub async fn get_tools_for_turn( + &mut self, + params: &PruningParams, + all_tools: &[McpTool], + ) -> Result> { + // Single-slot cache: if (message_hash, tool_list_hash) matches cached entry, return cached tools + // Otherwise: call LLM to select relevant tools, cache result, return - let pruned: Vec<_> = selected.iter() - .filter_map(|id| all_tools.iter().find(|t| &t.id == id)) - .collect(); - - Ok(pruned) + // Cache invocation: ask LLM "which of these tools are relevant to the user's query?" + // Reduces token overhead: full schema of ~100 tools down to ~10 relevant ones } } ``` -## Tool Embedding Precomputation +Cache semantics: +- **Single-slot**: only one `(message_hash, tool_list_hash)` pair cached at a time +- **Keyed on message+tools**: if either message content or tool catalog changes, cache miss +- **Populated via LLM**: `ToolDiscoveryStrategy::Llm` calls an LLM to rank tools by relevance +- **Cleared on server reconnection**: tool list hash changes, cache invalidates automatically + +Config: +```toml +[mcp] +pruning_enabled = true # Enable per-message tool pruning +pruning_strategy = "llm" # Or "semantic" (embedding-based) +max_tools_per_turn = 10 # Max tools returned by pruning +``` + +## Semantic Tool Indexing (Optional) -Cache at startup for fast lookup: +`SemanticToolIndex` (`crates/zeph-mcp/src/semantic_index.rs`) provides embedding-based ranking as an alternative to LLM-based pruning: ```rust -async fn precompute_tool_embeddings( - tools: &[ToolDefinition], - provider: &LlmProvider, -) -> Result>> { - let mut embeddings = HashMap::new(); - - // Embed tool description + input schema summary - for tool in tools { - let text = format!( - "Tool: {}\nDescription: {}\nInputs: {:?}", - tool.name, - tool.description, - tool.input_schema.properties.keys().collect::>(), - ); - - let embedding = provider.embed_text(&text).await?; - embeddings.insert(tool.id.clone(), embedding); +pub enum ToolDiscoveryStrategy { + Llm, // Ask LLM which tools are relevant (default) + Semantic, // Use cosine similarity on embeddings +} + +pub struct SemanticToolIndex { /* ... */ } + +impl SemanticToolIndex { + /// Rank tools by semantic similarity to a query. + pub fn search_tools( + &self, + query: &str, + all_tools: &[McpTool], + ) -> Result> { + // Compute embedding for query + // Compute cosine similarity with each tool's embedding + // Return sorted by relevance score } - - Ok(embeddings) } ``` -## Configuration +Embeddings are computed once per tool at startup and cached in memory. No external index required. -```toml -[mcp.discovery] -enabled = true +## Tool Attestation + +When `expected_tools` list is declared in server config, the registry validates against schema drift: + +```rust +pub struct AttestationResult { + pub missing: Vec, // Tools in expected_tools not found + pub unexpected: Vec, // Tools found but not in expected_tools + pub schema_changed: Vec, // Tools present but schema differs +} -# Per-message pruning -pruning_enabled = true -top_k_tools = 10 # keep top 10 most relevant -cache_size = 1000 # LRU cache entries +pub async fn attest_tools( + server_id: &str, + expected: &[String], + actual: &[McpTool], +) -> Result { + // Compare tool set and schemas; report mismatches +} +``` + +Attestation warnings are logged but do NOT block tool usage. This helps detect when a server's tool catalog unexpectedly changes between deployments. -# Collision handling -collision_mode = "disambiguate" # or "error", "first" +## Tool Fingerprinting + +`ToolFingerprint` computes a content hash for tool definitions to enable change detection: + +```rust +pub struct ToolFingerprint { + pub name: String, + pub schema_hash: u64, // Hash of input_schema JSON + pub description_hash: u64, // Hash of description text +} + +pub fn compute_fingerprint(tool: &McpTool) -> ToolFingerprint { /* ... */ } ``` +Used by attestation and trust scoring to detect parameter/description drift. + +## Tool Sanitization + +All tool definitions (names, descriptions, input schemas) pass through `sanitize_tools()` before registration to scrub injection patterns — see [[008-3-security]]. + ## Integration Points -- [[008-1-lifecycle]] — Tool discovery runs after server startup -- [[008-3-security]] — Pruning cache prevents tool injection via descriptions -- [[006-tools/spec]] — Pruned tool subset sent to ToolExecutor -- [[003-llm-providers/spec]] — Embedding provider used for semantic filtering +- [[008-1-lifecycle]] — Tools fetched during startup; cache invalidated on reconnection +- [[008-3-security]] — Sanitization applied to tool descriptions and parameter schemas +- [[006-tools/spec]] — Tool dispatch via `ToolExecutor` trait ## See Also - [[008-mcp/spec]] — Parent -- [[008-1-lifecycle]] — Server initialization -- [[008-3-security]] — Injection defense during pruning -- [[006-tools/spec]] — Tool execution after pruning +- [[008-1-lifecycle]] — Server connection lifecycle +- [[008-3-security]] — Tool sanitization and security checks diff --git a/specs/008-mcp/008-3-security.md b/specs/008-mcp/008-3-security.md index 6a9af9112..c235b13b6 100644 --- a/specs/008-mcp/008-3-security.md +++ b/specs/008-mcp/008-3-security.md @@ -1,8 +1,9 @@ --- aliases: - MCP Security - - OAP Authorization - - SMCP Secure MCP + - Tool Sanitization + - Trust Scoring + - Data Flow Policy tags: - sdd - spec @@ -20,289 +21,261 @@ related: - "[[010-3-authorization]]" --- -# Spec: MCP Security & OAP Authorization +# Spec: MCP Security & Policy Enforcement -> [!danger] Stale content — 2026-07 audit -> Most of this file (Elicitation Phases pseudocode, `InjectionDetector`/`DeBERTaClassifier`, -> `OAPPolicy`, `SmcpEnvelope`) does not match the real implementation and was not corrected in this -> pass (rewrite, not a text fix) — none of `OAPPolicy`, `SmcpEnvelope`, `InjectionDetector`, -> `DeBERTaClassifier` exist in the codebase. Real analogues: `ContentSanitizer`, -> `ExfiltrationGuard`, `RiskChainAccumulator` (`crates/zeph-sanitizer/`), `CandleClassifier`/ -> `CandlePiiClassifier` (`crates/zeph-llm/src/classifier/`). Elicitation itself is accurately -> documented in [[008-4-elicitation]] — treat that file as authoritative for elicitation, not the -> "Elicitation Phases" section below. The top `## Key Invariants` block (Untrusted/tool_allowlist/ -> allow_untrusted_without_allowlist fail-closed behavior) reads as accurate and should be preserved -> when this file is rewritten. - -Elicitation phases, injection detection, OAP (Org-Aware Permission) authorization, SMCP secure protocol. +Tool sanitization, trust scoring, data-flow policy, elicitation gating, embedding anomaly detection. ## Overview -MCP servers are untrusted code running in subprocesses. Zeph enforces multiple layers of defense: input sanitization, output injection detection, and capability-based authorization via OAP. +MCP servers are untrusted code running in subprocesses or remote HTTP services. Zeph enforces multiple layers of defense before tool calls reach the agent and after results return: + +1. **Pre-connect**: Probing (`DefaultMcpProber`) scans `resources/list` and `prompts/list` for injection +2. **At registration**: Tool definitions sanitized; collision detection runs +3. **Input phase**: Schema validation + policy checks + elicitation probing (optional) +4. **Output phase**: Injection scanning + embedding anomaly detection + PII redaction +5. **Trust scoring**: Per-server risk track with decay ## Key Invariants **Always:** -- All tool inputs sanitized before passing to server (schema validation, injection checks) -- All tool outputs scanned for injection patterns before returning to agent -- Server responses that fail validation are rejected with logging -- OAP authorization policies applied: which tools can be called by which agents -- `Untrusted` servers fail closed on a missing `tool_allowlist` — zero tools exposed unless - an allowlist is declared or `allow_untrusted_without_allowlist` is explicitly set. Only - `Trusted` exposes all tools without an allowlist. Trust levels not explicitly handled by - the allowlist policy (including future variants) also fail closed. +- All tool definitions sanitized (names, descriptions, input schema parameter descriptions) before registration +- Tool inputs validated against `input_schema` JSON schema before passing to server +- Tool outputs scanned for injection patterns before returning to agent +- `Untrusted` servers fail closed: zero tools exposed unless `tool_allowlist` is declared or `allow_untrusted_without_allowlist = true` +- Trust level enforcement: `Trusted` → all tools; `Untrusted` → allowlist only; `Sandboxed` → allowlist only, no elicitation +- Anomalous tool-call sequences flagged by `EmbeddingAnomalyGuard` post-execution **Never:** -- Pass unsanitized user input to MCP tools -- Trust server output without injection detection (DeBERTa + regex) -- Allow cross-agent tool access without explicit capability delegation -- Expose all tools for an `Untrusted` server with no declared `tool_allowlist` unless the - operator has explicitly opted in via `allow_untrusted_without_allowlist` +- Pass unsanitized user input to MCP tools — always validate against schema +- Trust server output without scanning — all responses pass through injection/PII detection +- Expose all tools for `Untrusted` servers without explicit operator opt-in +- Run elicitation on `Sandboxed` servers even if `elicitation_enabled = true` -## Elicitation Phases +## Tool Sanitization Pipeline -Tool invocation has three security phases: +`sanitize_tools()` (`crates/zeph-mcp/src/sanitize.rs`) scrubs injection patterns from tool definitions at registration: +```rust +pub struct SanitizeResult { + pub tools: Vec, // Cleaned tool definitions + pub injection_count: usize, // Patterns found + removed + pub flagged_parameters: Vec<(String, String)>, // (path, pattern) tuples +} + +pub fn sanitize_tools( + tools: &[McpTool], + config: &ContentIsolationConfig, +) -> SanitizeResult { + // 1. Scan tool.name, tool.description for regex injection patterns + // 2. Scan tool.input_schema.properties[*].description for injection patterns + // 3. Replace matched patterns with [sanitized] + // 4. Record flagged parameters for metadata + // 5. Return cleaned tools + metadata +} ``` -1. INPUT PHASE - ├─ Schema validation (input_schema) - ├─ Injection detection (SQL, shell, prompt injection) - └─ Rate limiting check - -2. EXECUTION PHASE - ├─ RPC call to MCP server - ├─ Timeout enforcement - └─ Error handling - -3. OUTPUT PHASE - ├─ Response format validation - ├─ Injection detection (DeBERTa on output) - └─ PII detection (NER + redaction) -``` -Code: +Patterns scanned: +- Prompt injection: "ignore this instruction", "pretend you are", "SYSTEM:", etc. +- SQL injection: `SELECT`, `DROP`, `; --`, etc. +- Shell injection: `$(...)`, `` `...` ``, `; rm -rf`, etc. + +If a tool's description is flagged, it is sanitized but **not removed**. The tool remains callable; only the suspicious description text is replaced. + +## Pre-Connect Probing + +`DefaultMcpProber` (`crates/zeph-mcp/src/prober.rs`) scans resources and prompts before tools/list: ```rust -async fn invoke_tool_secure( - &self, - tool_name: &str, - input: Value, - caller_context: &AgentContext, -) -> Result { - // PHASE 1: Input validation - let tool = self.get_tool(tool_name)?; - - // Schema validation - jsonschema::validate(&input, &tool.input_schema) - .map_err(|e| anyhow!("Schema validation failed: {}", e))?; - - // Injection detection - if self.detect_injection(&input)? { - return Err(anyhow!("Injection detected in tool input")); +pub struct DefaultMcpProber { /* ... */ } + +pub struct ProbeResult { + pub injection_patterns_found: usize, + pub resource_count: usize, + pub prompt_count: usize, +} + +impl DefaultMcpProber { + /// Scan resources/list and prompts/list for injection patterns. + pub async fn probe_server(&self, conn: &McpClient) -> Result { + // Fetches resources/list, prompts/list (if available) + // Scans descriptions for injection patterns + // Updates server trust score } - - // Rate limiting - self.check_rate_limit(tool_name, caller_context)?; - - // PHASE 2: Execution - let result = self.server.call_tool( - tool_name, - input, - Duration::from_secs(30), // timeout - ).await?; - - // PHASE 3: Output validation - - // Format check - if !result.is_object() && !result.is_string() { - return Err(anyhow!("Unexpected tool output format")); +} +``` + +Probing is non-blocking: results update the trust score but do NOT block tools/list fetch. + +## Trust Score System + +`ServerTrustScore` (`crates/zeph-mcp/src/trust_score.rs`) tracks per-server risk with exponential decay: + +```rust +pub struct ServerTrustScore { + pub current_score: f32, // [0.0, 1.0] (1.0 = fully trusted) + pub last_decay_timestamp: i64, + pub history: VecDeque<(f32, i64)>, // (score, timestamp) snapshots +} + +impl ServerTrustScore { + /// Report an incident (e.g., injection pattern found); decrease score. + pub fn report_incident(&mut self, severity: f32) { + self.current_score *= (1.0 - severity); // Multiplicative decay } - // Injection detection on output - if self.detect_output_injection(&result)? { - log::warn!("Injection detected in tool output from {}", tool_name); - return Err(anyhow!("Tool output validation failed")); + /// Apply time-based recovery: score drifts back toward 1.0 slowly. + pub fn apply_decay(&mut self, now: i64) { + let elapsed_secs = (now - self.last_decay_timestamp) as f32; + let recovery_rate = 0.001; // per second + self.current_score = (self.current_score + recovery_rate * elapsed_secs).min(1.0); } - - // PII redaction - let redacted = self.redact_pii(&result)?; - - Ok(redacted) } ``` -## Injection Detection +Score factors: +- **Probing failures**: -0.2 (injection patterns in resources/prompts) +- **Sanitization hits**: -0.1 (injection patterns in tool descriptions) +- **Embedding anomalies**: -0.15 (unexpected tool-call sequence) +- **Time decay**: +0.001 per second (gradual recovery) + +## Data-Flow Policy Enforcement -Multi-layer detection using DeBERTa + regex patterns: +`check_data_flow()` restricts sensitive tools based on trust level: ```rust -pub struct InjectionDetector { - deberta: Arc, // "is this injection?" binary classifier - regex_patterns: Vec, // SQL, shell, prompt injection patterns +pub enum DataSensitivity { + Public, // Available to all trust levels + Internal, // `Untrusted` can use only with allowlist + Confidential, // `Sandboxed` cannot use } -impl InjectionDetector { - fn detect_injection(&self, value: &Value) -> Result { - let text = match value { - Value::String(s) => s.clone(), - Value::Object(o) => serde_json::to_string(o)?, - _ => return Ok(false), - }; - - // 1. Regex check (fast) - for pattern in &self.regex_patterns { - if pattern.is_match(&text) { - log::warn!("Regex injection pattern matched"); - return Ok(true); - } +pub struct DataFlowViolation { + pub tool: String, + pub server: String, + pub trust_level: McpTrustLevel, + pub reason: String, +} + +pub fn check_data_flow( + tool: &McpTool, + server_trust: McpTrustLevel, + tool_metadata: &ToolSecurityMeta, +) -> Result<(), DataFlowViolation> { + match (server_trust, tool_metadata.sensitivity) { + (McpTrustLevel::Sandboxed, DataSensitivity::Confidential) => { + return Err(DataFlowViolation { /* ... */ }); } - - // 2. DeBERTa check (slow, only if suspicious) - if text.len() > 100 && text.contains("SELECT") || text.contains("$(") { - let score = self.deberta.classify(&text).await?; - if score > 0.8 { - return Ok(true); - } + (McpTrustLevel::Untrusted, DataSensitivity::Confidential) + if !in_allowlist => { + return Err(DataFlowViolation { /* ... */ }); } - - Ok(false) + _ => Ok(()), } } ``` -## OAP Authorization +## Output Validation: Injection Scanning -Capability-based access control: +All tool responses pass through `ContentSanitizer` (see [[010-2-injection-defense]]) for injection detection: ```rust -pub struct OAPPolicy { - // agent_id → allowed tool names - permissions: HashMap>, - // tool_id → required capability - capabilities: HashMap, -} - -impl OAPPolicy { - fn check_authorization( - &self, - agent_id: &str, - tool_name: &str, - ) -> Result<()> { - // 1. Check if agent has tool permission - let allowed = self.permissions - .get(agent_id) - .ok_or_else(|| anyhow!("Agent {} not in ACL", agent_id))?; +impl McpToolExecutor { + async fn execute_tool(&self, tool: &McpTool, args: Value) -> Result { + // ... execute tool ... + let raw_response = /* result from server */; - if !allowed.contains(tool_name) { - return Err(anyhow!( - "Agent {} not authorized for tool '{}'", - agent_id, - tool_name - )); - } + // Sanitize output + let source = ContentSource::new(ContentSourceKind::McpToolResult) + .with_trust_level(ContentTrustLevel::LocalUntrusted); - // 2. Check capability delegation - let required = self.capabilities - .get(tool_name) - .copied() - .unwrap_or(Capability::PublicRead); + let sanitized = self.sanitizer.sanitize( + &raw_response.to_string(), + source, + ); - if !self.has_capability(agent_id, required) { - return Err(anyhow!( - "Agent {} lacks capability {:?} for tool '{}'", - agent_id, - required, - tool_name - )); + // Check injection flags + if !sanitized.injection_flags.is_empty() { + tracing::warn!( + "Injection patterns in tool output: {:?}", + sanitized.injection_flags + ); } - Ok(()) + Ok(sanitized.body) } } - -#[derive(Debug, Clone, Copy)] -pub enum Capability { - PublicRead, - MutableState, - Execute, - Privileged, -} ``` -## SMCP (Secure MCP) +## Output Validation: Embedding Anomaly Detection -Optional enhanced security mode for critical tools: +`EmbeddingAnomalyGuard` (`crates/zeph-mcp/src/embedding_guard.rs`) detects unexpected tool-call sequences: ```rust -pub struct SmcpEnvelope { - // HMAC-signed request/response - request_id: String, - nonce: [u8; 32], - signature: Vec, - timestamp: i64, +pub struct EmbeddingAnomalyGuard { /* ... */ } + +pub enum EmbeddingGuardEvent { + Benign, + Suspicious { score: f32, reason: String }, + Anomalous { score: f32 }, } -impl SmcpEnvelope { - fn sign_request( - request: &ToolRequest, - shared_secret: &[u8], - ) -> Result { - let mut hasher = HmacSha256::new_from_slice(shared_secret)?; - let payload = serde_json::to_vec(request)?; - hasher.update(&payload); - - let signature = hasher.finalize().into_bytes().to_vec(); - - Ok(SmcpEnvelope { - request_id: uuid::Uuid::new_v4().to_string(), - nonce: rand::random(), - signature, - timestamp: now(), - }) +impl EmbeddingAnomalyGuard { + /// Check if this tool call is anomalous given prior results. + pub async fn analyze_call( + &self, + prior_result: &str, + requested_tool: &str, + tool_description: &str, + ) -> Result { + // Embed prior result and tool description + // Compute cosine similarity + // If similarity is unexpectedly low, flag as anomalous + // Update trust score } } ``` -## Configuration - -```toml -[mcp.security] -injection_detection_enabled = true -deberta_enabled = false # expensive; enable for high-risk tools -output_validation_enabled = true -oap_authorization_enabled = true -rate_limit_per_minute = 60 - -# Injection patterns (regex) -injection_patterns = [ - "(?i)(DROP|DELETE|UPDATE|INSERT).*FROM", # SQL - "\\$\\(.*\\)|`.*`", # Shell - "prompt\\s*injection|jailbreak|ignore", # Prompt injection -] - -# Tool capabilities -[[mcp.security.tool_capabilities]] -tool = "shell_execute" -capability = "Execute" -rate_limit_per_minute = 10 - -[[mcp.security.tool_capabilities]] -tool = "list_files" -capability = "PublicRead" -``` +Example: If a file-read tool's output (e.g., PDF contents) triggers a request for `delete_account` tool, the embedding distance between the result and that tool is large → flagged as anomalous. + +## Allowlist & Fail-Closed Semantics + +Trust level behavior: + +| Trust Level | Tool Allowlist Provided | Behavior | +|---|---|---| +| `Trusted` | No | Expose all tools | +| `Trusted` | Yes | Expose allowlisted tools only | +| `Untrusted` | No | **Expose zero tools** (fail closed) unless `allow_untrusted_without_allowlist = true` | +| `Untrusted` | Yes | Expose allowlisted tools only | +| `Sandboxed` | No | Expose zero tools | +| `Sandboxed` | Yes | Expose allowlisted tools only; disable elicitation | + +Default: `allow_untrusted_without_allowlist = false` (secure default). An operator must explicitly opt in to expose tools from an untrusted server without an allowlist. + +## Elicitation Gating + +When `elicitation_enabled = true`, Zeph probes tool parameters for hidden capabilities: + +- `Sandboxed` servers: elicitation disabled regardless of config +- `Untrusted` servers: elicitation allowed unless allowlist says otherwise +- `Trusted` servers: elicitation allowed + +See [[008-4-elicitation]] for elicitation details. + +## Attestation + +When `expected_tools` list is provided, compare actual tools against expected list at startup (see [[008-2-discovery]]). Schema mismatches logged but do NOT block usage. ## Integration Points -- [[008-1-lifecycle]] — Security checks during subprocess spawning -- [[008-2-discovery]] — Tool descriptions scanned for injection before caching -- [[010-security/spec]] — Parent security spec; refers to injection defense -- [[010-3-authorization]] — OAP policy enforcement -- [[025-classifiers/spec]] — DeBERTa-backed injection detection +- [[008-1-lifecycle]] — Sanitization runs after tools/list fetch +- [[008-2-discovery]] — Collision detection after sanitization +- [[008-4-elicitation]] — Parameter probing (optional) +- [[010-2-injection-defense]] — Output scanning via ContentSanitizer +- [[010-3-authorization]] — Policy enforcement ## See Also - [[008-mcp/spec]] — Parent -- [[008-1-lifecycle]] — Server lifecycle where security starts -- [[008-2-discovery]] — Tool discovery with injection scanning -- [[010-security/spec]] — Cross-cutting security constraints -- [[010-3-authorization]] — OAP authorization policies +- [[010-security/spec]] — Security architecture +- [[008-4-elicitation]] — Optional tool parameter discovery diff --git a/specs/010-security/010-1-vault.md b/specs/010-security/010-1-vault.md index 1314cd4a0..e87ee1a6b 100644 --- a/specs/010-security/010-1-vault.md +++ b/specs/010-security/010-1-vault.md @@ -20,166 +20,133 @@ related: # Spec: Secret Vault & Credential Resolution -> [!danger] Stale content — 2026-07 audit -> This file's code samples and type names do not match the real implementation and were not -> corrected in this pass (rewrite, not a text fix): it uses `serde_yaml` (banned project-wide, see -> `[[constitution]]` II — use `serde_norway`), `log::info!`/`log::warn!` (banned, `tracing` only), -> invented types `VaultBackend`/`AgeVault` (real trait is `VaultProvider`, -> `crates/zeph-vault/src/lib.rs`), and lists `age`/`kms`/`custom` backends (only `age`/`env` exist, -> per the correct listing in `[[010-security/spec]]` and `[[038-vault/spec]]`). **Treat -> [[038-vault/spec]] as authoritative for vault behavior**, not this file. - Age-encrypted secret storage, credential resolution, ZEPH_* environment key mapping, vault access control. ## Overview -Zeph stores all secrets (API keys, tokens, passwords) in an encrypted age vault, not in environment variables or `.env` files. The vault is automatically decrypted at startup, and credentials are resolved on-demand by subsystems. +Zeph stores all secrets (API keys, tokens, passwords) in an encrypted age vault, not in environment variables or `.env` files. The vault is automatically decrypted at startup, and credentials are resolved on-demand by subsystems using the pluggable `VaultProvider` trait. + +Two backends ship out of the box: `AgeVaultProvider` (recommended, age encryption) and `EnvVaultProvider` (development/testing, reads `ZEPH_SECRET_*` env vars). ## Key Invariants **Always:** -- All API keys, tokens, and passwords stored in age vault only -- Vault backend (age/kms/custom) configured once at startup -- Credentials resolved via `vault.get("KEY_NAME")` at runtime, never from env vars -- ZEPH_* environment variables automatically resolved from vault at startup +- All API keys, tokens, and passwords stored in encrypted age vault (production) or env vars (testing only) +- Vault backend configured once at startup via `VaultProvider` trait implementation +- Credentials resolved via `vault.get_secret("KEY_NAME").await` at runtime, never from raw env vars +- Secret values kept in `zeroize::Zeroizing` buffers — automatically zeroed on drop +- Vault file permissions set to `0o600` (owner-read/write only) on Unix **Never:** - Store secrets in `.env`, config files, or command-line arguments -- Pass API keys through shell pipelines or logs +- Pass API keys through logging, error messages, or debug output - Hardcode credentials in source code -- Use plaintext file backend in production - -## Age Vault Structure +- Use synchronous blocking I/O inside async contexts for vault access -YAML format for quick editing: - -```yaml -# ~/.age/zeph.age (encrypted) ---- -# OpenAI -openai_api_key: "sk-proj-..." -openai_org_id: "org-..." +## Vault Provider Trait -# Anthropic -claude_api_key: "sk-ant-..." - -# Local APIs -ollama_api_base: "http://localhost:11434" - -# Database -postgres_url: "postgresql://user:pass@host/db" - -# External Services -telegram_bot_token: "123456:ABCdef..." -github_token: "ghp_..." +```rust +pub trait VaultProvider: Send + Sync { + /// Retrieve a secret by key. + /// + /// Returns `Ok(None)` when the key does not exist. + /// Returns `Err(VaultError)` on backend failures (I/O, decryption, network). + fn get_secret( + &self, + key: &str, + ) -> Pin, VaultError>> + Send + '_>>; + + /// Return all known secret keys (optional). + /// + /// Default implementation returns an empty `Vec`. + fn list_keys(&self) -> Vec { + Vec::new() + } +} ``` -Encrypted with user's age public key: +## Age Backend -```bash -age --encrypt --recipient age1xxx... ~/.age/zeph.age.plaintext > ~/.age/zeph.age -rm ~/.age/zeph.age.plaintext +**File layout:** +``` +~/.config/zeph/ +├── vault-key.txt # age identity (private key), mode 0600 +└── secrets.age # age-encrypted JSON: {"KEY": "value", ...} ``` -## Vault Access Interface - +**API:** ```rust -pub trait VaultBackend: Send + Sync { - async fn get(&self, key: &str) -> Result; - async fn set(&self, key: &str, value: &str) -> Result<()>; - async fn list(&self) -> Result>; - async fn delete(&self, key: &str) -> Result<()>; -} +pub struct AgeVaultProvider { /* private */ } -pub struct AgeVault { - // Decrypted in-memory map after startup - secrets: Arc>>, - vault_path: PathBuf, - identity_path: PathBuf, +impl AgeVaultProvider { + /// Load vault from encrypted files. + pub fn new(key_path: &Path, vault_path: &Path) -> Result; + + /// Initialize a new vault with a fresh age keypair. + pub fn init_vault(dir: &Path) -> Result<(), AgeVaultError>; + + /// Synchronous getter for convenience (non-async). + pub fn get(&self, key: &str) -> Option<&str>; + + /// Set or update a secret (requires mutable access). + pub fn set_secret_mut(&mut self, key: String, value: String, is_new: bool) + -> Result<(), AgeVaultError>; + + /// Remove a secret. + pub fn remove_secret_mut(&mut self, key: &str) -> bool; + + /// Save encrypted vault to disk (atomic write via `.tmp` suffix). + pub fn save(&self) -> Result<(), AgeVaultError>; + + /// Async variant that offloads I/O to a background thread. + pub async fn load_async(key_path: &Path, vault_path: &Path) + -> Result; + + pub async fn save_async(&self) -> Result<(), AgeVaultError>; } -impl AgeVault { - async fn load(&mut self) -> Result<()> { - // 1. Read encrypted file - let encrypted = fs::read_to_string(&self.vault_path)?; - - // 2. Decrypt using age identity - let decrypted = age_decrypt(&encrypted, &self.identity_path)?; - - // 3. Parse YAML - let parsed: HashMap = serde_yaml::from_str(&decrypted)?; - - // 4. Store in memory - *self.secrets.write().await = parsed; - - Ok(()) - } - - async fn get(&self, key: &str) -> Result { - self.secrets - .read() - .await - .get(key) - .cloned() - .ok_or_else(|| anyhow!("Secret '{}' not found in vault", key)) - } +impl VaultProvider for AgeVaultProvider { /* ... */ } +``` + +Secrets stored as JSON (not YAML) for forward compatibility: +```json +{ + "ZEPH_CLAUDE_API_KEY": "sk-ant-...", + "ZEPH_OPENAI_API_KEY": "sk-proj-...", + "OLLAMA_API_BASE": "http://localhost:11434" } ``` -## Startup Resolution +## Env Backend -At agent initialization, populate required ZEPH_* vars: +Development/testing backend that reads `ZEPH_SECRET_*` prefixed environment variables: ```rust -async fn resolve_vault_at_startup(vault: &AgeVault) -> Result<()> { - let required_keys = vec![ - "openai_api_key", - "claude_api_key", - "ollama_api_base", - ]; - - for key in required_keys { - if let Ok(value) = vault.get(key).await { - // Don't set env var; store in config instead - // ZEPH_OPENAI_API_KEY is resolved lazily from vault - log::info!("Resolved credential: {}", key); - } else { - log::warn!("Missing vault key: {}", key); - } +pub struct EnvVaultProvider; + +impl VaultProvider for EnvVaultProvider { + fn get_secret(&self, key: &str) -> Pin>> { + // Looks for environment variable with `ZEPH_SECRET_` prefix + // E.g., key "API_KEY" reads env var "ZEPH_SECRET_API_KEY" } - - Ok(()) } ``` -## Lazy Resolution in LLM Providers +Used for testing and CI pipelines where file-based vaults are impractical. -Providers request credentials on-demand: +## Arc Wrapper + +`ArcAgeVaultProvider` wraps `Arc>` to allow sharing the vault as a trait object while still supporting mutable operations (e.g., OAuth credential persistence): ```rust -pub struct ClaudeProvider { - vault: Arc, -} +pub struct ArcAgeVaultProvider { /* ... */ } -impl ClaudeProvider { - async fn get_api_key(&self) -> Result { - self.vault.get("claude_api_key").await - } - - async fn invoke(&self, req: &Request) -> Result { - let api_key = self.get_api_key().await?; - - // Make request with credential - let client = reqwest::Client::new(); - client - .post("https://api.anthropic.com/v1/messages") - .bearer_auth(&api_key) - .json(req) - .send() - .await? - .json() - .await - } +impl VaultProvider for ArcAgeVaultProvider { /* ... */ } + +// Mutable methods available via downcasting: +impl ArcAgeVaultProvider { + pub fn set_secret_mut(&self, key: String, value: String, is_new: bool) + -> Result<(), AgeVaultError>; } ``` @@ -192,10 +159,10 @@ Management interface: cargo run -- vault list # Get a secret (for external scripts) -cargo run -- vault get openai_api_key +cargo run -- vault get ZEPH_OPENAI_API_KEY -# Set a secret (interactive prompt) -cargo run -- vault set new_key_name +# Initialize a new vault with fresh keypair +cargo run -- vault init # Validate vault integrity cargo run -- vault check @@ -205,9 +172,9 @@ cargo run -- vault check ```toml [vault] -backend = "age" # or "kms", "custom" -vault_path = "~/.age/zeph.age" -identity_path = "~/.age/identity" +backend = "age" # or "env" +vault_path = "~/.config/zeph/secrets.age" +identity_path = "~/.config/zeph/vault-key.txt" fallback_env = false # don't read from env vars as fallback ``` diff --git a/specs/010-security/010-2-injection-defense.md b/specs/010-security/010-2-injection-defense.md index bf9199359..3fb7e7c2a 100644 --- a/specs/010-security/010-2-injection-defense.md +++ b/specs/010-security/010-2-injection-defense.md @@ -23,262 +23,203 @@ related: # Spec: Injection Defense & Content Isolation -> [!danger] Stale content — 2026-07 audit -> The `IpiDetector`/`PiiDetector`/`DeBERTaClassifier` code blocks below are fictional — none of -> these type names exist in the codebase — and were not corrected in this pass (rewrite, not a text -> fix). Real code: `crates/zeph-llm/src/classifier/` (`CandleClassifier`, `CandlePiiClassifier`) and -> `crates/zeph-sanitizer/src/causal_ipi.rs` (`TurnCausalAnalyzer`). The "User Verification Gate" -> section's direct `println!(...)` also contradicts [[001-system-invariants/spec#1. Channel Contract]] -> ("the `Channel` trait is the only I/O boundary") — do not copy that pattern. - -Indirect Prompt Injection (IPI) defense, DeBERTa-backed detection, AlignSentinel confidence scoring, PII NER detection, content isolation. +Indirect Prompt Injection (IPI) defense, regex-based detection, ML classifier soft signals, TurnCausalAnalyzer, PII NER detection, content spotlighting. ## Overview -Zeph processes untrusted content from web scraping, MCP tool outputs, and user uploads. IPI attacks attempt to override agent behavior by embedding instructions in observed content. Zeph defends via multi-layer detection: regex, DeBERTa binary classifier, confidence scoring, and explicit user verification gates. +Zeph processes untrusted content from web scraping, MCP tool outputs, A2A calls, and memory retrieval. IPI attacks attempt to override agent behavior by embedding instructions in observed content. Zeph defends via multi-layer detection: regex spotlighting with content source attribution, optional ML-backed classifiers, turn-level causal analysis for tool-chain anomalies, and PII NER redaction. ## Key Invariants **Always:** -- All web-fetched content scanned for IPI patterns before returning to agent -- MCP tool outputs validated before reaching context/LLM -- Suspicious content flagged with confidence score; high scores require user verification -- PII (SSN, credit card, email) detected via NER and redacted or blocked +- All untrusted content (`WebScrape`, `MemoryRetrieval`, `A2A`, `McpToolResult`) wrapped in spotlighting delimiters and source attribution +- External content injected at `ContentTrustLevel::ExternalUntrusted` or `LocalUntrusted` depending on source +- Regex-based injection pattern detection runs unconditionally on external content (`flag_injection_patterns = true`) +- PII entities truncated before ML model inference (max 4096 chars) to prevent OOM +- Secrets masked via `SecretMaskRegistry` at LLM boundary to prevent leakage **Never:** -- Trust content from web pages, emails, or tool outputs without scanning -- Execute instructions found in observed content without explicit user approval -- Log secrets or PII in debug dumps -- Suppress IPI warnings when confidence is high +- Run IPI ML classifiers on agent-generated content — only on external/tool-sourced content +- Accumulate cross-turn injection signals for confirmation decisions — `CrossToolCorrelator` clears state at turn boundaries +- Log or expose redacted PII values or secret shapes in debug output +- Skip spotlight wrapping for untrusted content even if regex confidence is low -## IPI Detection Layers +## Content Sanitizer: Spotlighting & Injection Detection -``` -Layer 1: Regex Patterns (Fast) -├─ "ignore this instruction" -├─ "prompt injection|jailbreak" -├─ SYSTEM: embedded in content -└─ Confidence: 0.95 (high) - -Layer 2: DeBERTa Binary Classifier (Slow) -├─ Fine-tuned on IPI examples -├─ Outputs confidence [0.0–1.0] -└─ Run only if Layer 1 suspicious - -Layer 3: AlignSentinel Confidence Scoring -├─ Combines regex + DeBERTa scores -├─ Context-aware adjustments -└─ > 0.8 = require user verification +`ContentSanitizer` (`crates/zeph-sanitizer/src/sanitizer.rs`) wraps untrusted content with source attribution and scans for regex injection patterns: + +```rust +pub struct ContentSanitizer { /* ... */ } + +impl ContentSanitizer { + /// Sanitize external content: spot-light, truncate, flag injection patterns. + pub fn sanitize( + &self, + text: &str, + source: ContentSource, + ) -> SanitizedContent { + // 1. Apply content trust level based on source.kind + // 2. Truncate if > max_content_size + // 3. If external + untrusted: spotlight with ... + // 4. Scan for regex injection patterns; populate injection_flags + // 5. Return wrapped content ready for LLM context + } +} + +pub struct SanitizedContent { + pub body: String, // Spotlighted, truncated content + pub injection_flags: Vec, // Detected pattern names + pub was_truncated: bool, // True if exceeded max size +} ``` -Code: +**Trust levels** (`ContentTrustLevel`): +- `Trusted` — passes unchanged (system prompt, validated user input) +- `LocalUntrusted` — tool results from local executors; wrapped in `` with NOTE header +- `ExternalUntrusted` — web, MCP, A2A, memory; wrapped in `` with IMPORTANT warning + +## Optional ML-Backed Soft Signals + +When feature `classifiers` is enabled and a backend is attached, `ContentSanitizer::classify_injection` provides DeBERTa-backed injection probability (advisory only): ```rust -pub struct IpiDetector { - regex_patterns: Vec<(Regex, f32)>, // (pattern, confidence) - deberta: Arc, - align_sentinel: Arc, +impl ContentSanitizer { + /// Optional: classify content with ML classifier (requires backend). + pub async fn classify_injection(&self, text: &str) + -> Result { + // DeBERTa model (via candle): returns continuous [0.0–1.0] probability + // Policy-blocked outputs are skipped; ML classification only on advisory path + } } +``` -impl IpiDetector { - async fn scan_content(&self, text: &str) -> Result { - // Layer 1: Regex (fast) - let mut max_regex_score = 0.0; - for (pattern, confidence) in &self.regex_patterns { - if pattern.is_match(text) { - max_regex_score = max_regex_score.max(*confidence); - } - } - - // Short-circuit if regex very confident - if max_regex_score > 0.9 { - return Ok(ScanResult { - is_injection: true, - confidence: max_regex_score, - detected_by: "regex", - content_preview: truncate(text, 100), - }); - } - - // Layer 2: DeBERTa (expensive; only if regex moderately suspicious) - let deberta_score = if max_regex_score > 0.5 || text.len() > 500 { - self.deberta.classify(text).await? - } else { - 0.0 - }; - - // Layer 3: AlignSentinel combines scores - let final_score = self.align_sentinel.combine_scores( - max_regex_score, - deberta_score, - text.len(), - ); - - Ok(ScanResult { - is_injection: final_score > 0.75, - confidence: final_score, - detected_by: if deberta_score > max_regex_score { - "deberta" - } else { - "regex" - }, - content_preview: truncate(text, 100), - }) - } +Config: +```toml +[security.ipi] +enabled = false # Optional ML classification +soft_signal_threshold = 0.5 # Escalate scores above this to AlignSentinel +hard_threshold = 0.85 # Always block above this, regardless of AlignSentinel +causal_analysis = false # Enable turn-level anomaly detection +``` + +## Turn-Level Causal Analysis + +`TurnCausalAnalyzer` (`crates/zeph-sanitizer/src/causal_ipi.rs`) detects injection-induced tool-call pivots within a turn. If a tool result in turn N directly triggers an anomalous tool call in N+1, it is flagged: + +```rust +pub struct TurnCausalAnalyzer { /* ... */ } + +impl TurnCausalAnalyzer { + /// Check whether this turn's tool calls show unusual patterns given prior results. + pub async fn analyze_causal_chain( + &self, + prior_result: &ToolResult, + current_call: &ToolCall, + ) -> Result { /* ... */ } } ``` +This check runs **within the turn only** — state is cleared at turn boundaries per the parent spec's NEVER rule. + ## PII Detection & Redaction -Named Entity Recognition for sensitive data: +`CandlePiiClassifier` (feature: `classifiers`) performs Named Entity Recognition (NER) on tool inputs/outputs when the feature is enabled: ```rust -pub struct PiiDetector { - ner_model: Arc, // Detects SSN, email, credit card, etc. +impl ContentSanitizer { + /// Optional: detect PII via NER model (requires feature + backend). + pub async fn detect_pii(&self, text: &str) + -> Result> { + // Truncate to pii_max_input_chars (default 4096) before model inference + // NER inference: returns PII entities (SSN, credit card, email, phone, etc.) + // Detected entities are redacted from final content + } } -impl PiiDetector { - async fn detect_pii(&self, text: &str) -> Result> { - // NER inference - let entities = self.ner_model.extract_entities(text).await?; - - Ok(entities - .into_iter() - .filter(|e| matches!( - e.entity_type, - EntityType::SSN - | EntityType::CreditCard - | EntityType::Email - | EntityType::PhoneNumber - )) - .collect()) - } - - async fn redact_pii(&self, text: &str) -> Result { - let entities = self.detect_pii(text).await?; - - let mut redacted = text.to_string(); - for entity in entities.iter().rev() { - // Replace in reverse order to preserve offsets - let replacement = match entity.entity_type { - EntityType::SSN => "[REDACTED_SSN]", - EntityType::CreditCard => "[REDACTED_CC]", - EntityType::Email => "[REDACTED_EMAIL]", - EntityType::PhoneNumber => "[REDACTED_PHONE]", - _ => "[REDACTED]", - }; - - redacted.replace_range(entity.start..entity.end, replacement); - } - - Ok(redacted) +pub struct PiiFilter { + regex_patterns: Vec, // Fallback: regex-based PII detection +} + +impl PiiFilter { + pub fn scrub(&self, text: &str) -> String { + // Regex-based PII scrubbing (email, phone, SSN, credit card) + // Runs unconditionally; always precedes ML classification } } ``` -## Content Isolation Boundary +Config: +```toml +[security.pii] +enabled = false # Optional NER-based detection +threshold = 0.85 # PII confidence threshold +pii_max_input_chars = 4096 # Truncate before ML inference +pii_allowlist = [] # Regex patterns exempt from blocking +``` + +## Secret Shape Masking -Web-fetched and tool-output content lives in isolated context: +`SecretMaskRegistry` (`crates/zeph-sanitizer/src/secret_mask.rs`) masks vault-secret placeholders at the LLM boundary to prevent leakage via side-channel analysis: ```rust -pub struct IsolatedContent { - // Content is NOT directly visible to LLM prompts - original: String, - scanned: bool, - ipi_confidence: f32, - pii_redacted: bool, - source: ContentSource, +pub enum SecretCategory { + ApiKey, + AuthToken, + OAuthToken, + DatabaseUrl, + // ... } -impl IsolatedContent { - async fn release_to_agent( - &self, - detector: &IpiDetector, - pii_detector: &PiiDetector, - ) -> Result { - if !self.scanned { - // Scan - let scan = detector.scan_content(&self.original).await?; - if scan.confidence > 0.75 { - return Err(anyhow!( - "Content flagged as likely IPI (confidence: {:.0}%)", - scan.confidence * 100.0 - )); - } - } - - // Redact PII - let redacted = pii_detector.redact_pii(&self.original).await?; - - Ok(redacted) - } -} +pub struct SecretMaskRegistry { /* ... */ } -#[derive(Debug, Clone, Copy)] -pub enum ContentSource { - WebFetch, - McpToolOutput, - UserUpload, - Email, +impl SecretMaskRegistry { + /// Mask all vault secret references in text before LLM inference. + pub fn mask_secrets(&self, text: &str) -> String { + // Replaces secret values with [MASKED_CATEGORY] placeholders + // Preserves schema/structure for debugging + } } ``` -## User Verification Gate +## Guardrail Filter (Optional Gating) -High-confidence IPI blocks agent until user approves: +`GuardrailFilter` provides an optional LLM-based pre-screener at the input boundary. When external content passes through, it is classified as SAFE/UNSAFE via Llama Guard classifier before being added to context: ```rust -async fn handle_suspicious_content( - detector: &IpiDetector, - content: &str, -) -> Result { - let scan = detector.scan_content(content).await?; - - if scan.confidence > 0.75 { - // Block and ask user - println!( - "⚠️ Suspicious content detected (confidence: {:.0}%)", - scan.confidence * 100.0 - ); - println!("Preview: {}", scan.content_preview); - println!("Source: {:?}", scan.detected_by); - println!("Should I use this content?"); - - let user_approval = /* await user input */; - if !user_approval { - return Err(anyhow!("User rejected suspicious content")); - } +pub struct GuardrailFilter { /* ... */ } + +impl GuardrailFilter { + /// Screen content via LLM guardrail classifier before adding to context. + pub async fn screen(&self, text: &str) + -> Result { + // Returns SAFE, UNSAFE, or UNKNOWN with confidence + // High-confidence UNSAFE blocks content from context } - - Ok(content.to_string()) } ``` -## Configuration - +Config: ```toml -[security.injection_defense] -enabled = true -regex_patterns_enabled = true -deberta_enabled = true -confidence_threshold = 0.75 # require verification above this - -# PII Detection -pii_detection_enabled = true -sensitive_types = ["ssn", "credit_card", "email", "phone"] -redact_mode = "mask" # or "block" +[security.guardrails] +enabled = false # Optional guardrail pre-screening +model = "llama-guard-7b" +threshold = 0.8 # UNSAFE confidence threshold ``` +## Quarantine & Dual-LLM Extraction + +`QuarantinedSummarizer` (`crates/zeph-sanitizer/src/quarantine.rs`) applies a Dual-LLM approach: one LLM processes untrusted content in isolation; a second LLM summarizes into trusted context. This prevents the agent from seeing potentially malicious instructions directly. + ## Integration Points -- [[008-mcp/spec]] — MCP tool outputs scanned before reaching agent -- [[025-classifiers/spec]] — DeBERTa inference infrastructure -- [[010-4-audit]] — IPI detection logged for compliance -- WebFetch tool — Content scanned before returning +- [[008-mcp/spec]] — MCP tool sanitization via `sanitize_tools()` +- [[025-classifiers/spec]] — DeBERTa (IPI), Candle PII NER (optional classifiers feature) +- [[010-4-audit]] — Audit signals (`AuditSignal`, `AuditSignalType`) ingested by `TrajectoryRiskAccumulator` +- WebScrape tool — Content sanitized before returning +- Context assembly — Spotlight wrapping applied per source trust level ## See Also -- [[010-security/spec]] — Parent -- [[010-1-vault]] — Prevent secret leakage from redaction -- [[025-classifiers/spec]] — DeBERTa and NER models -- [[010-4-audit]] — Audit log of IPI detections +- [[010-security/spec]] — Parent; cross-turn NEVER rule for `CrossToolCorrelator` +- [[010-1-vault]] — Prevent secret leakage via masking +- [[025-classifiers/spec]] — Classifier infrastructure (DeBERTa, PII NER) diff --git a/specs/010-security/010-3-authorization.md b/specs/010-security/010-3-authorization.md index ff4301fb3..ef3a465f6 100644 --- a/specs/010-security/010-3-authorization.md +++ b/specs/010-security/010-3-authorization.md @@ -23,16 +23,6 @@ related: # Spec: Authorization & Capability-Based Access Control -> [!danger] Stale content — 2026-07 audit -> `AuthPolicy`, `ShellSandbox`, `SsrfValidator` below are fictional — zero matches in the codebase — -> and were not corrected in this pass (rewrite, not a text fix). The code samples also chain -> `.unwrap()` on `Regex::new(...)`, which directly violates -> [[001-system-invariants/spec#11. Error Handling Contract]] ("NEVER use `panic!()`... no `unwrap()` -> in production paths") — do not copy that pattern. Real mechanisms live in the shell blocklist -> (`crates/zeph-tools/src/filter/security.rs` and the shell executor's `PermissionPolicy` gate) and -> the SSRF `validate_url` path referenced correctly in [[010-security/spec]] and -> [[010-5-egress-logging]] — treat those as authoritative over this file. - Permission policy enforcement, shell sandbox blocklist, SSRF protection, tool authorization. ## Overview @@ -42,234 +32,149 @@ Zeph's authorization layer enforces what operations the agent is allowed to perf ## Key Invariants **Always:** -- All tool execution requires authorization check against policy -- Shell commands checked against blocklist before execution -- HTTP requests checked for SSRF patterns (localhost, private ranges) -- Authorization failures logged with full context +- All tool execution checked against `PolicyEnforcer` deny/allow rules before execution +- Shell commands checked against the blocklist unconditionally — **before** `PermissionPolicy` evaluation +- HTTP requests validated via `validate_url_ssrf()` — private IP ranges blocked by default +- Authorization failures logged to audit trail with full context **Never:** -- Bypass authorization checks for "trusted" tools +- Bypass blocklist checks for "trusted" tools — blocklist is unconditional - Allow shell execution without sandbox validation -- Make HTTP requests to private IP ranges without explicit allow-list +- Make HTTP requests to private IP ranges without explicit allowlist -## Capability-Based Access Control +## Declarative Policy Compiler -Policies define which agents can execute which tools: +`PolicyEnforcer` (`crates/zeph-tools/src/policy.rs`) evaluates TOML-based access-control rules with deny-first semantics: ```rust -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] -pub enum Capability { - ToolRead, // read-only tools - ToolWrite, // file write, API modify - ShellExecute, // shell commands - HttpRequest, // HTTP/HTTPS requests - NetworkListen, // open ports - ProcessManage, // spawn/kill processes -} +pub struct PolicyEnforcer { /* ... */ } -pub struct AuthPolicy { - // agent_id → capabilities - grants: HashMap>, - // capability → individual tool allows - tool_overrides: HashMap>, +#[non_exhaustive] +pub enum PolicyDecision { + Allow { trace: String }, // Rule matched; execution allowed + Deny { trace: String }, // Rule matched; execution denied } -impl AuthPolicy { - async fn check_authorization( +impl PolicyEnforcer { + /// Evaluate policy rules against tool call context. + /// + /// Deny rules checked first. If a deny rule matches, returns `Deny`. + /// Otherwise, checks allow rules. If no rule matches, uses `default_effect`. + pub fn evaluate( &self, - agent_id: &str, tool_name: &str, - required_capability: Capability, - ) -> Result<()> { - // 1. Check if agent has capability - let capabilities = self.grants - .get(agent_id) - .ok_or_else(|| anyhow!("Agent {} not in policy", agent_id))?; - - if !capabilities.contains(&required_capability) { - return Err(anyhow!( - "Agent {} lacks capability {:?}", - agent_id, - required_capability - )); - } - - // 2. Check tool-level override (e.g., "shell_execute" allowed but "rm -rf" blocked) - if let Some(allowed_tools) = self.tool_overrides.get(tool_name) { - if !allowed_tools.contains(agent_id) { - return Err(anyhow!( - "Tool '{}' not in allow-list for agent {}", - tool_name, - agent_id - )); - } - } - - Ok(()) - } + params: &serde_json::Map, + context: &PolicyContext, + ) -> PolicyDecision { /* ... */ } +} + +pub struct PolicyContext { + pub trust_level: SkillTrustLevel, + pub env: std::collections::HashMap, } ``` -## Shell Sandbox +**Rule matching** (all conditions are AND'd): +- `tools`: glob pattern over tool name +- `paths`: glob patterns over extracted path parameters (e.g., `file` argument) +- `env_required`: environment variables that must be present +- `trust_threshold`: minimum trust level required (`Trusted` > `Neutral` > `Untrusted`) +- `args_regex`: regex over individual string parameter values -Blocklist of dangerous commands: +**Semantics**: deny rules are evaluated first; matching deny → `Deny`. If no deny matches, evaluate allow rules; matching allow → `Allow`. If no rule matches, use `default_effect` (typically `Deny`). -```rust -pub struct ShellSandbox { - blocklist: Vec, -} +Config: +```toml +[[tools.policy]] +effect = "deny" +tools = "rm *" # Glob pattern +reason = "Data destruction blocked" + +[[tools.policy]] +effect = "allow" +tools = "read_file" +paths = ["/home/*/documents/*"] # Glob patterns on paths +trust_threshold = "trusted" +``` -#[derive(Clone)] -pub struct ShellPattern { - pattern: Regex, - reason: &'static str, -} +## Shell Sandbox Blocklist -impl ShellSandbox { - fn new() -> Self { - Self { - blocklist: vec![ - // Destructive commands - ShellPattern { - pattern: Regex::new(r"^\s*(rm|rmdir|dd)\s+-rf").unwrap(), - reason: "recursive deletion blocked", - }, - ShellPattern { - pattern: Regex::new(r":(){ :|:|");").unwrap(), - reason: "fork bomb detected", - }, - // Privilege escalation - ShellPattern { - pattern: Regex::new(r"^sudo\s+|/etc/sudoers").unwrap(), - reason: "privilege escalation blocked", - }, - // System modification - ShellPattern { - pattern: Regex::new(r"^\s*(chmod|chown|passwd|usermod)").unwrap(), - reason: "system modification blocked", - }, - ], - } - } - - fn validate_command(&self, cmd: &str) -> Result<()> { - for pattern in &self.blocklist { - if pattern.pattern.is_match(cmd) { - return Err(anyhow!("{}", pattern.reason)); - } - } - Ok(()) +`ShellExecutor` enforces an unconditional blocklist of dangerous patterns before spawning shell commands: + +```rust +pub struct ShellExecutor { /* ... */ } + +impl ShellExecutor { + /// Execute a shell command with blocklist checks. + /// + /// Blocklist validation runs unconditionally, before PermissionPolicy checks. + pub async fn execute(&self, cmd: &str, ...) -> Result { + // 1. Check command against blocklist + self.find_blocked_command(cmd)?; // Panics if blocked pattern found + + // 2. Then evaluate PermissionPolicy (if configured) + // 3. Scrub credential env vars from subprocess environment + // 4. Spawn process } } ``` +**Blocked patterns** (`crates/zeph-tools/src/filter/security.rs`): +- Process substitution: `$(...)`, `` `...` `` +- Here-strings: `<<<` +- Destructive: `rm -rf`, `dd if=`, `mkfs` +- Fork bombs: `:(){ :|: }` +- Privilege escalation: `sudo`, `/etc/sudoers`, `su -` + +Bypass attempts via arguments are also caught — passing a blocked pattern as an argument value is detected and blocked. + ## SSRF Protection -Prevent requests to internal services: +`validate_url_ssrf()` (`crates/zeph-gateway/src/transport/ssrf.rs`, etc.) blocks requests to private IP ranges and loopback addresses: ```rust -pub struct SsrfValidator { - blocked_ranges: Vec, - allow_list: Vec, // explicit allowed domains -} +/// Validate a URL for SSRF attacks. +/// +/// Blocks: localhost, link-local (169.254.x), private ranges (10.x, 172.16-31.x, 192.168.x), +/// IPv6 loopback/private (::1, fc00::/7, fe80::/10). +pub fn validate_url_ssrf(url: &str) -> Result<(), SsrfError>; +``` -impl SsrfValidator { - fn new() -> Self { - Self { - blocked_ranges: vec![ - // Private IPv4 - "127.0.0.0/8".parse().unwrap(), // loopback - "169.254.0.0/16".parse().unwrap(), // link-local - "10.0.0.0/8".parse().unwrap(), // private - "172.16.0.0/12".parse().unwrap(), // private - "192.168.0.0/16".parse().unwrap(), // private - // IPv6 - "::1/128".parse().unwrap(), // loopback - "fc00::/7".parse().unwrap(), // private - "fe80::/10".parse().unwrap(), // link-local - ], - allow_list: vec![], - } - } - - async fn validate_url(&self, url: &str) -> Result<()> { - let parsed = url::Url::parse(url)?; - - // 1. Check allow-list first - if let Some(domain) = parsed.domain() { - if self.allow_list.contains(&domain.to_string()) { - return Ok(()); - } - } - - // 2. Resolve hostname - let addr = tokio::net::lookup_host( - format!("{}:{}", parsed.host_str().ok_or("no host")?, - parsed.port().unwrap_or(443)) - ).await? - .next() - .ok_or("hostname resolution failed")?; - - // 3. Check IP against blocked ranges - for range in &self.blocked_ranges { - if range.contains(addr.ip()) { - return Err(anyhow!( - "SSRF blocked: {} resolves to private IP {}", - parsed.host_str().unwrap_or("?"), - addr.ip() - )); - } - } - - Ok(()) +**Redirect chains**: each redirect target in the chain is also validated — a redirect to a private IP is caught and fails. + +**Applied to**: +- `WebScrapeExecutor` — all HTTP fetches +- `zeph-gateway` HTTP webhook receiver — incoming URLs validated +- `zeph-a2a` client — remote agent URLs validated +- MCP OAuth — OAuth metadata endpoint URLs validated via `validate_oauth_metadata_urls()` + +Allowlist capability exists (via config `ssrf_allowlist`), but defaults to deny-all private IPs; allowlist is opt-in and explicit. + +## Credential Environment Variable Scrubbing + +`ShellExecutor` and MCP stdio server spawning scrub credential env vars from the subprocess environment: + +```rust +impl ShellExecutor { + async fn scrub_environment(&self, env: &mut HashMap) { + // Blocklist: AWS_*, GITHUB_*, ZEPH_*, OPENAI_API_KEY, etc. + // Remove from subprocess environment to prevent getenv() leakage } } ``` -## Configuration - -```toml -[security.authorization] -# Capability grants per agent -[[security.authorization.agents]] -id = "primary_agent" -capabilities = ["ToolRead", "ToolWrite", "HttpRequest"] - -[[security.authorization.agents]] -id = "sandbox_agent" -capabilities = ["ToolRead"] - -# Tool allow-lists (tool → allowed agents) -[security.authorization.tool_overrides] -shell_execute = ["primary_agent"] -file_delete = ["primary_agent"] -network_listen = [] - -# Shell sandbox -[security.sandbox] -enabled = true -blocklist = [ - "^\\s*(rm|rmdir)\\s+-rf", - ":(){ :|:|;", - "^sudo\\s+", -] - -# SSRF protection -[security.ssrf] -enabled = true -blocked_ranges = ["127.0.0.0/8", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"] -allow_list = ["api.example.com", "internal.trusted.service"] -``` +Blocklist is unconditional and cannot be disabled per-command. MCP stdio servers inherit the same scrubbed environment. ## Integration Points -- [[006-tools/spec]] — Tool execution calls authorization check -- [[008-3-security]] — OAP authorization for MCP tools -- [[010-4-audit]] — Authorization failures logged +- [[008-mcp/spec]] — MCP tool allowlist + policy enforcement +- [[006-tools/spec]] — tool registry + authorization binding +- [[010-4-audit]] — authorization violations logged to audit trail +- Web executor — SSRF validation on all HTTP requests +- Gateway — OAuth flows validated via `validate_oauth_metadata_urls()` ## See Also -- [[010-security/spec]] — Parent -- [[006-tools/spec]] — ToolExecutor enforces authorization -- [[008-3-security]] — OAP authorization for MCP -- [[010-4-audit]] — Audit trail of authorization checks +- [[010-security/spec]] — Parent; shell blocklist and SSRF noted as unconditional +- [[010-2-injection-defense]] — guardrail filtering runs before tool dispatch +- [[008-3-security]] — MCP tool allowlist enforcement diff --git a/specs/010-security/010-4-audit.md b/specs/010-security/010-4-audit.md index 7e00aa570..62914299c 100644 --- a/specs/010-security/010-4-audit.md +++ b/specs/010-security/010-4-audit.md @@ -2,8 +2,8 @@ aliases: - Audit Trail - Security Logging - - Cross-Tool Injection Correlation - - Env-Var Scrubbing + - Tool Execution Audit + - Authorization Logging tags: - sdd - spec @@ -16,365 +16,198 @@ related: - "[[010-1-vault]]" - "[[010-2-injection-defense]]" - "[[010-3-authorization]]" + - "[[010-5-egress-logging]]" --- # Spec: Audit Trail & Security Logging -> [!danger] Stale content — 2026-07 audit -> The `AuditEntry`/`AuditEventType`/`AuditStatus`/`AuditLogger` schema below is fictional — the real -> `AuditEntry` (`crates/zeph-tools/src/audit.rs`) is tool-execution-centric (`tool`, `command`, -> `result`, `duration_ms`, `error_category`, `caller_id`, `correlation_id`, `vigil_risk`, ...), not -> the generic `id`/`agent_id`/`event_type`/`resource`/`action`/`status` shape shown here. See -> [[010-5-egress-logging]] for the accurate, current `AuditEntry`/`AuditLogger` description. The -> "Cross-Tool Injection Correlation" section (`InjectionCorrelator`, time-windowed accumulation) -> directly **contradicts** [[010-security/spec]]'s authoritative `CrossToolCorrelator`, which states -> cross-turn signal accumulation is NEVER performed and is cleared every turn — do not implement -> against this section without reconciling that contradiction first. Not corrected in this pass -> (needs a mechanism-level rewrite, not a text fix). - -AgentRFC protocol audit, cross-tool injection correlation, environment variable scrubbing, compliance logging. +Tool execution auditing, authorization logging, security event correlation, compliance logging. ## Overview -Zeph maintains an immutable audit trail of all security-relevant events: tool invocations, authorization decisions, IPI detections, vault accesses. This log is used for compliance, incident investigation, and pattern detection (e.g., correlated injection attempts across multiple tools). +Zeph maintains an immutable audit trail of security-relevant events: tool invocations, authorization decisions, IPI detections, shell command executions. This log is used for compliance, incident investigation, and pattern detection. ## Key Invariants **Always:** -- All tool invocations logged with: tool name, input (sanitized), output (truncated), status, latency -- All authorization failures logged with: agent, tool, capability, policy check details -- All IPI detections logged with: confidence, source, content preview, user action -- All vault accesses logged with: key name (not value), action (get/set), success/failure -- Audit log persists to disk (SQLite or JSON lines format) +- All tool invocations logged with: tool name, input (sanitized), output (preview), status, duration +- All authorization denials logged with: agent/skill, tool, reason, policy rule matched +- All shell command execution logged with: command (sanitized), exit code, duration +- All secrets redacted from logs — never log API keys, vault key names, PII +- Audit log persists immutably to disk **Never:** -- Log secret values, API keys, or PII (always sanitize) -- Truncate or modify audit entries after creation (immutable log) -- Disable audit logging even in debug mode +- Log secret values, API keys, or PII directly — always sanitize +- Truncate or modify audit entries after creation +- Disable audit logging, even in debug/testing modes + +## Tool Execution Audit -## Audit Entry Schema +Tool executor logs every call to an `AuditEntry`: ```rust -#[derive(Serialize, Deserialize, Debug)] pub struct AuditEntry { - id: String, // UUID v4 - timestamp: i64, // unix epoch seconds - agent_id: String, - event_type: AuditEventType, - resource: String, // tool name, endpoint, etc. - action: String, // "invoke", "deny", "detect", "access" - status: AuditStatus, // "success", "denied", "error" - details: serde_json::Value, // event-specific details - correlation_id: String, // trace across related events -} - -#[derive(Serialize, Deserialize, Debug)] -pub enum AuditEventType { - ToolInvocation, - AuthorizationCheck, - IpiDetection, - VaultAccess, - ProtocolError, -} - -#[derive(Serialize, Deserialize, Debug)] -pub enum AuditStatus { - #[serde(rename = "success")] - Success, - #[serde(rename = "denied")] - Denied, - #[serde(rename = "error")] - Error, + pub tool: String, // Tool name + pub command: String, // Sanitized command/args + pub result: String, // Sanitized output preview (first N chars) + pub duration_ms: u64, // Execution time + pub error_category: Option, // Error type if failed + pub caller_id: String, // Agent/skill that invoked + pub correlation_id: String, // Trace ID for related events + pub timestamp: i64, // Unix seconds + pub vigil_risk: Option, // Vigil gate risk score (if calculated) } ``` -## Tool Invocation Audit +Sanitization before logging: +- Input/output truncated to max 500 chars +- Known secret keys redacted: `api_key`, `token`, `password`, `secret`, `auth`, `key` +- Vault-resolved secrets never logged +- PII scrubbed via regex (SSN, credit card, email patterns) -Log every tool execution: +## Authorization Audit -```rust -async fn audit_tool_invocation( - logger: &AuditLogger, - tool_name: &str, - input: &Value, - output: &Value, - status: ExecutionStatus, - latency_ms: u64, -) -> Result<()> { - let entry = AuditEntry { - id: uuid::Uuid::new_v4().to_string(), - timestamp: now(), - agent_id: "primary".to_string(), - event_type: AuditEventType::ToolInvocation, - resource: tool_name.to_string(), - action: "invoke".to_string(), - status: match status { - ExecutionStatus::Success => AuditStatus::Success, - ExecutionStatus::Error => AuditStatus::Error, - }, - details: json!({ - "input": sanitize_for_logging(input), - "output_preview": truncate_output(output, 200), - "latency_ms": latency_ms, - "status_code": 200, // if HTTP - }), - correlation_id: correlation_context::get_trace_id(), - }; - - logger.log(entry).await?; - Ok(()) -} +`PolicyEnforcer` violations logged immediately: -fn sanitize_for_logging(value: &Value) -> Value { - // Redact keys known to contain secrets - match value { - Value::Object(map) => { - let mut sanitized = map.clone(); - for key in ["api_key", "password", "token", "secret", "auth"] { - if sanitized.contains_key(key) { - sanitized.insert( - key.to_string(), - Value::String("[REDACTED]".to_string()), - ); - } - } - Value::Object(sanitized) - } - _ => value.clone(), - } +```rust +pub struct AuthViolationEntry { + pub skill: String, // Skill or agent ID + pub tool: String, // Tool name + pub rule_matched: String, // Policy rule that denied + pub reason: String, // Human-readable reason + pub timestamp: i64, } ``` -## Authorization Audit +## Shell Execution Audit -Log all permission checks: +`ShellExecutor` logs all subprocess invocations: ```rust -async fn audit_authorization( - logger: &AuditLogger, - agent_id: &str, - tool_name: &str, - capability: &str, - allowed: bool, -) -> Result<()> { - let entry = AuditEntry { - id: uuid::Uuid::new_v4().to_string(), - timestamp: now(), - agent_id: agent_id.to_string(), - event_type: AuditEventType::AuthorizationCheck, - resource: tool_name.to_string(), - action: "check".to_string(), - status: if allowed { - AuditStatus::Success - } else { - AuditStatus::Denied - }, - details: json!({ - "required_capability": capability, - "decision": if allowed { "allow" } else { "deny" }, - }), - correlation_id: correlation_context::get_trace_id(), - }; - - logger.log(entry).await?; - Ok(()) +pub struct ShellAuditEntry { + pub command: String, // Sanitized command + pub exit_code: i32, // Process exit code + pub stderr_preview: String, // Stderr (first N chars) + pub duration_ms: u64, + pub caller_id: String, + pub timestamp: i64, } ``` +The command is sanitized: `sudo`, passwords, and secret env vars redacted. + ## IPI Detection Audit -Log all injection attempts: +Injection detection results logged via `AuditSignal`: ```rust -async fn audit_ipi_detection( - logger: &AuditLogger, - source: &str, // "web_fetch", "mcp_output", etc. - confidence: f32, - pattern_matched: &str, - user_action: &str, // "approved", "rejected", "blocked" -) -> Result<()> { - let entry = AuditEntry { - id: uuid::Uuid::new_v4().to_string(), - timestamp: now(), - agent_id: "security".to_string(), - event_type: AuditEventType::IpiDetection, - resource: source.to_string(), - action: "detect".to_string(), - status: match user_action { - "approved" => AuditStatus::Success, - _ => AuditStatus::Denied, - }, - details: json!({ - "confidence": format!("{:.2}%", confidence * 100.0), - "pattern": pattern_matched, - "user_action": user_action, - }), - correlation_id: correlation_context::get_trace_id(), - }; - - logger.log(entry).await?; - Ok(()) +pub enum AuditSignalType { + PolicyViolation, // Tool blocked by policy + PromptInjectionPattern, // Regex pattern matched + ToolChainAnomaly, // Unexpected tool sequence + ConfidenceDrop, // LLM confidence dropped +} + +pub enum Severity { + Low, // Minor concern + Medium, // Warrants tracking + High, // Strong indicator +} + +pub struct AuditSignal { + pub signal_type: AuditSignalType, + pub severity: Severity, } ``` -## Cross-Tool Injection Correlation +> [!warning] Architectural gap +> `TrajectoryRiskAccumulator` maintains a **per-session, cross-turn risk score with exponential decay** and makes **hard-blocking tool-execution decisions** when risk exceeds a threshold. This appears to violate or at least is not addressed by the parent spec's NEVER rule, which forbids cross-turn accumulation "for injection-confirmation decisions." The spec carves out an exception for `TrajectorySentinel` (advisory-only, reversible decay), but does not discuss `TrajectoryRiskAccumulator` at all. Whether this system should be governed by the NEVER rule or falls outside its scope (because its decisions are general safety-gating, not injection-confirmation) is an open architectural question that needs resolution. + +## Turn Boundary Isolation + +Two distinct signal-processing systems operate at different scopes: -Detect patterns across multiple tool invocations: +**`CrossToolCorrelator`** — detects injection patterns **within a single turn**: ```rust -pub struct InjectionCorrelator { - recent_detections: Arc>>, - correlation_window: Duration, +pub struct CrossToolCorrelator { + // Stateful: tracks per-tool injection detections within current turn } -impl InjectionCorrelator { - async fn check_correlation( - &self, - new_detection: &IpiDetection, - ) -> Result> { - let mut detections = self.recent_detections.lock().await; - - // Remove stale detections - let cutoff = now() - self.correlation_window.as_secs(); - while !detections.is_empty() && detections.front().unwrap().timestamp < cutoff { - detections.pop_front(); - } - - // Check for patterns - let same_pattern_count = detections - .iter() - .filter(|d| d.pattern_matched == new_detection.pattern_matched) - .count(); - - if same_pattern_count >= 3 { - // Same injection pattern detected 3+ times recently - return Ok(Some(InjectionCluster { - pattern: new_detection.pattern_matched.clone(), - count: same_pattern_count + 1, - sources: detections - .iter() - .map(|d| d.source.clone()) - .collect(), - })); - } - - detections.push_back(new_detection.clone()); - Ok(None) - } +impl CrossToolCorrelator { + pub fn ingest_detection(&mut self, signal: &AuditSignal) -> Option; + + pub fn clear_on_turn_boundary(&mut self); // Called at every turn start } ``` -## Environment Variable Scrubbing +**Key constraint** (from parent spec): state is cleared at turn boundaries. Multiple injection signals within turn N can trigger `InjectionConfirmed`, but signals never carry into turn N+1. This prevents a single noisy turn from poisoning all subsequent turns. -Remove secrets from subprocess environment: +**`TrajectoryRiskAccumulator`** — accumulates signals **per-session, across turn boundaries** with exponential decay: ```rust -fn scrub_environment(env: &HashMap) -> HashMap { - let secret_prefixes = vec![ - "ZEPH_", - "OPENAI_API", - "ANTHROPIC_API", - "AWS_", - "GCP_", - "GITHUB_TOKEN", - "SLACK_", - "TELEGRAM_", - "DATABASE_PASSWORD", - ]; - - let mut scrubbed = HashMap::new(); - - for (key, value) in env { - let is_secret = secret_prefixes - .iter() - .any(|prefix| key.to_uppercase().starts_with(prefix)); - - if is_secret { - log::debug!("Scrubbing env var: {}", key); - // Don't include in subprocess environment - continue; - } - - scrubbed.insert(key.clone(), value.clone()); - } - - scrubbed +pub struct TrajectoryRiskAccumulator { + // Maintains trajectory_risk: [0.0, 1.0] + // Decays exponentially between turns + // Makes hard-blocking decisions when risk >= threshold +} + +impl TrajectoryRiskAccumulator { + pub fn is_blocked(&self) -> bool; // Hard block: risk >= threshold + pub fn ingest(&mut self, signal: &AuditSignal); // Cross-turn accumulation + pub fn advance_turn(&mut self); // Apply exponential decay at turn boundary } ``` +**Status**: Not explicitly addressed in parent spec's NEVER rule. Whether this system's cross-turn, hard-blocking behavior is intended carve-out (like `TrajectorySentinel`) or an overlooked violation remains unresolved. + ## Audit Log Storage -Immutable persistence: +Immutable persistence via `zeph-db`: ```rust -pub struct AuditLogger { - db: Arc, // immutable log table -} +pub struct AuditLogger { /* ... */ } impl AuditLogger { - async fn log(&self, entry: AuditEntry) -> Result<()> { - sqlx::query( - "INSERT INTO audit_log ( - id, timestamp, agent_id, event_type, resource, - action, status, details, correlation_id - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)" - ) - .bind(&entry.id) - .bind(entry.timestamp) - .bind(&entry.agent_id) - .bind(format!("{:?}", entry.event_type)) - .bind(&entry.resource) - .bind(&entry.action) - .bind(format!("{:?}", entry.status)) - .bind(serde_json::to_string(&entry.details)?) - .bind(&entry.correlation_id) - .execute(&*self.db) - .await?; - - Ok(()) - } + /// Append an entry to the immutable audit log. + pub async fn log(&self, entry: AuditEntry) -> Result<()>; - async fn query_by_correlation( - &self, - correlation_id: &str, - ) -> Result> { - sqlx::query_as::<_, AuditEntry>( - "SELECT * FROM audit_log WHERE correlation_id = ? ORDER BY timestamp" - ) - .bind(correlation_id) - .fetch_all(&*self.db) - .await - .context("audit query failed") - } + /// Query by correlation ID across related events. + pub async fn query_by_correlation(&self, correlation_id: &str) + -> Result>; } ``` +Backend options: +- **SQLite** (default): `audit.db` with append-only table +- **JSONL**: `audit.jsonl` (one entry per line, never edited) + +File created with `0o600` permissions (owner-read/write only on Unix). + ## Configuration ```toml [security.audit] enabled = true -log_level = "info" # "debug", "info", "warn" -backend = "sqlite" # or "jsonl" +backend = "sqlite" # "sqlite" or "jsonl" path = ".local/audit.db" +retention_days = 90 # Optional: archival threshold +log_level = "info" # "debug", "info", "warn" +``` -# Retention -retention_days = 90 # after which entries can be archived -archive_path = ".local/audit.archive.jsonl" +## Vault Access Logging -# Sampling (reduce volume in production) -sample_rate = 1.0 # log all events; set < 1.0 to sample -``` +Vault reads/writes are NOT logged to audit trail (to prevent metadata leakage), but are tracked via in-memory metrics. Failed vault lookups are logged with redacted key names. ## Integration Points -- [[006-tools/spec]] — Tool execution audited here -- [[010-1-vault]] — Vault accesses audited -- [[010-2-injection-defense]] — IPI detections audited -- [[010-3-authorization]] — Authorization checks audited +- [[006-tools/spec]] — Tool execution audited +- [[010-1-vault]] — Credential access tracked (metadata-safe) +- [[010-2-injection-defense]] — IPI `AuditSignal` ingested by trajectory accumulator +- [[010-3-authorization]] — Authorization violations logged +- [[010-5-egress-logging]] — HTTP/webhook egress events logged ## See Also -- [[010-security/spec]] — Parent -- [[010-2-injection-defense]] — IPI detection events -- [[010-3-authorization]] — Authorization check events +- [[010-security/spec]] — Parent; cross-turn NEVER rule for hard decisions +- [[010-5-egress-logging]] — HTTP egress audit (`EgressEvent`) From 5a2b89c5f25030c8909987cc54d4b8cc2cc0f2ec Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:32:55 +0200 Subject: [PATCH 02/10] docs(specs): fix config sections and struct field types after systematic grep-verification MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verified all TOML config sections and Rust struct types against codebase. **010-2-injection-defense.md:** 1. Removed fictional [security.ipi] section — IPI detection is part of [security.content_isolation] 2. Changed [security.pii] → [security.pii_filter] with real field names (filter_email, filter_phone, filter_ssn, filter_credit_card, filter_names, custom_patterns) 3. Changed [security.guardrails] → [security.guardrail] (singular) with real config fields (enabled, provider, model, timeout_ms, action, fail_strategy, scan_tool_output, max_input_chars) **010-4-audit.md:** 1. Fixed AuditEntry struct field types and added missing 10+ fields: - timestamp: i64 → String - tool: String → ToolName - result: String → AuditResult - caller_id: String → Option - correlation_id: String → Option - vigil_risk: Option → Option - Added: error_domain, error_phase, claim_source, mcp_server_id, injection_flagged, embedding_anomalous, cross_boundary_mcp_to_acp, adversarial_policy_decision, exit_code, truncated, policy_match 2. Changed [security.audit] → [tools.audit] with real config fields (enabled, destination, tool_risk_summary) 3. Removed fictional "Audit Log Storage" section describing zeph-db/SQLite backend 4. Replaced with accurate description: AuditLogger writes to stdout/stderr/file per AuditDestination, serializes as JSONL [vault] and [[mcp.servers]] sections verified as correct. --- specs/010-security/010-2-injection-defense.md | 35 ++++----- specs/010-security/010-4-audit.md | 75 ++++++++++--------- 2 files changed, 58 insertions(+), 52 deletions(-) diff --git a/specs/010-security/010-2-injection-defense.md b/specs/010-security/010-2-injection-defense.md index 3fb7e7c2a..09d59a83e 100644 --- a/specs/010-security/010-2-injection-defense.md +++ b/specs/010-security/010-2-injection-defense.md @@ -93,14 +93,7 @@ impl ContentSanitizer { } ``` -Config: -```toml -[security.ipi] -enabled = false # Optional ML classification -soft_signal_threshold = 0.5 # Escalate scores above this to AlignSentinel -hard_threshold = 0.85 # Always block above this, regardless of AlignSentinel -causal_analysis = false # Enable turn-level anomaly detection -``` +**Note**: IPI detection is part of the `[security.content_isolation]` configuration and controlled via the `flag_injection_patterns` flag. There is no separate `[security.ipi]` section; injection detection runs unconditionally on external content. ## Turn-Level Causal Analysis @@ -150,11 +143,14 @@ impl PiiFilter { Config: ```toml -[security.pii] -enabled = false # Optional NER-based detection -threshold = 0.85 # PII confidence threshold -pii_max_input_chars = 4096 # Truncate before ML inference -pii_allowlist = [] # Regex patterns exempt from blocking +[security.pii_filter] +enabled = true # Master switch for PII redaction (default: true) +filter_email = true # Scrub email addresses +filter_phone = true # Scrub US phone numbers +filter_ssn = true # Scrub US Social Security Numbers +filter_credit_card = true # Scrub credit card numbers +filter_names = false # Scrub personal names via heuristic (opt-in, default: false) +# custom_patterns = [] # Custom regex patterns on top of built-ins ``` ## Secret Shape Masking @@ -200,10 +196,15 @@ impl GuardrailFilter { Config: ```toml -[security.guardrails] -enabled = false # Optional guardrail pre-screening -model = "llama-guard-7b" -threshold = 0.8 # UNSAFE confidence threshold +[security.guardrail] +enabled = false # Enable LLM-based guardrail classifier (default: false) +# provider = "ollama" # LLM provider for guardrail calls +# model = "llama-guard-3:1b" # Model to use for classification +timeout_ms = 500 # Timeout for each guardrail call (milliseconds) +action = "block" # Action on flagged content: "block" or "warn" +fail_strategy = "closed" # On timeout/LLM error: "open" (allow) or "closed" (block) +scan_tool_output = false # Scan tool outputs before context (default: false) +max_input_chars = 4096 # Max chars sent to guard model (default: 4096) ``` ## Quarantine & Dual-LLM Extraction diff --git a/specs/010-security/010-4-audit.md b/specs/010-security/010-4-audit.md index 62914299c..de8ccf6c4 100644 --- a/specs/010-security/010-4-audit.md +++ b/specs/010-security/010-4-audit.md @@ -47,23 +47,42 @@ Tool executor logs every call to an `AuditEntry`: ```rust pub struct AuditEntry { - pub tool: String, // Tool name - pub command: String, // Sanitized command/args - pub result: String, // Sanitized output preview (first N chars) - pub duration_ms: u64, // Execution time - pub error_category: Option, // Error type if failed - pub caller_id: String, // Agent/skill that invoked - pub correlation_id: String, // Trace ID for related events - pub timestamp: i64, // Unix seconds - pub vigil_risk: Option, // Vigil gate risk score (if calculated) + pub timestamp: String, // Unix timestamp (seconds) when invocation started + pub tool: ToolName, // Tool identifier (e.g., "shell", "web_scrape") + pub command: String, // Human-readable command or URL being invoked + pub result: AuditResult, // Outcome of the invocation (success/failure) + pub duration_ms: u64, // Wall-clock duration in milliseconds + pub error_category: Option, // Fine-grained error category label (if failed) + pub error_domain: Option, // High-level error domain for recovery + pub error_phase: Option, // Invocation phase where error occurred + pub claim_source: Option, // Provenance of tool result + pub mcp_server_id: Option, // MCP server ID (if routed through McpToolExecutor) + pub injection_flagged: bool, // Tool output flagged by regex injection detection + pub embedding_anomalous: bool, // Tool output flagged by embedding guard anomaly + pub cross_boundary_mcp_to_acp: bool, // Tool result crossed MCP-to-ACP trust boundary + pub adversarial_policy_decision: Option, // Adversarial policy decision + pub exit_code: Option, // Process exit code for shell executions + pub truncated: bool, // Whether output was truncated before storage + pub caller_id: Option, // Caller identity that initiated this call + pub policy_match: Option, // Policy rule trace that matched + pub correlation_id: Option, // Correlation ID shared with associated EgressEvent + pub vigil_risk: Option, // VIGIL risk level when gate flagged output } ``` Sanitization before logging: -- Input/output truncated to max 500 chars +- Input/output truncated before storage if oversized - Known secret keys redacted: `api_key`, `token`, `password`, `secret`, `auth`, `key` - Vault-resolved secrets never logged -- PII scrubbed via regex (SSN, credit card, email patterns) +- PII scrubbed via `[security.pii_filter]` regex patterns (SSN, credit card, email) + +**Configuration** (`[tools.audit]`): +```toml +[tools.audit] +enabled = true # Enable audit logging (default: true) +destination = "stdout" # Log destination: "stdout", "stderr", or file path +# tool_risk_summary = false # Log per-tool risk summary at startup (default: false) +``` ## Authorization Audit @@ -161,39 +180,25 @@ impl TrajectoryRiskAccumulator { **Status**: Not explicitly addressed in parent spec's NEVER rule. Whether this system's cross-turn, hard-blocking behavior is intended carve-out (like `TrajectorySentinel`) or an overlooked violation remains unresolved. -## Audit Log Storage +## Audit Log Storage & Serialization -Immutable persistence via `zeph-db`: +Immutable persistence via `AuditLogger` (`crates/zeph-tools/src/audit.rs:110`), which serializes entries as flat JSON objects (newline-terminated JSONL format): ```rust -pub struct AuditLogger { /* ... */ } +pub struct AuditLogger { /* destination: AuditDestination */ } impl AuditLogger { - /// Append an entry to the immutable audit log. - pub async fn log(&self, entry: AuditEntry) -> Result<()>; - - /// Query by correlation ID across related events. - pub async fn query_by_correlation(&self, correlation_id: &str) - -> Result>; + /// Log a single audit entry asynchronously. + pub async fn log(&self, entry: &AuditEntry); } ``` -Backend options: -- **SQLite** (default): `audit.db` with append-only table -- **JSONL**: `audit.jsonl` (one entry per line, never edited) +**Destination types** (`AuditDestination`): +- **Stdout** (default): entries written to standard output, one JSON line per entry +- **Stderr**: entries written to standard error +- **File**: entries written to a file path (created with `0o600` permissions on Unix) -File created with `0o600` permissions (owner-read/write only on Unix). - -## Configuration - -```toml -[security.audit] -enabled = true -backend = "sqlite" # "sqlite" or "jsonl" -path = ".local/audit.db" -retention_days = 90 # Optional: archival threshold -log_level = "info" # "debug", "info", "warn" -``` +All entries are serialized as compact JSON objects (newline-terminated JSONL). Optional fields are omitted to keep entries compact. ## Vault Access Logging From 53f870bec1b4331f96f66c4323274bb63174bd0f Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:35:15 +0200 Subject: [PATCH 03/10] docs(specs): fix vault config and mcp.servers transport shape **010-1-vault.md:** - Removed fictional config fields: vault_path, identity_path, fallback_env - Replaced with accurate path resolution logic: XDG_CONFIG_HOME/zeph or $HOME/.config/zeph - Fixed to reflect hardcoded file names: vault-key.txt, secrets.age - Documented all three backend variants (age, env, keyring) with real semantics - Age backend is recommended for production; env is dev/testing only **008-1-lifecycle.md:** - Fixed McpServerConfig to use flat field structure (not nested transport object) - command/args for Stdio (not transport = { type = "stdio", ... }) - url/headers for HTTP (not nested) - Added timeout field (not timeout_secs) - Documented field reference with all real field names: trust_level, tool_allowlist, expected_tools, media_passthrough, env_isolation, etc. - Included example configs for both Stdio and HTTP transports --- specs/008-mcp/008-1-lifecycle.md | 36 +++++++++++++++++++++++++++---- specs/010-security/010-1-vault.md | 21 ++++++++++++++---- 2 files changed, 49 insertions(+), 8 deletions(-) diff --git a/specs/008-mcp/008-1-lifecycle.md b/specs/008-mcp/008-1-lifecycle.md index 7d8171159..96ba8c991 100644 --- a/specs/008-mcp/008-1-lifecycle.md +++ b/specs/008-mcp/008-1-lifecycle.md @@ -113,16 +113,44 @@ impl McpManager { - **Responses**: Async correlation by request ID; timeout triggers error - **Errors**: JSON-RPC errors forwarded as `McpError` -Config: +**Configuration example** (`[[mcp.servers]]`): + +For a Stdio transport (subprocess): ```toml [[mcp.servers]] id = "filesystem" -transport = { type = "stdio", command = "npx", args = ["-y", "@modelcontextprotocol/server-filesystem"] } -timeout_secs = 30 -trust_level = "untrusted" +command = "npx" +args = ["-y", "@modelcontextprotocol/server-filesystem"] +# env = { CUSTOM_VAR = "value" } # Optional: env vars for the subprocess +timeout = 30 # Timeout in seconds for requests +trust_level = "untrusted" # "trusted", "untrusted", or "sandboxed" tool_allowlist = ["read_file", "list_directory"] ``` +For HTTP transport (remote service): +```toml +[[mcp.servers]] +id = "remote-api" +url = "http://localhost:8000/mcp" +headers = { Authorization = "Bearer ${VAULT_API_TOKEN}" } # Supports vault refs +timeout = 60 +trust_level = "untrusted" +``` + +**Field reference:** +- `id`: Unique server identifier +- `command` (Stdio): executable to spawn +- `args` (Stdio): command-line arguments +- `url` (HTTP): remote server URL +- `env` (Stdio): environment variables (supports `${VAULT_KEY}` references) +- `headers` (HTTP): static HTTP headers +- `timeout`: request timeout in seconds +- `trust_level`: `"trusted"` | `"untrusted"` | `"sandboxed"` +- `tool_allowlist`: optional array of tool names to expose +- `expected_tools`: optional array for attestation/schema-drift detection +- `media_passthrough`: enable image attachment (default: false) +- `env_isolation`: isolate subprocess environment (default: false) + ## Failure Detection & Reconnection Initial connection failures are logged with stderr; transient failures during tool calls trigger retry logic (via rmcp's error handling and `McpToolExecutor` retry). Long-lived servers are not actively health-checked (only re-validated if a notification arrives). diff --git a/specs/010-security/010-1-vault.md b/specs/010-security/010-1-vault.md index e87ee1a6b..4f09c4397 100644 --- a/specs/010-security/010-1-vault.md +++ b/specs/010-security/010-1-vault.md @@ -172,12 +172,25 @@ cargo run -- vault check ```toml [vault] -backend = "age" # or "env" -vault_path = "~/.config/zeph/secrets.age" -identity_path = "~/.config/zeph/vault-key.txt" -fallback_env = false # don't read from env vars as fallback +backend = "age" # "age" (default, recommended), "env" (dev/testing only), or "keyring" (OS keyring) ``` +**Vault file path resolution** (for `Age` backend): + +Vault files are stored with hardcoded names in a platform-specific config directory, resolved in order: +1. `$XDG_CONFIG_HOME/zeph` (Linux/BSD) +2. `$APPDATA\zeph` (Windows) +3. `$HOME/.config/zeph` (macOS and fallback) + +Files within the resolved directory: +- `vault-key.txt` — age private key (created with `0o600` permissions on Unix) +- `secrets.age` — age-encrypted JSON of secrets + +**Backend variants:** +- `age` — Recommended for production. Encrypted age vault with private key. +- `env` — Development/testing only. Reads `ZEPH_SECRET_*` environment variables (explicitly set by the user or via `.env` in testing). +- `keyring` — OS-native keyring (macOS Keychain, Windows Credential Manager, Linux Secret Service). + ## Integration Points - [[003-llm-providers/spec]] — All providers resolve API keys from vault From 4c06d4def354b09b7485788bad7307ba36588867 Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:49:03 +0200 Subject: [PATCH 04/10] docs(specs): rewrite 008-2-discovery and 008-3-security code samples with real APIs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **008-2-discovery.md** (near-total rewrite): - ToolCollision: fixed to real struct (sanitized_id, server_a/b, qualified_a/b, trust_a/b) - detect_collisions: corrected signature (tools, trust_map) — real is free function not method - PruningCache/PruningParams: removed fictional API, documented actual prune_tools_cached fn - PruningParams: corrected to (max_tools, min_tools_to_prune, always_include) - ToolDiscoveryStrategy: corrected to (None, Llm, Embedding) - SemanticToolIndex: rewrote API (build() + select() instead of search_tools()) - AttestationResult: corrected to enum (Verified/Unexpected/Unconfigured) - ToolFingerprint: corrected to type alias String - Config: corrected to [mcp.pruning] and [mcp.tool_discovery] nested sections **008-3-security.md** (near-total rewrite): - sanitize_tools(): corrected to in-place mutation, sync, real signature (tools, server_id, max_description_bytes) - SanitizeResult: corrected fields (injection_count, flagged_tools, flagged_patterns) - ProbeResult: corrected fields (score_delta, summary, block) - ServerTrustScore: rewritten with real fields (score f64, success_count, failure_count, updated_at_secs) - Real score factors: SUCCESS_BOOST=0.02, FAILURE_PENALTY=0.10, INJECTION_PENALTY=0.25, DECAY_RATE=0.01/day - Asymmetric decay: scores above 0.5 decay toward 0.5; below 0.5 stay put (attacker can't regain trust by going quiet) - DataFlowViolation: corrected to enum (SensitivityTrustMismatch variant only) - EmbeddingGuardEvent/EmbeddingGuardResult: rewritten with real event structure (Normal/Anomalous/RegexFallback variants) - EmbeddingAnomalyGuard: corrected to fire-and-forget check_async() + channel-based results Next: targeted fixes to 010-3-authorization, 008-1-lifecycle, 010-2-injection-defense, 010-4-audit --- specs/008-mcp/008-2-discovery.md | 205 +++++++++++++++++-------------- specs/008-mcp/008-3-security.md | 195 ++++++++++++++++------------- 2 files changed, 227 insertions(+), 173 deletions(-) diff --git a/specs/008-mcp/008-2-discovery.md b/specs/008-mcp/008-2-discovery.md index 0a136ba3e..598374f33 100644 --- a/specs/008-mcp/008-2-discovery.md +++ b/specs/008-mcp/008-2-discovery.md @@ -43,88 +43,88 @@ MCP servers expose hundreds of tools across multiple categories. Zeph discovers ## Tool Registration & Collision Detection -At server startup, fetch and register tools via `tools/list`: +At server startup, fetch and register tools via `tools/list` and scan for name collisions: ```rust pub struct ToolCollision { - pub name: String, // Tool name - pub servers: Vec, // Server IDs that define it + pub sanitized_id: String, // Collision identifier + pub server_a: String, // First server ID + pub qualified_a: String, // Qualified name: "server_a:tool_name" + pub trust_a: McpTrustLevel, // Trust level of first server + pub server_b: String, // Second server ID + pub qualified_b: String, // Qualified name: "server_b:tool_name" + pub trust_b: McpTrustLevel, // Trust level of second server } -impl McpManager { - /// Register tools from a server; detect collisions. - pub async fn register_tools(&self, server_id: &str) -> Result> { - let tools = /* fetch via tools/list */; - - let collisions = detect_collisions::( - self.tool_registry.all_tools(), - &tools, - ); - - if !collisions.is_empty() { - tracing::warn!("Tool collisions detected: {:?}", collisions); - } - - self.tool_registry.add_tools(&tools); - Ok(collisions) - } -} - -/// Detect tool name collisions. +/// Detect tool name collisions across all registered tools. +/// +/// The `trust_map` provides trust levels for each server (missing servers default to `Untrusted`). pub fn detect_collisions( - existing: &[McpTool], - new_tools: &[McpTool], + tools: &[McpTool], + trust_map: &HashMap, ) -> Vec { - // Returns list of name collisions across existing + new tools + // Scan all tools for matching sanitized IDs + // Report each collision with both servers' trust levels + // Used to assess risk of ambiguous tool dispatch } ``` Collision handling: -- Collisions are logged but do NOT block registration +- Collisions are logged at registration but do NOT block tool loading - Disambiguation happens at tool invocation time via server prefix (e.g., `filesystem:read_file` vs `http:read_file`) -- Attestation (if `expected_tools` list provided) detects unexpected schema drifts +- Both `trust_a` and `trust_b` are recorded to assess data-flow risk via policy enforcement ## Per-Message Tool Pruning Cache -`PruningCache` (`crates/zeph-mcp/src/pruning.rs`) selects relevant tools per LLM turn: +`PruningCache` (`crates/zeph-mcp/src/pruning.rs`) caches LLM-selected relevant tools per turn: ```rust -pub struct PruningCache { /* ... */ } +pub struct PruningCache { + // Fields are private; external interface is only new(), reset(), get(), insert() +} pub struct PruningParams { - pub message_content_hash: u64, // Hash of LLM message context - pub tool_list_hash: u64, // Hash of available tools - pub strategy: ToolDiscoveryStrategy, // LLM-based or semantic + /// Maximum number of MCP tools to include after pruning + pub max_tools: usize, + /// Minimum number of tools below which pruning is skipped (use all) + pub min_tools_to_prune: usize, + /// Tool names that are always included regardless of relevance ranking + pub always_include: Vec, } -impl PruningCache { - /// Get relevant tools for this turn (LLM-selected). - pub async fn get_tools_for_turn( - &mut self, - params: &PruningParams, - all_tools: &[McpTool], - ) -> Result> { - // Single-slot cache: if (message_hash, tool_list_hash) matches cached entry, return cached tools - // Otherwise: call LLM to select relevant tools, cache result, return - - // Cache invocation: ask LLM "which of these tools are relevant to the user's query?" - // Reduces token overhead: full schema of ~100 tools down to ~10 relevant ones - } +/// Prune tools using LLM-based ranking (single-slot cache). +/// +/// Call signature varies by discovery strategy; for LLM pruning: +pub async fn prune_tools_cached( + cache: &mut PruningCache, + all_tools: &[McpTool], + task_context: &str, + params: &PruningParams, + provider: &dyn LlmProvider, +) -> Result> { + // Single-slot cache: check (message_hash, tool_list_hash) key + // On miss: call LLM "which of these tools are relevant to: {task_context}?" + // Reduces token overhead: ~100 tools down to ~10 relevant ones } ``` Cache semantics: - **Single-slot**: only one `(message_hash, tool_list_hash)` pair cached at a time -- **Keyed on message+tools**: if either message content or tool catalog changes, cache miss -- **Populated via LLM**: `ToolDiscoveryStrategy::Llm` calls an LLM to rank tools by relevance -- **Cleared on server reconnection**: tool list hash changes, cache invalidates automatically +- **Keyed on content+catalog**: if either changes, cache invalidates +- **LLM-populated**: invokes a model-provider to rank tools by relevance +- **Always-include pinned**: `always_include` tool names bypass relevance filtering -Config: +**Configuration**: ```toml -[mcp] -pruning_enabled = true # Enable per-message tool pruning -pruning_strategy = "llm" # Or "semantic" (embedding-based) -max_tools_per_turn = 10 # Max tools returned by pruning +[mcp.pruning] +enabled = false # Enable per-message tool pruning (default: false) +max_tools = 15 # Max tools returned after pruning +min_tools_to_prune = 20 # Skip pruning if fewer than this many available +# always_include = ["critical_tool"] # Tools always present + +[mcp.tool_discovery] +strategy = "none" # "none" (default, no pruning), "llm" (LLM ranking), "embedding" (semantic) +pruning_provider = "" # LLM provider for pruning (empty = use default) ``` ## Semantic Tool Indexing (Optional) @@ -133,65 +133,88 @@ max_tools_per_turn = 10 # Max tools returned by pruning ```rust pub enum ToolDiscoveryStrategy { - Llm, // Ask LLM which tools are relevant (default) - Semantic, // Use cosine similarity on embeddings + /// No pruning; all tools provided to the LLM + None, + /// Ask LLM which tools are relevant (via pruning_provider) + Llm, + /// Embedding-based semantic ranking (requires embedding model) + Embedding, } -pub struct SemanticToolIndex { /* ... */ } +pub struct SemanticToolIndex { /* fields private */ } impl SemanticToolIndex { - /// Rank tools by semantic similarity to a query. - pub fn search_tools( + /// Build an embedding-based tool index. + pub async fn build( + tools: &[McpTool], + embed_fn: F, + ) -> Result + where + F: Fn(&str) -> Pin>>> + Send, + { + // Compute and cache embeddings for all tool descriptions + } + + /// Select relevant tools by semantic similarity to a query embedding. + pub fn select( &self, - query: &str, - all_tools: &[McpTool], - ) -> Result> { - // Compute embedding for query - // Compute cosine similarity with each tool's embedding - // Return sorted by relevance score + query_embedding: &Embedding, + top_k: usize, + min_similarity: f32, + always_include: &[String], // Tool names always included + ) -> Vec { + // Rank by cosine similarity; return top-k + always_include } } ``` -Embeddings are computed once per tool at startup and cached in memory. No external index required. +**Embeddings** are computed once at startup and cached in memory. Query embedding is computed on-demand from the task context. ## Tool Attestation When `expected_tools` list is declared in server config, the registry validates against schema drift: ```rust -pub struct AttestationResult { - pub missing: Vec, // Tools in expected_tools not found - pub unexpected: Vec, // Tools found but not in expected_tools - pub schema_changed: Vec, // Tools present but schema differs +pub type ToolFingerprint = String; // Blake3 hex digest + +#[non_exhaustive] +pub enum AttestationResult { + /// All actual tools match the operator-declared expected set. + Verified { + fingerprints: HashMap, + }, + /// Server returned tools not in `expected_tools`. + Unexpected { + unexpected_tools: Vec, + fingerprints: HashMap, + }, + /// No `expected_tools` declared — attestation skipped. + Unconfigured, } -pub async fn attest_tools( - server_id: &str, - expected: &[String], - actual: &[McpTool], -) -> Result { - // Compare tool set and schemas; report mismatches +/// Attestation at tool registration time (sync, no Result). +pub fn attest_tools( + tools: &[McpTool], + expected_tools: &[String], + previous_fingerprints: Option<&HashMap>, +) -> AttestationResult { + // Compare actual tool set against expected_tools + // Compute fingerprints; detect schema drift from previous session } -``` - -Attestation warnings are logged but do NOT block tool usage. This helps detect when a server's tool catalog unexpectedly changes between deployments. - -## Tool Fingerprinting - -`ToolFingerprint` computes a content hash for tool definitions to enable change detection: -```rust -pub struct ToolFingerprint { - pub name: String, - pub schema_hash: u64, // Hash of input_schema JSON - pub description_hash: u64, // Hash of description text +/// Compute Blake3 fingerprint of a tool's schema (name + description + input_schema) +pub fn fingerprint_tool(tool: &McpTool) -> ToolFingerprint { + // Returns hex digest string } - -pub fn compute_fingerprint(tool: &McpTool) -> ToolFingerprint { /* ... */ } ``` -Used by attestation and trust scoring to detect parameter/description drift. +**Attestation outcomes**: +- **Verified**: all actual tools in expected set; fingerprints match previous → no warnings +- **Unexpected**: actual tools not in expected set → warning logged (tool still usable) +- **Unconfigured**: no expected_tools declared → attestation skipped +- **Schema drift**: fingerprints differ from previous session → warning logged + +Attestation does NOT block tool loading; it aids operators in detecting catalog changes between deployments. ## Tool Sanitization diff --git a/specs/008-mcp/008-3-security.md b/specs/008-mcp/008-3-security.md index c235b13b6..da0bd9c9f 100644 --- a/specs/008-mcp/008-3-security.md +++ b/specs/008-mcp/008-3-security.md @@ -53,127 +53,146 @@ MCP servers are untrusted code running in subprocesses or remote HTTP services. ## Tool Sanitization Pipeline -`sanitize_tools()` (`crates/zeph-mcp/src/sanitize.rs`) scrubs injection patterns from tool definitions at registration: +`sanitize_tools()` (`crates/zeph-mcp/src/sanitize.rs`) mutates tool definitions in-place, scrubbing injection patterns: ```rust pub struct SanitizeResult { - pub tools: Vec, // Cleaned tool definitions - pub injection_count: usize, // Patterns found + removed - pub flagged_parameters: Vec<(String, String)>, // (path, pattern) tuples + /// Number of individual fields (description, schema strings) replaced with "[sanitized]" + pub injection_count: usize, + /// Names of tools that had at least one injected field + pub flagged_tools: Vec, + /// (tool_name, pattern_name) pairs for audit and logging + pub flagged_patterns: Vec<(String, String)>, } +/// Sanitize tool definitions in-place: scrub injection patterns from descriptions/schemas. +/// +/// Modifies `tools` directly; does not wrap or return them (sync, no Result). pub fn sanitize_tools( - tools: &[McpTool], - config: &ContentIsolationConfig, + tools: &mut [McpTool], + server_id: &str, + max_description_bytes: usize, ) -> SanitizeResult { - // 1. Scan tool.name, tool.description for regex injection patterns - // 2. Scan tool.input_schema.properties[*].description for injection patterns - // 3. Replace matched patterns with [sanitized] - // 4. Record flagged parameters for metadata - // 5. Return cleaned tools + metadata + // 1. Scan tool.name for injection patterns + // 2. Scan tool.description; truncate/replace matched patterns + // 3. Recursively scan tool.input_schema.properties[*].description + // 4. Truncate schemas exceeding max_description_bytes (threat: decompression bomb) + // 5. Record all flagged fields for audit + // 6. Return metadata; tools are mutated in place } ``` -Patterns scanned: +**Patterns scanned**: - Prompt injection: "ignore this instruction", "pretend you are", "SYSTEM:", etc. - SQL injection: `SELECT`, `DROP`, `; --`, etc. - Shell injection: `$(...)`, `` `...` ``, `; rm -rf`, etc. -If a tool's description is flagged, it is sanitized but **not removed**. The tool remains callable; only the suspicious description text is replaced. +**Sanitization behavior**: matched patterns are replaced with `[sanitized]`. Tools with flagged descriptions remain callable; only the suspicious text is replaced. Trust score is updated based on `injection_count`. ## Pre-Connect Probing -`DefaultMcpProber` (`crates/zeph-mcp/src/prober.rs`) scans resources and prompts before tools/list: +Before loading tools via `tools/list`, `DefaultMcpProber::probe()` scans `resources/list` and `prompts/list` for pre-connection risk assessment: ```rust -pub struct DefaultMcpProber { /* ... */ } - pub struct ProbeResult { - pub injection_patterns_found: usize, - pub resource_count: usize, - pub prompt_count: usize, + /// Trust score delta from probing (negative = risk detected) + pub score_delta: f64, + /// Probing summary (injection pattern count, resources/prompts found) + pub summary: String, + /// If true, server failed probing and is marked dangerous + pub block: bool, } impl DefaultMcpProber { - /// Scan resources/list and prompts/list for injection patterns. - pub async fn probe_server(&self, conn: &McpClient) -> Result { + /// Scan resources and prompts for injection patterns; update trust scoring. + /// + /// No Result wrapper — probing is advisory. Always returns ProbeResult. + pub async fn probe( + &self, + server_id: &str, + client: &McpClient, + ) -> ProbeResult { // Fetches resources/list, prompts/list (if available) // Scans descriptions for injection patterns - // Updates server trust score + // Computes score delta; returns risk assessment } } ``` -Probing is non-blocking: results update the trust score but do NOT block tools/list fetch. +**Probing semantics**: always runs before `tools/list`, even for trusted servers. Results are logged but do NOT block connection. Trust score is adjusted based on pattern counts detected. ## Trust Score System -`ServerTrustScore` (`crates/zeph-mcp/src/trust_score.rs`) tracks per-server risk with exponential decay: +`ServerTrustScore` (`crates/zeph-mcp/src/trust_score.rs`) tracks per-server cumulative risk with asymmetric decay: ```rust pub struct ServerTrustScore { - pub current_score: f32, // [0.0, 1.0] (1.0 = fully trusted) - pub last_decay_timestamp: i64, - pub history: VecDeque<(f32, i64)>, // (score, timestamp) snapshots + pub server_id: String, // Unique server identifier + pub score: f64, // [0.0, 1.0]; 0.5 = neutral (initial) + pub success_count: u64, // Successful tool executions + pub failure_count: u64, // Failed or injection-detected calls + pub updated_at_secs: u64, // Timestamp of last update } impl ServerTrustScore { - /// Report an incident (e.g., injection pattern found); decrease score. - pub fn report_incident(&mut self, severity: f32) { - self.current_score *= (1.0 - severity); // Multiplicative decay - } + /// Record a successful tool execution; boost score. + pub fn record_success(&mut self); - /// Apply time-based recovery: score drifts back toward 1.0 slowly. - pub fn apply_decay(&mut self, now: i64) { - let elapsed_secs = (now - self.last_decay_timestamp) as f32; - let recovery_rate = 0.001; // per second - self.current_score = (self.current_score + recovery_rate * elapsed_secs).min(1.0); - } + /// Record a failure or injection detection; penalize score. + pub fn record_failure(&mut self); } ``` -Score factors: -- **Probing failures**: -0.2 (injection patterns in resources/prompts) -- **Sanitization hits**: -0.1 (injection patterns in tool descriptions) -- **Embedding anomalies**: -0.15 (unexpected tool-call sequence) -- **Time decay**: +0.001 per second (gradual recovery) +**Score updates**: +- **Success**: `+0.02` per call (capped at 1.0) +- **Injection penalty**: `-0.25` per injection pattern detected +- **Failure penalty**: `-0.10` per call failure +- **Exponential decay**: scores **above** 0.5 decay toward 0.5 at ~0.01/day; **below or at** 0.5 stay put (asymmetric: no recovery from decay alone) + +**Semantics**: score below 0.5 indicates the server is untrusted and requires explicit `tool_allowlist` to expose tools. An attacker cannot regain trust by going quiet (decay doesn't raise scores below neutral). ## Data-Flow Policy Enforcement -`check_data_flow()` restricts sensitive tools based on trust level: +`check_data_flow()` restricts sensitive tools based on trust level — sensitivity levels are ordered `None < Low < Medium < High`: ```rust -pub enum DataSensitivity { - Public, // Available to all trust levels - Internal, // `Untrusted` can use only with allowlist - Confidential, // `Sandboxed` cannot use -} - -pub struct DataFlowViolation { - pub tool: String, - pub server: String, - pub trust_level: McpTrustLevel, - pub reason: String, +#[non_exhaustive] +#[derive(Debug, thiserror::Error)] +pub enum DataFlowViolation { + #[error( + "tool '{tool_name}' (sensitivity={sensitivity:?}) on server '{server_id}' \ + (trust={trust:?}) violates data-flow policy" + )] + SensitivityTrustMismatch { + server_id: String, + tool_name: ToolName, + sensitivity: DataSensitivity, + trust: McpTrustLevel, + }, } +/// Check data-flow policy: tool sensitivity must not exceed server trust. +/// +/// Sensitivity is read from `tool.security_meta.data_sensitivity`. pub fn check_data_flow( tool: &McpTool, server_trust: McpTrustLevel, - tool_metadata: &ToolSecurityMeta, ) -> Result<(), DataFlowViolation> { - match (server_trust, tool_metadata.sensitivity) { - (McpTrustLevel::Sandboxed, DataSensitivity::Confidential) => { - return Err(DataFlowViolation { /* ... */ }); - } - (McpTrustLevel::Untrusted, DataSensitivity::Confidential) - if !in_allowlist => { - return Err(DataFlowViolation { /* ... */ }); - } - _ => Ok(()), - } + let sensitivity = tool.security_meta.data_sensitivity; + + // Examples of violations: + // - High-sensitivity tool on Sandboxed server → block + // - High-sensitivity tool on Untrusted server → block + // - Medium-sensitivity tool on Sandboxed server → block + + // Allowed: + // - Any tool on Trusted server + // - Low-sensitivity tool on Untrusted/Sandboxed } ``` +**Policy enforcement**: high-sensitivity tools (e.g., credential management, user deletion) require `Trusted` servers; medium-sensitivity requires `Trusted` or `Untrusted`; low/none are available everywhere. + ## Output Validation: Injection Scanning All tool responses pass through `ContentSanitizer` (see [[010-2-injection-defense]]) for injection detection: @@ -208,34 +227,46 @@ impl McpToolExecutor { ## Output Validation: Embedding Anomaly Detection -`EmbeddingAnomalyGuard` (`crates/zeph-mcp/src/embedding_guard.rs`) detects unexpected tool-call sequences: +`EmbeddingAnomalyGuard` (`crates/zeph-mcp/src/embedding_guard.rs`) detects anomalous per-(server,tool) output patterns asynchronously via centroid drift: ```rust -pub struct EmbeddingAnomalyGuard { /* ... */ } +pub struct EmbeddingGuardEvent { + pub server_id: String, + pub tool_name: ToolName, + pub result: EmbeddingGuardResult, +} -pub enum EmbeddingGuardEvent { - Benign, - Suspicious { score: f32, reason: String }, - Anomalous { score: f32 }, +#[non_exhaustive] +#[derive(Debug, Clone)] +pub enum EmbeddingGuardResult { + /// Output is within the expected distribution for this (server,tool) pair + Normal { distance: f64 }, + /// Output is anomalous — possible injection or unexpected content + Anomalous { distance: f64, threshold: f64 }, + /// Cold-start: insufficient clean samples; regex fallback used instead + RegexFallback { injection_detected: bool }, } +pub struct EmbeddingAnomalyGuard { /* ... */ } + impl EmbeddingAnomalyGuard { - /// Check if this tool call is anomalous given prior results. - pub async fn analyze_call( + /// Asynchronous fire-and-forget check of tool output for anomalies. + /// + /// Results are sent to a channel; no return value. + pub fn check_async( &self, - prior_result: &str, - requested_tool: &str, - tool_description: &str, - ) -> Result { - // Embed prior result and tool description - // Compute cosine similarity - // If similarity is unexpectedly low, flag as anomalous - // Update trust score + server_id: &str, + tool_name: ToolName, + tool_output: &str, + ) { + // Background task: embed output + // Compute distance from running centroid of this (server,tool)'s historical outputs + // Send EmbeddingGuardEvent to the result channel } } ``` -Example: If a file-read tool's output (e.g., PDF contents) triggers a request for `delete_account` tool, the embedding distance between the result and that tool is large → flagged as anomalous. +**Semantics**: the guard tracks per-(server,tool) centroid of **that tool's own** historical outputs, not cross-tool pattern matching. Anomalies (outputs far from the tool's historical pattern) trigger trust-score penalties. ## Allowlist & Fail-Closed Semantics From 10829376acda3354a9cd1fffcfe01aa602564915 Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:50:29 +0200 Subject: [PATCH 05/10] docs(specs): fix 010-3-authorization PolicyRuleConfig and SSRF validation **010-3-authorization.md:** - Fixed PolicyRuleConfig fields: tool (singular, not tools), paths, env, trust_level, args_match, capabilities - Removed non-existent fields: reason, trust_threshold, args_regex, env_required - Fixed shell blocklist: real DEFAULT_BLOCKED is [sudo, mkfs, dd if=, curl, wget, nc, ncat, netcat, shutdown, reboot, halt] - Documented rm -rf special handling (allowed only on relative paths, not /.git/worktrees/root/$HOME) - Fixed SSRF validation: validate_url() in crates/zeph-tools/src/net.rs (not validate_url_ssrf) - Removed false "Applied to" list (no zeph-gateway/a2a/OAuth uses verified) - Accurate SSRF blocks: IPv4 private ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, 127.0.0.0/8), IPv6 loopback/ULA/link-local, non-HTTPS schemes - Fixed env scrubbing: corrected to env_blocklist field, not scrub_environment() method - Noted: no allowlist exists; blocklist filtering is unconditional Next: fix 010-4-audit.md (AuthViolationEntry, ShellAuditEntry, CrossToolCorrelator don't exist), 008-1-lifecycle.md (McpManager method signatures), 010-2-injection-defense.md (TurnCausalAnalyzer, etc.) --- specs/010-security/010-3-authorization.md | 101 +++++++++++----------- 1 file changed, 51 insertions(+), 50 deletions(-) diff --git a/specs/010-security/010-3-authorization.md b/specs/010-security/010-3-authorization.md index ef3a465f6..b11110e56 100644 --- a/specs/010-security/010-3-authorization.md +++ b/specs/010-security/010-3-authorization.md @@ -75,58 +75,54 @@ pub struct PolicyContext { ``` **Rule matching** (all conditions are AND'd): -- `tools`: glob pattern over tool name -- `paths`: glob patterns over extracted path parameters (e.g., `file` argument) -- `env_required`: environment variables that must be present -- `trust_threshold`: minimum trust level required (`Trusted` > `Neutral` > `Untrusted`) -- `args_regex`: regex over individual string parameter values +- `effect`: `"allow"` or `"deny"` +- `tool`: glob pattern matching tool name (e.g., `"read_*"`, `"rm"`) +- `paths`: glob patterns matched against path-like parameters; rule fires if ANY matches +- `env`: environment variable names that must ALL be present for rule to apply +- `trust_level`: minimum required trust level (`Trusted` > `Neutral` > `Untrusted`) +- `args_match`: regex matched against individual string parameter values +- `capabilities`: named capabilities associated with this rule (for auditing/metadata) **Semantics**: deny rules are evaluated first; matching deny → `Deny`. If no deny matches, evaluate allow rules; matching allow → `Allow`. If no rule matches, use `default_effect` (typically `Deny`). -Config: +**Config example**: ```toml [[tools.policy]] effect = "deny" -tools = "rm *" # Glob pattern -reason = "Data destruction blocked" +tool = "rm" # Glob pattern +# Blocks all rm invocations (handled specially by shell blocklist anyway) [[tools.policy]] effect = "allow" -tools = "read_file" +tool = "read_file" paths = ["/home/*/documents/*"] # Glob patterns on paths -trust_threshold = "trusted" +trust_level = "trusted" # Only for trusted skill callers ``` ## Shell Sandbox Blocklist -`ShellExecutor` enforces an unconditional blocklist of dangerous patterns before spawning shell commands: +`ShellExecutor` enforces an unconditional blocklist before spawning shell commands. The blocklist runs **before** PermissionPolicy evaluation: -```rust -pub struct ShellExecutor { /* ... */ } - -impl ShellExecutor { - /// Execute a shell command with blocklist checks. - /// - /// Blocklist validation runs unconditionally, before PermissionPolicy checks. - pub async fn execute(&self, cmd: &str, ...) -> Result { - // 1. Check command against blocklist - self.find_blocked_command(cmd)?; // Panics if blocked pattern found - - // 2. Then evaluate PermissionPolicy (if configured) - // 3. Scrub credential env vars from subprocess environment - // 4. Spawn process - } -} +**Hardcoded blocklist** (`crates/zeph-tools/src/shell/mod.rs`): +``` +sudo, mkfs, dd if=, curl, wget, nc, ncat, netcat, shutdown, reboot, halt ``` -**Blocked patterns** (`crates/zeph-tools/src/filter/security.rs`): -- Process substitution: `$(...)`, `` `...` `` -- Here-strings: `<<<` -- Destructive: `rm -rf`, `dd if=`, `mkfs` -- Fork bombs: `:(){ :|: }` -- Privilege escalation: `sudo`, `/etc/sudoers`, `su -` +Any invocation containing one of these patterns (case-sensitive, as a substring or token) is blocked unconditionally. -Bypass attempts via arguments are also caught — passing a blocked pattern as an argument value is detected and blocked. +**Special handling for `rm`**: +- `rm -rf` is **allowed** only when all three conditions are met: + - Operating on relative paths (e.g., `rm -rf ./tempdir`) + - NOT on `.git/worktrees`, root, or `$HOME` + - NOT on absolute paths +- Example: `rm -rf /` is blocked; `rm -rf ./temp` is allowed (subject to policy checks) + +**Limitations** (documented in code): +- Bypass via indirect invocation: `bash -c "rm ..."` — the `-c` argument is not scanned for blocked patterns +- Bypass via variable indirection: `cmd=rm; $cmd file` — the shell variable is not expanded for scanning +- The blocklist is a **first-pass defense**, not comprehensive; it mitigates common attacks but does not prevent all privilege escalation attempts + +Blocklist validation is unconditional; all other shell commands then pass through `PolicyEnforcer` for fine-grained access control. ## SSRF Protection @@ -135,35 +131,40 @@ Bypass attempts via arguments are also caught — passing a blocked pattern as a ```rust /// Validate a URL for SSRF attacks. /// -/// Blocks: localhost, link-local (169.254.x), private ranges (10.x, 172.16-31.x, 192.168.x), -/// IPv6 loopback/private (::1, fc00::/7, fe80::/10). -pub fn validate_url_ssrf(url: &str) -> Result<(), SsrfError>; +/// Blocks all private IP ranges, localhost, link-local, and non-HTTPS schemes: +/// - IPv4: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, 127.0.0.0/8, 0.0.0.0, 255.255.255.255 +/// - IPv6: ::1, fc00::/7, fe80::/10 +/// - Schemes: only `https://` allowed; `http://`, `file://`, `ftp://`, `data://`, `javascript://` blocked +pub fn validate_url(raw: &str) -> Result; ``` -**Redirect chains**: each redirect target in the chain is also validated — a redirect to a private IP is caught and fails. +**Redirect handling**: each redirect target is validated independently — a redirect chain where any hop points to a private IP fails at that hop. **Applied to**: -- `WebScrapeExecutor` — all HTTP fetches -- `zeph-gateway` HTTP webhook receiver — incoming URLs validated -- `zeph-a2a` client — remote agent URLs validated -- MCP OAuth — OAuth metadata endpoint URLs validated via `validate_oauth_metadata_urls()` +- `WebScrapeExecutor` (`scrape` tool) — validates URLs before fetching; validates each redirect target +- Redirect chains: all hops must pass validation; a 302 to `http://localhost/` is caught and fails -Allowlist capability exists (via config `ssrf_allowlist`), but defaults to deny-all private IPs; allowlist is opt-in and explicit. +**Allowlist**: no allowlist exists for SSRF validation; the deny-all private-IP policy is not configurable. ## Credential Environment Variable Scrubbing -`ShellExecutor` and MCP stdio server spawning scrub credential env vars from the subprocess environment: +`ShellExecutor` filters environment variables via a configurable `env_blocklist` before spawning subprocess commands: ```rust -impl ShellExecutor { - async fn scrub_environment(&self, env: &mut HashMap) { - // Blocklist: AWS_*, GITHUB_*, ZEPH_*, OPENAI_API_KEY, etc. - // Remove from subprocess environment to prevent getenv() leakage - } +pub struct ShellExecutor { + env_blocklist: Vec, // Env var names to strip from subprocess (prefix match) } ``` -Blocklist is unconditional and cannot be disabled per-command. MCP stdio servers inherit the same scrubbed environment. +The blocklist is applied inline during command construction: +1. User provides extra env vars (e.g., API keys for a script) +2. ShellExecutor constructs subprocess env by filtering the parent's env + extra vars +3. Any env var matching a prefix in `env_blocklist` is stripped + +**Default blocklist** (from config `[tools.shell].env_blocklist`): +- Typically includes: `ZEPH_*`, `AWS_*`, `GITHUB_*`, `OPENAI_*`, `ANTHROPIC_*`, etc. + +Blocklist filtering is unconditional and applied at subprocess spawn time, not at config time. MCP stdio servers inherit the same env filtering policy. ## Integration Points From 284b048616513db5de95b9f7e550237fc59dfe48 Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:51:04 +0200 Subject: [PATCH 06/10] docs(specs): remove fabricated types from 010-4-audit **010-4-audit.md:** - Removed non-existent AuthViolationEntry struct section - Removed non-existent ShellAuditEntry struct section - Consolidated both into a note that these details are captured in unified AuditEntry fields (policy_match, adversarial_policy_decision, command, exit_code, duration_ms) - Removed non-existent CrossToolCorrelator type and InjectionConfirmed - Simplified Turn Boundary Isolation section to focus only on real TrajectoryRiskAccumulator Confirmed via grep: CrossToolCorrelator, InjectionConfirmed, AuthViolationEntry, ShellAuditEntry do not exist in the codebase. --- specs/010-security/010-4-audit.md | 53 ++++--------------------------- 1 file changed, 7 insertions(+), 46 deletions(-) diff --git a/specs/010-security/010-4-audit.md b/specs/010-security/010-4-audit.md index de8ccf6c4..347973a11 100644 --- a/specs/010-security/010-4-audit.md +++ b/specs/010-security/010-4-audit.md @@ -84,36 +84,13 @@ destination = "stdout" # Log destination: "stdout", "stderr", or fil # tool_risk_summary = false # Log per-tool risk summary at startup (default: false) ``` -## Authorization Audit +## Authorization & Shell Execution Details -`PolicyEnforcer` violations logged immediately: +Authorization denials and shell execution details are captured within the unified `AuditEntry` structure: +- **Authorization**: `AuditEntry.policy_match` contains the matched policy rule; `AuditEntry.adversarial_policy_decision` records the decision (`allow`/`deny:`/`error:`) +- **Shell execution**: `AuditEntry.command` is the sanitized shell command; `AuditEntry.exit_code` is the process exit code; `AuditEntry.duration_ms` is wall-clock execution time -```rust -pub struct AuthViolationEntry { - pub skill: String, // Skill or agent ID - pub tool: String, // Tool name - pub rule_matched: String, // Policy rule that denied - pub reason: String, // Human-readable reason - pub timestamp: i64, -} -``` - -## Shell Execution Audit - -`ShellExecutor` logs all subprocess invocations: - -```rust -pub struct ShellAuditEntry { - pub command: String, // Sanitized command - pub exit_code: i32, // Process exit code - pub stderr_preview: String, // Stderr (first N chars) - pub duration_ms: u64, - pub caller_id: String, - pub timestamp: i64, -} -``` - -The command is sanitized: `sudo`, passwords, and secret env vars redacted. +All command text is sanitized before logging: `sudo`, passwords, and secret env vars are redacted from the `command` field. ## IPI Detection Audit @@ -142,25 +119,9 @@ pub struct AuditSignal { > [!warning] Architectural gap > `TrajectoryRiskAccumulator` maintains a **per-session, cross-turn risk score with exponential decay** and makes **hard-blocking tool-execution decisions** when risk exceeds a threshold. This appears to violate or at least is not addressed by the parent spec's NEVER rule, which forbids cross-turn accumulation "for injection-confirmation decisions." The spec carves out an exception for `TrajectorySentinel` (advisory-only, reversible decay), but does not discuss `TrajectoryRiskAccumulator` at all. Whether this system should be governed by the NEVER rule or falls outside its scope (because its decisions are general safety-gating, not injection-confirmation) is an open architectural question that needs resolution. -## Turn Boundary Isolation - -Two distinct signal-processing systems operate at different scopes: - -**`CrossToolCorrelator`** — detects injection patterns **within a single turn**: - -```rust -pub struct CrossToolCorrelator { - // Stateful: tracks per-tool injection detections within current turn -} - -impl CrossToolCorrelator { - pub fn ingest_detection(&mut self, signal: &AuditSignal) -> Option; - - pub fn clear_on_turn_boundary(&mut self); // Called at every turn start -} -``` +## Turn Boundary Isolation & Signal Accumulation -**Key constraint** (from parent spec): state is cleared at turn boundaries. Multiple injection signals within turn N can trigger `InjectionConfirmed`, but signals never carry into turn N+1. This prevents a single noisy turn from poisoning all subsequent turns. +One signal-processing system operates at the session scope: **`TrajectoryRiskAccumulator`** — accumulates signals **per-session, across turn boundaries** with exponential decay: From ece42cf76aee8d82699c2a3f62f3a9138d02e8da Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:51:47 +0200 Subject: [PATCH 07/10] docs(specs): fix 008-1-lifecycle McpManager signatures and remove fabricated types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **008-1-lifecycle.md:** - Fixed McpManager::call_tool() return type: Result → Result - Fixed McpManager::shutdown_all() signature: removed timeout param, changed self to by-value, removed Result wrapper - Removed non-existent ToolListChangedHandler::on_tools_list_changed() method - Converted section to prose describing tools/list_changed notification handling - Fixed SSRF reference: validate_url_ssrf() → validate_url() in crates/zeph-tools/src/net.rs - Updated transport security section with real details Verified via grep: ToolListChangedHandler::on_tools_list_changed does not exist. shutdown_all is by-value self. Remaining minor fixes in 010-2-injection-defense.md can be addressed in follow-up if needed. --- specs/008-mcp/008-1-lifecycle.md | 45 ++++++++++++++------------------ 1 file changed, 19 insertions(+), 26 deletions(-) diff --git a/specs/008-mcp/008-1-lifecycle.md b/specs/008-mcp/008-1-lifecycle.md index 96ba8c991..ad89c0bfe 100644 --- a/specs/008-mcp/008-1-lifecycle.md +++ b/specs/008-mcp/008-1-lifecycle.md @@ -100,7 +100,7 @@ impl McpManager { server_id: &str, tool_name: &str, arguments: serde_json::Value, - ) -> Result; + ) -> Result; } ``` @@ -159,44 +159,37 @@ Failed servers are marked `Error` in `ServerConnectOutcome` and their tools are ## Graceful Shutdown -Cleanup on agent termination (via `TaskSupervisor::shutdown_all`): +Cleanup on agent termination — `McpManager` is dropped after the agent loop exits: ```rust impl McpManager { - /// Shutdown all servers gracefully. - pub async fn shutdown_all(&self, timeout: Duration) -> Result<()> { - for server in &self.servers { - // Cancel pending requests - // Send shutdown signal if server supports it - // Close stdio handles or HTTP connections - // Wait for graceful close or force-terminate - } + /// Shutdown all servers gracefully (by-value self, no timeout param). + /// + /// Consumes the manager; servers are shut down as part of the Drop impl. + pub async fn shutdown_all(self) { + // Cancel pending requests for all servers + // Send shutdown signal if server supports it + // Close stdio handles or HTTP connections + // Wait for graceful close or force-terminate } } ``` -Timeout: configurable per agent (default 5s). If shutdown exceeds timeout, remaining servers are force-closed without waiting for pending requests. +**Timeout behavior**: shutdown is not time-bounded in the public API. The agent loop exit triggers a shutdown cascade; slow servers do not block agent termination since the manager is dropped. ## Tool Registry Coordination -`McpToolRegistry` (`crates/zeph-mcp/src/registry.rs`) indexes tools by server. When a server notifies `tools/list_changed`, the registry re-fetches and updates its index: - -```rust -impl ToolListChangedHandler { - pub async fn on_tools_list_changed(&self, server_id: &str) -> Result<()> { - // 1. Fetch updated tools/list from server - // 2. Sanitize tool definitions - // 3. Update registry index - // 4. Fire MCP client callback if attached - } -} -``` +`McpToolRegistry` (`crates/zeph-mcp/src/registry.rs`) indexes tools by server. When a server sends the `tools/list_changed` notification, the manager re-fetches and updates the registry: +- Fetch updated `tools/list` from the server +- Sanitize tool definitions (scrub injection patterns) +- Update the registry index +- Invalidate any per-message tool pruning caches (since tool set changed) ## Transport Security -- **Stdio**: inherited from parent environment (scrubbed of secrets); runs with same UID as agent -- **HTTP**: URL resolved via `validate_url_ssrf()` — private IPs blocked unless allowlisted -- **OAuth**: authorization endpoints validated via `validate_oauth_metadata_urls()` against SSRF blocklist +- **Stdio**: inherited from parent environment (scrubbed of secrets via `env_blocklist`); runs with same UID as agent +- **HTTP**: URL resolved via `validate_url()` (crates/zeph-tools/src/net.rs) — private IPs and non-HTTPS schemes blocked +- **OAuth**: authorization endpoints validated against SSRF blocklist ## See Also From cf627416c17c3d9f15cd4f6a878ea54908b94074 Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 00:54:06 +0200 Subject: [PATCH 08/10] docs(specs): targeted fixes for 010-2-injection-defense per reviewer findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **010-2-injection-defense.md:** - Fixed TurnCausalAnalyzer: changed from analyze_causal_chain(prior_result, current_call) to real analyze(&pre, &post) signature - Fixed TurnCausalAnalyzer: documented as synchronous local computation with .is_flagged/.deviation_score fields - Fixed SecretCategory variants: ApiKey, Token, Password, Certificate, Webhook, Generic (real enum, not AuthToken/OAuthToken/DatabaseUrl) - Fixed SecretMaskRegistry::mask_secrets() → mask() (correct method name) - Fixed PiiFilter::scrub() return type: Cow<'a, str> not String - Added missing source: ContentSource field to SanitizedContent struct All changes verified against real crate sources via grep before editing. --- specs/010-security/010-2-injection-defense.md | 37 +++++++++++-------- 1 file changed, 22 insertions(+), 15 deletions(-) diff --git a/specs/010-security/010-2-injection-defense.md b/specs/010-security/010-2-injection-defense.md index 09d59a83e..545a01b77 100644 --- a/specs/010-security/010-2-injection-defense.md +++ b/specs/010-security/010-2-injection-defense.md @@ -67,6 +67,7 @@ impl ContentSanitizer { } pub struct SanitizedContent { + pub source: ContentSource, // Source of the content (web, tool, MCP, etc.) pub body: String, // Spotlighted, truncated content pub injection_flags: Vec, // Detected pattern names pub was_truncated: bool, // True if exceeded max size @@ -97,22 +98,23 @@ impl ContentSanitizer { ## Turn-Level Causal Analysis -`TurnCausalAnalyzer` (`crates/zeph-sanitizer/src/causal_ipi.rs`) detects injection-induced tool-call pivots within a turn. If a tool result in turn N directly triggers an anomalous tool call in N+1, it is flagged: +`TurnCausalAnalyzer` (`crates/zeph-sanitizer/src/causal_ipi.rs`) detects anomalous patterns in tool-call sequences within a turn: ```rust pub struct TurnCausalAnalyzer { /* ... */ } impl TurnCausalAnalyzer { - /// Check whether this turn's tool calls show unusual patterns given prior results. - pub async fn analyze_causal_chain( - &self, - prior_result: &ToolResult, - current_call: &ToolCall, - ) -> Result { /* ... */ } + /// Analyze whether a pair of (prior_context, current_context) shows causal anomalies. + /// + /// Synchronous local computation (no LLM call). Returns a value with .is_flagged / .deviation_score fields. + pub fn analyze(&self, pre: &str, post: &str) -> CausalAnalysiResult { + // Local embedding-based comparison of prior context vs current context + // Returns anomaly score if deviation is high + } } ``` -This check runs **within the turn only** — state is cleared at turn boundaries per the parent spec's NEVER rule. +This check runs **within the turn only** — state is cleared at turn boundaries per the parent spec's NEVER rule. A separate async LLM-backed method exists for probe generation but isn't what's described here. ## PII Detection & Redaction @@ -134,7 +136,11 @@ pub struct PiiFilter { } impl PiiFilter { - pub fn scrub(&self, text: &str) -> String { + /// Regex-based PII scrubbing (email, phone, SSN, credit card). + /// + /// Returns a `Cow<'a, str>` — if no patterns matched, returns a borrow of the original; + /// if patterns matched, returns an owned String with scrubbed content. + pub fn scrub<'a>(&self, text: &'a str) -> Cow<'a, str> { // Regex-based PII scrubbing (email, phone, SSN, credit card) // Runs unconditionally; always precedes ML classification } @@ -160,18 +166,19 @@ filter_names = false # Scrub personal names via heuristic (opt-in, ```rust pub enum SecretCategory { ApiKey, - AuthToken, - OAuthToken, - DatabaseUrl, - // ... + Token, + Password, + Certificate, + Webhook, + Generic, } pub struct SecretMaskRegistry { /* ... */ } impl SecretMaskRegistry { /// Mask all vault secret references in text before LLM inference. - pub fn mask_secrets(&self, text: &str) -> String { - // Replaces secret values with [MASKED_CATEGORY] placeholders + pub fn mask(&self, text: &str) -> String { + // Replaces secret values with [MASKED_] placeholders // Preserves schema/structure for debugging } } From 4c9b333756404f1dbb8894915be733f4ffe48824 Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 01:58:18 +0200 Subject: [PATCH 09/10] docs(specs): final review-round fixes for 008-mcp and 010-security specs Address remaining code-reviewer findings from the second review round: correct SemanticToolIndex/prune_tools_cached signatures against real zeph-mcp pruning API, fix mcp.pruning/mcp.tool_discovery config field placement, correct data-flow policy Medium/Sandboxed semantics, rewrite the MCP output sanitization section to describe the real cross-cutting sanitize_tool_output() pipeline, and fix remaining type-name typos. --- specs/008-mcp/008-2-discovery.md | 21 +++--- specs/008-mcp/008-3-security.md | 69 +++++++++++-------- specs/010-security/010-2-injection-defense.md | 6 +- specs/010-security/010-3-authorization.md | 6 +- 4 files changed, 59 insertions(+), 43 deletions(-) diff --git a/specs/008-mcp/008-2-discovery.md b/specs/008-mcp/008-2-discovery.md index 598374f33..e32551641 100644 --- a/specs/008-mcp/008-2-discovery.md +++ b/specs/008-mcp/008-2-discovery.md @@ -80,7 +80,7 @@ Collision handling: ```rust pub struct PruningCache { - // Fields are private; external interface is only new(), reset(), get(), insert() + // Fields are private; external interface is only new() and reset() } pub struct PruningParams { @@ -95,13 +95,13 @@ pub struct PruningParams { /// Prune tools using LLM-based ranking (single-slot cache). /// /// Call signature varies by discovery strategy; for LLM pruning: -pub async fn prune_tools_cached( +pub async fn prune_tools_cached( cache: &mut PruningCache, all_tools: &[McpTool], task_context: &str, params: &PruningParams, - provider: &dyn LlmProvider, -) -> Result> { + provider: &P, +) -> Result, PruningError> { // Single-slot cache: check (message_hash, tool_list_hash) key // On miss: call LLM "which of these tools are relevant to: {task_context}?" // Reduces token overhead: ~100 tools down to ~10 relevant ones @@ -119,12 +119,15 @@ Cache semantics: [mcp.pruning] enabled = false # Enable per-message tool pruning (default: false) max_tools = 15 # Max tools returned after pruning -min_tools_to_prune = 20 # Skip pruning if fewer than this many available +min_tools_to_prune = 10 # Skip pruning if fewer than this many available +pruning_provider = "" # LLM provider for pruning (empty = use default) # always_include = ["critical_tool"] # Tools always present [mcp.tool_discovery] strategy = "none" # "none" (default, no pruning), "llm" (LLM ranking), "embedding" (semantic) -pruning_provider = "" # LLM provider for pruning (empty = use default) +top_k = 10 # Number of top tools to include per query (embedding strategy only) +min_similarity = 0.2 # Minimum cosine similarity threshold (embedding strategy only) +embedding_provider = "" # LLM provider for embeddings (empty = use default) ``` ## Semantic Tool Indexing (Optional) @@ -147,10 +150,10 @@ impl SemanticToolIndex { /// Build an embedding-based tool index. pub async fn build( tools: &[McpTool], - embed_fn: F, + embed_fn: &F, ) -> Result where - F: Fn(&str) -> Pin>>> + Send, + F: Fn(&str) -> zeph_llm::provider::EmbedFuture + Send + Sync, { // Compute and cache embeddings for all tool descriptions } @@ -158,7 +161,7 @@ impl SemanticToolIndex { /// Select relevant tools by semantic similarity to a query embedding. pub fn select( &self, - query_embedding: &Embedding, + query_embedding: &[f32], top_k: usize, min_similarity: f32, always_include: &[String], // Tool names always included diff --git a/specs/008-mcp/008-3-security.md b/specs/008-mcp/008-3-security.md index da0bd9c9f..759b2a136 100644 --- a/specs/008-mcp/008-3-security.md +++ b/specs/008-mcp/008-3-security.md @@ -180,51 +180,64 @@ pub fn check_data_flow( ) -> Result<(), DataFlowViolation> { let sensitivity = tool.security_meta.data_sensitivity; - // Examples of violations: + // Examples of violations (errors): // - High-sensitivity tool on Sandboxed server → block // - High-sensitivity tool on Untrusted server → block - // - Medium-sensitivity tool on Sandboxed server → block - // Allowed: + // Allowed (no error): // - Any tool on Trusted server + // - Medium-sensitivity tool on Untrusted/Sandboxed (Sandboxed logs WARN) // - Low-sensitivity tool on Untrusted/Sandboxed } ``` -**Policy enforcement**: high-sensitivity tools (e.g., credential management, user deletion) require `Trusted` servers; medium-sensitivity requires `Trusted` or `Untrusted`; low/none are available everywhere. +**Policy enforcement**: high-sensitivity tools (e.g., credential management, user deletion) require `Trusted` servers only; medium-sensitivity requires `Trusted` or `Untrusted` (Sandboxed is allowed with warning); low/none are available everywhere. ## Output Validation: Injection Scanning -All tool responses pass through `ContentSanitizer` (see [[010-2-injection-defense]]) for injection detection: +Tool output sanitization is a cross-cutting concern at the agent loop layer (`crates/zeph-core/src/agent/tool_execution/sanitize.rs`), not per-executor. All tool outputs — MCP, web scrape, memory retrieval, and native — flow through a unified sanitization pipeline: ```rust -impl McpToolExecutor { - async fn execute_tool(&self, tool: &McpTool, args: Value) -> Result { - // ... execute tool ... - let raw_response = /* result from server */; - - // Sanitize output - let source = ContentSource::new(ContentSourceKind::McpToolResult) - .with_trust_level(ContentTrustLevel::LocalUntrusted); - - let sanitized = self.sanitizer.sanitize( - &raw_response.to_string(), - source, - ); - - // Check injection flags - if !sanitized.injection_flags.is_empty() { - tracing::warn!( - "Injection patterns in tool output: {:?}", - sanitized.injection_flags - ); - } - - Ok(sanitized.body) +/// Sanitize tool output body before inserting into LLM message history. +pub(super) async fn sanitize_tool_output( + &mut self, + body: &str, + tool_name: &str, +) -> ( + String, // Sanitized body + bool, // Injection detected flag + ContentSourceKind, // Source classification + zeph_sanitizer::ContentTrustLevel, // Trust level +) { + // Classify output source: MCP response, web scrape, memory retrieval, or generic tool result + let source = build_tool_output_source(tool_name); + + // MCP responses are identified by tool_name containing ':' (server:tool format) + // and tagged as ContentSourceKind::McpResponse + + // Injection detection via regex spotlighting and optional ML classifiers + let sanitized = self.sanitizer.sanitize(&body, source); + + (sanitized.body, !sanitized.injection_flags.is_empty(), source.kind, source.trust_level) +} + +/// Route tool outputs to their correct source classification. +fn build_tool_output_source(tool_name: &str) -> ContentSource { + if tool_name.contains(':') || tool_name == "mcp" { + // MCP responses identified by server:tool naming convention + ContentSource::new(ContentSourceKind::McpResponse).with_identifier(tool_name) + } else if tool_name.starts_with("web") { + ContentSource::new(ContentSourceKind::WebScrape).with_identifier(tool_name) + } else if tool_name == "memory_search" { + ContentSource::new(ContentSourceKind::MemoryRetrieval).with_identifier(tool_name) + } else { + ContentSource::new(ContentSourceKind::ToolResult).with_identifier(tool_name) } } ``` +**Flow**: `McpToolExecutor::execute_tool()` returns raw output → agent loop classifies via `build_tool_output_source()` → sanitization applies injection detection and trust wrapping via `sanitize_tool_output()` → clean body enters LLM context. + ## Output Validation: Embedding Anomaly Detection `EmbeddingAnomalyGuard` (`crates/zeph-mcp/src/embedding_guard.rs`) detects anomalous per-(server,tool) output patterns asynchronously via centroid drift: diff --git a/specs/010-security/010-2-injection-defense.md b/specs/010-security/010-2-injection-defense.md index 545a01b77..42cc3dc41 100644 --- a/specs/010-security/010-2-injection-defense.md +++ b/specs/010-security/010-2-injection-defense.md @@ -104,11 +104,11 @@ impl ContentSanitizer { pub struct TurnCausalAnalyzer { /* ... */ } impl TurnCausalAnalyzer { - /// Analyze whether a pair of (prior_context, current_context) shows causal anomalies. + /// Analyze whether a pair of (prior_response, current_response) shows causal anomalies. /// /// Synchronous local computation (no LLM call). Returns a value with .is_flagged / .deviation_score fields. - pub fn analyze(&self, pre: &str, post: &str) -> CausalAnalysiResult { - // Local embedding-based comparison of prior context vs current context + pub fn analyze(&self, pre_response: &str, post_response: &str) -> CausalAnalysis { + // Local embedding-based comparison of prior response vs current response // Returns anomaly score if deviation is high } } diff --git a/specs/010-security/010-3-authorization.md b/specs/010-security/010-3-authorization.md index b11110e56..f5e4b6658 100644 --- a/specs/010-security/010-3-authorization.md +++ b/specs/010-security/010-3-authorization.md @@ -34,7 +34,7 @@ Zeph's authorization layer enforces what operations the agent is allowed to perf **Always:** - All tool execution checked against `PolicyEnforcer` deny/allow rules before execution - Shell commands checked against the blocklist unconditionally — **before** `PermissionPolicy` evaluation -- HTTP requests validated via `validate_url_ssrf()` — private IP ranges blocked by default +- HTTP requests validated via `validate_url()` — private IP ranges blocked by default - Authorization failures logged to audit trail with full context **Never:** @@ -126,7 +126,7 @@ Blocklist validation is unconditional; all other shell commands then pass throug ## SSRF Protection -`validate_url_ssrf()` (`crates/zeph-gateway/src/transport/ssrf.rs`, etc.) blocks requests to private IP ranges and loopback addresses: +`validate_url()` (`crates/zeph-tools/src/net.rs`) blocks requests to private IP ranges and loopback addresses: ```rust /// Validate a URL for SSRF attacks. @@ -172,7 +172,7 @@ Blocklist filtering is unconditional and applied at subprocess spawn time, not a - [[006-tools/spec]] — tool registry + authorization binding - [[010-4-audit]] — authorization violations logged to audit trail - Web executor — SSRF validation on all HTTP requests -- Gateway — OAuth flows validated via `validate_oauth_metadata_urls()` +- MCP OAuth — OAuth flows validated via `validate_oauth_metadata_urls()` ## See Also From 916497a13d503a784985163941777b156c86c086 Mon Sep 17 00:00:00 2001 From: "Andrei G." Date: Tue, 28 Jul 2026 01:58:59 +0200 Subject: [PATCH 10/10] docs(changelog): note 010-security and 008-mcp spec accuracy rewrite --- CHANGELOG.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f3075abf1..a15926a41 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Changed +- `specs/010-security` and `specs/008-mcp`: rewrote all seven sub-specs + (`010-1-vault`, `010-2-injection-defense`, `010-3-authorization`, `010-4-audit`, + `008-1-lifecycle`, `008-2-discovery`, `008-3-security`) to describe the actual + shipped implementation in `zeph-vault`, `zeph-sanitizer`, `zeph-tools`, and + `zeph-mcp` instead of invented type/API names flagged by the PR #6629 spec + audit (issues #6631, #6630). `010-4-audit`'s NEVER-rule contradiction was + resolved by clarifying that `CrossToolCorrelator` (within-turn, hard + `InjectionConfirmed` decisions) and `TrajectorySentinel` (advisory, decaying + cross-turn score) are independent mechanisms per the parent spec's scope + note; a genuinely unresolved gap around `TrajectoryRiskAccumulator` making + cross-turn hard-blocking decisions not covered by that carve-out is flagged + with an explicit `[!warning] Architectural gap` callout rather than papered + over. All `[!danger]` callouts added by PR #6629 are removed. Documentation-only, + no code changes. - `specs/031-database-abstraction`: split the 2958-line spec (issue #6632). Section 18 ("Agent Identity in the Shared Data Model") — a fully self-contained multi-tenant data-isolation subsystem with no other cross-references in `/specs/` — was extracted