Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ nxd resume → dispatcher → executor → agents (parallel per wave)
| `internal/cli/logs.go` | `nxd logs <story-id>` — trace JSONL viewer with `--follow`, `--lines`, `--raw` |
| `internal/cli/diff.go` | `nxd diff <story-id>` — worktree diff against base branch with `--stat`, `--cached` |
| `internal/cli/dashboard.go` | Wires event bus into WebSocket hub |
| `internal/security/` | Security agent brains (LLM-free): growable OWASP Top 10 + CWE `KnowledgeBase` (JSON-persisted, immutable `Add`, `Covers` by ID-or-CWE, `Checklist` for prompts), multi-tool scanner runner (gosec/govulncheck/gitleaks/semgrep/npm-audit, graceful degrade, real parsers), language detection, findings/report |
| `internal/engine/security_gate.go` | `SecurityGate`: `ScanRepo` (standalone `nxd security scan`) + `ReviewStory` (per-story pre-merge gate in `postExecutionPipeline`, after QA before merge). Scanners ∪ LLM threat-model review against the KB; pauses on findings ≥ gate severity; self-upskills the KB from confirmed findings |

## Build & Test

Expand All @@ -54,6 +56,15 @@ make mempalace-check # smoke the MemPalace bridge end-to-end
- Native Windows: all read-only commands work (`status`, `dashboard`, `doctor`, `config`, `events`, `metrics`, `report`, `projects`). Full agent pipeline (`req`/`resume`) needs tmux → run inside WSL2.
- Platform-specific code lives in `_unix.go` / `_windows.go` build-tagged pairs: `internal/cli/req_*.go` (daemon detach), `internal/engine/lockfile_*.go` (advisory lock + process liveness), `internal/devdb/docker/host_*.go` (docker default host). Shell command exec goes through `internal/shellexec` (`sh -c` on Unix, `cmd.exe /C` on Windows, override with `NXD_SHELL`).

## Current State (2026-06-26) — security agent (ported from VXD)

Self-upskilling security agent, mirrored from vortex-dispatch (offline-friendly: scanners are local binaries, LLM layer uses the configured Ollama/cloud client).
- **`internal/security/`** (LLM-free, unit-tested): growable OWASP Top 10 + CWE `KnowledgeBase` (JSON at `<state_dir>/security/knowledge.json`; `Add` immutable/versioned/dedup, `Covers` matches ID or CWE, `Checklist` renders for prompts) + a runner orchestrating **gosec, govulncheck, gitleaks, semgrep, npm audit** (language-aware applicability, PATH detection, graceful degrade — skipped tools are *listed*, never silently dropped; pure per-tool parsers → real findings).
- **`engine/security_gate.go`** `SecurityGate`: `ScanRepo` (standalone) + `ReviewStory` (per-story, in `postExecutionPipeline` after QA before merge via `Monitor.SetSecurityGate`, wired in `resume.go` — `TestResume_WiresSecurityGate`). Findings ≥ `security.gate_severity` (default **critical**) pause the requirement; never escalate. Self-upskilling: confirmed high+ findings of a new vuln class → learned KB rule + `SECURITY_RULE_LEARNED`.
- **CLI:** `nxd security scan [path] [--json] [--llm] [--min <sev>]` + `nxd security kb` (falls back to DefaultConfig so it runs in any repo). Planner prompt now carries the OWASP Top 10 so every story is designed secure.
- **Config:** `security.{disable_gate, gate_severity (default critical), auto_learn (default true), kb_path}`.
- **Verified:** scanned NXD itself (Go/JS/Python/Shell, all 5 scanners ran, crit=0). Full suite 31 pkgs + vet + golangci-lint clean. Mirrors VXD `internal/security` verbatim (zero VXD refs).

## Core Infrastructure: MemPalace

MemPalace is the local-first semantic memory layer NXD relies on for
Expand Down Expand Up @@ -240,6 +251,11 @@ Architectural ceilings (cannot reach 95% without major refactor):

## Event Types

Security agent events (added 2026-06-26):
- `STORY_SECURITY_PASSED` / `STORY_SECURITY_FAILED` — per-story security gate result; a FAILED gate pauses the requirement (human decision) rather than escalating
- `SECURITY_SCAN_COMPLETED` — a standalone `nxd security scan` finished (findings count, max severity)
- `SECURITY_RULE_LEARNED` — the agent added a new vulnerability class to the knowledge base from a confirmed finding (self-upskilling)

Controller events (added 2026-04-12):
- `CONTROLLER_ANALYSIS` — emitted each tick with stories_checked and actions_taken counts
- `CONTROLLER_ACTION` — emitted per corrective action with kind (cancel/restart/reprioritize) and reason
Expand Down
15 changes: 15 additions & 0 deletions docs/reference/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,21 @@ nxd learn [--force] [--pass <1|2|3>]

---

### nxd security

Security agent: scan a repository with deterministic scanners (gosec, govulncheck, gitleaks, semgrep, npm audit) plus an optional LLM threat-model review against a growable OWASP/CWE knowledge base that learns new vulnerability classes from confirmed findings. The same agent runs as a per-story pre-merge gate during `nxd resume`.

```bash
nxd security scan [path] [--json] [--llm] [--min <critical|high|medium|low>]
nxd security kb [--json]
```

- `scan` reports findings by severity and lists applicable scanners that are not installed (no silent gaps). Exit code is non-zero when a finding meets `--min` (default `high`) — CI-friendly.
- `kb` shows the knowledge base: version, baseline rules, and `+`-marked learned rules.
- Configure via the `security:` section of `nxd.yaml` (`disable_gate`, `gate_severity` default `critical`, `auto_learn`, `kb_path`).

---

### nxd spec

Spec-driven development scaffolding. `nxd spec init` creates a `.spec/` folder with 8 markdown files covering the 5W1H dimensions; the planner picks them up as structured context.
Expand Down
15 changes: 15 additions & 0 deletions internal/cli/resume.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import (
"github.com/tzone85/nexus-dispatch/internal/routing"
"github.com/tzone85/nexus-dispatch/internal/runtime"
"github.com/tzone85/nexus-dispatch/internal/scratchboard"
"github.com/tzone85/nexus-dispatch/internal/security"
"github.com/tzone85/nexus-dispatch/internal/state"
"github.com/tzone85/nexus-dispatch/internal/tmux"
)
Expand Down Expand Up @@ -479,6 +480,20 @@ func runResume(cmd *cobra.Command, args []string) error {
engine.WithMonDevDBLifecycle(devdbLifecycle),
)

// Enable the per-story security gate: after QA and before merge, run the
// security agent (scanners + LLM threat-model review against the growable
// knowledge base) and pause the requirement when a finding meets the gate
// severity. Skipped in dry-run and when security.disable_gate is set.
if !dryRun && !s.Config.Security.DisableGate {
gateSev := security.ParseSeverity(s.Config.Security.GateSeverity)
senior := s.Config.Models.Senior
monitor.SetSecurityGate(engine.NewSecurityGate(
llmClient, senior.Model, senior.MaxTokens, securityKBPath(s.Config),
gateSev, s.Config.Security.AutoLearn, s.Events, s.Proj,
))
log.Printf("[resume] security gate enabled (block at %s+, auto-learn=%v)", gateSev, s.Config.Security.AutoLearn)
}

rc := &engine.RunContext{
ReqID: reqID,
PlannedStories: plannedStories,
Expand Down
17 changes: 17 additions & 0 deletions internal/cli/resume_wiring_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,20 @@ func TestResume_WiresTechLeadFixer(t *testing.T) {
}
}
}

// TestResume_WiresSecurityGate guards the per-story security gate against the
// dead-wire class: the gate scans + reviews each story before merge, but only if
// runResume constructs and attaches it.
func TestResume_WiresSecurityGate(t *testing.T) {
src, err := os.ReadFile("resume.go")
if err != nil {
t.Fatalf("read resume.go: %v", err)
}
code := string(src)

for _, want := range []string{"NewSecurityGate(", "SetSecurityGate("} {
if !strings.Contains(code, want) {
t.Errorf("resume.go must wire the security gate: missing %q", want)
}
}
}
1 change: 1 addition & 0 deletions internal/cli/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ func init() {
rootCmd.AddCommand(newEscalationsCmd())
rootCmd.AddCommand(newGCCmd())
rootCmd.AddCommand(newConfigCmd())
rootCmd.AddCommand(newSecurityCmd())
rootCmd.AddCommand(newEventsCmd())
rootCmd.AddCommand(newDashboardCmd())
rootCmd.AddCommand(newArchiveCmd())
Expand Down
209 changes: 209 additions & 0 deletions internal/cli/security.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
package cli

import (
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"

"github.com/spf13/cobra"
"github.com/tzone85/nexus-dispatch/internal/config"
"github.com/tzone85/nexus-dispatch/internal/engine"
"github.com/tzone85/nexus-dispatch/internal/llm"
"github.com/tzone85/nexus-dispatch/internal/security"
"github.com/tzone85/nexus-dispatch/internal/state"
)

func newSecurityCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "security",
Short: "Security agent: scan repositories and inspect the knowledge base",
Long: `The security agent combines deterministic scanners (gosec, govulncheck,
gitleaks, semgrep, npm audit) with an optional LLM threat-model review driven by
a growable knowledge base (OWASP Top 10 + CWE baseline that learns new
vulnerability classes from confirmed findings).`,
SilenceUsage: true,
}
cmd.AddCommand(newSecurityScanCmd())
cmd.AddCommand(newSecurityKBCmd())
return cmd
}

func newSecurityScanCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "scan [repo-path]",
Short: "Run a security scan on a repository",
Long: `Scans a repository with every applicable, installed scanner and (optionally)
an LLM threat-model review. Findings are reported by severity; applicable
scanners that are not installed are listed so coverage gaps are never silent.

Exit code is non-zero when a finding meets or exceeds --min (default: high),
so the command is CI-friendly.`,
Args: cobra.MaximumNArgs(1),
RunE: runSecurityScan,
}
cmd.Flags().Bool("json", false, "Output the report as JSON")
cmd.Flags().Bool("llm", false, "Add an LLM threat-model review (requires a configured model; reads source files)")
cmd.Flags().String("min", "high", "Severity that makes the command exit non-zero: critical|high|medium|low")
cmd.SilenceUsage = true
return cmd
}

func runSecurityScan(cmd *cobra.Command, args []string) error {
jsonOut, _ := cmd.Flags().GetBool("json")
useLLM, _ := cmd.Flags().GetBool("llm")
minStr, _ := cmd.Flags().GetString("min")

repoPath, err := resolveScanPath(args)
if err != nil {
return err
}

cfgPath, _ := cmd.Flags().GetString("config")
cfg := loadConfigOrDefault(cfgPath)

kbPath := securityKBPath(cfg)

// Event/projection stores so scans are auditable. Use an in-memory
// projection (scan results are informational; the event log is the record).
es, err := state.NewFileStore(filepath.Join(expandHome(cfg.Workspace.StateDir), "events.jsonl"))
if err != nil {
return fmt.Errorf("open event store: %w", err)
}
defer func() { _ = es.Close() }()
ps, err := state.NewSQLiteStore(":memory:")
if err != nil {
return fmt.Errorf("open projection store: %w", err)
}
defer func() { _ = ps.Close() }()

// LLM review is opt-in: it needs file access, so it runs godmode (skip
// permission prompts) to stay non-interactive. Default is scanners-only
// (nil client ⇒ the gate runs deterministic scanners only).
var llmClient llm.Client
model := cfg.Models.Senior.Model
maxTokens := cfg.Models.Senior.MaxTokens
if useLLM {
built, buildErr := buildLLMClient(cfg.Models.Senior.Provider, true)
if buildErr != nil {
fmt.Fprintf(cmd.OutOrStdout(), "warning: LLM review unavailable (%v) — running scanners only\n", buildErr)
} else {
llmClient = built
}
}

gate := engine.NewSecurityGate(
llmClient, model, maxTokens, kbPath,
security.ParseSeverity(cfg.Security.GateSeverity),
cfg.Security.AutoLearn, es, ps,
)

report, err := gate.ScanRepo(context.Background(), repoPath)
if err != nil {
return fmt.Errorf("scan: %w", err)
}

if jsonOut {
enc := json.NewEncoder(cmd.OutOrStdout())
enc.SetIndent("", " ")
if err := enc.Encode(report); err != nil {
return err
}
} else {
fmt.Fprintln(cmd.OutOrStdout(), report.FormatMarkdown())
}

// CI-friendly exit code.
min := security.ParseSeverity(minStr)
if report.HasAtLeast(min) {
return fmt.Errorf("security scan found %s+ findings", min)
}
return nil
}

func newSecurityKBCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "kb",
Short: "Show the security knowledge base (version, rules, learned classes)",
Args: cobra.NoArgs,
RunE: runSecurityKB,
}
cmd.Flags().Bool("json", false, "Output the knowledge base as JSON")
cmd.SilenceUsage = true
return cmd
}

func runSecurityKB(cmd *cobra.Command, args []string) error {
cfgPath, _ := cmd.Flags().GetString("config")
cfg := loadConfigOrDefault(cfgPath)
kb, err := security.LoadKnowledgeBase(securityKBPath(cfg))
if err != nil {
return fmt.Errorf("load knowledge base: %w", err)
}
jsonOut, _ := cmd.Flags().GetBool("json")
if jsonOut {
enc := json.NewEncoder(cmd.OutOrStdout())
enc.SetIndent("", " ")
return enc.Encode(kb)
}
out := cmd.OutOrStdout()
baseline, learned := 0, 0
for _, r := range kb.Rules {
if r.Source == security.RuleLearned {
learned++
} else {
baseline++
}
}
fmt.Fprintf(out, "Security knowledge base v%d — %d rules (%d baseline, %d learned)\n\n",
kb.Version, len(kb.Rules), baseline, learned)
for _, r := range kb.Rules {
marker := " "
if r.Source == security.RuleLearned {
marker = "+"
}
fmt.Fprintf(out, " %s [%s] %s — %s\n", marker, r.ID, r.Title, r.Severity)
}
return nil
}

// resolveScanPath resolves the target repo to an absolute path (cwd default).
func resolveScanPath(args []string) (string, error) {
p := ""
if len(args) > 0 {
p = args[0]
} else {
cwd, err := os.Getwd()
if err != nil {
return "", fmt.Errorf("get working directory: %w", err)
}
p = cwd
}
abs, err := filepath.Abs(p)
if err != nil {
return "", fmt.Errorf("resolve path: %w", err)
}
return abs, nil
}

// loadConfigOrDefault loads the project config, falling back to DefaultConfig
// when none exists. The security scan is a standalone tool meant to run in ANY
// repo (which usually has no nxd.yaml), and it only needs model/KB/severity
// defaults — so a missing config is not an error here.
func loadConfigOrDefault(cfgPath string) config.Config {
cfg, err := loadConfig(cfgPath)
if err != nil {
return config.DefaultConfig()
}
return cfg
}

// securityKBPath resolves where the knowledge base persists: the configured path
// or <state_dir>/security/knowledge.json.
func securityKBPath(cfg config.Config) string {
if cfg.Security.KBPath != "" {
return expandHome(cfg.Security.KBPath)
}
return filepath.Join(expandHome(cfg.Workspace.StateDir), "security", "knowledge.json")
}
21 changes: 21 additions & 0 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ type Config struct {
Memory MemoryConfig `yaml:"memory"`
Investigation InvestigationConfig `yaml:"investigation"`
QA QAConfig `yaml:"qa"`
Security SecurityConfig `yaml:"security,omitempty"`
Runtimes map[string]RuntimeConfig `yaml:"runtimes"`
Plugins PluginConfig `yaml:"plugins"`
Methodology MethodologyConfig `yaml:"methodology"`
Expand Down Expand Up @@ -233,6 +234,26 @@ type QAConfig struct {
SuccessCriteria []SuccessCriterion `yaml:"success_criteria"`
}

// SecurityConfig controls the security agent: the per-story pre-merge security
// gate, the standalone `nxd security scan`, and the self-upskilling knowledge
// base shared by both.
type SecurityConfig struct {
// DisableGate turns OFF the per-story pre-merge security gate. The gate runs
// deterministic scanners + an LLM threat-model review on each story and
// pauses the requirement (for a human decision) when a finding meets/exceeds
// GateSeverity. Default (false) = gate ON. The standalone scan always works.
DisableGate bool `yaml:"disable_gate,omitempty"`
// GateSeverity is the build-pausing block threshold: critical|high|medium|low.
// Empty ⇒ "critical" (pause only on secrets / confirmed injection).
GateSeverity string `yaml:"gate_severity,omitempty"`
// AutoLearn grows the knowledge base from confirmed high+ findings so future
// builds inherit vulnerability classes seen in past ones. Default true.
AutoLearn bool `yaml:"auto_learn"`
// KBPath overrides where the knowledge base persists. Empty ⇒
// <state_dir>/security/knowledge.json.
KBPath string `yaml:"kb_path,omitempty"`
}

// SuccessCriterion defines a single declarative success check in config YAML.
type SuccessCriterion struct {
Kind string `yaml:"kind"`
Expand Down
9 changes: 9 additions & 0 deletions internal/config/loader.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,15 @@ func DefaultConfig() Config {
{Kind: "test_passes", Value: "go test ./..."},
},
},
Security: SecurityConfig{
// The pipeline gate pauses a build only on CRITICAL findings (leaked
// secrets, LLM-confirmed injection/hardcoded credentials) so it is
// high-signal and does not stall builds on context-dependent SAST
// noise. The standalone `nxd security scan` reports high/medium too
// (default --min high). Tighten the gate via security.gate_severity.
GateSeverity: "critical",
AutoLearn: true,
},
// DDD + TDD on by default. Per-requirement opt-out via the
// `methodology: relaxed` directive in the requirement text or a
// `.spec/methodology.md` file. Operators can disable globally by
Expand Down
Loading
Loading