From c9eadf8e3e8e44a51e741a95efca8936c0ea6501 Mon Sep 17 00:00:00 2001 From: cluic Date: Fri, 10 Jul 2026 11:46:58 +0800 Subject: [PATCH 01/44] docs: add product evolution harness --- .codex/hooks.json | 16 +++ .gitignore | 1 + AGENTS.md | 35 +++++++ docs/AGENT_COORDINATION.md | 32 ++++++ docs/AI_HANDOFF.md | 58 +++++++++++ docs/API.md | 64 ++++++++++++ docs/ARCHITECTURE.md | 74 ++++++++++++++ docs/DATA_MODEL.md | 61 ++++++++++++ docs/DECISIONS.md | 13 +++ docs/PERMISSIONS.md | 41 ++++++++ docs/PRD.md | 63 ++++++++++++ docs/ROADMAP.md | 33 +++++++ docs/STATUS.md | 32 ++++++ docs/TESTING.md | 53 ++++++++++ docs/UIUX.md | 57 +++++++++++ ...2026-07-10-crp-product-evolution-design.md | 98 +++++++++++++++++++ 16 files changed, 731 insertions(+) create mode 100644 .codex/hooks.json create mode 100644 AGENTS.md create mode 100644 docs/AGENT_COORDINATION.md create mode 100644 docs/AI_HANDOFF.md create mode 100644 docs/API.md create mode 100644 docs/ARCHITECTURE.md create mode 100644 docs/DATA_MODEL.md create mode 100644 docs/DECISIONS.md create mode 100644 docs/PERMISSIONS.md create mode 100644 docs/PRD.md create mode 100644 docs/ROADMAP.md create mode 100644 docs/STATUS.md create mode 100644 docs/TESTING.md create mode 100644 docs/UIUX.md create mode 100644 docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md diff --git a/.codex/hooks.json b/.codex/hooks.json new file mode 100644 index 0000000..555c5e9 --- /dev/null +++ b/.codex/hooks.json @@ -0,0 +1,16 @@ +{ + "hooks": { + "Stop": [ + { + "hooks": [ + { + "type": "command", + "command": "test -f docs/STATUS.md && test -f docs/AI_HANDOFF.md && test -f AGENTS.md", + "timeout": 10, + "statusMessage": "Checking CRP harness docs exist" + } + ] + } + ] + } +} diff --git a/.gitignore b/.gitignore index 501d03f..19d46c3 100644 --- a/.gitignore +++ b/.gitignore @@ -39,3 +39,4 @@ coverage/ /local_docs /python/ /__pycache__/ +.superpowers/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..ecdde53 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,35 @@ +# AGENTS.md + +## Project Map + +- CLI and supervisor entrypoint: `node/bin/crp.mjs` +- Proxy worker: `node/src/server.mjs` +- Provider, credential, and control-plane modules: `node/src/` (target architecture; see `docs/ARCHITECTURE.md`) +- Local Web management UI: `node/ui/` (target architecture) +- Tests: `node/test/` +- Living docs: `docs/AI_HANDOFF.md`, `docs/STATUS.md`, `docs/ROADMAP.md` + +## Working Rules + +- Read this file, `docs/AI_HANDOFF.md`, and `docs/STATUS.md` before editing. +- Do not implement outside the current vertical slice in `docs/ROADMAP.md`. +- Keep Codex `model_provider` and the proxy address stable; provider switching belongs inside CRP. +- Never return, log, capture, or commit complete API keys. +- Update affected API, data, permissions, UI/UX, testing, status, and handoff docs with behavior changes. +- Do not run parallel writable agents without scopes and no-edit areas in `docs/AGENT_COORDINATION.md`. +- Record reusable work mistakes as one concise required or prohibited sentence. + +## Required Checks + +- Current test suite: `cd node && npm test` +- Runtime dependency audit: `cd node && npm audit --omit=dev` +- Future UI-bearing changes: run browser E2E and attach visual evidence defined in `docs/TESTING.md`. +- Future credential, config migration, or lifecycle changes are L3 and require expert confirmation. + +## Done Means + +- Relevant deterministic checks pass. +- Changed behavior has tests or a documented reason. +- Sensitive values are absent from logs, API responses, fixtures, and diffs. +- Living docs and `docs/STATUS.md` reflect the resulting facts. +- The diff contains no unrelated changes and its merge risk is classified. diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md new file mode 100644 index 0000000..5d605cb --- /dev/null +++ b/docs/AGENT_COORDINATION.md @@ -0,0 +1,32 @@ +# Agent Coordination + +## Active Workstreams + +| Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | +| --- | --- | --- | --- | --- | --- | --- | +| Harness and design | Primary Codex | Documentation only | `AGENTS.md`, `.codex/`, `docs/` | In review | User design approval | Product code under `node/` | + +## Shared Contracts + +- Stable Codex provider and proxy URL invariants: `docs/ARCHITECTURE.md`. +- Provider schema: `docs/DATA_MODEL.md`. +- Admin HTTP contract: `docs/API.md`. +- Security boundary: `docs/PERMISSIONS.md`. +- Acceptance gate: `docs/TESTING.md`. + +## Integration Points + +- Supervisor owns persistent state and spawns the worker. +- Worker receives versioned immutable snapshots over authenticated parent-child IPC. +- UI and CLI use the same Admin API semantics. +- Credential adapters share one contract but have platform-specific implementations. + +## Locks / Avoid Editing + +- No parallel writable work is authorized while the written spec is under review. +- Future agents must not change shared contracts without first updating the owning doc and coordination row. +- Credential, migration, and lifecycle work must be isolated from UI styling work until contracts pass review. + +## Decisions Needed + +No blocking decision remains for planning. File ownership and worktree assignments must be added before any parallel implementation begins. diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md new file mode 100644 index 0000000..003d1ba --- /dev/null +++ b/docs/AI_HANDOFF.md @@ -0,0 +1,58 @@ +# AI Handoff + +## Product Summary + +CRP preserves ChatGPT login/remote features while routing Codex model traffic to an OpenAI-compatible upstream. The approved next milestone adds named providers, reliable lifecycle management, and a local Web UI for ordinary users. + +## Current Scope + +Documentation and harness only. No product implementation from the approved design has landed. Read `docs/PRD.md` and the formal design spec before changing code. + +## Architecture + +Target: long-lived supervisor control plane plus independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. + +## Data and API + +- Non-secret profiles live in a schema-versioned registry. +- API keys live in native credential stores with explicit `0600` fallback. +- Local API contract is in `docs/API.md`; data contract is in `docs/DATA_MODEL.md`. + +## Permissions + +One authenticated local OS user. Admin API is loopback-only, origin/host checked, CSRF protected, and never returns full keys. See `docs/PERMISSIONS.md`. + +## Current Progress + +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. Written docs await final user review. + +## How To Run Current Code + +```bash +cd node +npm ci +npm test +node bin/crp.mjs --help +``` + +Do not run `crp start` against a real home directory during tests because it modifies Codex configuration. + +## Verification + +- Current baseline: `cd node && npm test` +- Runtime audit: `cd node && npm audit --omit=dev` +- Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. + +## Known Risks + +Credential migration, localhost browser security, worker IPC, port release races, in-flight activation semantics, and secret leakage. + +## Recent Decisions + +- Use harness-builder `iterate` mode. +- Target ordinary users with CLI + local Web UI. +- Support macOS/Windows UI first and preserve Linux CLI. +- Use Supervisor + Proxy Worker. +- Keep Codex provider and proxy URL stable. +- Use guided utility console UI. +- Classify future V1 implementation as L3. diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..00dbf14 --- /dev/null +++ b/docs/API.md @@ -0,0 +1,64 @@ +# Local Admin API + +## Conventions + +- Base URL: `http://127.0.0.1:15101/api/v1`. +- JSON request and response bodies. +- Mutations are idempotent where an idempotency key is practical. +- Complete credentials are write-only. + +## Authentication and Browser Security + +- Bind only to loopback and reject unexpected `Host` and `Origin` values. +- Disable CORS. +- Establish a local SameSite session and require a CSRF token for mutations. +- CLI uses a `0600` local control token. +- Read endpoints return only `credentialConfigured` and a masked preview. + +## Error Format + +```json +{ + "error": { + "code": "PROVIDER_AUTH_FAILED", + "message": "The provider rejected the API key.", + "action": "Update the API key and test again.", + "requestId": "local-request-id", + "details": {} + } +} +``` + +`details` must be sanitized and safe to display. + +## Endpoints + +| Method | Path | Purpose | +| --- | --- | --- | +| GET | `/status` | Supervisor, worker, Codex bootstrap, and active-provider status | +| GET | `/providers` | List masked provider summaries | +| POST | `/providers` | Create a profile and write its credential | +| GET | `/providers/:id` | Read one masked profile | +| PATCH | `/providers/:id` | Update metadata and optionally replace its credential | +| DELETE | `/providers/:id` | Delete an inactive profile and credential | +| POST | `/providers/:id/test` | Run connection and Responses compatibility tests | +| POST | `/providers/:id/activate` | Atomically activate a tested profile | +| POST | `/proxy/start` | Start worker on the fixed proxy port | +| POST | `/proxy/stop` | Gracefully stop worker | +| POST | `/proxy/restart` | Drain, stop, release port, restart, and health-check worker | +| GET | `/activity` | Page through sanitized lifecycle activity | +| GET | `/settings` | Read non-secret local settings | +| PATCH | `/settings` | Update supported settings with validation | +| POST | `/codex/bootstrap` | Back up and idempotently set the fixed Codex provider/proxy entry | +| POST | `/diagnostics/export` | Produce a redacted local diagnostic bundle | + +## State Conflict Rules + +- Activating an untested or failed provider returns `409 PROVIDER_NOT_READY`. +- Deleting the active provider returns `409 PROVIDER_ACTIVE`. +- Lifecycle commands already in progress return their current operation instead of starting a second one. +- Changing the fixed proxy port requires successful Codex bootstrap and an explicit UI warning. + +## Contract Change Rules + +Update this document and contract tests in the same change. Breaking API changes require a new `/api/vN` prefix and a migration path for the bundled UI and CLI. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..a9c4fd7 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,74 @@ +# Architecture + +## Current State + +Version 0.2.2 is a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, patches Codex to use the `OpenAI` provider section, and spawns the proxy. Only capture enablement hot-reloads. + +## Target Overview + +The approved design splits the local application into a long-lived supervisor control plane and a replaceable proxy worker data plane. + +```text +Browser UI / crp CLI + | + v +Supervisor (127.0.0.1:15101) + - Admin API and static UI + - Provider registry + - Credential adapters + - Codex config adapter + - Worker manager and activity log + | + | IPC + immutable config snapshots + v +Proxy Worker (127.0.0.1:15100) + - HTTP/SSE forwarding + - Authorization rewrite + - Per-request provider snapshot + | + v +Active OpenAI-compatible upstream +``` + +## Invariants + +1. Codex keeps `model_provider = "OpenAI"` so existing OpenAI-tagged threads remain visible. +2. Codex keeps the fixed proxy address `http://127.0.0.1:15100` after bootstrap. +3. Provider activation affects new requests only; in-flight requests retain their starting snapshot. +4. The supervisor remains available while the worker restarts. +5. The worker never receives unrelated provider credentials. + +## Module Boundaries + +- `supervisor`: owns state transitions, admin server, activity records, and child-process lifecycle. +- `provider-registry`: validates and persists non-secret profiles and the active provider ID. +- `credential-store`: exposes get/set/delete by opaque credential reference; implements Keychain, Credential Manager, and file fallback adapters. +- `codex-config`: backs up and idempotently bootstraps the fixed OpenAI provider entry. +- `worker-protocol`: versioned IPC messages for configure, drain, shutdown, health, and events. +- `proxy-worker`: forwards traffic from immutable provider snapshots; does not own persistent configuration. +- `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. +- `web-ui`: static local app with onboarding and daily management views. + +## Lifecycle + +- Activate: validate profile → resolve secret → optionally re-test → persist `activeProviderId` → send snapshot generation N+1 → wait for worker acknowledgement. +- Restart: mark restarting → ask worker to drain → enforce timeout → wait for exit and port release → spawn → send active snapshot → require health success. +- Crash: record sanitized error → restart with capped exponential backoff → stop looping after the configured threshold. + +## Storage and Deployment + +- Package remains distributed through npm. +- Supervisor metadata: `~/.codex-remote-proxy/providers.json` and `state.json`, mode `0600` where supported. +- Credentials: native OS store; `~/.codex-remote-proxy/secrets.json` only after explicit fallback consent. +- Activity events: bounded local JSONL or SQLite store without request/response bodies. +- macOS and Windows receive UI support; Linux uses the same supervisor and CLI without an initial UI guarantee. + +## Risks + +- Credential backend differences across platforms. +- Atomic activation and in-flight request correctness. +- Port release races during restart. +- Localhost CSRF and DNS-rebinding-style attacks. +- Safe migration from the existing flat secret-bearing configuration. + +These areas require integration tests and L3 review before release. diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md new file mode 100644 index 0000000..c9b8dc6 --- /dev/null +++ b/docs/DATA_MODEL.md @@ -0,0 +1,61 @@ +# Data Model + +## ProviderProfile + +| Field | Type | Rules | +| --- | --- | --- | +| `id` | UUID string | Immutable | +| `name` | string | Required, unique case-insensitively | +| `baseUrl` | HTTPS/HTTP URL | HTTP allowed only for loopback | +| `credentialRef` | opaque string | Points to a credential adapter entry | +| `authHeader` | string | Default `authorization` | +| `authScheme` | string | Default `Bearer` | +| `extraHeaders` | string map | Sensitive-looking names are rejected | +| `modelMode` | enum | `passthrough` or `override` | +| `modelOverride` | string/null | Required when mode is `override` | +| `lastTestAt` | ISO timestamp/null | Set after compatibility test | +| `lastTestStatus` | enum | `untested`, `passed`, `failed` | +| `lastTestCode` | string/null | Sanitized stable error code | +| `createdAt` / `updatedAt` | ISO timestamp | Supervisor-owned | + +## RegistryDocument + +```json +{ + "schemaVersion": 2, + "activeProviderId": "uuid-or-null", + "providers": [], + "settings": { + "proxyHost": "127.0.0.1", + "proxyPort": 15100, + "adminHost": "127.0.0.1", + "adminPort": 15101, + "captureEnabled": false + } +} +``` + +## RuntimeState + +Runtime state includes supervisor PID, worker PID, worker status, snapshot generation, start timestamps, restart count, and the last sanitized error. It is observational and can be reconstructed. + +## ActivityEvent + +Activity events record timestamp, category, action, provider ID, result, stable error code, and sanitized details. They never contain complete keys, authorization headers, cookies, or request/response bodies. + +## Relationships + +- One registry has zero or one active provider. +- Each provider has exactly one credential reference after it is saved. +- A worker snapshot references one provider and one resolved credential for its process lifetime. + +## Lifecycle and Deletion + +- A profile must pass a Responses compatibility test before first activation. +- Deleting the active profile is rejected until another profile is activated or the proxy is stopped. +- Deleting a profile removes its native credential entry and writes an activity event. +- Activity retention defaults to 30 days or 10,000 events, whichever limit is reached first. + +## Migration + +On first version-2 start, back up existing CRP files, convert the flat upstream to a provider named `Default`, move the API key into the selected credential backend, persist schema version 2, and verify the resulting profile before removing the old secret field. Migration must be transactional and recoverable from its backup. diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md new file mode 100644 index 0000000..684ae80 --- /dev/null +++ b/docs/DECISIONS.md @@ -0,0 +1,13 @@ +# Decisions + +| Date | Decision | Context | Consequences | +| --- | --- | --- | --- | +| 2026-07-10 | Use `iterate` harness mode | Existing v0.2.2 project is being expanded | Preserve current behavior while adding living contracts | +| 2026-07-10 | Target ordinary users first | CLI-only setup is a product barrier | Add guided local Web UI while retaining CLI | +| 2026-07-10 | Use Supervisor + Proxy Worker | Reliable restart and stable management access are required | Adds IPC/state coordination but isolates lifecycle | +| 2026-07-10 | Keep Codex provider as `OpenAI` | Thread metadata and listing are provider-bound | Existing history remains visible during upstream switches | +| 2026-07-10 | Keep a fixed loopback proxy address | Rewriting Codex config on every switch is disruptive | Provider activation happens inside CRP | +| 2026-07-10 | Prefer native credential stores | Product will handle multiple long-lived API keys | Platform adapters and L3 review are required | +| 2026-07-10 | Use guided utility console UI | New users need onboarding; experienced users need fast switching | First-run wizard transitions to low-density dashboard | +| 2026-07-10 | Keep Admin API loopback-only | Management actions expose credentials and process control | Reject remote binds and enforce browser-origin protections | +| 2026-07-10 | Defer remote access and failover | They expand security and operational scope | MVP stays local and single-active-provider | diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md new file mode 100644 index 0000000..d2c5b51 --- /dev/null +++ b/docs/PERMISSIONS.md @@ -0,0 +1,41 @@ +# Permissions and Security + +## Roles + +The MVP has one role: the local operating-system user. It has no accounts, remote users, or administrator delegation. + +## Sensitive Actions + +- Store, replace, or delete an API key. +- Activate a provider. +- Patch Codex configuration. +- Start, stop, or restart the proxy worker. +- Change listening ports or credential backend. +- Export diagnostics. + +## Access Matrix + +| Actor | Can Do | Cannot Do | Notes | +| --- | --- | --- | --- | +| Authenticated local UI session | Manage providers and worker | Read complete keys; bind remote interfaces | CSRF and origin checks required | +| Local CLI with control token | Same management actions | Read complete keys | Designed for automation | +| Proxy worker | Use active snapshot | Read registry or other provider keys | Receives least-privilege configuration | +| Remote network client | Nothing | Access UI, API, or proxy management | Admin server is loopback-only | + +## Audit Requirements + +Record provider creation/update/deletion, test result, activation, Codex bootstrap, lifecycle commands, migration, credential fallback choice, and diagnostic export. Store stable IDs and sanitized error codes, never secret values. + +## Security Requirements + +- Prefer native credential stores. +- Set metadata, token, state, and fallback secret files to `0600` where supported. +- Never accept sensitive keys inside `extraHeaders`; use the credential field. +- Redact authorization, cookie, token, secret, and API-key headers and fields. +- Reject non-loopback admin binds. +- Validate URL scheme and block accidental credential forwarding to an unvalidated target. +- Treat diagnostics and capture content as sensitive local data. + +## Security Risks + +Credential migration, localhost browser attacks, log leakage, symlink/path attacks, and worker IPC spoofing make the first implementation an L3 merge-risk change. diff --git a/docs/PRD.md b/docs/PRD.md new file mode 100644 index 0000000..6456b27 --- /dev/null +++ b/docs/PRD.md @@ -0,0 +1,63 @@ +# Product Requirements + +## Product Summary + +Codex Remote Proxy keeps Codex signed in with ChatGPT while routing model traffic through a user-selected OpenAI-compatible upstream. The next product milestone adds a durable local control plane, named provider profiles, reliable proxy lifecycle management, and a browser-based management UI without changing Codex's provider identity during routine use. + +## Target Users + +- Primary: ordinary Codex Desktop users who have an OpenAI-compatible base URL and API key but do not want to edit configuration files. +- Secondary: developers who want CLI automation, diagnostics, and multiple provider profiles. +- Initial platform: management UI on macOS and Windows; CLI remains supported on Linux. + +## Core Problems + +1. Changing Codex `model_provider` changes which historical threads the app lists. +2. The project has no explicit, reliable restart operation. +3. It persists only one flat upstream configuration and has no named provider lifecycle. +4. CLI-only setup excludes users who are uncomfortable with terminals and config files. + +## MVP Scope + +- `crp ui` starts the local supervisor and opens the management UI. +- Add, edit, test, activate, and delete named provider profiles. +- Store API keys in macOS Keychain or Windows Credential Manager, with an explicit `0600` file fallback. +- Keep Codex on `model_provider = "OpenAI"` and a fixed loopback proxy URL. +- Atomically switch the active upstream for new requests without restarting Codex. +- Start, stop, and reliably restart the proxy worker from CLI and UI. +- Show supervisor, worker, active provider, health, and actionable lifecycle errors. +- Migrate the existing single-provider CRP configuration without losing settings. + +## Non-Goals + +- LAN or internet-accessible administration. +- User accounts, cloud sync, or shared team configuration. +- Provider load balancing or automatic failover. +- Electron/Tauri desktop packaging. +- Viewing captured request or response bodies in the UI. +- Automatic launch at login in the first vertical slice. + +## Core User Flows + +1. First run: launch UI → add provider → run compatibility test → save and activate → patch Codex once → restart Codex once → complete a request. +2. Routine switch: choose another tested provider → activate atomically → new requests use it while history remains visible. +3. Recovery: select Restart Proxy → worker drains or terminates safely → port is released → worker restarts → health check passes or an actionable error appears. + +## Responsibilities + +- Client UI: onboarding, provider forms, status, lifecycle actions, activity display, safe secret entry. +- Supervisor/API: validation, provider persistence, credential access, Codex bootstrap, worker lifecycle, audit events. +- Proxy worker: request forwarding, authorization rewrite, stable ingress, in-flight configuration snapshots, optional capture. +- CLI: automation-compatible access to the same supervisor contracts. + +## Success Criteria + +- A new user configures a working provider in under five minutes without manually editing a file. +- Two providers can be tested and switched while Codex threads remain visible. +- The proxy worker can restart on the same port without taking down the management UI. +- Full API keys never appear in read APIs, logs, capture headers, diagnostics, or repository files. +- macOS and Windows E2E flows pass; Linux CLI regression tests pass. + +## Open Decisions + +No decision blocks the design. Launch-at-login, high-fidelity styling, traffic inspection, and failover are intentionally deferred. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md new file mode 100644 index 0000000..789c9e2 --- /dev/null +++ b/docs/ROADMAP.md @@ -0,0 +1,33 @@ +# Roadmap + +## Current Milestone + +M1: design and harness for the multi-provider local management experience. + +## MVP Vertical Slices + +| Slice | User Value | Status | Verification | +| --- | --- | --- | --- | +| V0: Harness and approved design | Shared product, architecture, security, UI, and test contracts | In review | Doc self-review and user approval | +| V1: Provider lifecycle end to end | Add, test, activate, switch, and restart through local UI without changing Codex provider | Planned | Full acceptance flow in `docs/TESTING.md` | +| V2: Existing-user migration | Upgrade v0.2.2 flat config without losing provider access | Planned | Migration/rollback fixtures on all platforms | +| V3: Cross-platform hardening | Reliable macOS/Windows UI and Linux CLI | Planned | Platform CI, E2E, screenshots, accessibility | + +## V1 Internal Delivery Order + +1. Supervisor/worker protocol and deterministic process tests. +2. Provider registry, credential adapters, and migration-safe storage. +3. Admin API and security boundary. +4. Onboarding and guided utility UI. +5. Codex bootstrap, end-to-end activation, and restart verification. + +## Later + +- Launch at login and system tray integrations. +- High-fidelity activity metrics and safe capture tooling. +- Optional provider health monitoring and manual fallback controls. +- Signed installers or a desktop shell only if npm onboarding remains a measured barrier. + +## Explicit Non-Goals + +Remote administration, accounts, cloud sync, team sharing, load balancing, automatic failover, and capture-body viewing are outside the MVP. diff --git a/docs/STATUS.md b/docs/STATUS.md new file mode 100644 index 0000000..28b02cc --- /dev/null +++ b/docs/STATUS.md @@ -0,0 +1,32 @@ +# Status + +## Current Milestone + +M1 design and harness review. + +## In Progress + +- User review of the written product and architecture contracts. + +## Done + +- Audited v0.2.2 behavior and verified the four reported product gaps. +- Selected Supervisor + Proxy Worker architecture. +- Defined stable Codex provider and fixed proxy invariants. +- Defined provider/credential model, local Admin API, guided utility UI, security boundary, and verification path. +- Created the project harness and living docs. + +## Blocked + +- Product implementation is blocked until the written specification is approved and an implementation plan is reviewed. + +## Next + +1. Approve or revise `docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md`. +2. Produce the detailed implementation plan. +3. Begin V1 with tests for supervisor/worker lifecycle and stable provider behavior. + +## Risks + +- Future V1 implementation is L3 because it handles credentials, local browser security, Codex configuration, and process lifecycle. +- Cross-platform credential APIs and restart semantics require real macOS and Windows verification. diff --git a/docs/TESTING.md b/docs/TESTING.md new file mode 100644 index 0000000..3bbda16 --- /dev/null +++ b/docs/TESTING.md @@ -0,0 +1,53 @@ +# Testing + +## Environment Requirements + +- Node.js 22.13 or newer. +- macOS runner with Keychain access for platform integration tests. +- Windows runner with Credential Manager access for platform integration tests. +- Linux runner for CLI and proxy regression coverage. +- Chromium for browser E2E and screenshot comparison. + +## Required and Mocked Services + +- Deterministic local mock upstreams for JSON responses, SSE, timeouts, TLS errors, 401, 404, compressed requests, and disconnects. +- Native credential stores are exercised in platform integration jobs; unit tests use an in-memory adapter. +- No real API key is required in CI. + +## Current Commands + +- Existing suite: `cd node && npm test` +- Runtime audit: `cd node && npm audit --omit=dev` + +The implementation plan must add stable scripts for `test:unit`, `test:integration`, `test:e2e`, `lint`, and UI build before their respective modules land. + +## Test Matrix + +| Area | Required Tests | +| --- | --- | +| Provider registry | Validation, duplicate names, atomic persistence, schema migration, rollback | +| Credentials | Native adapter contract, file fallback permission, masking, deletion, log redaction | +| Worker protocol | Version mismatch, acknowledgement, stale generation rejection, crash handling | +| Proxy behavior | Auth rewrite, HTTP/SSE, compression, timeout, disconnect, optional model override | +| Activation | Failed test rejection, atomic new-request switch, in-flight old snapshot | +| Restart | Drain timeout, SIGTERM escalation, port release, same-port spawn, health failure | +| Codex bootstrap | Backup, idempotency, stable OpenAI provider, fixed URL, recovery | +| Admin API | Auth/session, CSRF, Host/Origin rejection, error contracts, secret write-only behavior | +| UI E2E | First-run flow, two-provider switch, restart, errors, keyboard and accessibility scan | +| Cross-platform | macOS and Windows UI path; Linux CLI regression | + +## First Vertical Slice Acceptance + +1. Start from an isolated temporary home directory. +2. Launch `crp ui` without a saved provider. +3. Create and test two mock providers through the browser. +4. Activate provider A and complete a proxied request. +5. Keep a request in flight while activating provider B; verify old/new snapshot behavior. +6. Verify the Codex provider key remains `OpenAI` and the proxy URL remains fixed. +7. Restart the worker from the UI and verify supervisor availability and same-port recovery. +8. Confirm API responses, logs, activity, capture headers, and diagnostics contain no complete key. +9. Capture approved macOS and Windows screenshots and run accessibility checks. + +## Verification Gate + +Credential, config migration, lifecycle, and browser-security tests must all pass before L3 expert review. Passing unit tests alone is insufficient. diff --git a/docs/UIUX.md b/docs/UIUX.md new file mode 100644 index 0000000..13b4bfd --- /dev/null +++ b/docs/UIUX.md @@ -0,0 +1,57 @@ +# UI/UX + +## Audience and Tone + +Primary users may not understand providers, processes, ports, or configuration files. Use calm, direct language, show one recommended action, and place technical details behind expandable areas. + +## Device Priorities + +1. Desktop browsers on macOS and Windows. +2. Keyboard-complete desktop interaction. +3. Linux browser compatibility after CLI parity is stable. + +The admin UI is not designed for mobile or remote access. + +## Page Map + +- Onboarding: provider details → compatibility test → activate → Codex bootstrap result. +- Overview: supervisor/worker health, active provider, fixed proxy address, recent error, switch and restart actions. +- Providers: list, create, edit, test, activate, delete, and replace credential. +- Activity: sanitized lifecycle events and diagnostic export. +- Settings: ports, credential backend status, capture toggle, Codex bootstrap state. + +## Core Interaction Rules + +- A provider cannot be activated until its compatibility test passes. +- Activation shows progress and completes only after the worker acknowledges the new generation. +- Restart uses one explicit confirmation only when in-flight requests may be interrupted. +- Destructive profile deletion requires confirmation and is unavailable for the active profile. +- Secret inputs are blank on edit; masked previews are informational, never form values. +- Every error includes a plain-language cause and a next action. + +## Visual Direction + +Approved direction: **guided utility console**. + +- First run uses a short step-by-step flow. +- Daily use uses a light, low-density sidebar layout. +- Status uses text plus icons, never color alone. +- Advanced process and HTTP details remain collapsed by default. + +## Initial Design Tokens + +- System UI font stack for native familiarity and zero font downloads. +- Neutral gray surfaces, blue primary actions, green healthy status, amber warning, red destructive/error. +- Minimum 44px interactive targets and visible keyboard focus. +- Eight-point spacing scale and restrained shadows. + +## Accessibility + +- Meet WCAG 2.2 AA contrast. +- Preserve full keyboard navigation and semantic form labels. +- Announce asynchronous test, activation, and restart state changes. +- Respect reduced-motion preferences. + +## Visual Evidence + +Low-fidelity architecture, provider flow, UI direction, and dashboard/error-state screens were reviewed and approved in the local visual companion on 2026-07-10. High-fidelity browser screenshots remain a required pre-implementation alignment gate, not a completed artifact. diff --git a/docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md b/docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md new file mode 100644 index 0000000..0a3d533 --- /dev/null +++ b/docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md @@ -0,0 +1,98 @@ +# Codex Remote Proxy Product Evolution Design + +Date: 2026-07-10 + +Status: Approved in conversation; written specification pending user review + +Mode: Cluic Harness Builder `iterate` + +## Idea Triage + +This is a current-milestone product enhancement, not a backlog note. It addresses four verified gaps in v0.2.2: provider-bound history visibility, incomplete restart semantics, single-provider persistence, and CLI-only onboarding. + +## Product Boundary + +- Users: ordinary Codex Desktop users first; developer automation second. +- Core problem: route Codex traffic to multiple compatible upstreams without changing the Codex provider identity or requiring config-file editing. +- MVP: local Web UI plus CLI for provider CRUD, compatibility testing, atomic activation, and reliable worker lifecycle. +- Non-goals: accounts, cloud sync, remote/LAN admin, load balancing, auto failover, desktop shell, and capture-body UI. +- Client: onboarding and guided local management. +- Server: loopback supervisor, provider registry, credentials, Codex bootstrap, worker management. +- Data plane: fixed-port proxy worker with immutable per-request provider snapshots. +- Roles: one authenticated local OS user. +- External services: OpenAI-compatible upstreams and native OS credential stores. +- Verification: two-provider end-to-end switch, history/provider stability, same-port restart, cross-platform E2E, and secret scans. + +## Considered Approaches + +### A. Single Node Process + +Proxy, UI, API, and persistence share one process. It minimizes initial code but takes the UI offline during restart and preserves the existing responsibility coupling. + +### B. Supervisor + Proxy Worker — Selected + +A long-lived supervisor serves the UI/API and owns persistence. A child worker owns only proxy traffic and can restart independently. This adds an IPC contract but directly satisfies reliable restart and stable management requirements. + +### C. Desktop Application + Daemon + +Electron or Tauri could provide the most native experience but adds packaging, signing, updater, and platform complexity before npm-based onboarding has been measured. + +## Approved Architecture + +The supervisor listens on `127.0.0.1:15101`; the worker listens on fixed `127.0.0.1:15100`. Codex is bootstrapped once to `model_provider = "OpenAI"` and the fixed worker URL. Provider activation persists the selected profile and pushes a monotonically increasing immutable snapshot to the worker. Requests capture their snapshot at start so activation never changes an in-flight target or credential. + +The supervisor owns worker draining, termination, port-release confirmation, respawn, snapshot delivery, and health verification. It remains reachable throughout a worker restart. Repeated crash recovery uses capped backoff and stops after a bounded threshold. + +## Provider and Credential Design + +Provider profiles store name, base URL, auth header/scheme, safe extra headers, model passthrough/override policy, credential reference, timestamps, and compatibility-test result. Complete keys are write-only through the Admin API and live in native platform stores. A permission-restricted file backend is an explicit fallback, not the default. + +A provider must pass both connection validation and a minimal Responses API request using a selected test model before first activation. Failure returns a stable code and a user action. + +## UI Design + +The approved direction is a guided utility console. First run presents provider details, test, activate, and Codex bootstrap as a short wizard. Later visits open an overview with active provider, supervisor/worker state, fixed proxy address, recent error, provider switch, and restart actions. + +Four pages are in scope: Overview, Providers, Activity, and Settings. Technical details are collapsed by default. Secret values are never prefilled. Status never relies on color alone, controls are keyboard-accessible, and async lifecycle changes are announced. + +## Error Handling + +- Authentication failure prompts key replacement without echoing the key. +- DNS, TLS, timeout, HTTP 404, and Responses incompatibility are separate errors. +- Port conflicts identify the safe next action and never silently change the fixed URL. +- Worker crashes trigger bounded recovery and sanitized activity events. +- Migration failures restore the prior configuration from backup. +- Concurrent activation/restart mutations return the existing operation instead of racing. + +## Impacted Areas + +- Code: CLI, proxy server, new supervisor, provider registry, credential adapters, worker protocol, Admin API, static UI. +- Contracts: new registry schema, credential adapter, IPC protocol, HTTP API, migration format. +- Existing behavior: proxy forwarding and optional capture must remain compatible. +- Docs: all harness living docs created with this design. +- Release: npm package contents, platform CI, security review, and migration notes. + +## Acceptance Criteria + +1. A clean user launches `crp ui` and configures a provider in under five minutes without editing files. +2. Two providers can be created, tested, and activated from the UI. +3. Provider B activation routes new requests to B while an in-flight A request completes against A. +4. Codex continues to use provider key `OpenAI` and the same local proxy URL. +5. Restart keeps the supervisor/UI available and restores the worker on the same port. +6. Existing v0.2.2 configuration migrates transactionally with rollback. +7. No full key appears in API reads, logs, activity, capture headers, diagnostics, fixtures, or diffs. +8. Current proxy tests plus new unit, integration, browser E2E, accessibility, macOS, Windows, and Linux CLI gates pass. + +## Scoped Implementation Shape + +The next plan will sequence tests and contracts before implementation: worker lifecycle protocol, provider/credential persistence, local API security, guided UI, migration, then cross-platform hardening. The first deliverable remains one end-to-end vertical slice rather than disconnected backend and UI projects. + +## Verification and Merge Classification + +- Documentation commit: L0; reviewable after doc checks and diff inspection. +- V1 implementation: L3 because it changes credential handling, browser-local security, Codex configuration, migration, and process lifecycle. +- V1 cannot auto-merge. It requires deterministic checks, visual evidence, AI review, rollback evidence, and expert confirmation. + +## Documentation Sync + +Architecture, API, data, permissions, UI/UX, testing, roadmap, status, handoff, coordination, and durable decisions are represented in the root `docs/` contracts. Changes to implementation facts must update the owning documents in the same review. From afc2ad1140bc5940a2e76b307108cbf94f50dc33 Mon Sep 17 00:00:00 2001 From: cluic Date: Fri, 10 Jul 2026 12:13:13 +0800 Subject: [PATCH 02/44] docs: add V1 implementation plan --- docs/AGENT_COORDINATION.md | 2 +- docs/AI_HANDOFF.md | 5 +- docs/ROADMAP.md | 2 +- docs/STATUS.md | 10 +- .../plans/2026-07-10-crp-v1-implementation.md | 920 ++++++++++++++++++ ...2026-07-10-crp-product-evolution-design.md | 2 +- 6 files changed, 932 insertions(+), 9 deletions(-) create mode 100644 docs/superpowers/plans/2026-07-10-crp-v1-implementation.md diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 5d605cb..232e9f5 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| Harness and design | Primary Codex | Documentation only | `AGENTS.md`, `.codex/`, `docs/` | In review | User design approval | Product code under `node/` | +| Harness, design, and plan | Primary Codex | Documentation only | `AGENTS.md`, `.codex/`, `docs/` | Ready | User execution-mode selection | Product code under `node/` | ## Shared Contracts diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index 003d1ba..88e00f3 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,7 +6,7 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -Documentation and harness only. No product implementation from the approved design has landed. Read `docs/PRD.md` and the formal design spec before changing code. +Documentation, harness, and implementation planning only. No product implementation from the approved design has landed. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. Written docs await final user review. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification is approved and the detailed V1 plan awaits execution-mode selection. ## How To Run Current Code @@ -56,3 +56,4 @@ Credential migration, localhost browser security, worker IPC, port release races - Keep Codex provider and proxy URL stable. - Use guided utility console UI. - Classify future V1 implementation as L3. +- Execute the approved design through the task sequence in `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md`. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 789c9e2..c6ef0ff 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -8,7 +8,7 @@ M1: design and harness for the multi-provider local management experience. | Slice | User Value | Status | Verification | | --- | --- | --- | --- | -| V0: Harness and approved design | Shared product, architecture, security, UI, and test contracts | In review | Doc self-review and user approval | +| V0: Harness and approved design | Shared product, architecture, security, UI, and test contracts | Approved | Doc self-review and user approval | | V1: Provider lifecycle end to end | Add, test, activate, switch, and restart through local UI without changing Codex provider | Planned | Full acceptance flow in `docs/TESTING.md` | | V2: Existing-user migration | Upgrade v0.2.2 flat config without losing provider access | Planned | Migration/rollback fixtures on all platforms | | V3: Cross-platform hardening | Reliable macOS/Windows UI and Linux CLI | Planned | Platform CI, E2E, screenshots, accessibility | diff --git a/docs/STATUS.md b/docs/STATUS.md index 28b02cc..b704da7 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ M1 design and harness review. ## In Progress -- User review of the written product and architecture contracts. +- User selection of the V1 execution mode. ## Done @@ -15,15 +15,17 @@ M1 design and harness review. - Defined stable Codex provider and fixed proxy invariants. - Defined provider/credential model, local Admin API, guided utility UI, security boundary, and verification path. - Created the project harness and living docs. +- Received written specification approval. +- Created and self-reviewed the detailed V1 implementation plan. ## Blocked -- Product implementation is blocked until the written specification is approved and an implementation plan is reviewed. +- Product implementation is blocked until the detailed implementation plan is reviewed and an execution mode is selected. ## Next -1. Approve or revise `docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md`. -2. Produce the detailed implementation plan. +1. Review `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md`. +2. Select subagent-driven or inline execution. 3. Begin V1 with tests for supervisor/worker lifecycle and stable provider behavior. ## Risks diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md new file mode 100644 index 0000000..9dfca48 --- /dev/null +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -0,0 +1,920 @@ +# CRP Multi-Provider Local Management V1 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a local Web management experience that can add, test, activate, and switch multiple OpenAI-compatible providers while keeping Codex on a fixed provider identity and reliably managing an independent proxy worker. + +**Architecture:** A loopback-only supervisor owns provider metadata, credentials, Codex bootstrap, activity, the Admin API, and a static UI. It controls a child proxy worker through a versioned IPC protocol; the worker applies monotonically increasing immutable settings snapshots so new requests switch atomically while in-flight requests remain pinned. + +**Tech Stack:** Node.js ESM 22.13+, built-in `node:http`, `node:test`, `child_process.fork`, JSON persistence, `@napi-rs/keyring`, vanilla HTML/CSS/ES modules, Playwright, npm/Changesets. + +--- + +## Scope and File Map + +### Existing files to modify + +- `node/bin/crp.mjs`: reduce to CLI parsing, supervisor discovery, and command dispatch while retaining compatibility aliases. +- `node/src/server.mjs`: obtain an immutable settings snapshot at request start and support IPC-hosted execution. +- `node/src/capture-store.mjs`: preserve capture behavior while receiving snapshot-safe paths. +- `node/package.json`: add credential/E2E dependencies, scripts, and packaged UI files. +- `node/package-lock.json`: lock new dependencies. +- `README.md`, `README.zh-CN.md`, `node/README.md`, `node/CHANGELOG.md`: document the new lifecycle and migration. +- Living contracts under `docs/`: synchronize facts after each task. + +### Focused modules to create + +- `node/src/shared/errors.mjs`: stable local error codes and safe serialization. +- `node/src/shared/paths.mjs`: all CRP home, state, registry, token, log, and Codex paths. +- `node/src/codex/codex-config.mjs`: idempotent backup/bootstrap of the fixed Codex provider entry. +- `node/src/providers/provider-schema.mjs`: provider validation and public masking. +- `node/src/providers/provider-registry.mjs`: atomic schema-versioned provider persistence. +- `node/src/credentials/native-keyring.mjs`: native keyring adapter. +- `node/src/credentials/file-credential-store.mjs`: explicit `0600` fallback adapter. +- `node/src/credentials/credential-store.mjs`: adapter contract and backend selection. +- `node/src/worker/runtime-settings.mjs`: monotonic immutable snapshot store. +- `node/src/worker/protocol.mjs`: parent/child message constructors and validators. +- `node/src/worker/worker-entry.mjs`: IPC-driven worker entrypoint. +- `node/src/supervisor/activity-store.mjs`: bounded sanitized lifecycle events. +- `node/src/supervisor/worker-manager.mjs`: start, drain, stop, restart, backoff, and health. +- `node/src/supervisor/migration.mjs`: transactional v0.2.2 migration. +- `node/src/supervisor/provider-service.mjs`: test/activate/delete orchestration. +- `node/src/supervisor/session-auth.mjs`: local control token, browser session, and CSRF. +- `node/src/supervisor/admin-server.mjs`: versioned Admin API and static UI serving. +- `node/src/supervisor/supervisor.mjs`: composition root and runtime state. +- `node/src/supervisor/supervisor-entry.mjs`: detached supervisor process entrypoint. +- `node/src/supervisor/supervisor-client.mjs`: CLI HTTP client. +- `node/ui/index.html`, `node/ui/styles.css`, `node/ui/app.js`: guided utility console. +- `node/playwright.config.mjs`: E2E configuration. +- `node/scripts/check-source.mjs`: portable syntax/lint gate. + +### Tests to create + +- Unit tests mirror each focused module under `node/test/`. +- Multi-process and HTTP tests live under `node/test/integration/`. +- Browser specs live under `node/test/e2e/`. +- Deterministic child/upstream fixtures live under `node/test/fixtures/`. + +## Task 1: Establish Portable Quality Gates and Dependencies + +**Files:** +- Modify: `node/package.json` +- Modify: `node/package-lock.json` +- Create: `node/scripts/check-source.mjs` +- Create: `node/scripts/run-test-group.mjs` +- Test: existing `node/test/*.test.mjs` + +- [ ] **Step 1: Record the fresh baseline** + +Run: + +```bash +cd node +npm test +npm audit --omit=dev +``` + +Expected: 12 tests pass and the runtime audit reports zero vulnerabilities. + +- [ ] **Step 2: Add the portable source-check script** + +Create `node/scripts/check-source.mjs`: + +```js +import { readdirSync, statSync } from "node:fs"; +import { join, relative } from "node:path"; +import { spawnSync } from "node:child_process"; +import { fileURLToPath } from "node:url"; + +const root = fileURLToPath(new URL("..", import.meta.url)); +const roots = [join(root, "bin"), join(root, "src"), join(root, "scripts"), join(root, "ui")]; +const files = []; + +function walk(dir) { + for (const name of readdirSync(dir, { withFileTypes: true })) { + const path = join(dir, name.name); + if (name.isDirectory()) walk(path); + if (name.isFile() && /\.(mjs|js)$/.test(name.name)) files.push(path); + } +} + +for (const dir of roots) { + if (statSync(dir, { throwIfNoEntry: false })?.isDirectory()) walk(dir); +} + +for (const file of files.sort()) { + const result = spawnSync(process.execPath, ["--check", file], { encoding: "utf8" }); + if (result.status !== 0) { + process.stderr.write(`${relative(root, file)}\n${result.stderr}`); + process.exit(result.status ?? 1); + } +} + +console.log(`Syntax checked ${files.length} source files.`); +``` + +- [ ] **Step 3: Add the cross-platform grouped-test runner** + +Create `node/scripts/run-test-group.mjs`: + +```js +import { readdirSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { spawnSync } from "node:child_process"; + +const group = process.argv[2]; +const testRoot = resolve("test"); +const selectedRoot = group === "integration" ? join(testRoot, "integration") : testRoot; +const recursive = group === "integration"; + +function collect(dir, descend) { + const files = []; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const path = join(dir, entry.name); + if (entry.isDirectory() && descend) files.push(...collect(path, true)); + if (entry.isFile() && entry.name.endsWith(".test.mjs")) files.push(path); + } + return files; +} + +const files = collect(selectedRoot, recursive).sort(); +if (files.length === 0) throw new Error(`No ${group} test files found`); +const result = spawnSync(process.execPath, ["--test", ...files], { stdio: "inherit" }); +process.exit(result.status ?? 1); +``` + +- [ ] **Step 4: Add exact dependencies and scripts** + +Run: + +```bash +cd node +npm install @napi-rs/keyring@1.3.0 +npm install --save-dev @playwright/test@1.61.1 +``` + +Set these scripts in `node/package.json`: + +```json +{ + "scripts": { + "start": "node src/server.mjs", + "check": "node bin/crp.mjs check", + "status": "node bin/crp.mjs status", + "guide": "node bin/crp.mjs guide", + "lint": "node scripts/check-source.mjs", + "test": "node --test", + "test:unit": "node scripts/run-test-group.mjs unit", + "test:integration": "node scripts/run-test-group.mjs integration", + "test:e2e": "playwright test", + "test:all": "npm run lint && npm run test && npm run test:e2e", + "changeset": "changeset", + "version-packages": "changeset version", + "release": "changeset publish" + }, + "files": ["bin/", "src/", "ui/", "proxy-config.example.json", "README.md"] +} +``` + +- [ ] **Step 5: Verify gates** + +Run: + +```bash +cd node +npm run lint +npm test +npm audit --omit=dev +``` + +Expected: syntax check succeeds, 12 tests pass, and runtime audit reports zero vulnerabilities. + +- [ ] **Step 6: Commit** + +```bash +git add node/package.json node/package-lock.json node/scripts/check-source.mjs node/scripts/run-test-group.mjs +git commit -m "build: add CRP quality gates" +``` + +## Task 2: Extract Stable Paths, Errors, and Codex Bootstrap + +**Files:** +- Create: `node/src/shared/paths.mjs` +- Create: `node/src/shared/errors.mjs` +- Create: `node/src/codex/codex-config.mjs` +- Modify: `node/bin/crp.mjs:14-27,603-669` +- Create: `node/test/codex-config.test.mjs` + +- [ ] **Step 1: Write failing idempotency and preservation tests** + +Create `node/test/codex-config.test.mjs` with tests that import `patchCodexConfigText` and assert: + +```js +const original = `model_provider = "custom"\n\n[model_providers.custom]\nname = "Custom"\nbase_url = "https://old.example/v1"\n`; +const once = patchCodexConfigText(original, "http://127.0.0.1:15100"); +const twice = patchCodexConfigText(once, "http://127.0.0.1:15100"); + +assert.match(once, /^model_provider = "OpenAI"/m); +assert.match(once, /\[model_providers\.OpenAI\]/); +assert.match(once, /base_url = "http:\/\/127\.0\.0\.1:15100"/); +assert.match(once, /\[model_providers\.custom\]/); +assert.equal(twice, once); +``` + +Add a temporary-file test asserting `bootstrapCodexConfig()` creates one backup, writes mode-preserving content, and returns `{ changed: true, backupPath }`; a second call returns `{ changed: false, backupPath: null }`. + +- [ ] **Step 2: Run the test and confirm the missing-module failure** + +```bash +cd node +node --test test/codex-config.test.mjs +``` + +Expected: FAIL with `ERR_MODULE_NOT_FOUND` for `src/codex/codex-config.mjs`. + +- [ ] **Step 3: Implement exact public contracts** + +Create `node/src/shared/errors.mjs`: + +```js +export class CrpError extends Error { + constructor(code, message, action, { status = 500, details = {}, cause } = {}) { + super(message, { cause }); + this.name = "CrpError"; + this.code = code; + this.action = action; + this.status = status; + this.details = details; + } +} + +export function toPublicError(error, requestId) { + const safe = error instanceof CrpError + ? error + : new CrpError("INTERNAL_ERROR", "CRP could not complete the operation.", "Open Activity for details."); + return { error: { code: safe.code, message: safe.message, action: safe.action, requestId, details: safe.details } }; +} +``` + +Create `node/src/shared/paths.mjs` exporting one `getPaths(home = os.homedir())` result containing `globalHome`, `registryPath`, `secretFallbackPath`, `statePath`, `controlTokenPath`, `activityPath`, `logPath`, `codexConfigPath`, and `authPath`. + +Move TOML section helpers, `patchCodexConfigText`, backup logic, and file writes from `crp.mjs` into `node/src/codex/codex-config.mjs`. Export: + +```js +export function patchCodexConfigText(text, proxyUrl) {} +export function bootstrapCodexConfig({ configPath, proxyUrl, now = () => new Date() }) {} +``` + +`bootstrapCodexConfig` must compare before writing, copy the original beside it with a UTC timestamp, write through a same-directory temporary file, then rename. + +- [ ] **Step 4: Import the adapter from the existing CLI** + +Replace the duplicated config-patching functions in `node/bin/crp.mjs` with: + +```js +import { bootstrapCodexConfig } from "../src/codex/codex-config.mjs"; +import { getPaths } from "../src/shared/paths.mjs"; +``` + +Keep command output and aliases unchanged in this task. + +- [ ] **Step 5: Verify focused and regression tests** + +```bash +cd node +node --test test/codex-config.test.mjs +npm test +``` + +Expected: new tests pass and the existing 12 tests remain green. + +- [ ] **Step 6: Commit** + +```bash +git add node/src/shared node/src/codex node/bin/crp.mjs node/test/codex-config.test.mjs +git commit -m "refactor: isolate Codex bootstrap" +``` + +## Task 3: Add the Atomic Provider Registry + +**Files:** +- Create: `node/src/providers/provider-schema.mjs` +- Create: `node/src/providers/provider-registry.mjs` +- Create: `node/test/provider-registry.test.mjs` + +- [ ] **Step 1: Write failing registry behavior tests** + +Cover create/list/get/update, case-insensitive duplicate names, activation, active-delete rejection, inactive deletion, reload, malformed JSON, and `0600` permissions. Use injected deterministic IDs and clocks: + +```js +const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => "2026-07-10T00:00:00.000Z" +}); +const created = registry.create({ + name: "OpenRouter", + baseUrl: "https://openrouter.ai/api/v1", + credentialRef: "provider-1", + authHeader: "authorization", + authScheme: "Bearer", + extraHeaders: {}, + modelMode: "passthrough", + modelOverride: null +}); +assert.equal(created.id, "provider-1"); +assert.equal(registry.list().length, 1); +``` + +- [ ] **Step 2: Verify the tests fail** + +```bash +cd node +node --test test/provider-registry.test.mjs +``` + +Expected: FAIL because the provider modules do not exist. + +- [ ] **Step 3: Implement validation and masking** + +Export from `provider-schema.mjs`: + +```js +export const TEST_STATUSES = new Set(["untested", "passed", "failed"]); +export function validateProviderInput(input) {} +export function normalizeProvider(input, { id, now }) {} +export function toPublicProvider(profile, credentialConfigured) { + const { credentialRef, ...safe } = profile; + return { ...safe, credentialConfigured }; +} +``` + +Reject missing names, duplicate names, credentials in `extraHeaders`, non-HTTP(S) URLs, non-loopback HTTP URLs, and missing override models. + +- [ ] **Step 4: Implement atomic persistence** + +`ProviderRegistry` must expose: + +```js +list(); +get(id); +create(input); +update(id, patch); +delete(id); +markTest(id, { status, code }); +setActive(id); +getActive(); +getDocument(); +``` + +Every mutation writes a complete schema-version-2 document to a same-directory temporary file with mode `0600`, fsyncs it, and renames it over the registry. Failed validation must leave the original file byte-for-byte unchanged. + +- [ ] **Step 5: Run focused and full tests** + +```bash +cd node +node --test test/provider-registry.test.mjs +npm test +``` + +Expected: all provider tests and regressions pass. + +- [ ] **Step 6: Commit** + +```bash +git add node/src/providers node/test/provider-registry.test.mjs +git commit -m "feat: add atomic provider registry" +``` + +## Task 4: Add Native and Explicit-Fallback Credential Stores + +**Files:** +- Create: `node/src/credentials/native-keyring.mjs` +- Create: `node/src/credentials/file-credential-store.mjs` +- Create: `node/src/credentials/credential-store.mjs` +- Create: `node/test/credential-store.test.mjs` + +- [ ] **Step 1: Write one shared adapter contract test** + +Run the same assertions against an in-memory test double and `FileCredentialStore`: + +```js +await store.set("provider-1", "sk-test-secret"); +assert.equal(await store.has("provider-1"), true); +assert.equal(await store.get("provider-1"), "sk-test-secret"); +await store.delete("provider-1"); +assert.equal(await store.has("provider-1"), false); +await assert.rejects(() => store.get("provider-1"), /CREDENTIAL_NOT_FOUND/); +``` + +Assert the fallback file is `0600`, stores no provider metadata, survives reload, and never returns all secrets through a list method. + +- [ ] **Step 2: Verify the contract tests fail** + +```bash +cd node +node --test test/credential-store.test.mjs +``` + +Expected: FAIL because credential modules do not exist. + +- [ ] **Step 3: Implement the adapters** + +Use `@napi-rs/keyring` exactly as follows in `native-keyring.mjs`: + +```js +import { Entry } from "@napi-rs/keyring"; +const SERVICE = "org.cluic.codex-remote-proxy"; + +export class NativeKeyringStore { + async set(ref, secret) { new Entry(SERVICE, ref).setPassword(secret); } + async get(ref) { + const password = new Entry(SERVICE, ref).getPassword(); + if (!password) throw new Error("CREDENTIAL_NOT_FOUND"); + return password; + } + async has(ref) { + try { return Boolean(await this.get(ref)); } catch { return false; } + } + async delete(ref) { + try { new Entry(SERVICE, ref).deletePassword(); } catch { return false; } + return true; + } +} +``` + +`FileCredentialStore` persists `{ "schemaVersion": 1, "credentials": { "provider-1": "secret" } }` atomically with `0600` mode. `createCredentialStore({ backend, fallbackConsent, paths })` must never silently choose file storage; native failure returns `CREDENTIAL_BACKEND_UNAVAILABLE` unless `fallbackConsent === true`. + +- [ ] **Step 4: Verify no secret is exposed by public helpers** + +Add assertions that `JSON.stringify(toPublicProvider(profile, true))` contains neither `credentialRef` nor `sk-test-secret`, then run: + +```bash +cd node +node --test test/credential-store.test.mjs test/provider-registry.test.mjs +``` + +Expected: all focused tests pass. + +- [ ] **Step 5: Commit** + +```bash +git add node/src/credentials node/test/credential-store.test.mjs +git commit -m "feat: add secure credential stores" +``` + +## Task 5: Make Proxy Settings Snapshot-Based + +**Files:** +- Create: `node/src/worker/runtime-settings.mjs` +- Modify: `node/src/server.mjs:48-94,242-518` +- Modify: `node/test/server.test.mjs` +- Create: `node/test/runtime-settings.test.mjs` + +- [ ] **Step 1: Write failing snapshot-generation tests** + +```js +const source = new RuntimeSettingsSource(); +source.apply({ generation: 1, settings: settingsA }); +assert.equal(source.current().generation, 1); +assert.throws(() => source.apply({ generation: 1, settings: settingsB }), /STALE_SNAPSHOT/); +source.apply({ generation: 2, settings: settingsB }); +assert.equal(source.current().settings.upstream.baseUrl, settingsB.upstream.baseUrl); +``` + +Assert returned snapshots are deeply frozen and callers cannot mutate headers. + +- [ ] **Step 2: Add an in-flight switch integration test to `server.test.mjs`** + +Start delayed upstream A and immediate upstream B. Send request 1, wait until A receives it, apply generation 2, send request 2, release A, then assert response 1 came from A and response 2 came from B. + +- [ ] **Step 3: Run both tests and verify failure** + +```bash +cd node +node --test test/runtime-settings.test.mjs test/server.test.mjs +``` + +Expected: runtime module is missing and the static server cannot switch settings. + +- [ ] **Step 4: Implement snapshot capture at request start** + +`RuntimeSettingsSource` exposes `apply(snapshot)`, `current()`, and `publicState()`. Change `createServer` to accept an optional `settingsSource`; for every non-health request, obtain exactly one snapshot before reading the body: + +```js +const active = settingsSource ? settingsSource.current() : { generation: 0, settings }; +const requestSettings = active.settings; +``` + +Use `requestSettings` for target URL, headers, timeout, SSL, capture context, and logs. Health uses `settingsSource.publicState()` and must not include credentials. + +- [ ] **Step 5: Verify snapshot behavior and regressions** + +```bash +cd node +node --test test/runtime-settings.test.mjs test/server.test.mjs +npm test +``` + +Expected: in-flight/new-request assertions pass and existing proxy/capture behavior remains green. + +- [ ] **Step 6: Commit** + +```bash +git add node/src/worker/runtime-settings.mjs node/src/server.mjs node/test/runtime-settings.test.mjs node/test/server.test.mjs +git commit -m "feat: switch proxy settings atomically" +``` + +## Task 6: Define IPC and Build the Worker Entrypoint + +**Files:** +- Create: `node/src/worker/protocol.mjs` +- Create: `node/src/worker/worker-entry.mjs` +- Create: `node/test/worker-protocol.test.mjs` +- Create: `node/test/integration/worker-entry.test.mjs` + +- [ ] **Step 1: Write failing protocol validation tests** + +Cover parent messages `configure`, `drain`, `shutdown`, `status` and child messages `ready`, `configured`, `drained`, `status`, `fatal`. Assert secrets are permitted only inside the parent-only configure payload and are removed by `sanitizeProtocolMessage()`. + +- [ ] **Step 2: Write a child-process integration test** + +Fork `worker-entry.mjs` with IPC, wait for `ready`, send generation 1 settings on port 0, wait for `configured`, issue a proxied request, request status, then send shutdown and assert exit code 0. + +- [ ] **Step 3: Run tests and verify failure** + +```bash +cd node +node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs +``` + +Expected: FAIL because protocol and entrypoint modules do not exist. + +- [ ] **Step 4: Implement the versioned protocol** + +Use `PROTOCOL_VERSION = 1`. Every message must contain `{ version: 1, type, requestId }`; configure additionally contains `{ generation, settings }`. Unknown versions/types return a `fatal` child message with stable code `WORKER_PROTOCOL_INVALID` and never echo the input payload. + +- [ ] **Step 5: Implement graceful worker behavior** + +`worker-entry.mjs` must: + +1. Create `RuntimeSettingsSource` and the HTTP server once. +2. Listen only after the first valid configure message. +3. Stop accepting connections on drain, wait for tracked in-flight requests, and emit `drained`. +4. Close capture resources and exit 0 on shutdown. +5. Emit sanitized fatal messages for uncaught startup errors. + +- [ ] **Step 6: Verify and commit** + +```bash +cd node +node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs +npm test +git add node/src/worker node/test/worker-protocol.test.mjs node/test/integration/worker-entry.test.mjs +git commit -m "feat: add proxy worker protocol" +``` + +Expected: focused and full suites pass before commit. + +## Task 7: Implement Reliable Worker Lifecycle Management + +**Files:** +- Create: `node/src/supervisor/worker-manager.mjs` +- Create: `node/test/fixtures/fake-worker.mjs` +- Create: `node/test/worker-manager.test.mjs` +- Create: `node/test/integration/worker-restart.test.mjs` + +- [ ] **Step 1: Write failing state-machine tests** + +Assert transitions: + +```text +stopped -> starting -> running +running -> draining -> stopped +running -> draining -> starting -> running +running -> crashed -> backoff -> starting +backoff -> failed after the configured crash threshold +``` + +Use injected `forkWorker`, `clock`, and `waitForPortFree` functions so unit tests contain no sleeps. + +- [ ] **Step 2: Write the fixed-port restart test** + +Start a real worker on a chosen free port, call `restart(snapshot)`, assert the PID changes, the supervisor-side manager remains available, the port is rebound, and health returns generation 1. + +- [ ] **Step 3: Run and verify failures** + +```bash +cd node +node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs +``` + +Expected: FAIL because `WorkerManager` does not exist. + +- [ ] **Step 4: Implement the lifecycle contract** + +Export a `WorkerManager` with: + +```js +start(snapshot); +applySnapshot(snapshot); +stop({ drainTimeoutMs = 5000 }); +restart(snapshot, { drainTimeoutMs = 5000 }); +getPublicState(); +close(); +``` + +Restart must wait for `drained`, escalate to `SIGTERM`, then `SIGKILL` after a bounded timeout, confirm port release by attempting an exclusive listen, spawn, configure, and require health before reporting success. Crash recovery uses delays 250ms, 500ms, 1000ms, 2000ms, 4000ms and enters `failed` after five crashes in 60 seconds. + +- [ ] **Step 5: Verify and commit** + +```bash +cd node +node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs +npm test +git add node/src/supervisor/worker-manager.mjs node/test/worker-manager.test.mjs node/test/integration/worker-restart.test.mjs node/test/fixtures/fake-worker.mjs +git commit -m "feat: manage proxy worker lifecycle" +``` + +## Task 8: Add Activity, Migration, and Provider Orchestration + +**Files:** +- Create: `node/src/supervisor/activity-store.mjs` +- Create: `node/src/supervisor/migration.mjs` +- Create: `node/src/supervisor/provider-service.mjs` +- Create: `node/test/activity-store.test.mjs` +- Create: `node/test/migration.test.mjs` +- Create: `node/test/provider-service.test.mjs` + +- [ ] **Step 1: Write failing activity redaction tests** + +Append an event containing authorization, cookie, token, secret, API key, and nested error fields. Assert the persisted JSONL contains `[REDACTED]`, never the test values, and retention truncates to the newest 10,000 rows. + +- [ ] **Step 2: Write transactional migration tests** + +Given the current flat `config.json`, assert migration creates one `Default` provider, stores its key through the injected credential adapter, backs up original files, and removes the secret from provider metadata. Inject a credential-write failure and assert original bytes are restored and no schema-2 file remains. + +- [ ] **Step 3: Write provider-service tests** + +Use two local mock upstreams and assert `testProvider(id, model)` classifies DNS/TLS/timeout/401/404/invalid-Responses errors, marks success, and that `activate(id)` rejects untested profiles, resolves only the selected credential, increments generation, persists active ID, and waits for worker acknowledgement. + +- [ ] **Step 4: Run tests and verify failure** + +```bash +cd node +node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs +``` + +Expected: FAIL because the orchestration modules do not exist. + +- [ ] **Step 5: Implement the three services** + +Use one safe recursive redactor shared by activity and errors. `ProviderService` exposes: + +```js +listProviders(); +createProvider(input, secret, { fallbackConsent }); +updateProvider(id, patch, replacementSecret); +deleteProvider(id); +testProvider(id, model); +activate(id); +getStatus(); +``` + +`testProvider` sends a minimal `POST /responses` request with `stream: false`, `input: "Reply with OK."`, and the selected model. Store only stable result codes and timestamps. + +- [ ] **Step 6: Verify and commit** + +```bash +cd node +node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs +npm test +git add node/src/supervisor/activity-store.mjs node/src/supervisor/migration.mjs node/src/supervisor/provider-service.mjs node/test/activity-store.test.mjs node/test/migration.test.mjs node/test/provider-service.test.mjs +git commit -m "feat: orchestrate provider lifecycle" +``` + +## Task 9: Build the Secured Loopback Admin API + +**Files:** +- Create: `node/src/supervisor/session-auth.mjs` +- Create: `node/src/supervisor/admin-server.mjs` +- Create: `node/src/supervisor/supervisor.mjs` +- Create: `node/src/supervisor/supervisor-entry.mjs` +- Create: `node/test/session-auth.test.mjs` +- Create: `node/test/integration/admin-server.test.mjs` + +- [ ] **Step 1: Write failing browser-auth tests** + +Assert a 32-byte control token is created with `0600` mode. Test `/api/v1/session` with a valid bearer token returns an HttpOnly `SameSite=Strict` cookie and CSRF token, then clears access when expired. Invalid Host, non-loopback Origin, missing CSRF, missing cookie, and CORS preflight must be rejected. + +- [ ] **Step 2: Write failing API contract tests** + +Exercise every endpoint in `docs/API.md` against injected provider and worker services. Assert mutation status codes, `409` conflicts, sanitized error format, activity pagination, static UI fallback, and that serialized responses never contain `credentialRef` or a complete key. + +- [ ] **Step 3: Run and verify failures** + +```bash +cd node +node --test test/session-auth.test.mjs test/integration/admin-server.test.mjs +``` + +Expected: FAIL because auth and Admin API modules do not exist. + +- [ ] **Step 4: Implement auth and route table** + +Use `crypto.randomBytes(32).toString("base64url")` for control/session/CSRF tokens. Accept bearer auth for CLI and cookie+CSRF for browser mutations. Route only the methods and paths listed in `docs/API.md`; unmatched API paths return `404 API_NOT_FOUND`. Serve `ui/index.html`, `ui/styles.css`, and `ui/app.js` with explicit MIME types and `Cache-Control: no-store`. + +- [ ] **Step 5: Compose the supervisor** + +`createSupervisor(options)` builds paths, migration, registry, credential store, activity store, worker manager, provider service, auth, and Admin API. `supervisor-entry.mjs` loads the selected home, listens on `127.0.0.1:15101`, writes state only after health is ready, and shuts down cleanly on `SIGTERM`/`SIGINT`. + +- [ ] **Step 6: Verify and commit** + +```bash +cd node +node --test test/session-auth.test.mjs test/integration/admin-server.test.mjs +npm test +git add node/src/supervisor node/test/session-auth.test.mjs node/test/integration/admin-server.test.mjs +git commit -m "feat: add secured local Admin API" +``` + +## Task 10: Route CLI Commands Through the Supervisor + +**Files:** +- Create: `node/src/supervisor/supervisor-client.mjs` +- Modify: `node/bin/crp.mjs` +- Modify: `node/test/crp.test.mjs` +- Create: `node/test/integration/crp-lifecycle.test.mjs` + +- [ ] **Step 1: Extend failing CLI tests** + +Assert help includes: + +```text +crp ui [--no-open] [--json] +crp restart [--json] +crp shutdown [--json] +crp provider list|add|test|activate|delete [--json] +``` + +Using a temporary HOME, assert `ui --no-open --json` starts a supervisor, `start` starts the worker, `restart` changes only worker PID, `stop` stops only the worker, and `shutdown` stops the supervisor. + +- [ ] **Step 2: Run and verify failures** + +```bash +cd node +node --test test/crp.test.mjs test/integration/crp-lifecycle.test.mjs +``` + +Expected: FAIL because new commands and client do not exist. + +- [ ] **Step 3: Implement supervisor discovery and client** + +`ensureSupervisor({ paths, adminPort })` checks state and `/api/v1/status`; if absent, spawn detached `supervisor-entry.mjs`, wait up to 8 seconds for health, and return its URL. `SupervisorClient` reads the control token and exposes `request(method, path, body)` with bearer auth. + +- [ ] **Step 4: Implement command semantics** + +- `ui`: ensure supervisor, create the fragment URL `/#token=`, open with macOS `open`, Windows `cmd /c start`, or Linux `xdg-open`; `--no-open` only prints JSON. +- `start`: ensure supervisor, bootstrap Codex if required, then POST `/proxy/start`. +- `stop`: POST `/proxy/stop` and leave supervisor running. +- `restart`: POST `/proxy/restart`. +- `shutdown`: terminate supervisor after stopping worker. +- `install` and `setup`: remain aliases for `start` with one deprecation field in JSON. +- `provider`: expose automation-safe CRUD/test/activate commands without printing complete keys. + +- [ ] **Step 5: Verify compatibility and commit** + +```bash +cd node +node --test test/crp.test.mjs test/integration/crp-lifecycle.test.mjs +npm test +git add node/bin/crp.mjs node/src/supervisor/supervisor-client.mjs node/test/crp.test.mjs node/test/integration/crp-lifecycle.test.mjs +git commit -m "feat: manage CRP through supervisor CLI" +``` + +## Task 11: Implement the Guided Utility Web UI + +**Files:** +- Create: `node/ui/index.html` +- Create: `node/ui/styles.css` +- Create: `node/ui/app.js` +- Create: `node/playwright.config.mjs` +- Create: `node/test/e2e/onboarding.spec.mjs` +- Create: `node/test/e2e/provider-switch.spec.mjs` +- Create: `node/test/e2e/restart-and-errors.spec.mjs` + +- [ ] **Step 1: Write failing Playwright onboarding test** + +The test starts the supervisor against a temporary HOME and mock upstream, opens the fragment-token URL, enters provider name/base URL/API key/test model, clicks `Test connection`, expects `Compatible`, activates, bootstraps Codex, and expects the Overview page. Assert the generated Codex config contains `model_provider = "OpenAI"` and fixed port 15100. + +- [ ] **Step 2: Write failing daily-flow tests** + +Create two providers through API fixtures, then use the UI to switch A → B, restart the worker, view an actionable 401 error, navigate with keyboard only, and assert no input contains the saved key. Add `axe`-free semantic assertions using Playwright roles and a screenshot for Overview at 1440×900. + +- [ ] **Step 3: Run and verify UI absence** + +```bash +cd node +npx playwright install chromium +npm run test:e2e +``` + +Expected: FAIL because the UI files and visible role names do not exist. + +- [ ] **Step 4: Build the accessible static shell** + +`index.html` contains semantic landmarks, a skip link, `aria-live="polite"` status region, onboarding and app roots, and no inline scripts. `styles.css` implements the approved guided utility console with system fonts, 8px spacing scale, 44px targets, visible focus, AA colors, status icons plus text, responsive desktop layout, and reduced-motion rules. + +- [ ] **Step 5: Implement UI state and API calls** + +`app.js` must: + +1. Exchange the fragment token at `/api/v1/session`, then remove the fragment with `history.replaceState`. +2. Keep the returned CSRF token only in memory. +3. Render onboarding when no provider exists. +4. Render Overview, Providers, Activity, and Settings without exposing secrets. +5. Disable duplicate mutations while one is pending. +6. Map stable API errors to cause/action panels and expandable sanitized details. +7. Confirm destructive deletion and restart only when in-flight requests exist. + +- [ ] **Step 6: Run visual and regression checks** + +```bash +cd node +npm run lint +npm run test:e2e +npm test +``` + +Expected: all browser flows pass and Playwright writes the approved Overview screenshot artifact. + +- [ ] **Step 7: Commit** + +```bash +git add node/ui node/playwright.config.mjs node/test/e2e node/package.json node/package-lock.json +git commit -m "feat: add guided local management UI" +``` + +## Task 12: Complete Migration, Cross-Platform Gates, Docs, and Release Readiness + +**Files:** +- Modify: `.github/workflows/release-preflight.yml` +- Create: `.github/workflows/platform-tests.yml` +- Modify: `README.md` +- Modify: `README.zh-CN.md` +- Modify: `node/README.md` +- Modify: `node/CHANGELOG.md` +- Create: `node/.changeset/multi-provider-local-ui.md` +- Modify: all affected `docs/*.md` +- Modify: `docs/STATUS.md` +- Modify: `docs/AI_HANDOFF.md` + +- [ ] **Step 1: Add failing package-content verification** + +Run `npm pack --dry-run --json` in a test and assert the package includes `bin/`, `src/`, `ui/index.html`, `ui/styles.css`, and `ui/app.js`, and excludes tests, `.superpowers`, credentials, state, and capture databases. + +- [ ] **Step 2: Add platform workflow matrix** + +Create a GitHub Actions matrix over `macos-latest`, `windows-latest`, and `ubuntu-latest` using Node 22. Each runs `npm ci`, `npm run lint`, and `npm test`. macOS and Windows also install Chromium and run E2E; native keyring jobs create/read/delete a synthetic CI credential and always clean it up. + +- [ ] **Step 3: Run the complete local deterministic gate** + +```bash +cd node +npm run lint +npm test +npm run test:integration +npm run test:e2e +npm audit --omit=dev +npm pack --dry-run +``` + +Expected: zero test failures, zero runtime vulnerabilities, and UI assets present in the tarball listing. + +- [ ] **Step 4: Perform manual security and visual review** + +Use temporary credentials only. Search generated logs, activity, diagnostics, capture records, API responses, screenshots, and the Git diff for the complete test key. Verify invalid Host/Origin requests fail, the Admin API is not reachable from a non-loopback bind, macOS/Windows screenshots match the approved guided console, and keyboard-only onboarding succeeds. + +- [ ] **Step 5: Synchronize living docs and release notes** + +Mark implemented facts rather than target facts, record verification commands/results, update migration and rollback instructions, document new commands, set `docs/STATUS.md` to expert review, and add a minor Changeset because the npm package gains user-visible provider/UI/lifecycle capabilities. + +- [ ] **Step 6: Request two-stage review** + +First request requirements review against the approved design spec. After all requirement findings are resolved, request code-quality/security review focused on credential boundaries, migration rollback, localhost security, worker lifecycle, and secret leakage. + +- [ ] **Step 7: Commit the release-readiness change** + +```bash +git add .github README.md README.zh-CN.md node/README.md node/CHANGELOG.md node/.changeset docs +git commit -m "docs: prepare multi-provider UI release" +``` + +## Final Verification and Merge Gate + +- [ ] Re-run every command from Task 12 Step 3 on the final tree. +- [ ] Confirm macOS, Windows, and Linux CI is green. +- [ ] Confirm zero unresolved requirements, quality, security, accessibility, or visual-review findings. +- [ ] Confirm migration rollback evidence and a redacted diagnostic bundle are attached to the review. +- [ ] Confirm `docs/STATUS.md`, `docs/AI_HANDOFF.md`, and `docs/DECISIONS.md` match implemented behavior. +- [ ] Classify the implementation as L3 and obtain expert confirmation; do not auto-merge. diff --git a/docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md b/docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md index 0a3d533..2558cf8 100644 --- a/docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md +++ b/docs/superpowers/specs/2026-07-10-crp-product-evolution-design.md @@ -2,7 +2,7 @@ Date: 2026-07-10 -Status: Approved in conversation; written specification pending user review +Status: Approved by user Mode: Cluic Harness Builder `iterate` From 7d7553e46e674265376ddbc7d72c644388115840 Mon Sep 17 00:00:00 2001 From: cluic Date: Fri, 10 Jul 2026 12:21:37 +0800 Subject: [PATCH 03/44] chore: ignore local worktrees --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 19d46c3..5fe9725 100644 --- a/.gitignore +++ b/.gitignore @@ -40,3 +40,4 @@ coverage/ /python/ /__pycache__/ .superpowers/ +.worktrees/ From e18ae09d8a069009408ebc2076672a0f4433921c Mon Sep 17 00:00:00 2001 From: cluic Date: Sun, 12 Jul 2026 18:01:48 +0800 Subject: [PATCH 04/44] build: add CRP quality gates --- docs/AGENT_COORDINATION.md | 6 +- docs/AI_HANDOFF.md | 12 +- docs/STATUS.md | 13 +- docs/TESTING.md | 10 +- .../plans/2026-07-10-crp-v1-implementation.md | 12 +- node/package-lock.json | 290 +++++++++++++++++- node/package.json | 10 +- node/scripts/check-source.mjs | 30 ++ node/scripts/run-test-group.mjs | 25 ++ node/test/capture-store.test.mjs | 57 +++- 10 files changed, 424 insertions(+), 41 deletions(-) create mode 100644 node/scripts/check-source.mjs create mode 100644 node/scripts/run-test-group.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 232e9f5..518f230 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| Harness, design, and plan | Primary Codex | Documentation only | `AGENTS.md`, `.codex/`, `docs/` | Ready | User execution-mode selection | Product code under `node/` | +| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 1 owns `node/package*.json`, `node/scripts/`, `node/test/capture-store.test.mjs`, and affected living docs | In progress on `codex/harness-product-design` | Approved design, plan, and Node 22 baseline | Every file outside the active task; no simultaneous writable agent | ## Shared Contracts @@ -23,10 +23,10 @@ ## Locks / Avoid Editing -- No parallel writable work is authorized while the written spec is under review. +- No parallel writable work is authorized; only one task implementer may write at a time, and review agents are read-only. - Future agents must not change shared contracts without first updating the owning doc and coordination row. - Credential, migration, and lifecycle work must be isolated from UI styling work until contracts pass review. ## Decisions Needed -No blocking decision remains for planning. File ownership and worktree assignments must be added before any parallel implementation begins. +Subagent-driven execution is selected. Work remains on the clean dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index 88e00f3..a967d51 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,7 +6,7 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -Documentation, harness, and implementation planning only. No product implementation from the approved design has landed. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Task 1 quality gates and dependencies have landed; no provider-lifecycle product behavior has landed yet. Task 2, stable paths, errors, and Codex bootstrap, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture @@ -24,14 +24,16 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification is approved and the detailed V1 plan awaits execution-mode selection. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Task 1 is complete. ## How To Run Current Code ```bash cd node npm ci +npm run lint npm test +npm run test:unit node bin/crp.mjs --help ``` @@ -39,8 +41,9 @@ Do not run `crp start` against a real home directory during tests because it mod ## Verification -- Current baseline: `cd node && npm test` -- Runtime audit: `cd node && npm audit --omit=dev` +- Node 22.19 baseline and Task 1 gate: `npm test` passes 12/12 tests and `npm audit --omit=dev` reports zero vulnerabilities. +- Node 22.19 additions: `npm run lint` syntax-checks the current source roots and `npm run test:unit` passes the 12 top-level tests. +- Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. ## Known Risks @@ -57,3 +60,4 @@ Credential migration, localhost browser security, worker IPC, port release races - Use guided utility console UI. - Classify future V1 implementation as L3. - Execute the approved design through the task sequence in `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md`. +- Keep file-watcher tests condition-based and cleanup-safe across supported Node versions. diff --git a/docs/STATUS.md b/docs/STATUS.md index b704da7..e35d22e 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -2,11 +2,11 @@ ## Current Milestone -M1 design and harness review. +V1 implementation: provider lifecycle end to end. ## In Progress -- User selection of the V1 execution mode. +- Sequential execution of the approved V1 plan; Task 1 is complete and Task 2 is next. ## Done @@ -17,16 +17,17 @@ M1 design and harness review. - Created the project harness and living docs. - Received written specification approval. - Created and self-reviewed the detailed V1 implementation plan. +- Completed Task 1 portable syntax, unit-test, E2E script, packaging, and dependency gates. +- Stabilized capture watcher tests with condition-based waits and assertion-safe cleanup on Node 22.19 and Node 24.2. ## Blocked -- Product implementation is blocked until the detailed implementation plan is reviewed and an execution mode is selected. +- No current blocker is recorded for Task 2. ## Next -1. Review `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md`. -2. Select subagent-driven or inline execution. -3. Begin V1 with tests for supervisor/worker lifecycle and stable provider behavior. +1. Execute Task 2: extract stable paths and errors, then add idempotent Codex bootstrap behavior. +2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/TESTING.md b/docs/TESTING.md index 3bbda16..f14f1bf 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -16,10 +16,16 @@ ## Current Commands -- Existing suite: `cd node && npm test` +- Portable syntax check: `cd node && npm run lint` +- Existing full suite: `cd node && npm test` +- Top-level unit suite: `cd node && npm run test:unit` - Runtime audit: `cd node && npm audit --omit=dev` -The implementation plan must add stable scripts for `test:unit`, `test:integration`, `test:e2e`, `lint`, and UI build before their respective modules land. +`npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. `npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. + +## Test Authoring Rules + +- File-watcher tests must wait for observable state and register cleanup before assertions. ## Test Matrix diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index 9dfca48..185d870 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -64,7 +64,7 @@ - Create: `node/scripts/run-test-group.mjs` - Test: existing `node/test/*.test.mjs` -- [ ] **Step 1: Record the fresh baseline** +- [x] **Step 1: Record the fresh baseline** Run: @@ -76,7 +76,7 @@ npm audit --omit=dev Expected: 12 tests pass and the runtime audit reports zero vulnerabilities. -- [ ] **Step 2: Add the portable source-check script** +- [x] **Step 2: Add the portable source-check script** Create `node/scripts/check-source.mjs`: @@ -113,7 +113,7 @@ for (const file of files.sort()) { console.log(`Syntax checked ${files.length} source files.`); ``` -- [ ] **Step 3: Add the cross-platform grouped-test runner** +- [x] **Step 3: Add the cross-platform grouped-test runner** Create `node/scripts/run-test-group.mjs`: @@ -143,7 +143,7 @@ const result = spawnSync(process.execPath, ["--test", ...files], { stdio: "inher process.exit(result.status ?? 1); ``` -- [ ] **Step 4: Add exact dependencies and scripts** +- [x] **Step 4: Add exact dependencies and scripts** Run: @@ -176,7 +176,7 @@ Set these scripts in `node/package.json`: } ``` -- [ ] **Step 5: Verify gates** +- [x] **Step 5: Verify gates** Run: @@ -189,7 +189,7 @@ npm audit --omit=dev Expected: syntax check succeeds, 12 tests pass, and runtime audit reports zero vulnerabilities. -- [ ] **Step 6: Commit** +- [x] **Step 6: Commit** ```bash git add node/package.json node/package-lock.json node/scripts/check-source.mjs node/scripts/run-test-group.mjs diff --git a/node/package-lock.json b/node/package-lock.json index bd23b90..c81aa44 100644 --- a/node/package-lock.json +++ b/node/package-lock.json @@ -1,20 +1,22 @@ { "name": "@cluic/codex-remote-proxy", - "version": "0.1.3", + "version": "0.2.2", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@cluic/codex-remote-proxy", - "version": "0.1.3", + "version": "0.2.2", "dependencies": { + "@napi-rs/keyring": "1.3.0", "fzstd": "^0.1.1" }, "bin": { "crp": "bin/crp.mjs" }, "devDependencies": { - "@changesets/cli": "^2.31.0" + "@changesets/cli": "^2.31.0", + "@playwright/test": "1.61.1" }, "engines": { "node": ">=22.13.0" @@ -366,6 +368,225 @@ "node": ">=6 <7 || >=8" } }, + "node_modules/@napi-rs/keyring": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring/-/keyring-1.3.0.tgz", + "integrity": "sha512-WrOw/bcXm0f9qHkumlT1QlArXSTWqaY9sunsDpOk+yCCorCKMxvWT/a3xko4EYHVdeZoh00yI2TydXn6eyICDA==", + "license": "MIT", + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + }, + "optionalDependencies": { + "@napi-rs/keyring-darwin-arm64": "1.3.0", + "@napi-rs/keyring-darwin-x64": "1.3.0", + "@napi-rs/keyring-freebsd-x64": "1.3.0", + "@napi-rs/keyring-linux-arm-gnueabihf": "1.3.0", + "@napi-rs/keyring-linux-arm64-gnu": "1.3.0", + "@napi-rs/keyring-linux-arm64-musl": "1.3.0", + "@napi-rs/keyring-linux-riscv64-gnu": "1.3.0", + "@napi-rs/keyring-linux-x64-gnu": "1.3.0", + "@napi-rs/keyring-linux-x64-musl": "1.3.0", + "@napi-rs/keyring-win32-arm64-msvc": "1.3.0", + "@napi-rs/keyring-win32-ia32-msvc": "1.3.0", + "@napi-rs/keyring-win32-x64-msvc": "1.3.0" + } + }, + "node_modules/@napi-rs/keyring-darwin-arm64": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-darwin-arm64/-/keyring-darwin-arm64-1.3.0.tgz", + "integrity": "sha512-pl76hJvdYUBn6I24bXiOBMA9nbDapo3I5B+f3OorjDU4dUMSypXeKbOVehJe8fhgTiH24flMyTS3aAIy43xegQ==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-darwin-x64": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-darwin-x64/-/keyring-darwin-x64-1.3.0.tgz", + "integrity": "sha512-YcJtEV5LA3cvA4z3BurgxH5IhTsW1JfIvcAAcqcecwk06Si9F9NqkxbZVIfDwQ8oRHgaBmT3zZJnLAotCrVahw==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-freebsd-x64": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-freebsd-x64/-/keyring-freebsd-x64-1.3.0.tgz", + "integrity": "sha512-vlLf31TGhfRAaxLDBhg8b89ss0HHD/lyNmL5F3UjSaz5CUXElsJmKYq9fqA/B+cZKUEUcLHHGhF0I/CqcFdaVw==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-linux-arm-gnueabihf": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-linux-arm-gnueabihf/-/keyring-linux-arm-gnueabihf-1.3.0.tgz", + "integrity": "sha512-KiWdMMu/Inz/bHHIAGrnF7r54FZDYXuHO6UFF/rhIrshUsxbMG1Rl9lEymNtqqsVo927G0VYcb02FzWQ3iBQRQ==", + "cpu": [ + "arm" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-linux-arm64-gnu": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-linux-arm64-gnu/-/keyring-linux-arm64-gnu-1.3.0.tgz", + "integrity": "sha512-eyKGpY40lm9Jvs1aD294XRH4y7+TlJM0YVAryZeXA6TX0mb4gMkxVXwSQv7MCwgah7raeUd0dKUb4BPAYIgcMg==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-linux-arm64-musl": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-linux-arm64-musl/-/keyring-linux-arm64-musl-1.3.0.tgz", + "integrity": "sha512-iIK6JWHXAJqDrEyLY3TmswwloVyt2vj+04TZnew+uSJ9gnDO8EwRbp3/iw3LpWaXiDO7VomGO6y8I0Id8uBZSw==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-linux-riscv64-gnu": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-linux-riscv64-gnu/-/keyring-linux-riscv64-gnu-1.3.0.tgz", + "integrity": "sha512-/PGqrwn6EwgtK6vccASSXJRfOSP4vN1F4ASsIQ+7MdrK6hNvAJ1FZPrIuD5gGGdxezo3F++To2Wq7DbuGIeuNQ==", + "cpu": [ + "riscv64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-linux-x64-gnu": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-linux-x64-gnu/-/keyring-linux-x64-gnu-1.3.0.tgz", + "integrity": "sha512-2PDK1WKWTu9lBGq9VvNEkSlQD3O7YwVpmnyN2M3cy4v7NJ/8gDMd9GXv3G+FVXN13uhp4gnnPBS+ScefmEeD2A==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-linux-x64-musl": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-linux-x64-musl/-/keyring-linux-x64-musl-1.3.0.tgz", + "integrity": "sha512-oJ2HkX8YUo46QBkn0pG+HuIKQNqr523q6vBobCn+P95s4C4K6/kLBqHY/1bg5J4ap31DzsznhnFKcfBNBsjCnw==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-win32-arm64-msvc": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-win32-arm64-msvc/-/keyring-win32-arm64-msvc-1.3.0.tgz", + "integrity": "sha512-tOd3c/uAaeoE4ycVlmAdSvygz0Zt3zdca6Y7gokBeIbaRDWpjDIUOpU3MvML59XAaqyuKGsVVu0F/DZb1lHPmw==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-win32-ia32-msvc": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-win32-ia32-msvc/-/keyring-win32-ia32-msvc-1.3.0.tgz", + "integrity": "sha512-sPSqeAFZMGqP1R++M2JTza7GQJJ/TpCo6JU6Vcd4jnebvOaEDs9b7eipakU1PJdSvhpC2yXMCNRk9gXfrhuwHQ==", + "cpu": [ + "ia32" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@napi-rs/keyring-win32-x64-msvc": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/@napi-rs/keyring-win32-x64-msvc/-/keyring-win32-x64-msvc-1.3.0.tgz", + "integrity": "sha512-4DnCWXwDc0HRKwyRlG5y0VhKZW2tNRQfKKfyj6IX/KWfDNyq9hn4n+GL1auyDcOO/v8PwnhmYo2+rOOqCkvvOg==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, "node_modules/@nodelib/fs.scandir": { "version": "2.1.5", "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", @@ -404,6 +625,22 @@ "node": ">= 8" } }, + "node_modules/@playwright/test": { + "version": "1.61.1", + "resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.61.1.tgz", + "integrity": "sha512-8nKv6+0RJSL9FE4jYOEGXnPeM/Hg12qZpmqzZjRh3qM0Y7c3z1mrOTfFLids72RDQYVh9WpLEfR5WdpNX4fkig==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "playwright": "1.61.1" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/@types/node": { "version": "25.9.0", "resolved": "https://registry.npmjs.org/@types/node/-/node-25.9.0.tgz", @@ -628,6 +865,21 @@ "node": ">=6 <7 || >=8" } }, + "node_modules/fsevents": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", + "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, "node_modules/fzstd": { "version": "0.1.1", "resolved": "https://registry.npmmirror.com/fzstd/-/fzstd-0.1.1.tgz", @@ -991,6 +1243,38 @@ "node": ">=6" } }, + "node_modules/playwright": { + "version": "1.61.1", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.61.1.tgz", + "integrity": "sha512-DWnY5o3YbLWK4GovuAVwpqL+1VwGNdUGrRr++8j8PtQQzvAVZUIMjKQ90fY689sEJZJBbZVw1rXaOKSTitkzPQ==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "playwright-core": "1.61.1" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "fsevents": "2.3.2" + } + }, + "node_modules/playwright-core": { + "version": "1.61.1", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.61.1.tgz", + "integrity": "sha512-h7Qlt6m4REp25qvIdvbDtVmD4LqVXfpRxhORv9L0jzETM05p4fuPJ3dKyuSXQxDSbXnmS79HAgi9589lGSpLkg==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "playwright-core": "cli.js" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/prettier": { "version": "2.8.8", "resolved": "https://registry.npmjs.org/prettier/-/prettier-2.8.8.tgz", diff --git a/node/package.json b/node/package.json index 8f469eb..fa8a5b8 100644 --- a/node/package.json +++ b/node/package.json @@ -14,6 +14,7 @@ "files": [ "bin/", "src/", + "ui/", "proxy-config.example.json", "README.md" ], @@ -25,7 +26,12 @@ "check": "node bin/crp.mjs check", "status": "node bin/crp.mjs status", "guide": "node bin/crp.mjs guide", + "lint": "node scripts/check-source.mjs", "test": "node --test", + "test:unit": "node scripts/run-test-group.mjs unit", + "test:integration": "node scripts/run-test-group.mjs integration", + "test:e2e": "playwright test", + "test:all": "npm run lint && npm run test && npm run test:e2e", "changeset": "changeset", "version-packages": "changeset version", "release": "changeset publish" @@ -37,9 +43,11 @@ "node": ">=22.13.0" }, "dependencies": { + "@napi-rs/keyring": "1.3.0", "fzstd": "^0.1.1" }, "devDependencies": { - "@changesets/cli": "^2.31.0" + "@changesets/cli": "^2.31.0", + "@playwright/test": "1.61.1" } } diff --git a/node/scripts/check-source.mjs b/node/scripts/check-source.mjs new file mode 100644 index 0000000..f35eb36 --- /dev/null +++ b/node/scripts/check-source.mjs @@ -0,0 +1,30 @@ +import { readdirSync, statSync } from "node:fs"; +import { join, relative } from "node:path"; +import { spawnSync } from "node:child_process"; +import { fileURLToPath } from "node:url"; + +const root = fileURLToPath(new URL("..", import.meta.url)); +const roots = [join(root, "bin"), join(root, "src"), join(root, "scripts"), join(root, "ui")]; +const files = []; + +function walk(dir) { + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const path = join(dir, entry.name); + if (entry.isDirectory()) walk(path); + if (entry.isFile() && /\.(mjs|js)$/.test(entry.name)) files.push(path); + } +} + +for (const dir of roots) { + if (statSync(dir, { throwIfNoEntry: false })?.isDirectory()) walk(dir); +} + +for (const file of files.sort()) { + const result = spawnSync(process.execPath, ["--check", file], { encoding: "utf8" }); + if (result.status !== 0) { + process.stderr.write(`${relative(root, file)}\n${result.stderr}`); + process.exit(result.status ?? 1); + } +} + +console.log(`Syntax checked ${files.length} source files.`); diff --git a/node/scripts/run-test-group.mjs b/node/scripts/run-test-group.mjs new file mode 100644 index 0000000..c6df129 --- /dev/null +++ b/node/scripts/run-test-group.mjs @@ -0,0 +1,25 @@ +import { readdirSync, statSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { spawnSync } from "node:child_process"; + +const group = process.argv[2]; +const testRoot = resolve("test"); +const selectedRoot = group === "integration" ? join(testRoot, "integration") : testRoot; +const recursive = group === "integration"; + +function collect(dir, descend) { + const files = []; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const path = join(dir, entry.name); + if (entry.isDirectory() && descend) files.push(...collect(path, true)); + if (entry.isFile() && entry.name.endsWith(".test.mjs")) files.push(path); + } + return files; +} + +const files = statSync(selectedRoot, { throwIfNoEntry: false })?.isDirectory() + ? collect(selectedRoot, recursive).sort() + : []; +if (files.length === 0) throw new Error(`No ${group} test files found`); +const result = spawnSync(process.execPath, ["--test", ...files], { stdio: "inherit" }); +process.exit(result.status ?? 1); diff --git a/node/test/capture-store.test.mjs b/node/test/capture-store.test.mjs index 2e5184f..644bfc1 100644 --- a/node/test/capture-store.test.mjs +++ b/node/test/capture-store.test.mjs @@ -18,8 +18,15 @@ function makeTempDir(prefix) { return join(os.tmpdir(), `${prefix}-${Date.now()}-${Math.random().toString(16).slice(2)}`); } -function wait(ms = 700) { - return new Promise((resolvePromise) => setTimeout(resolvePromise, ms)); +async function waitFor(condition, description, { timeoutMs = 5000, intervalMs = 25 } = {}) { + const deadline = Date.now() + timeoutMs; + + while (!condition()) { + if (Date.now() >= deadline) { + throw new Error(`Timed out waiting for ${description} after ${timeoutMs}ms`); + } + await new Promise((resolvePromise) => setTimeout(resolvePromise, intervalMs)); + } } test("normalizeCaptureConfig applies defaults", () => { @@ -118,9 +125,17 @@ test("capture manager writes a complete request/response record", async () => { rmSync(dir, { recursive: true, force: true }); }); -test("capture manager hot-disables when runtime config changes", async () => { +test("capture manager hot-disables when runtime config changes", async (t) => { const dir = makeTempDir("crp-hot-disable"); mkdirSync(dir, { recursive: true }); + let manager; + t.after(() => { + try { + manager?.close(); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); const runtimeConfigPath = join(dir, "proxy-config.json"); const dbPath = join(dir, "traffic.sqlite3"); writeFileSync(runtimeConfigPath, `${JSON.stringify({ @@ -130,13 +145,14 @@ test("capture manager hot-disables when runtime config changes", async () => { } }, null, 2)}\n`, "utf8"); - const manager = new CaptureManager({ + manager = new CaptureManager({ configPath: runtimeConfigPath, capture: { enabled: true, dbPath } - }).start(); + }); + manager.start(); assert.equal(manager.getPublicState().captureActive, true); writeFileSync(runtimeConfigPath, `${JSON.stringify({ @@ -145,18 +161,26 @@ test("capture manager hot-disables when runtime config changes", async () => { dbPath } }, null, 2)}\n`, "utf8"); - await wait(); + await waitFor( + () => manager.getPublicState().captureActive === false, + "capture recording to become inactive" + ); assert.equal(manager.getPublicState().captureActive, false); assert.equal(manager.getPublicState().captureState, "disabled"); - - manager.close(); - rmSync(dir, { recursive: true, force: true }); }); -test("capture manager marks restart required when db path changes", async () => { +test("capture manager marks restart required when db path changes", async (t) => { const dir = makeTempDir("crp-db-change"); mkdirSync(dir, { recursive: true }); + let manager; + t.after(() => { + try { + manager?.close(); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); const runtimeConfigPath = join(dir, "proxy-config.json"); const dbPath = join(dir, "traffic.sqlite3"); const nextDbPath = join(dir, "traffic-next.sqlite3"); @@ -167,13 +191,14 @@ test("capture manager marks restart required when db path changes", async () => } }, null, 2)}\n`, "utf8"); - const manager = new CaptureManager({ + manager = new CaptureManager({ configPath: runtimeConfigPath, capture: { enabled: true, dbPath } - }).start(); + }); + manager.start(); writeFileSync(runtimeConfigPath, `${JSON.stringify({ capture: { @@ -181,15 +206,15 @@ test("capture manager marks restart required when db path changes", async () => dbPath: nextDbPath } }, null, 2)}\n`, "utf8"); - await wait(); + await waitFor( + () => manager.getPublicState().captureRestartRequired === true, + "capture restart to become required" + ); const state = manager.getPublicState(); assert.equal(state.captureRestartRequired, true); assert.equal(resolve(state.captureRuntimeDbPath), resolve(dbPath)); assert.equal(resolve(state.captureDbPath), resolve(nextDbPath)); - - manager.close(); - rmSync(dir, { recursive: true, force: true }); }); test("loadRuntimeCaptureConfig validates malformed config", () => { From 587707c4a92af8152bbe5d69e8dda6318413aa21 Mon Sep 17 00:00:00 2001 From: cluic Date: Sun, 12 Jul 2026 18:35:10 +0800 Subject: [PATCH 05/44] refactor: isolate Codex bootstrap --- docs/AGENT_COORDINATION.md | 4 +- docs/AI_HANDOFF.md | 11 +- docs/ARCHITECTURE.md | 15 +- docs/STATUS.md | 7 +- docs/TESTING.md | 6 +- .../plans/2026-07-10-crp-v1-implementation.md | 14 +- node/bin/crp.mjs | 95 +-- node/src/codex/codex-config.mjs | 264 +++++++++ node/src/shared/errors.mjs | 29 + node/src/shared/paths.mjs | 19 + node/test/codex-config.test.mjs | 546 ++++++++++++++++++ 11 files changed, 909 insertions(+), 101 deletions(-) create mode 100644 node/src/codex/codex-config.mjs create mode 100644 node/src/shared/errors.mjs create mode 100644 node/src/shared/paths.mjs create mode 100644 node/test/codex-config.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 518f230..e6e50b8 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 1 owns `node/package*.json`, `node/scripts/`, `node/test/capture-store.test.mjs`, and affected living docs | In progress on `codex/harness-product-design` | Approved design, plan, and Node 22 baseline | Every file outside the active task; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 2 owns `node/src/shared/`, `node/src/codex/`, `node/bin/crp.mjs`, `node/test/codex-config.test.mjs`, and affected living docs | Task 2 complete on `codex/harness-product-design`; Task 3 next | Task 1 quality gates | Every file outside the active task; no simultaneous writable agent | ## Shared Contracts @@ -29,4 +29,4 @@ ## Decisions Needed -Subagent-driven execution is selected. Work remains on the clean dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. +Subagent-driven execution is selected. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. Task 3 requires a fresh file scope before writable work begins. diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index a967d51..48418b4 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,11 +6,11 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Task 1 quality gates and dependencies have landed; no provider-lifecycle product behavior has landed yet. Task 2, stable paths, errors, and Codex bootstrap, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 and 2 have landed; no provider-registry or provider-lifecycle product behavior has landed yet. Task 3, the atomic provider registry, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Target: long-lived supervisor control plane plus independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, and idempotent Codex bootstrap with source-EOL preservation, CRP sidecar locking, exclusive adjacent backups, external-change detection, atomic replacement, and permission preservation. Target: long-lived supervisor control plane plus independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Task 1 is complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 and 2 are complete. ## How To Run Current Code @@ -42,13 +42,13 @@ Do not run `crp start` against a real home directory during tests because it mod ## Verification - Node 22.19 baseline and Task 1 gate: `npm test` passes 12/12 tests and `npm audit --omit=dev` reports zero vulnerabilities. -- Node 22.19 additions: `npm run lint` syntax-checks the current source roots and `npm run test:unit` passes the 12 top-level tests. +- Node 22.19 Task 2 gate: `node --test test/codex-config.test.mjs` passes 15/15, including deterministic rename failure, exclusive same-timestamp backup collision, busy lock, external source change, CRLF preservation, guide semantics, and all three start aliases; `npm test` passes 27/27, `npm run lint` syntax-checks 9 source files, and `npm audit --omit=dev` reports zero vulnerabilities. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. ## Known Risks -Credential migration, localhost browser security, worker IPC, port release races, in-flight activation semantics, and secret leakage. +Credential migration, localhost browser security, worker IPC, port release races, in-flight activation semantics, secret leakage, and cross-platform atomic rename and permission semantics. ## Recent Decisions @@ -61,3 +61,4 @@ Credential migration, localhost browser security, worker IPC, port release races - Classify future V1 implementation as L3. - Execute the approved design through the task sequence in `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md`. - Keep file-watcher tests condition-based and cleanup-safe across supported Node versions. +- Atomic configuration writes must compare content first and preserve source file permissions. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index a9c4fd7..5daf0f1 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 is a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, patches Codex to use the `OpenAI` provider section, and spawns the proxy. Only capture enablement hot-reloads. +Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Task 2 has landed shared path and public-error contracts plus an idempotent Codex configuration adapter; the supervisor, provider registry, credential stores, Admin API, and independent worker remain target-state architecture. ## Target Overview @@ -40,10 +40,17 @@ Active OpenAI-compatible upstream ## Module Boundaries +Landed in Task 2: + +- `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. +- `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. +- `codex-config`: preserves custom providers and source line endings while idempotently bootstrapping the fixed OpenAI provider entry; a sidecar CRP lock serializes writers, changed files receive an exclusive timestamped adjacent backup, and the source is rechecked before atomic mode-preserving replacement. + +Remaining target-state boundaries: + - `supervisor`: owns state transitions, admin server, activity records, and child-process lifecycle. - `provider-registry`: validates and persists non-secret profiles and the active provider ID. - `credential-store`: exposes get/set/delete by opaque credential reference; implements Keychain, Credential Manager, and file fallback adapters. -- `codex-config`: backs up and idempotently bootstraps the fixed OpenAI provider entry. - `worker-protocol`: versioned IPC messages for configure, drain, shutdown, health, and events. - `proxy-worker`: forwards traffic from immutable provider snapshots; does not own persistent configuration. - `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. @@ -58,9 +65,11 @@ Active OpenAI-compatible upstream ## Storage and Deployment - Package remains distributed through npm. -- Supervisor metadata: `~/.codex-remote-proxy/providers.json` and `state.json`, mode `0600` where supported. +- The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; modules other than the existing CLI state path have not landed yet. +- Supervisor metadata target: `~/.codex-remote-proxy/providers.json` and `state.json`, mode `0600` where supported. - Credentials: native OS store; `~/.codex-remote-proxy/secrets.json` only after explicit fallback consent. - Activity events: bounded local JSONL or SQLite store without request/response bodies. +- Codex configuration replacement holds an exclusive sidecar lock, compares bytes before writing, creates backups with exclusive-copy semantics, rechecks the source before rename, and preserves the source permission mode through same-directory temporary-file `fsync` and rename; unchanged content is neither rewritten nor backed up. - macOS and Windows receive UI support; Linux uses the same supervisor and CLI without an initial UI guarantee. ## Risks diff --git a/docs/STATUS.md b/docs/STATUS.md index e35d22e..2a7ef72 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Task 1 is complete and Task 2 is next. +- Sequential execution of the approved V1 plan; Tasks 1 and 2 are complete and Task 3 is next. ## Done @@ -19,14 +19,15 @@ V1 implementation: provider lifecycle end to end. - Created and self-reviewed the detailed V1 implementation plan. - Completed Task 1 portable syntax, unit-test, E2E script, packaging, and dependency gates. - Stabilized capture watcher tests with condition-based waits and assertion-safe cleanup on Node 22.19 and Node 24.2. +- Completed Task 2 shared path and public-error contracts plus line-ending-preserving, lock-serialized, idempotent and atomic Codex bootstrap behavior, covered by 15/15 focused and 27/27 full Node 22.19 tests. ## Blocked -- No current blocker is recorded for Task 2. +- No current blocker is recorded for Task 3. ## Next -1. Execute Task 2: extract stable paths and errors, then add idempotent Codex bootstrap behavior. +1. Execute Task 3: add the atomic provider registry. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/TESTING.md b/docs/TESTING.md index f14f1bf..15e60f4 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -19,13 +19,15 @@ - Portable syntax check: `cd node && npm run lint` - Existing full suite: `cd node && npm test` - Top-level unit suite: `cd node && npm run test:unit` +- Task 2 focused suite: `cd node && node --test test/codex-config.test.mjs` - Runtime audit: `cd node && npm audit --omit=dev` -`npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. `npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. +`npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. On Node 22.19, the Task 2 focused suite passes 15/15 tests and the current full suite passes 27/27 tests. The focused coverage verifies OpenAI provider creation and update, custom-provider and CRLF preservation, byte idempotency, one-time adjacent backup, exclusive same-timestamp backup collision handling, CRP lock contention, external source-change rejection, atomic mode-preserving replacement, deterministic rename-failure cleanup and original preservation, all nine injected-home paths, safe public error serialization, `start`/`install`/`setup` JSON and managed-state backup propagation, and accurate guide backup semantics. `npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. ## Test Authoring Rules - File-watcher tests must wait for observable state and register cleanup before assertions. +- Configuration persistence tests must verify no-op writes, exclusive backup collisions, lock cleanup, external source changes, and source permission preservation. ## Test Matrix @@ -57,3 +59,5 @@ ## Verification Gate Credential, config migration, lifecycle, and browser-security tests must all pass before L3 expert review. Passing unit tests alone is insufficient. + +Task 2 does not remove the L3 requirement, and atomic rename and permission behavior remain unverified on real Windows and Linux hosts. diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index 185d870..22c0586 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -205,7 +205,7 @@ git commit -m "build: add CRP quality gates" - Modify: `node/bin/crp.mjs:14-27,603-669` - Create: `node/test/codex-config.test.mjs` -- [ ] **Step 1: Write failing idempotency and preservation tests** +- [x] **Step 1: Write failing idempotency and preservation tests** Create `node/test/codex-config.test.mjs` with tests that import `patchCodexConfigText` and assert: @@ -223,7 +223,7 @@ assert.equal(twice, once); Add a temporary-file test asserting `bootstrapCodexConfig()` creates one backup, writes mode-preserving content, and returns `{ changed: true, backupPath }`; a second call returns `{ changed: false, backupPath: null }`. -- [ ] **Step 2: Run the test and confirm the missing-module failure** +- [x] **Step 2: Run the test and confirm the missing-module failure** ```bash cd node @@ -232,7 +232,7 @@ node --test test/codex-config.test.mjs Expected: FAIL with `ERR_MODULE_NOT_FOUND` for `src/codex/codex-config.mjs`. -- [ ] **Step 3: Implement exact public contracts** +- [x] **Step 3: Implement exact public contracts** Create `node/src/shared/errors.mjs`: @@ -267,7 +267,7 @@ export function bootstrapCodexConfig({ configPath, proxyUrl, now = () => new Dat `bootstrapCodexConfig` must compare before writing, copy the original beside it with a UTC timestamp, write through a same-directory temporary file, then rename. -- [ ] **Step 4: Import the adapter from the existing CLI** +- [x] **Step 4: Import the adapter from the existing CLI** Replace the duplicated config-patching functions in `node/bin/crp.mjs` with: @@ -278,7 +278,7 @@ import { getPaths } from "../src/shared/paths.mjs"; Keep command output and aliases unchanged in this task. -- [ ] **Step 5: Verify focused and regression tests** +- [x] **Step 5: Verify focused and regression tests** ```bash cd node @@ -288,7 +288,9 @@ npm test Expected: new tests pass and the existing 12 tests remain green. -- [ ] **Step 6: Commit** +Actual Node 22.19 Task 2 verification: the focused suite passes 15/15 and the full suite passes 27/27 after adding deterministic atomic-write failure, exclusive backup-collision, CRP lock, external source-change, CRLF, guide semantics, and CLI alias propagation coverage. + +- [x] **Step 6: Commit** ```bash git add node/src/shared node/src/codex node/bin/crp.mjs node/test/codex-config.test.mjs diff --git a/node/bin/crp.mjs b/node/bin/crp.mjs index 07ed13a..e141b58 100644 --- a/node/bin/crp.mjs +++ b/node/bin/crp.mjs @@ -1,22 +1,25 @@ #!/usr/bin/env node import { spawn, spawnSync } from "node:child_process"; -import { chmodSync, closeSync, copyFileSync, existsSync, mkdirSync, openSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { chmodSync, closeSync, existsSync, mkdirSync, openSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { resolve } from "node:path"; import net from "node:net"; import readline from "node:readline/promises"; -import os from "node:os"; import { setTimeout as delay } from "node:timers/promises"; import { DEFAULT_CAPTURE_DB_PATH } from "../src/capture-config.mjs"; +import { bootstrapCodexConfig } from "../src/codex/codex-config.mjs"; +import { getPaths } from "../src/shared/paths.mjs"; const PACKAGE_ROOT = resolve(import.meta.dirname, ".."); -const DEFAULT_CODEX_CONFIG_PATH = resolve(os.homedir(), ".codex", "config.toml"); -const DEFAULT_AUTH_PATH = resolve(os.homedir(), ".codex", "auth.json"); -const GLOBAL_HOME = resolve(os.homedir(), ".codex-remote-proxy"); +const { + codexConfigPath: DEFAULT_CODEX_CONFIG_PATH, + authPath: DEFAULT_AUTH_PATH, + globalHome: GLOBAL_HOME, + statePath: STATE_FILE, + logPath: LOG_FILE +} = getPaths(); const BIN_DIR = resolve(GLOBAL_HOME, "bin"); const CRP_SHIM_PATH = resolve(BIN_DIR, "crp"); -const STATE_FILE = resolve(GLOBAL_HOME, "state.json"); -const LOG_FILE = resolve(GLOBAL_HOME, "proxy.log"); const USER_CONFIG_FILE = resolve(GLOBAL_HOME, "config.json"); const NODE_RUNTIME_CONFIG_PATH = resolve(GLOBAL_HOME, "node", "proxy-config.json"); const OPENAI_SECTION_HEADER = "[model_providers.OpenAI]"; @@ -493,7 +496,7 @@ function buildGuideData() { "Use status --json to confirm the proxy is healthy." ], notes: [ - "The start command modifies ~/.codex/config.toml and creates a backup.", + "The start command creates a backup only when it changes ~/.codex/config.toml.", "The proxy configuration and state are stored under ~/.codex-remote-proxy/.", "Use CRP_UPSTREAM_BASE_URL and CRP_UPSTREAM_API_KEY when you want non-interactive start without exposing secrets in later AI interactions.", "The optional ~/.codex/config.toml [codex_remote_proxy] section supports upstream_base_url, upstream_api_key, capture_enabled, and capture_db_path as non-interactive sources." @@ -600,75 +603,6 @@ function writeProxyConfig(path, config) { writeFileSync(path, `${JSON.stringify(config, null, 2)}\n`, "utf8"); } -function backupFile(path) { - const timestamp = new Date().toISOString().replace(/[-:]/g, "").replace(/\..+/, "").replace("T", "-"); - const backupPath = `${path}.${timestamp}.bak`; - copyFileSync(path, backupPath); - return backupPath; -} - -function renderTomlString(value) { - return JSON.stringify(value); -} - -function upsertKey(lines, startIndex, endIndex, key, value) { - const rendered = typeof value === "boolean" ? (value ? "true" : "false") : renderTomlString(String(value)); - for (let index = startIndex; index < endIndex; index += 1) { - const stripped = lines[index].trim(); - if (!stripped || stripped.startsWith("#") || !stripped.includes("=")) { - continue; - } - const currentKey = stripped.split("=", 1)[0].trim(); - if (currentKey === key) { - lines[index] = `${key} = ${rendered}`; - return lines; - } - } - lines.splice(endIndex, 0, `${key} = ${rendered}`); - return lines; -} - -function firstSectionIndex(lines) { - for (let index = 0; index < lines.length; index += 1) { - const stripped = lines[index].trim(); - if (stripped.startsWith("[") && stripped.endsWith("]")) { - return index; - } - } - return lines.length; -} - -function patchCodexConfigText(text, proxyUrl) { - const lines = splitLines(text); - const topEnd = firstSectionIndex(lines); - upsertKey(lines, 0, topEnd, "model_provider", "OpenAI"); - let sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); - - if (!sectionRange) { - if (lines.length && lines[lines.length - 1].trim()) { - lines.push(""); - } - lines.push( - OPENAI_SECTION_HEADER, - 'name = "OpenAI"', - `base_url = ${renderTomlString(proxyUrl)}`, - 'wire_api = "responses"', - "requires_openai_auth = true" - ); - return `${lines.join("\n")}\n`; - } - - const [sectionStart] = sectionRange; - upsertKey(lines, sectionStart + 1, sectionRange[1], "name", "OpenAI"); - sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); - upsertKey(lines, sectionStart + 1, sectionRange[1], "base_url", proxyUrl); - sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); - upsertKey(lines, sectionStart + 1, sectionRange[1], "wire_api", "responses"); - sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); - upsertKey(lines, sectionStart + 1, sectionRange[1], "requires_openai_auth", true); - return `${lines.join("\n")}\n`; -} - function resolveConfigValue({ cliValue, envKey, savedValues = [], defaultValue = "" }) { if (typeof cliValue === "string" && cliValue.trim()) { return { value: cliValue.trim(), source: "cli" }; @@ -811,9 +745,7 @@ async function installCommand(options) { if (!existsSync(codexConfigPath)) { throw new Error(`Codex config not found: ${codexConfigPath}`); } - const codexBackup = backupFile(codexConfigPath); - const patchedText = patchCodexConfigText(readFileSync(codexConfigPath, "utf8"), proxyUrl); - writeFileSync(codexConfigPath, patchedText, "utf8"); + const { backupPath } = bootstrapCodexConfig({ configPath: codexConfigPath, proxyUrl }); const existingState = loadManagedState(); if (existingState?.pid && isProcessAlive(existingState.pid)) { @@ -834,7 +766,7 @@ async function installCommand(options) { proxyConfigPath, logFile: startResult.logFile || null, upstreamBaseUrl, - codexConfigBackup: codexBackup, + codexConfigBackup: backupPath, startedAt: new Date().toISOString() }; saveManagedState(managedState); @@ -864,6 +796,7 @@ async function installCommand(options) { upstreamBaseUrl, codexConfigPath, proxyConfigPath, + codexConfigBackup: backupPath, configSource: { upstreamBaseUrl: resolved.upstreamBaseUrl.source, apiKey: resolved.apiKey.source, diff --git a/node/src/codex/codex-config.mjs b/node/src/codex/codex-config.mjs new file mode 100644 index 0000000..5c3d920 --- /dev/null +++ b/node/src/codex/codex-config.mjs @@ -0,0 +1,264 @@ +import { randomUUID } from "node:crypto"; +import { + chmodSync, + closeSync, + constants, + copyFileSync, + fsyncSync, + openSync, + readFileSync, + renameSync, + rmSync, + statSync, + writeFileSync +} from "node:fs"; +import { basename, dirname, join } from "node:path"; + +const OPENAI_SECTION_HEADER = "[model_providers.OpenAI]"; +const DEFAULT_FILE_OPERATIONS = { + chmodSync, + closeSync, + constants, + copyFileSync, + fsyncSync, + openSync, + readFileSync, + renameSync, + rmSync, + statSync, + writeFileSync +}; + +function splitLines(text) { + const lines = text.split(/\r?\n/); + if (lines.at(-1) === "") { + lines.pop(); + } + return lines; +} + +function findSectionRange(lines, sectionHeader) { + for (let start = 0; start < lines.length; start += 1) { + if (lines[start].trim() !== sectionHeader) { + continue; + } + let end = lines.length; + for (let index = start + 1; index < lines.length; index += 1) { + const stripped = lines[index].trim(); + if (stripped.startsWith("[") && stripped.endsWith("]")) { + end = index; + break; + } + } + return [start, end]; + } + return null; +} + +function renderTomlString(value) { + return JSON.stringify(value); +} + +function upsertKey(lines, startIndex, endIndex, key, value) { + const rendered = typeof value === "boolean" + ? (value ? "true" : "false") + : renderTomlString(String(value)); + + for (let index = startIndex; index < endIndex; index += 1) { + const stripped = lines[index].trim(); + if (!stripped || stripped.startsWith("#") || !stripped.includes("=")) { + continue; + } + const currentKey = stripped.split("=", 1)[0].trim(); + if (currentKey === key) { + lines[index] = `${key} = ${rendered}`; + return; + } + } + lines.splice(endIndex, 0, `${key} = ${rendered}`); +} + +function firstSectionIndex(lines) { + for (let index = 0; index < lines.length; index += 1) { + const stripped = lines[index].trim(); + if (stripped.startsWith("[") && stripped.endsWith("]")) { + return index; + } + } + return lines.length; +} + +function makeBackupStem(configPath, date) { + const timestamp = date.toISOString() + .replace(/[-:]/g, "") + .replace(/\..+/, "") + .replace("T", "-"); + return `${configPath}.${timestamp}`; +} + +function copyBackupExclusively(configPath, date, fileOperations) { + const stem = makeBackupStem(configPath, date); + let suffix = 1; + let backupPath = `${stem}.bak`; + + while (true) { + try { + fileOperations.copyFileSync( + configPath, + backupPath, + fileOperations.constants.COPYFILE_EXCL + ); + return backupPath; + } catch (error) { + if (error?.code !== "EEXIST") { + throw error; + } + backupPath = `${stem}.${suffix}.bak`; + suffix += 1; + } + } +} + +function createConfigError(code, message, cause) { + const error = new Error(message, { cause }); + error.code = code; + return error; +} + +function acquireConfigLock(lockPath, fileOperations) { + try { + return fileOperations.openSync(lockPath, "wx", 0o600); + } catch (error) { + if (error?.code === "EEXIST") { + throw createConfigError( + "CODEX_CONFIG_BUSY", + "Codex configuration is already being updated.", + error + ); + } + throw error; + } +} + +function releaseConfigLock(lockPath, fileDescriptor, fileOperations) { + let cleanupError; + try { + fileOperations.closeSync(fileDescriptor); + } catch (error) { + cleanupError = error; + } + try { + fileOperations.rmSync(lockPath, { force: true }); + } catch (error) { + cleanupError ??= error; + } + if (cleanupError) { + throw cleanupError; + } +} + +function writeFileAtomically(path, text, mode, fileOperations, beforeRename) { + const tempPath = join( + dirname(path), + `.${basename(path)}.${process.pid}.${randomUUID()}.tmp` + ); + let fileDescriptor; + + try { + fileDescriptor = fileOperations.openSync(tempPath, "wx", mode); + fileOperations.writeFileSync(fileDescriptor, text, "utf8"); + fileOperations.fsyncSync(fileDescriptor); + fileOperations.closeSync(fileDescriptor); + fileDescriptor = undefined; + fileOperations.chmodSync(tempPath, mode); + beforeRename(); + fileOperations.renameSync(tempPath, path); + } catch (error) { + if (fileDescriptor !== undefined) { + try { + fileOperations.closeSync(fileDescriptor); + } catch { + // Preserve the original write failure. + } + } + try { + fileOperations.rmSync(tempPath, { force: true }); + } catch { + // Preserve the original write failure. + } + throw error; + } +} + +export function patchCodexConfigText(text, proxyUrl) { + const lineEnding = text.match(/\r\n|\n/)?.[0] ?? "\n"; + const lines = splitLines(text); + const topEnd = firstSectionIndex(lines); + upsertKey(lines, 0, topEnd, "model_provider", "OpenAI"); + let sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); + + if (!sectionRange) { + if (lines.length && lines.at(-1).trim()) { + lines.push(""); + } + lines.push( + OPENAI_SECTION_HEADER, + 'name = "OpenAI"', + `base_url = ${renderTomlString(proxyUrl)}`, + 'wire_api = "responses"', + "requires_openai_auth = true" + ); + return `${lines.join(lineEnding)}${lineEnding}`; + } + + const [sectionStart] = sectionRange; + upsertKey(lines, sectionStart + 1, sectionRange[1], "name", "OpenAI"); + sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); + upsertKey(lines, sectionStart + 1, sectionRange[1], "base_url", proxyUrl); + sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); + upsertKey(lines, sectionStart + 1, sectionRange[1], "wire_api", "responses"); + sectionRange = findSectionRange(lines, OPENAI_SECTION_HEADER); + upsertKey(lines, sectionStart + 1, sectionRange[1], "requires_openai_auth", true); + return `${lines.join(lineEnding)}${lineEnding}`; +} + +export function bootstrapCodexConfig({ + configPath, + proxyUrl, + now = () => new Date(), + fileOperations = DEFAULT_FILE_OPERATIONS +}) { + const lockPath = `${configPath}.crp.lock`; + let lockFileDescriptor; + + try { + lockFileDescriptor = acquireConfigLock(lockPath, fileOperations); + const originalText = fileOperations.readFileSync(configPath, "utf8"); + const patchedText = patchCodexConfigText(originalText, proxyUrl); + if (patchedText === originalText) { + return { changed: false, backupPath: null }; + } + + const sourceMode = fileOperations.statSync(configPath).mode & 0o7777; + const backupPath = copyBackupExclusively(configPath, now(), fileOperations); + writeFileAtomically( + configPath, + patchedText, + sourceMode, + fileOperations, + () => { + if (fileOperations.readFileSync(configPath, "utf8") !== originalText) { + throw createConfigError( + "CODEX_CONFIG_CHANGED", + "Codex configuration changed during bootstrap." + ); + } + } + ); + return { changed: true, backupPath }; + } finally { + if (lockFileDescriptor !== undefined) { + releaseConfigLock(lockPath, lockFileDescriptor, fileOperations); + } + } +} diff --git a/node/src/shared/errors.mjs b/node/src/shared/errors.mjs new file mode 100644 index 0000000..d7dc61a --- /dev/null +++ b/node/src/shared/errors.mjs @@ -0,0 +1,29 @@ +export class CrpError extends Error { + constructor(code, message, action, { status = 500, details = {}, cause } = {}) { + super(message, { cause }); + this.name = "CrpError"; + this.code = code; + this.action = action; + this.status = status; + this.details = details; + } +} + +export function toPublicError(error, requestId) { + const safe = error instanceof CrpError + ? error + : new CrpError( + "INTERNAL_ERROR", + "CRP could not complete the operation.", + "Open Activity for details." + ); + return { + error: { + code: safe.code, + message: safe.message, + action: safe.action, + requestId, + details: safe.details + } + }; +} diff --git a/node/src/shared/paths.mjs b/node/src/shared/paths.mjs new file mode 100644 index 0000000..5e8133b --- /dev/null +++ b/node/src/shared/paths.mjs @@ -0,0 +1,19 @@ +import os from "node:os"; +import { resolve } from "node:path"; + +export function getPaths(home = os.homedir()) { + const resolvedHome = resolve(home); + const globalHome = resolve(resolvedHome, ".codex-remote-proxy"); + + return { + globalHome, + registryPath: resolve(globalHome, "providers.json"), + secretFallbackPath: resolve(globalHome, "secrets.json"), + statePath: resolve(globalHome, "state.json"), + controlTokenPath: resolve(globalHome, "control-token"), + activityPath: resolve(globalHome, "activity.jsonl"), + logPath: resolve(globalHome, "supervisor.log"), + codexConfigPath: resolve(resolvedHome, ".codex", "config.toml"), + authPath: resolve(resolvedHome, ".codex", "auth.json") + }; +} diff --git a/node/test/codex-config.test.mjs b/node/test/codex-config.test.mjs new file mode 100644 index 0000000..8d2fd3b --- /dev/null +++ b/node/test/codex-config.test.mjs @@ -0,0 +1,546 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import { randomUUID } from "node:crypto"; +import * as realFileOperations from "node:fs"; +import { + chmodSync, + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + statSync, + writeFileSync +} from "node:fs"; +import os from "node:os"; +import { dirname, join, resolve } from "node:path"; + +import { + bootstrapCodexConfig, + patchCodexConfigText +} from "../src/codex/codex-config.mjs"; +import { CrpError, toPublicError } from "../src/shared/errors.mjs"; +import { getPaths } from "../src/shared/paths.mjs"; + +const PROXY_URL = "http://127.0.0.1:15100"; +const PACKAGE_ROOT = resolve(import.meta.dirname, ".."); +const CLI_PATH = join(PACKAGE_ROOT, "bin", "crp.mjs"); +const START_PAYLOAD_KEYS = [ + "captureActive", + "captureConfigured", + "captureDbPath", + "captureRestartRequired", + "captureState", + "codexConfigBackup", + "codexConfigPath", + "configSource", + "failedWriteCount", + "health", + "implementation", + "lastWriteErrorAt", + "lastWriteErrorMessage", + "listenHost", + "listenPort", + "logFile", + "managedStatePath", + "message", + "ok", + "pid", + "proxyConfigPath", + "proxyUrl", + "upstreamBaseUrl" +].sort(); + +function makeHomeEnv(homeDir) { + return { + ...process.env, + HOME: homeDir, + USERPROFILE: homeDir + }; +} + +function runCrp(args, env) { + return spawnSync(process.execPath, [CLI_PATH, ...args], { + cwd: PACKAGE_ROOT, + env, + encoding: "utf8", + timeout: 20_000, + killSignal: "SIGKILL" + }); +} + +function isProcessAlive(pid) { + if (!Number.isInteger(pid) || pid <= 0) { + return false; + } + try { + process.kill(pid, 0); + return true; + } catch { + return false; + } +} + +async function waitForProcessExit(pid, timeoutMs = 2_000) { + const deadline = Date.now() + timeoutMs; + while (isProcessAlive(pid) && Date.now() < deadline) { + await new Promise((resolveWait) => setTimeout(resolveWait, 25)); + } + return !isProcessAlive(pid); +} + +async function cleanupCliHome({ homeDir, env, pid }) { + const statePath = join(homeDir, ".codex-remote-proxy", "state.json"); + let managedPid = pid; + let cleanupError; + + try { + if (!managedPid && existsSync(statePath)) { + try { + managedPid = JSON.parse(readFileSync(statePath, "utf8")).pid; + } catch { + managedPid = null; + } + } + if (existsSync(statePath)) { + runCrp(["stop", "--json"], env); + } + if (isProcessAlive(managedPid)) { + try { + process.kill(managedPid, "SIGTERM"); + } catch { + // The process may exit between the liveness check and the signal. + } + } + if (isProcessAlive(managedPid) && !(await waitForProcessExit(managedPid))) { + try { + process.kill(managedPid, "SIGKILL"); + } catch { + // The process may exit between the liveness check and the signal. + } + if (!(await waitForProcessExit(managedPid))) { + cleanupError = new Error(`Could not stop test proxy process ${managedPid}`); + } + } + } finally { + rmSync(homeDir, { recursive: true, force: true }); + } + + if (cleanupError) { + throw cleanupError; + } +} + +test("patchCodexConfigText creates the fixed OpenAI provider and preserves custom providers", () => { + const original = [ + 'model_provider = "custom"', + "", + "[model_providers.custom]", + 'name = "Custom"', + 'base_url = "https://old.example/v1"', + "" + ].join("\n"); + + const once = patchCodexConfigText(original, PROXY_URL); + const twice = patchCodexConfigText(once, PROXY_URL); + + assert.match(once, /^model_provider = "OpenAI"$/m); + assert.match( + once, + /\[model_providers\.OpenAI\]\nname = "OpenAI"\nbase_url = "http:\/\/127\.0\.0\.1:15100"\nwire_api = "responses"\nrequires_openai_auth = true/ + ); + assert.match( + once, + /\[model_providers\.custom\]\nname = "Custom"\nbase_url = "https:\/\/old\.example\/v1"/ + ); + assert.equal(twice, once); +}); + +test("patchCodexConfigText updates every fixed OpenAI provider field", () => { + const original = [ + 'model_provider = "legacy"', + "", + "[model_providers.OpenAI]", + 'name = "Legacy"', + 'base_url = "https://wrong.example/v1"', + 'wire_api = "chat"', + "requires_openai_auth = false", + "" + ].join("\n"); + + const patched = patchCodexConfigText(original, PROXY_URL); + + assert.match(patched, /^model_provider = "OpenAI"$/m); + assert.match(patched, /^name = "OpenAI"$/m); + assert.match(patched, /^base_url = "http:\/\/127\.0\.0\.1:15100"$/m); + assert.match(patched, /^wire_api = "responses"$/m); + assert.match(patched, /^requires_openai_auth = true$/m); + assert.equal(patchCodexConfigText(patched, PROXY_URL), patched); +}); + +test("patchCodexConfigText preserves CRLF line endings byte-for-byte after the first patch", () => { + const original = [ + 'model_provider = "custom"', + "", + "[model_providers.custom]", + 'name = "Custom"', + "" + ].join("\r\n"); + + const once = patchCodexConfigText(original, PROXY_URL); + const twice = patchCodexConfigText(once, PROXY_URL); + + assert.equal(once.includes("\r\n"), true); + assert.equal(once.replaceAll("\r\n", "").includes("\n"), false); + assert.equal(twice, once); +}); + +test("bootstrapCodexConfig backs up and atomically writes only changed content", () => { + const tempDir = mkdtempSync(join(os.tmpdir(), "crp-codex-config-")); + const configPath = join(tempDir, "config.toml"); + const original = 'model_provider = "custom"\n'; + + try { + writeFileSync(configPath, original, "utf8"); + chmodSync(configPath, 0o640); + + const first = bootstrapCodexConfig({ + configPath, + proxyUrl: PROXY_URL, + now: () => new Date("2026-07-10T12:34:56.789Z") + }); + + assert.deepEqual(first, { + changed: true, + backupPath: `${configPath}.20260710-123456.bak` + }); + assert.equal(dirname(first.backupPath), tempDir); + assert.equal(readFileSync(first.backupPath, "utf8"), original); + assert.equal(readFileSync(configPath, "utf8"), patchCodexConfigText(original, PROXY_URL)); + assert.equal(statSync(configPath).mode & 0o777, 0o640); + assert.equal(readdirSync(tempDir).filter((name) => name.endsWith(".bak")).length, 1); + + const firstWriteMtime = statSync(configPath, { bigint: true }).mtimeNs; + const second = bootstrapCodexConfig({ + configPath, + proxyUrl: PROXY_URL, + now: () => new Date("2026-07-10T12:35:56.789Z") + }); + + assert.deepEqual(second, { changed: false, backupPath: null }); + assert.equal(statSync(configPath, { bigint: true }).mtimeNs, firstWriteMtime); + assert.equal(readdirSync(tempDir).filter((name) => name.endsWith(".bak")).length, 1); + assert.deepEqual( + readdirSync(tempDir).filter((name) => name.endsWith(".tmp")), + [] + ); + assert.equal(existsSync(`${configPath}.crp.lock`), false); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } +}); + +test("bootstrapCodexConfig rejects an existing CRP lock without touching the config", () => { + const tempDir = mkdtempSync(join(os.tmpdir(), "crp-codex-busy-")); + const configPath = join(tempDir, "config.toml"); + const lockPath = `${configPath}.crp.lock`; + const originalBytes = Buffer.from('model_provider = "custom"\n', "utf8"); + const lockBytes = Buffer.from("existing lock owner\n", "utf8"); + + try { + writeFileSync(configPath, originalBytes); + chmodSync(configPath, 0o640); + writeFileSync(lockPath, lockBytes); + const originalMode = statSync(configPath).mode & 0o777; + + assert.throws( + () => bootstrapCodexConfig({ configPath, proxyUrl: PROXY_URL }), + (error) => error?.code === "CODEX_CONFIG_BUSY" + ); + assert.deepEqual(readFileSync(configPath), originalBytes); + assert.equal(statSync(configPath).mode & 0o777, originalMode); + assert.deepEqual(readFileSync(lockPath), lockBytes); + assert.deepEqual( + readdirSync(tempDir).filter((name) => name.endsWith(".bak") || name.endsWith(".tmp")), + [] + ); + + rmSync(lockPath); + const result = bootstrapCodexConfig({ configPath, proxyUrl: PROXY_URL }); + assert.equal(result.changed, true); + assert.equal(existsSync(result.backupPath), true); + assert.equal(existsSync(lockPath), false); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } +}); + +test("bootstrapCodexConfig preserves the original and removes its temp file when rename fails", () => { + const tempDir = mkdtempSync(join(os.tmpdir(), "crp-codex-rename-failure-")); + const configPath = join(tempDir, "config.toml"); + const originalBytes = Buffer.from('model_provider = "custom"\n', "utf8"); + const renameError = new Error("forced atomic rename failure"); + + try { + writeFileSync(configPath, originalBytes); + chmodSync(configPath, 0o640); + const originalMode = statSync(configPath).mode & 0o777; + + assert.throws( + () => bootstrapCodexConfig({ + configPath, + proxyUrl: PROXY_URL, + now: () => new Date("2026-07-10T12:36:56.789Z"), + fileOperations: { + ...realFileOperations, + renameSync() { + throw renameError; + } + } + }), + (error) => error === renameError + ); + + assert.deepEqual(readFileSync(configPath), originalBytes); + assert.equal(statSync(configPath).mode & 0o777, originalMode); + assert.deepEqual( + readdirSync(tempDir).filter((name) => name.endsWith(".tmp")), + [] + ); + assert.equal(existsSync(`${configPath}.crp.lock`), false); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } +}); + +test("bootstrapCodexConfig preserves an external source change detected before rename", () => { + const tempDir = mkdtempSync(join(os.tmpdir(), "crp-codex-source-change-")); + const configPath = join(tempDir, "config.toml"); + const originalBytes = Buffer.from('model_provider = "custom"\n', "utf8"); + const externalBytes = Buffer.from('model_provider = "external"\n', "utf8"); + let sourceReadCount = 0; + let renameCalled = false; + + try { + writeFileSync(configPath, originalBytes); + + assert.throws( + () => bootstrapCodexConfig({ + configPath, + proxyUrl: PROXY_URL, + fileOperations: { + ...realFileOperations, + readFileSync(path, ...args) { + if (path === configPath) { + sourceReadCount += 1; + if (sourceReadCount === 2) { + realFileOperations.writeFileSync(configPath, externalBytes); + } + } + return realFileOperations.readFileSync(path, ...args); + }, + renameSync(...args) { + renameCalled = true; + return realFileOperations.renameSync(...args); + } + } + }), + (error) => error?.code === "CODEX_CONFIG_CHANGED" + ); + + assert.equal(sourceReadCount, 2); + assert.equal(renameCalled, false); + assert.deepEqual(readFileSync(configPath), externalBytes); + assert.deepEqual( + readdirSync(tempDir).filter((name) => name.endsWith(".tmp")), + [] + ); + assert.equal(existsSync(`${configPath}.crp.lock`), false); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } +}); + +test("bootstrapCodexConfig keeps both backups when changes share a timestamp", () => { + const tempDir = mkdtempSync(join(os.tmpdir(), "crp-codex-backup-collision-")); + const configPath = join(tempDir, "config.toml"); + const original = 'model_provider = "custom"\n'; + const fixedNow = () => new Date("2026-07-10T12:37:56.789Z"); + + try { + writeFileSync(configPath, original, "utf8"); + + const first = bootstrapCodexConfig({ + configPath, + proxyUrl: PROXY_URL, + now: fixedNow + }); + const firstPatched = readFileSync(configPath, "utf8"); + const secondProxyUrl = "http://127.0.0.1:15101"; + const second = bootstrapCodexConfig({ + configPath, + proxyUrl: secondProxyUrl, + now: fixedNow + }); + + assert.equal(first.backupPath, `${configPath}.20260710-123756.bak`); + assert.equal(second.backupPath, `${configPath}.20260710-123756.1.bak`); + assert.equal(readFileSync(first.backupPath, "utf8"), original); + assert.equal(readFileSync(second.backupPath, "utf8"), firstPatched); + assert.equal( + readFileSync(configPath, "utf8"), + patchCodexConfigText(firstPatched, secondProxyUrl) + ); + } finally { + rmSync(tempDir, { recursive: true, force: true }); + } +}); + +for (const command of ["start", "install", "setup"]) { + test(`${command} preserves the JSON contract and propagates the Codex backup`, async (t) => { + const homeDir = mkdtempSync(join(os.tmpdir(), `crp-${command}-home-`)); + const env = makeHomeEnv(homeDir); + const codexDir = join(homeDir, ".codex"); + const codexConfigPath = join(codexDir, "config.toml"); + const statePath = join(homeDir, ".codex-remote-proxy", "state.json"); + const originalConfig = 'model_provider = "custom"\n'; + const placeholderCredential = ["placeholder", "credential", randomUUID()].join("-"); + const upstreamBaseUrl = "http://127.0.0.1:1/v1"; + let managedPid = null; + let cleanupPromise; + const cleanup = () => { + cleanupPromise ??= cleanupCliHome({ homeDir, env, pid: managedPid }); + return cleanupPromise; + }; + t.after(cleanup); + + try { + mkdirSync(codexDir, { recursive: true }); + writeFileSync(codexConfigPath, originalConfig, "utf8"); + + const result = runCrp([ + command, + "--json", + "--upstream-base-url", + upstreamBaseUrl, + "--api-key", + placeholderCredential + ], env); + + assert.equal(result.status, 0, result.stderr); + const payload = JSON.parse(result.stdout); + managedPid = payload.pid; + + assert.deepEqual(Object.keys(payload).sort(), START_PAYLOAD_KEYS); + assert.equal(payload.ok, true); + assert.equal(payload.implementation, "node"); + assert.equal(payload.listenHost, "127.0.0.1"); + assert.equal(payload.upstreamBaseUrl, upstreamBaseUrl); + assert.equal(payload.codexConfigPath, codexConfigPath); + assert.equal(payload.configSource.upstreamBaseUrl, "cli"); + assert.equal(payload.configSource.apiKey, "cli"); + assert.equal(payload.configSource.listenPort, "auto"); + assert.equal(payload.message, "Proxy configured and started"); + assert.match(payload.proxyUrl, /^http:\/\/127\.0\.0\.1:\d+$/); + + const state = JSON.parse(readFileSync(statePath, "utf8")); + assert.equal(state.pid, payload.pid); + assert.equal(state.proxyUrl, payload.proxyUrl); + assert.equal(state.codexConfigBackup, payload.codexConfigBackup); + assert.equal(existsSync(payload.codexConfigBackup), true); + assert.equal(readFileSync(payload.codexConfigBackup, "utf8"), originalConfig); + assert.equal( + readFileSync(codexConfigPath, "utf8"), + patchCodexConfigText(originalConfig, payload.proxyUrl) + ); + } finally { + await cleanup(); + } + }); +} + +test("guide explains that Codex backups are created only for config changes", () => { + const homeDir = mkdtempSync(join(os.tmpdir(), "crp-guide-home-")); + try { + const result = runCrp(["guide", "--json"], makeHomeEnv(homeDir)); + + assert.equal(result.status, 0, result.stderr); + const payload = JSON.parse(result.stdout); + assert.equal( + payload.notes.find((note) => note.includes("backup")), + "The start command creates a backup only when it changes ~/.codex/config.toml." + ); + } finally { + rmSync(homeDir, { recursive: true, force: true }); + } +}); + +test("getPaths derives every managed path from the injected home", () => { + const injectedHome = resolve(os.tmpdir(), "crp-injected-home"); + + assert.deepEqual(getPaths(injectedHome), { + globalHome: resolve(injectedHome, ".codex-remote-proxy"), + registryPath: resolve(injectedHome, ".codex-remote-proxy", "providers.json"), + secretFallbackPath: resolve(injectedHome, ".codex-remote-proxy", "secrets.json"), + statePath: resolve(injectedHome, ".codex-remote-proxy", "state.json"), + controlTokenPath: resolve(injectedHome, ".codex-remote-proxy", "control-token"), + activityPath: resolve(injectedHome, ".codex-remote-proxy", "activity.jsonl"), + logPath: resolve(injectedHome, ".codex-remote-proxy", "supervisor.log"), + codexConfigPath: resolve(injectedHome, ".codex", "config.toml"), + authPath: resolve(injectedHome, ".codex", "auth.json") + }); + assert.notEqual(getPaths(injectedHome).globalHome, getPaths().globalHome); +}); + +test("CrpError retains stable fields and toPublicError exposes only safe fields", () => { + const cause = new Error("private-cause-message"); + const error = new CrpError( + "PROVIDER_CONFLICT", + "That provider already exists.", + "Choose a different provider name.", + { status: 409, details: { field: "name" }, cause } + ); + + assert.equal(error.name, "CrpError"); + assert.equal(error.code, "PROVIDER_CONFLICT"); + assert.equal(error.message, "That provider already exists."); + assert.equal(error.action, "Choose a different provider name."); + assert.equal(error.status, 409); + assert.deepEqual(error.details, { field: "name" }); + assert.equal(error.cause, cause); + + const publicError = toPublicError(error, "request-known"); + assert.deepEqual(publicError, { + error: { + code: "PROVIDER_CONFLICT", + message: "That provider already exists.", + action: "Choose a different provider name.", + requestId: "request-known", + details: { field: "name" } + } + }); + assert.doesNotMatch(JSON.stringify(publicError), /cause|stack|private-cause-message/); +}); + +test("toPublicError replaces unknown errors without leaking their message or stack", () => { + const error = new Error("upstream returned a private credential"); + error.stack = "private stack trace"; + + const publicError = toPublicError(error, "request-unknown"); + + assert.deepEqual(publicError, { + error: { + code: "INTERNAL_ERROR", + message: "CRP could not complete the operation.", + action: "Open Activity for details.", + requestId: "request-unknown", + details: {} + } + }); + assert.doesNotMatch( + JSON.stringify(publicError), + /private credential|private stack trace|cause|stack/ + ); +}); From 57c3896863c6d5e285b43e1843c0a1211ca4e61f Mon Sep 17 00:00:00 2001 From: cluic Date: Sun, 12 Jul 2026 19:40:19 +0800 Subject: [PATCH 06/44] feat: add atomic provider registry --- docs/AGENT_COORDINATION.md | 4 +- docs/AI_HANDOFF.md | 13 +- docs/ARCHITECTURE.md | 14 +- docs/DATA_MODEL.md | 26 +- docs/STATUS.md | 8 +- docs/TESTING.md | 15 +- .../plans/2026-07-10-crp-v1-implementation.md | 14 +- node/src/providers/provider-registry.mjs | 631 +++++++++++ node/src/providers/provider-schema.mjs | 346 ++++++ node/test/provider-registry.test.mjs | 1003 +++++++++++++++++ 10 files changed, 2043 insertions(+), 31 deletions(-) create mode 100644 node/src/providers/provider-registry.mjs create mode 100644 node/src/providers/provider-schema.mjs create mode 100644 node/test/provider-registry.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index e6e50b8..2ddb184 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 2 owns `node/src/shared/`, `node/src/codex/`, `node/bin/crp.mjs`, `node/test/codex-config.test.mjs`, and affected living docs | Task 2 complete on `codex/harness-product-design`; Task 3 next | Task 1 quality gates | Every file outside the active task; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 3 owns `node/src/providers/`, `node/test/provider-registry.test.mjs`, and affected living docs | Task 3 complete on `codex/harness-product-design`; Task 4 next | Task 2 shared paths, errors, and Codex bootstrap | Every file outside the active task; no simultaneous writable agent | ## Shared Contracts @@ -29,4 +29,4 @@ ## Decisions Needed -Subagent-driven execution is selected. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. Task 3 requires a fresh file scope before writable work begins. +Subagent-driven execution is selected. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. Task 4 requires a fresh file scope before writable work begins. diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index 48418b4..929c161 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,16 +6,16 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 and 2 have landed; no provider-registry or provider-lifecycle product behavior has landed yet. Task 3, the atomic provider registry, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 3 have landed, including the atomic provider metadata registry; provider credentials and lifecycle orchestration have not landed. Task 4, native and explicit-fallback credential stores, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, and idempotent Codex bootstrap with source-EOL preservation, CRP sidecar locking, exclusive adjacent backups, external-change detection, atomic replacement, and permission preservation. Target: long-lived supervisor control plane plus independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, and a lock-serialized atomic schema-version-2 provider registry with refreshed defensive reads. Target: credential stores, provider-service orchestration, long-lived supervisor control plane, and independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API -- Non-secret profiles live in a schema-versioned registry. -- API keys live in native credential stores with explicit `0600` fallback. +- Non-secret profiles now live in the implemented schema-versioned registry. +- API keys will live in native credential stores with explicit `0600` fallback after Task 4 lands. - Local API contract is in `docs/API.md`; data contract is in `docs/DATA_MODEL.md`. ## Permissions @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 and 2 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 3 are complete. ## How To Run Current Code @@ -43,6 +43,7 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 baseline and Task 1 gate: `npm test` passes 12/12 tests and `npm audit --omit=dev` reports zero vulnerabilities. - Node 22.19 Task 2 gate: `node --test test/codex-config.test.mjs` passes 15/15, including deterministic rename failure, exclusive same-timestamp backup collision, busy lock, external source change, CRLF preservation, guide semantics, and all three start aliases; `npm test` passes 27/27, `npm run lint` syntax-checks 9 source files, and `npm audit --omit=dev` reports zero vulnerabilities. +- Node 22.19 Task 3 gate: `node --test test/provider-registry.test.mjs` passes 23/23, including multi-instance lock serialization, strict schema and header validation, test-state invalidation, primary-error preservation, degraded lock cleanup, refreshed defensive copies, and public allowlisting; `npm test` passes 50/50, `npm run lint` syntax-checks 11 source files, and `npm audit --omit=dev` reports zero vulnerabilities. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. @@ -62,3 +63,5 @@ Credential migration, localhost browser security, worker IPC, port release races - Execute the approved design through the task sequence in `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md`. - Keep file-watcher tests condition-based and cleanup-safe across supported Node versions. - Atomic configuration writes must compare content first and preserve source file permissions. +- Registry mutation must persist successfully before replacing in-memory state. +- Registry mutations must reload while holding the registry lock before replacing state. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 5daf0f1..87c8206 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Task 2 has landed shared path and public-error contracts plus an idempotent Codex configuration adapter; the supervisor, provider registry, credential stores, Admin API, and independent worker remain target-state architecture. +Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 and 3 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, and an atomic schema-version-2 provider registry. The supervisor, credential stores, provider service, Admin API, and independent worker remain target-state architecture. ## Target Overview @@ -40,17 +40,19 @@ Active OpenAI-compatible upstream ## Module Boundaries -Landed in Task 2: +Landed in Tasks 2 and 3: - `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. - `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. - `codex-config`: preserves custom providers and source line endings while idempotently bootstrapping the fixed OpenAI provider entry; a sidecar CRP lock serializes writers, changed files receive an exclusive timestamped adjacent backup, and the source is rechecked before atomic mode-preserving replacement. +- `provider-schema`: canonicalizes safe URLs, validates auth and extra-header syntax plus complete stored profiles, and builds public provider objects from an explicit allowlist. +- `provider-registry`: synchronously manages provider CRUD, test state, and the active provider ID through strict schema-version-2 documents, lock-serialized reload-before-mutate writes, refreshed defensive reads, and same-directory fsynced `0600` atomic replacement. Remaining target-state boundaries: - `supervisor`: owns state transitions, admin server, activity records, and child-process lifecycle. -- `provider-registry`: validates and persists non-secret profiles and the active provider ID. - `credential-store`: exposes get/set/delete by opaque credential reference; implements Keychain, Credential Manager, and file fallback adapters. +- `provider-service`: coordinates compatibility tests, credentials, activation, and credential-aware deletion above the metadata registry. - `worker-protocol`: versioned IPC messages for configure, drain, shutdown, health, and events. - `proxy-worker`: forwards traffic from immutable provider snapshots; does not own persistent configuration. - `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. @@ -65,11 +67,13 @@ Remaining target-state boundaries: ## Storage and Deployment - Package remains distributed through npm. -- The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; modules other than the existing CLI state path have not landed yet. -- Supervisor metadata target: `~/.codex-remote-proxy/providers.json` and `state.json`, mode `0600` where supported. +- The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; the provider registry now owns `providers.json`, while the other new stores remain target state. +- Provider metadata: `~/.codex-remote-proxy/providers.json`, atomically replaced with mode `0600` where supported after complete document validation. +- Supervisor runtime metadata target: `~/.codex-remote-proxy/state.json`, mode `0600` where supported. - Credentials: native OS store; `~/.codex-remote-proxy/secrets.json` only after explicit fallback consent. - Activity events: bounded local JSONL or SQLite store without request/response bodies. - Codex configuration replacement holds an exclusive sidecar lock, compares bytes before writing, creates backups with exclusive-copy semantics, rechecks the source before rename, and preserves the source permission mode through same-directory temporary-file `fsync` and rename; unchanged content is neither rewritten nor backed up. +- Provider-registry mutation holds an exclusive `0600` sidecar lock across disk reload, complete validation, same-directory temporary-file `fsync`, `chmod 0600`, rename, and in-memory replacement; validation or persistence failure leaves the prior in-memory document unchanged. Bounded cleanup preserves primary errors and distinguishes a durable committed/degraded result from a retryable failure; permanent residual locks require explicit repair and restart and are never auto-removed. - macOS and Windows receive UI support; Linux uses the same supervisor and CLI without an initial UI guarantee. ## Risks diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index c9b8dc6..400927e 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -1,19 +1,21 @@ # Data Model +Task 3 has implemented strict schema-version-2 provider validation and atomic metadata-registry persistence. Credential adapters, provider-service orchestration, and migration remain target-state work. + ## ProviderProfile | Field | Type | Rules | | --- | --- | --- | -| `id` | UUID string | Immutable | +| `id` | opaque string | Stable, non-empty, and immutable | | `name` | string | Required, unique case-insensitively | -| `baseUrl` | HTTPS/HTTP URL | HTTP allowed only for loopback | -| `credentialRef` | opaque string | Points to a credential adapter entry | +| `baseUrl` | canonical HTTPS/HTTP URL | C0/DEL rejected before parsing; HTTP allowed only for loopback | +| `credentialRef` | opaque string | Points to a credential adapter entry; immutable after profile creation | | `authHeader` | string | Default `authorization` | -| `authScheme` | string | Default `Bearer` | -| `extraHeaders` | string map | Sensitive-looking names are rejected | +| `authScheme` | string | Default `Bearer`; empty for raw keys, otherwise an HTTP token | +| `extraHeaders` | string map | Sensitive compacted names and values rejected by Node header validation are not allowed | | `modelMode` | enum | `passthrough` or `override` | | `modelOverride` | string/null | Required when mode is `override` | -| `lastTestAt` | ISO timestamp/null | Set after compatibility test | +| `lastTestAt` | ISO timestamp/null | Set after compatibility test and bounded by `createdAt` / `updatedAt` | | `lastTestStatus` | enum | `untested`, `passed`, `failed` | | `lastTestCode` | string/null | Sanitized stable error code | | `createdAt` / `updatedAt` | ISO timestamp | Supervisor-owned | @@ -23,7 +25,7 @@ ```json { "schemaVersion": 2, - "activeProviderId": "uuid-or-null", + "activeProviderId": "provider-id-or-null", "providers": [], "settings": { "proxyHost": "127.0.0.1", @@ -35,6 +37,14 @@ } ``` +The implemented registry accepts only complete schema-version-2 documents with the exact fixed settings above. Provider IDs and case-folded names must be unique, and a non-null `activeProviderId` must reference an existing profile. A missing file starts as this empty document in memory and is created only by the first successful mutation. + +Every mutation acquires an exclusive `0600` sidecar lock, reloads the registry while holding that lock, validates a cloned complete document, writes it through a same-directory exclusive `0600` temporary file, fsyncs the file, and renames it over the registry before replacing in-memory state and releasing the lock. Registry reads refresh committed disk state and return defensive copies. Automatic schema migration is not part of Task 3. + +Lock close and removal receive bounded cleanup retries. Cleanup failure never masks an earlier mutation failure; permanent cleanup failure after a durable rename reports a non-retryable committed/degraded error while retaining the committed disk and in-memory state. An instance that records a permanent residual lock never inspects or auto-removes that path; explicit operator repair and restart are required. + +Changing `baseUrl`, authentication fields, extra headers, or model policy resets the compatibility-test state to `untested`; changing only the display name preserves it. Credential replacement can explicitly record the same reset. Public provider projection uses an explicit field allowlist and requires a boolean credential-configured flag. + ## RuntimeState Runtime state includes supervisor PID, worker PID, worker status, snapshot generation, start timestamps, restart count, and the last sanitized error. It is observational and can be reconstructed. @@ -51,6 +61,8 @@ Activity events record timestamp, category, action, provider ID, result, stable ## Lifecycle and Deletion +The following provider-service lifecycle behavior remains target state; Task 3 implements only profile test-state recording, active-ID persistence, active-profile delete rejection, and inactive-profile metadata deletion. + - A profile must pass a Responses compatibility test before first activation. - Deleting the active profile is rejected until another profile is activated or the proxy is stopped. - Deleting a profile removes its native credential entry and writes an activity event. diff --git a/docs/STATUS.md b/docs/STATUS.md index 2a7ef72..dbbfb70 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 and 2 are complete and Task 3 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 3 are complete and Task 4 is next. ## Done @@ -20,17 +20,19 @@ V1 implementation: provider lifecycle end to end. - Completed Task 1 portable syntax, unit-test, E2E script, packaging, and dependency gates. - Stabilized capture watcher tests with condition-based waits and assertion-safe cleanup on Node 22.19 and Node 24.2. - Completed Task 2 shared path and public-error contracts plus line-ending-preserving, lock-serialized, idempotent and atomic Codex bootstrap behavior, covered by 15/15 focused and 27/27 full Node 22.19 tests. +- Completed Task 3 strict provider validation and lock-serialized atomic schema-version-2 registry behavior, covered by 23/23 focused and 50/50 full Node 22.19 tests. ## Blocked -- No current blocker is recorded for Task 3. +- No current blocker is recorded for Task 4. ## Next -1. Execute Task 3: add the atomic provider registry. +1. Execute Task 4: add native and explicit-fallback credential stores. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks - Future V1 implementation is L3 because it handles credentials, local browser security, Codex configuration, and process lifecycle. - Cross-platform credential APIs and restart semantics require real macOS and Windows verification. +- Provider-registry atomic rename and permission semantics remain unverified on real Windows and Linux hosts. diff --git a/docs/TESTING.md b/docs/TESTING.md index 15e60f4..c84e9e8 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -20,20 +20,29 @@ - Existing full suite: `cd node && npm test` - Top-level unit suite: `cd node && npm run test:unit` - Task 2 focused suite: `cd node && node --test test/codex-config.test.mjs` +- Task 3 focused suite: `cd node && node --test test/provider-registry.test.mjs` - Runtime audit: `cd node && npm audit --omit=dev` -`npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. On Node 22.19, the Task 2 focused suite passes 15/15 tests and the current full suite passes 27/27 tests. The focused coverage verifies OpenAI provider creation and update, custom-provider and CRLF preservation, byte idempotency, one-time adjacent backup, exclusive same-timestamp backup collision handling, CRP lock contention, external source-change rejection, atomic mode-preserving replacement, deterministic rename-failure cleanup and original preservation, all nine injected-home paths, safe public error serialization, `start`/`install`/`setup` JSON and managed-state backup propagation, and accurate guide backup semantics. `npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. +`npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. On Node 22.19, the Task 2 focused suite passes 15/15 tests. Its coverage verifies OpenAI provider creation and update, custom-provider and CRLF preservation, byte idempotency, one-time adjacent backup, exclusive same-timestamp backup collision handling, CRP lock contention, external source-change rejection, atomic mode-preserving replacement, deterministic rename-failure cleanup and original preservation, all nine injected-home paths, safe public error serialization, `start`/`install`/`setup` JSON and managed-state backup propagation, and accurate guide backup semantics. + +The Node 22.19 Task 3 focused suite passes 23/23 tests and the current full suite passes 50/50 tests; `npm run lint` syntax-checks 11 source files. Task 3 coverage includes normalized CRUD, multi-instance lock serialization and read refresh, stale uniqueness rejection, immutable fields, canonical safe URLs, header and auth-scheme validation, model policy defaults, test-state invalidation/reset, timestamp ordering, activation and deletion, strict reload rejection, lazy first persistence, `0600` lock and file modes, same-directory fsync/rename ordering, validation and rename rollback, transient and permanent lock cleanup, primary-error preservation, foreign-lock and temporary-file preservation, defensive copies, and exact public projection. Real Windows and Linux rename and permission semantics remain unverified. + +`npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. ## Test Authoring Rules - File-watcher tests must wait for observable state and register cleanup before assertions. - Configuration persistence tests must verify no-op writes, exclusive backup collisions, lock cleanup, external source changes, and source permission preservation. +- Registry mutation must persist successfully before replacing in-memory state. +- Registry mutations must reload while holding the registry lock before replacing state. +- Lock cleanup must never mask a primary registry error or make a durable mutation appear retryable. +- A recorded residual registry lock must never be inspected or removed automatically. ## Test Matrix | Area | Required Tests | | --- | --- | -| Provider registry | Validation, duplicate names, atomic persistence, schema migration, rollback | +| Provider registry | Validation, duplicate names, strict schema rejection, atomic persistence, rollback; migration is covered by the future migration suite | | Credentials | Native adapter contract, file fallback permission, masking, deletion, log redaction | | Worker protocol | Version mismatch, acknowledgement, stale generation rejection, crash handling | | Proxy behavior | Auth rewrite, HTTP/SSE, compression, timeout, disconnect, optional model override | @@ -60,4 +69,4 @@ Credential, config migration, lifecycle, and browser-security tests must all pass before L3 expert review. Passing unit tests alone is insufficient. -Task 2 does not remove the L3 requirement, and atomic rename and permission behavior remain unverified on real Windows and Linux hosts. +Tasks 2 and 3 do not remove the L3 requirement, and atomic rename and permission behavior remain unverified on real Windows and Linux hosts. diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index 22c0586..72a00b3 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -304,7 +304,7 @@ git commit -m "refactor: isolate Codex bootstrap" - Create: `node/src/providers/provider-registry.mjs` - Create: `node/test/provider-registry.test.mjs` -- [ ] **Step 1: Write failing registry behavior tests** +- [x] **Step 1: Write failing registry behavior tests** Cover create/list/get/update, case-insensitive duplicate names, activation, active-delete rejection, inactive deletion, reload, malformed JSON, and `0600` permissions. Use injected deterministic IDs and clocks: @@ -328,7 +328,7 @@ assert.equal(created.id, "provider-1"); assert.equal(registry.list().length, 1); ``` -- [ ] **Step 2: Verify the tests fail** +- [x] **Step 2: Verify the tests fail** ```bash cd node @@ -337,7 +337,7 @@ node --test test/provider-registry.test.mjs Expected: FAIL because the provider modules do not exist. -- [ ] **Step 3: Implement validation and masking** +- [x] **Step 3: Implement validation and masking** Export from `provider-schema.mjs`: @@ -353,7 +353,7 @@ export function toPublicProvider(profile, credentialConfigured) { Reject missing names, duplicate names, credentials in `extraHeaders`, non-HTTP(S) URLs, non-loopback HTTP URLs, and missing override models. -- [ ] **Step 4: Implement atomic persistence** +- [x] **Step 4: Implement atomic persistence** `ProviderRegistry` must expose: @@ -371,7 +371,7 @@ getDocument(); Every mutation writes a complete schema-version-2 document to a same-directory temporary file with mode `0600`, fsyncs it, and renames it over the registry. Failed validation must leave the original file byte-for-byte unchanged. -- [ ] **Step 5: Run focused and full tests** +- [x] **Step 5: Run focused and full tests** ```bash cd node @@ -381,7 +381,9 @@ npm test Expected: all provider tests and regressions pass. -- [ ] **Step 6: Commit** +Actual Node 22.19 Task 3 verification: the focused suite passes 23/23 and the full suite passes 50/50 after adding strict profile and document validation, multi-instance lock serialization, stale-state refresh, test-state invalidation, primary-error preservation, deterministic rollback and degraded-lock cleanup, defensive-copy, and public-allowlist coverage. + +- [x] **Step 6: Commit** ```bash git add node/src/providers node/test/provider-registry.test.mjs diff --git a/node/src/providers/provider-registry.mjs b/node/src/providers/provider-registry.mjs new file mode 100644 index 0000000..8705e9d --- /dev/null +++ b/node/src/providers/provider-registry.mjs @@ -0,0 +1,631 @@ +import { randomUUID } from "node:crypto"; +import { + chmodSync, + closeSync, + existsSync, + fsyncSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +} from "node:fs"; +import { basename, dirname, join } from "node:path"; +import { isDeepStrictEqual } from "node:util"; + +import { CrpError } from "../shared/errors.mjs"; +import { + normalizeProvider, + validateProviderInput, + validateStoredProvider +} from "./provider-schema.mjs"; + +const DEFAULT_SETTINGS = Object.freeze({ + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: 15101, + captureEnabled: false +}); +const DOCUMENT_FIELDS = new Set([ + "schemaVersion", + "activeProviderId", + "providers", + "settings" +]); +const SETTINGS_FIELDS = new Set(Object.keys(DEFAULT_SETTINGS)); +const EDITABLE_FIELDS = new Set([ + "name", + "baseUrl", + "authHeader", + "authScheme", + "extraHeaders", + "modelMode", + "modelOverride" +]); +const IMMUTABLE_FIELDS = new Set(["id", "createdAt", "credentialRef"]); +const TEST_INVALIDATING_FIELDS = [ + "baseUrl", + "authHeader", + "authScheme", + "extraHeaders", + "modelMode", + "modelOverride" +]; +const TEST_CODE_PATTERN = /^[A-Z][A-Z0-9_]*$/; +const LOCK_CLEANUP_ATTEMPTS = 2; +const DEFAULT_FILE_OPERATIONS = { + chmodSync, + closeSync, + existsSync, + fsyncSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +}; + +function clone(value) { + return structuredClone(value); +} + +function emptyDocument() { + return { + schemaVersion: 2, + activeProviderId: null, + providers: [], + settings: { ...DEFAULT_SETTINGS } + }; +} + +function isPlainObject(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return false; + } + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function registryInvalid(cause) { + return new CrpError( + "PROVIDER_REGISTRY_INVALID", + "The provider registry is invalid.", + "Restore a valid provider registry or remove it after making a backup.", + { status: 500, cause } + ); +} + +function inputError(code, message, action, status = 400) { + return new CrpError(code, message, action, { status }); +} + +function normalizedName(name) { + return name.toLowerCase(); +} + +function validateExactFields(value, fields) { + return isPlainObject(value) + && Object.keys(value).length === fields.size + && Object.keys(value).every((key) => fields.has(key)); +} + +function validateDocument(document) { + try { + if (!validateExactFields(document, DOCUMENT_FIELDS)) { + throw new Error("invalid document fields"); + } + if (document.schemaVersion !== 2) { + throw new Error("unsupported schema version"); + } + if (!Array.isArray(document.providers)) { + throw new Error("providers must be an array"); + } + if (!validateExactFields(document.settings, SETTINGS_FIELDS)) { + throw new Error("invalid settings fields"); + } + for (const [key, value] of Object.entries(DEFAULT_SETTINGS)) { + if (document.settings[key] !== value) { + throw new Error("fixed settings changed"); + } + } + if (document.activeProviderId !== null && typeof document.activeProviderId !== "string") { + throw new Error("invalid active provider id"); + } + + const ids = new Set(); + const names = new Set(); + for (const profile of document.providers) { + validateStoredProvider(profile); + if (ids.has(profile.id)) { + throw new Error("duplicate provider id"); + } + const nameKey = normalizedName(profile.name); + if (names.has(nameKey)) { + throw new Error("duplicate provider name"); + } + ids.add(profile.id); + names.add(nameKey); + } + if (document.activeProviderId !== null && !ids.has(document.activeProviderId)) { + throw new Error("active provider does not exist"); + } + return true; + } catch (error) { + if (error instanceof CrpError && error.code === "PROVIDER_REGISTRY_INVALID") { + throw error; + } + throw registryInvalid(error); + } +} + +function parseDocument(bytes) { + let document; + try { + document = JSON.parse(bytes); + } catch (error) { + throw registryInvalid(error); + } + validateDocument(document); + return document; +} + +function providerNotFound() { + return inputError( + "PROVIDER_NOT_FOUND", + "The provider does not exist.", + "Refresh the provider list and try again.", + 404 + ); +} + +function registryBusy(cause) { + return new CrpError( + "PROVIDER_REGISTRY_BUSY", + "The provider registry is already being updated.", + "Wait for the current registry update to finish and try again.", + { status: 409, cause } + ); +} + +function committedLockDegraded() { + return new CrpError( + "PROVIDER_REGISTRY_COMMITTED_LOCK_DEGRADED", + "The provider change was saved, but its registry lock could not be fully released.", + "Stop CRP, explicitly repair the residual registry lock, then restart CRP.", + { status: 500, details: { committed: true } } + ); +} + +function registryLockDegraded() { + return new CrpError( + "PROVIDER_REGISTRY_LOCK_DEGRADED", + "The provider registry lock could not be safely recovered.", + "Stop CRP, explicitly repair the residual registry lock, then restart CRP.", + { status: 500, details: { committed: false } } + ); +} + +function assertPatch(patch) { + if (!isPlainObject(patch)) { + throw inputError( + "PROVIDER_INPUT_INVALID", + "Provider settings are invalid.", + "Submit a provider settings object and try again." + ); + } + for (const key of Object.keys(patch)) { + if (IMMUTABLE_FIELDS.has(key)) { + throw inputError( + "PROVIDER_IMMUTABLE_FIELD", + "An immutable provider field cannot be changed.", + "Create a new provider when its identity or credential reference must change." + ); + } + if (!EDITABLE_FIELDS.has(key)) { + throw inputError( + "PROVIDER_INPUT_INVALID", + "Provider settings are invalid.", + "Remove system-managed fields and try again." + ); + } + if (patch[key] === undefined) { + throw inputError( + "PROVIDER_INPUT_INVALID", + "Provider settings are invalid.", + "Provide an explicit value for every updated field." + ); + } + } +} + +export class ProviderRegistry { + constructor({ + path, + createId = randomUUID, + now = () => new Date().toISOString(), + fileOperations + }) { + this.path = path; + this.lockPath = `${path}.crp.lock`; + this.createId = createId; + this.now = now; + this.fileOperations = { ...DEFAULT_FILE_OPERATIONS, ...fileOperations }; + this.degradedLock = null; + this.document = this.#load(); + } + + #load() { + if (!this.fileOperations.existsSync(this.path)) { + return emptyDocument(); + } + let bytes; + try { + bytes = this.fileOperations.readFileSync(this.path, "utf8"); + } catch (error) { + throw new CrpError( + "PROVIDER_REGISTRY_READ_FAILED", + "The provider registry could not be read.", + "Check the registry file permissions and try again.", + { status: 500, cause: error } + ); + } + return parseDocument(bytes); + } + + #findIndex(document, id) { + return document.providers.findIndex((profile) => profile.id === id); + } + + #refresh() { + const document = this.#load(); + this.document = document; + return document; + } + + #acquireLock() { + this.fileOperations.mkdirSync(dirname(this.path), { recursive: true }); + if (this.degradedLock !== null) { + throw registryLockDegraded(); + } + + let fileDescriptor; + const token = `${randomUUID()}\n`; + try { + fileDescriptor = this.fileOperations.openSync(this.lockPath, "wx", 0o600); + } catch (error) { + if (error?.code === "EEXIST") { + throw registryBusy(error); + } + throw error; + } + + const lock = { fileDescriptor, token }; + try { + this.fileOperations.writeFileSync(fileDescriptor, token, "utf8"); + return lock; + } catch (error) { + const cleanup = this.#releaseLock(lock); + if (cleanup.residualLock) { + this.degradedLock = { token }; + } + throw error; + } + } + + #closeLock(fileDescriptor) { + let error = null; + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + try { + this.fileOperations.closeSync(fileDescriptor); + return { closed: true, error: null }; + } catch (caught) { + if (attempt > 0 && caught?.code === "EBADF") { + return { closed: true, error: null }; + } + error = caught; + } + } + return { closed: false, error }; + } + + #removeOwnedLock(token) { + let error = null; + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + let currentToken; + try { + currentToken = this.fileOperations.readFileSync(this.lockPath, "utf8"); + } catch (caught) { + if (caught?.code === "ENOENT") { + return { removed: true, residualLock: false, foreign: false, error: null }; + } + error = caught; + continue; + } + if (currentToken !== token) { + return { removed: false, residualLock: true, foreign: true, error: null }; + } + try { + this.fileOperations.rmSync(this.lockPath, { force: true }); + return { removed: true, residualLock: false, foreign: false, error: null }; + } catch (caught) { + error = caught; + } + } + return { + removed: false, + residualLock: this.fileOperations.existsSync(this.lockPath), + foreign: false, + error + }; + } + + #releaseLock(lock) { + const close = this.#closeLock(lock.fileDescriptor); + const removal = this.#removeOwnedLock(lock.token); + return { + ok: close.closed && removal.removed, + closeError: close.error, + removalError: removal.error, + residualLock: removal.residualLock, + foreignLock: removal.foreign + }; + } + + #recordCleanupState(lock, cleanup) { + if (cleanup.residualLock) { + this.degradedLock = { token: lock.token }; + } else { + this.degradedLock = null; + } + } + + #getIndex(document, id) { + const index = this.#findIndex(document, id); + if (index === -1) { + throw providerNotFound(); + } + return index; + } + + #assertUniqueName(document, name, excludedId = null) { + const nameKey = normalizedName(name); + if (document.providers.some((profile) => ( + profile.id !== excludedId && normalizedName(profile.name) === nameKey + ))) { + throw inputError( + "PROVIDER_NAME_CONFLICT", + "A provider with this name already exists.", + "Choose a different provider name.", + 409 + ); + } + } + + #persist(document) { + const bytes = `${JSON.stringify(document, null, 2)}\n`; + const parent = dirname(this.path); + const tempPath = join( + parent, + `.${basename(this.path)}.${process.pid}.${randomUUID()}.tmp` + ); + let fileDescriptor; + + this.fileOperations.mkdirSync(parent, { recursive: true }); + try { + fileDescriptor = this.fileOperations.openSync(tempPath, "wx", 0o600); + this.fileOperations.writeFileSync(fileDescriptor, bytes, "utf8"); + this.fileOperations.fsyncSync(fileDescriptor); + this.fileOperations.closeSync(fileDescriptor); + fileDescriptor = undefined; + this.fileOperations.chmodSync(tempPath, 0o600); + this.fileOperations.renameSync(tempPath, this.path); + } catch (error) { + if (fileDescriptor !== undefined) { + try { + this.fileOperations.closeSync(fileDescriptor); + } catch { + // Preserve the original persistence error. + } + } + try { + this.fileOperations.rmSync(tempPath, { force: true }); + } catch { + // Preserve the original persistence error. + } + throw error; + } + } + + #commit(mutator) { + let lock; + let result; + let primaryError; + let committed = false; + try { + lock = this.#acquireLock(); + const candidate = clone(this.#load()); + const mutationResult = mutator(candidate); + validateDocument(candidate); + result = clone(mutationResult); + this.#persist(candidate); + this.document = candidate; + committed = true; + } catch (error) { + primaryError = error; + } + + let cleanup = { ok: true, residualLock: false }; + if (lock !== undefined) { + cleanup = this.#releaseLock(lock); + this.#recordCleanupState(lock, cleanup); + } + + if (primaryError !== undefined) { + throw primaryError; + } + if (!cleanup.ok) { + if (committed) { + throw committedLockDegraded(); + } + throw registryLockDegraded(); + } + return result; + } + + list() { + return clone(this.#refresh().providers); + } + + get(id) { + const document = this.#refresh(); + const index = this.#getIndex(document, id); + return clone(document.providers[index]); + } + + create(input) { + validateProviderInput(input); + const id = this.createId(); + const profile = normalizeProvider(input, { id, now: this.now() }); + return this.#commit((document) => { + if (this.#findIndex(document, profile.id) !== -1) { + throw inputError( + "PROVIDER_ID_CONFLICT", + "A provider identity conflict occurred.", + "Retry creating the provider.", + 409 + ); + } + this.#assertUniqueName(document, profile.name); + document.providers.push(profile); + return profile; + }); + } + + update(id, patch) { + assertPatch(patch); + const timestamp = this.now(); + + return this.#commit((document) => { + const index = this.#getIndex(document, id); + const current = document.providers[index]; + const normalized = normalizeProvider({ + name: current.name, + baseUrl: current.baseUrl, + credentialRef: current.credentialRef, + authHeader: current.authHeader, + authScheme: current.authScheme, + extraHeaders: current.extraHeaders, + modelMode: current.modelMode, + modelOverride: current.modelOverride, + ...patch + }, { id: current.id, now: timestamp }); + this.#assertUniqueName(document, normalized.name, id); + const invalidatesTest = TEST_INVALIDATING_FIELDS.some((field) => ( + !isDeepStrictEqual(current[field], normalized[field]) + )); + const updated = { + ...normalized, + credentialRef: current.credentialRef, + lastTestAt: invalidatesTest ? null : current.lastTestAt, + lastTestStatus: invalidatesTest ? "untested" : current.lastTestStatus, + lastTestCode: invalidatesTest ? null : current.lastTestCode, + createdAt: current.createdAt, + updatedAt: timestamp + }; + document.providers[index] = updated; + return updated; + }); + } + + delete(id) { + return this.#commit((document) => { + const index = this.#getIndex(document, id); + if (document.activeProviderId === id) { + throw inputError( + "PROVIDER_ACTIVE", + "The active provider cannot be deleted.", + "Activate another provider or stop the proxy first.", + 409 + ); + } + const [deleted] = document.providers.splice(index, 1); + return deleted; + }); + } + + markTest(id, { status, code = null } = {}) { + if (status !== "untested" && status !== "passed" && status !== "failed") { + throw inputError( + "PROVIDER_TEST_RESULT_INVALID", + "The provider test result is invalid.", + "Record an untested, passed, or failed compatibility test result." + ); + } + if (status === "untested" && code !== null) { + throw inputError( + "PROVIDER_TEST_RESULT_INVALID", + "The provider test result is invalid.", + "Do not include an error code when resetting the test result." + ); + } + if (status === "passed" && code !== null) { + throw inputError( + "PROVIDER_TEST_RESULT_INVALID", + "The provider test result is invalid.", + "Do not include an error code for a passed test." + ); + } + if (status === "failed" && ( + typeof code !== "string" || !TEST_CODE_PATTERN.test(code) + )) { + throw inputError( + "PROVIDER_TEST_RESULT_INVALID", + "The provider test result is invalid.", + "Record a stable error code for a failed test." + ); + } + const timestamp = this.now(); + return this.#commit((document) => { + const index = this.#getIndex(document, id); + const updated = { + ...document.providers[index], + lastTestAt: status === "untested" ? null : timestamp, + lastTestStatus: status, + lastTestCode: status === "untested" ? null : code, + updatedAt: timestamp + }; + document.providers[index] = updated; + return updated; + }); + } + + setActive(id) { + if (id === null) { + return this.#commit((document) => { + document.activeProviderId = null; + return null; + }); + } + return this.#commit((document) => { + const index = this.#getIndex(document, id); + document.activeProviderId = id; + return document.providers[index]; + }); + } + + getActive() { + const document = this.#refresh(); + if (document.activeProviderId === null) { + return null; + } + const index = this.#getIndex(document, document.activeProviderId); + return clone(document.providers[index]); + } + + getDocument() { + return clone(this.#refresh()); + } +} diff --git a/node/src/providers/provider-schema.mjs b/node/src/providers/provider-schema.mjs new file mode 100644 index 0000000..79f8285 --- /dev/null +++ b/node/src/providers/provider-schema.mjs @@ -0,0 +1,346 @@ +import { CrpError } from "../shared/errors.mjs"; +import { validateHeaderValue } from "node:http"; + +export const TEST_STATUSES = new Set(["untested", "passed", "failed"]); + +const INPUT_FIELDS = new Set([ + "name", + "baseUrl", + "credentialRef", + "authHeader", + "authScheme", + "extraHeaders", + "modelMode", + "modelOverride" +]); +const PROFILE_FIELDS = new Set([ + "id", + ...INPUT_FIELDS, + "lastTestAt", + "lastTestStatus", + "lastTestCode", + "createdAt", + "updatedAt" +]); +const SENSITIVE_HEADER_TERMS = [ + "authorization", + "cookie", + "token", + "secret", + "apikey" +]; +const HEADER_NAME_PATTERN = /^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/; +const TEST_CODE_PATTERN = /^[A-Z][A-Z0-9_]*$/; +const CONTROL_CHARACTER_PATTERN = /[\u0000-\u001f\u007f]/; + +function inputError(field, reason) { + throw new CrpError( + "PROVIDER_INPUT_INVALID", + "Provider settings are invalid.", + "Review the provider settings and try again.", + { status: 400, details: { field, reason } } + ); +} + +function isPlainObject(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return false; + } + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function assertExactFields(value, allowedFields, field) { + for (const key of Object.keys(value)) { + if (!allowedFields.has(key)) { + inputError(field, "contains an unsupported field"); + } + } +} + +function normalizeRequiredString(value, field) { + if (typeof value !== "string" || value.trim().length === 0) { + inputError(field, "must be a non-empty string"); + } + return value.trim(); +} + +function normalizeBaseUrl(value) { + if (typeof value !== "string" || value.trim().length === 0) { + inputError("baseUrl", "must be a non-empty string"); + } + if (CONTROL_CHARACTER_PATTERN.test(value)) { + inputError("baseUrl", "must not contain control characters"); + } + const baseUrl = value.trim(); + let parsed; + try { + parsed = new URL(baseUrl); + } catch { + inputError("baseUrl", "must be a valid URL"); + } + + if (parsed.protocol !== "http:" && parsed.protocol !== "https:") { + inputError("baseUrl", "must use HTTP or HTTPS"); + } + if (authorityContainsUserInfo(baseUrl) || parsed.username || parsed.password) { + inputError("baseUrl", "must not contain credentials"); + } + if (parsed.protocol === "http:" && !isLoopbackHostname(parsed.hostname)) { + inputError("baseUrl", "HTTP is allowed only for loopback hosts"); + } + return parsed.toString(); +} + +function authorityContainsUserInfo(value) { + const authorityStart = value.indexOf("://"); + if (authorityStart === -1) { + return false; + } + const remainder = value.slice(authorityStart + 3); + const authorityEnd = remainder.search(/[/?#]/); + const authority = authorityEnd === -1 ? remainder : remainder.slice(0, authorityEnd); + return authority.includes("@"); +} + +function isLoopbackHostname(hostname) { + const lower = hostname.toLowerCase(); + if (lower === "localhost" || lower === "::1" || lower === "[::1]") { + return true; + } + const octets = lower.split("."); + if (octets.length !== 4 || octets.some((octet) => !/^\d{1,3}$/.test(octet))) { + return false; + } + const numbers = octets.map(Number); + return numbers[0] === 127 && numbers.every((octet) => octet >= 0 && octet <= 255); +} + +function normalizeAuthHeader(value) { + const authHeader = value === undefined + ? "authorization" + : normalizeRequiredString(value, "authHeader"); + if (!HEADER_NAME_PATTERN.test(authHeader)) { + inputError("authHeader", "must be a valid HTTP header name"); + } + return authHeader; +} + +function normalizeAuthScheme(value) { + if (value === undefined) { + return "Bearer"; + } + if (typeof value !== "string") { + inputError("authScheme", "must be a string"); + } + if (CONTROL_CHARACTER_PATTERN.test(value)) { + inputError("authScheme", "must not contain control characters"); + } + const authScheme = value.trim(); + if (authScheme && !HEADER_NAME_PATTERN.test(authScheme)) { + inputError("authScheme", "must be empty or an HTTP token"); + } + return authScheme; +} + +function isSensitiveHeaderName(name) { + const compact = name.toLowerCase().replace(/[^a-z0-9]/g, ""); + return SENSITIVE_HEADER_TERMS.some((term) => compact.includes(term)); +} + +function normalizeExtraHeaders(value) { + if (value === undefined) { + return {}; + } + if (!isPlainObject(value)) { + inputError("extraHeaders", "must be a string map"); + } + + const extraHeaders = {}; + for (const [name, headerValue] of Object.entries(value)) { + if (!HEADER_NAME_PATTERN.test(name)) { + inputError("extraHeaders", "contains an invalid HTTP header name"); + } + if (isSensitiveHeaderName(name)) { + inputError("extraHeaders", "contains a sensitive header name"); + } + if (typeof headerValue !== "string") { + inputError("extraHeaders", "must contain only string values"); + } + try { + validateHeaderValue(name, headerValue); + } catch { + inputError("extraHeaders", "contains an invalid HTTP header value"); + } + extraHeaders[name] = headerValue; + } + return extraHeaders; +} + +function normalizeModelPolicy(modeValue, overrideValue) { + const modelMode = modeValue === undefined ? "passthrough" : modeValue; + if (modelMode !== "passthrough" && modelMode !== "override") { + inputError("modelMode", "must be passthrough or override"); + } + + let modelOverride = overrideValue === undefined ? null : overrideValue; + if (modelOverride !== null) { + if (typeof modelOverride !== "string" || modelOverride.trim().length === 0) { + inputError("modelOverride", "must be a non-empty string or null"); + } + modelOverride = modelOverride.trim(); + } + if (modelMode === "override" && modelOverride === null) { + inputError("modelOverride", "is required in override mode"); + } + return { modelMode, modelOverride }; +} + +function normalizeInput(input) { + if (!isPlainObject(input)) { + inputError("provider", "must be an object"); + } + assertExactFields(input, INPUT_FIELDS, "provider"); + const modelPolicy = normalizeModelPolicy(input.modelMode, input.modelOverride); + return { + name: normalizeRequiredString(input.name, "name"), + baseUrl: normalizeBaseUrl(input.baseUrl), + credentialRef: normalizeRequiredString(input.credentialRef, "credentialRef"), + authHeader: normalizeAuthHeader(input.authHeader), + authScheme: normalizeAuthScheme(input.authScheme), + extraHeaders: normalizeExtraHeaders(input.extraHeaders), + ...modelPolicy + }; +} + +function isIsoTimestamp(value) { + if (typeof value !== "string") { + return false; + } + try { + return new Date(value).toISOString() === value; + } catch { + return false; + } +} + +function assertStoredValue(condition, field, reason) { + if (!condition) { + inputError(field, reason); + } +} + +export function validateProviderInput(input) { + normalizeInput(input); + return true; +} + +export function normalizeProvider(input, { id, now }) { + const normalized = normalizeInput(input); + const providerId = normalizeRequiredString(id, "id"); + if (!isIsoTimestamp(now)) { + inputError("now", "must be an ISO timestamp"); + } + return { + id: providerId, + ...normalized, + lastTestAt: null, + lastTestStatus: "untested", + lastTestCode: null, + createdAt: now, + updatedAt: now + }; +} + +export function validateStoredProvider(profile) { + if (!isPlainObject(profile)) { + inputError("provider", "must be an object"); + } + assertExactFields(profile, PROFILE_FIELDS, "provider"); + assertStoredValue( + Object.keys(profile).length === PROFILE_FIELDS.size, + "provider", + "is missing required fields" + ); + + const normalized = normalizeInput({ + name: profile.name, + baseUrl: profile.baseUrl, + credentialRef: profile.credentialRef, + authHeader: profile.authHeader, + authScheme: profile.authScheme, + extraHeaders: profile.extraHeaders, + modelMode: profile.modelMode, + modelOverride: profile.modelOverride + }); + assertStoredValue( + typeof profile.id === "string" && profile.id.trim() === profile.id && profile.id.length > 0, + "id", + "must be a normalized non-empty string" + ); + for (const key of INPUT_FIELDS) { + assertStoredValue( + JSON.stringify(profile[key]) === JSON.stringify(normalized[key]), + key, + "must be normalized" + ); + } + assertStoredValue(isIsoTimestamp(profile.createdAt), "createdAt", "must be an ISO timestamp"); + assertStoredValue(isIsoTimestamp(profile.updatedAt), "updatedAt", "must be an ISO timestamp"); + assertStoredValue(profile.updatedAt >= profile.createdAt, "updatedAt", "must not precede createdAt"); + assertStoredValue(TEST_STATUSES.has(profile.lastTestStatus), "lastTestStatus", "is invalid"); + assertStoredValue( + profile.lastTestAt === null || isIsoTimestamp(profile.lastTestAt), + "lastTestAt", + "must be null or an ISO timestamp" + ); + if (profile.lastTestAt !== null) { + assertStoredValue( + profile.lastTestAt >= profile.createdAt && profile.lastTestAt <= profile.updatedAt, + "lastTestAt", + "must be between createdAt and updatedAt" + ); + } + assertStoredValue( + profile.lastTestCode === null + || (typeof profile.lastTestCode === "string" && TEST_CODE_PATTERN.test(profile.lastTestCode)), + "lastTestCode", + "must be null or a stable error code" + ); + + if (profile.lastTestStatus === "untested") { + assertStoredValue(profile.lastTestAt === null, "lastTestAt", "must be null when untested"); + assertStoredValue(profile.lastTestCode === null, "lastTestCode", "must be null when untested"); + } else { + assertStoredValue(profile.lastTestAt !== null, "lastTestAt", "is required after a test"); + } + if (profile.lastTestStatus === "passed") { + assertStoredValue(profile.lastTestCode === null, "lastTestCode", "must be null after a passed test"); + } + if (profile.lastTestStatus === "failed") { + assertStoredValue(profile.lastTestCode !== null, "lastTestCode", "is required after a failed test"); + } + return true; +} + +export function toPublicProvider(profile, credentialConfigured) { + if (typeof credentialConfigured !== "boolean") { + inputError("credentialConfigured", "must be a boolean"); + } + return { + id: profile.id, + name: profile.name, + baseUrl: profile.baseUrl, + authHeader: profile.authHeader, + authScheme: profile.authScheme, + extraHeaders: { ...profile.extraHeaders }, + modelMode: profile.modelMode, + modelOverride: profile.modelOverride, + lastTestAt: profile.lastTestAt, + lastTestStatus: profile.lastTestStatus, + lastTestCode: profile.lastTestCode, + createdAt: profile.createdAt, + updatedAt: profile.updatedAt, + credentialConfigured + }; +} diff --git a/node/test/provider-registry.test.mjs b/node/test/provider-registry.test.mjs new file mode 100644 index 0000000..c15b240 --- /dev/null +++ b/node/test/provider-registry.test.mjs @@ -0,0 +1,1003 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import * as realFileOperations from "node:fs"; +import { + existsSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + statSync, + writeFileSync +} from "node:fs"; +import os from "node:os"; +import { basename, dirname, join } from "node:path"; + +import { ProviderRegistry } from "../src/providers/provider-registry.mjs"; +import { + TEST_STATUSES, + normalizeProvider, + toPublicProvider, + validateProviderInput +} from "../src/providers/provider-schema.mjs"; +import { CrpError, toPublicError } from "../src/shared/errors.mjs"; + +const FIXED_NOW = "2026-07-10T00:00:00.000Z"; +const LATER_NOW = "2026-07-10T01:00:00.000Z"; +const DEFAULT_SETTINGS = { + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: 15101, + captureEnabled: false +}; + +function makeTempRegistry(t, prefix = "crp-provider-registry-") { + const tempDir = mkdtempSync(join(os.tmpdir(), prefix)); + const registryPath = join(tempDir, "providers.json"); + t.after(() => rmSync(tempDir, { recursive: true, force: true })); + return { tempDir, registryPath }; +} + +function makeIds(...ids) { + let index = 0; + return () => ids[index++] ?? `provider-${index}`; +} + +function makeClock(...timestamps) { + let index = 0; + return () => timestamps[index++] ?? timestamps.at(-1) ?? FIXED_NOW; +} + +function validInput(overrides = {}) { + return { + name: "OpenRouter", + baseUrl: "https://openrouter.ai/api/v1", + credentialRef: "provider-1", + ...overrides + }; +} + +function assertCrpError(expectedCode, expectedStatus) { + return (error) => { + assert.ok(error instanceof CrpError); + assert.equal(error.code, expectedCode); + assert.equal(error.status, expectedStatus); + assert.equal(typeof error.action, "string"); + assert.notEqual(error.action.length, 0); + return true; + }; +} + +function listTempFiles(tempDir, registryPath) { + const registryName = basename(registryPath); + return readdirSync(tempDir).filter((name) => ( + name.startsWith(`.${registryName}.`) && name.endsWith(".tmp") + )); +} + +function makeFileError(message, code) { + const error = new Error(message); + error.code = code; + return error; +} + +test("creates, lists, gets, and updates normalized providers", (t) => { + const { registryPath } = makeTempRegistry(t); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: makeClock(FIXED_NOW, LATER_NOW) + }); + + assert.deepEqual(registry.getDocument(), { + schemaVersion: 2, + activeProviderId: null, + providers: [], + settings: DEFAULT_SETTINGS + }); + + const created = registry.create(validInput({ name: " OpenRouter " })); + assert.deepEqual(created, { + id: "provider-1", + name: "OpenRouter", + baseUrl: "https://openrouter.ai/api/v1", + credentialRef: "provider-1", + authHeader: "authorization", + authScheme: "Bearer", + extraHeaders: {}, + modelMode: "passthrough", + modelOverride: null, + lastTestAt: null, + lastTestStatus: "untested", + lastTestCode: null, + createdAt: FIXED_NOW, + updatedAt: FIXED_NOW + }); + assert.deepEqual(registry.list(), [created]); + assert.deepEqual(registry.get("provider-1"), created); + + const updated = registry.update("provider-1", { + name: "Router Primary", + authHeader: "x-provider-auth", + authScheme: "Token", + extraHeaders: { "x-region": "us-east" }, + modelMode: "override", + modelOverride: "gpt-compatible" + }); + assert.equal(updated.id, "provider-1"); + assert.equal(updated.credentialRef, "provider-1"); + assert.equal(updated.createdAt, FIXED_NOW); + assert.equal(updated.updatedAt, LATER_NOW); + assert.equal(updated.name, "Router Primary"); + assert.equal(updated.modelMode, "override"); + assert.equal(updated.modelOverride, "gpt-compatible"); + assert.deepEqual(updated.extraHeaders, { "x-region": "us-east" }); +}); + +test("requires and trims names and rejects case-insensitive duplicates", (t) => { + const { registryPath } = makeTempRegistry(t); + const registry = new ProviderRegistry({ + path: registryPath, + createId: makeIds("provider-1", "provider-2", "provider-3"), + now: () => FIXED_NOW + }); + + assert.throws( + () => registry.create(validInput({ name: " " })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); + registry.create(validInput()); + assert.throws( + () => registry.create(validInput({ + name: " openrouter ", + credentialRef: "provider-2" + })), + assertCrpError("PROVIDER_NAME_CONFLICT", 409) + ); + + registry.create(validInput({ + name: "Backup", + baseUrl: "https://backup.example/v1", + credentialRef: "provider-3" + })); + assert.throws( + () => registry.update("provider-3", { name: "OPENROUTER" }), + assertCrpError("PROVIDER_NAME_CONFLICT", 409) + ); +}); + +test("serializes multi-instance mutations and refreshes existing readers", (t) => { + const { registryPath } = makeTempRegistry(t, "crp-provider-multi-instance-"); + const first = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + const second = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-2", + now: () => FIXED_NOW + }); + const staleDuplicate = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-3", + now: () => FIXED_NOW + }); + + first.create(validInput({ name: "Primary" })); + second.create(validInput({ + name: "Backup", + baseUrl: "https://backup.example/v1", + credentialRef: "provider-2" + })); + + assert.deepEqual( + first.list().map(({ id, name }) => ({ id, name })), + [ + { id: "provider-1", name: "Primary" }, + { id: "provider-2", name: "Backup" } + ] + ); + assert.equal(first.get("provider-2").name, "Backup"); + assert.throws( + () => staleDuplicate.create(validInput({ + name: " PRIMARY ", + credentialRef: "provider-3" + })), + assertCrpError("PROVIDER_NAME_CONFLICT", 409) + ); + assert.equal(first.getDocument().providers.length, 2); +}); + +test("rejects a foreign registry lock without removing it", (t) => { + const { tempDir, registryPath } = makeTempRegistry(t, "crp-provider-foreign-lock-"); + const lockPath = `${registryPath}.crp.lock`; + const foreignLockBytes = Buffer.from("foreign-owner\n", "utf8"); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + writeFileSync(lockPath, foreignLockBytes, { mode: 0o600 }); + + assert.throws( + () => registry.create(validInput()), + assertCrpError("PROVIDER_REGISTRY_BUSY", 409) + ); + assert.deepEqual(readFileSync(lockPath), foreignLockBytes); + assert.equal(existsSync(registryPath), false); + assert.deepEqual(listTempFiles(tempDir, registryPath), []); +}); + +test("rejects missing providers and immutable profile fields", (t) => { + const { registryPath } = makeTempRegistry(t); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + + assert.throws( + () => registry.get("missing"), + assertCrpError("PROVIDER_NOT_FOUND", 404) + ); + assert.throws( + () => registry.update("missing", { name: "Missing" }), + assertCrpError("PROVIDER_NOT_FOUND", 404) + ); + assert.throws( + () => registry.delete("missing"), + assertCrpError("PROVIDER_NOT_FOUND", 404) + ); + assert.throws( + () => registry.setActive("missing"), + assertCrpError("PROVIDER_NOT_FOUND", 404) + ); + + registry.create(validInput()); + for (const patch of [ + { id: "provider-2" }, + { createdAt: LATER_NOW }, + { credentialRef: "provider-2" } + ]) { + assert.throws( + () => registry.update("provider-1", patch), + assertCrpError("PROVIDER_IMMUTABLE_FIELD", 400) + ); + } +}); + +test("accepts, rejects, and canonically persists provider URLs", (t) => { + for (const baseUrl of [ + "https://provider.example/v1", + "https://provider.example/v1/@scope", + "https://provider.example/v1?contact=user@example.com", + "http://localhost:8080/v1", + "http://127.0.0.1:8080/v1", + "http://127.42.0.9/v1", + "http://[::1]:8080/v1" + ]) { + assert.doesNotThrow(() => validateProviderInput(validInput({ baseUrl }))); + } + + for (const baseUrl of [ + "ftp://provider.example/v1", + "http://provider.example/v1", + "http://localhost.example/v1", + "http://128.0.0.1/v1", + "https://user:password@provider.example/v1", + "https://@provider.example/v1", + "https://:@provider.example/v1", + "https://provider.example/v1\rignored", + "https://provider.example/v1\nignored", + "https://provider.example/v1\0ignored", + "https://provider.example/v1\x7fignored", + "not a url" + ]) { + assert.throws( + () => validateProviderInput(validInput({ baseUrl })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); + } + + const { registryPath } = makeTempRegistry(t, "crp-provider-canonical-url-"); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + const created = registry.create(validInput({ + baseUrl: "HTTPS://Provider.Example:443/a/../v1" + })); + assert.equal(created.baseUrl, "https://provider.example/v1"); + assert.equal( + JSON.parse(readFileSync(registryPath, "utf8")).providers[0].baseUrl, + "https://provider.example/v1" + ); +}); + +test("requires extraHeaders to be a string map with non-sensitive names", () => { + for (const extraHeaders of [ + [], + { "x-region": 1 }, + { authorization: "hidden" }, + { "Proxy-Authorization": "hidden" }, + { COOKIE: "hidden" }, + { "Set-Cookie": "hidden" }, + { "x-auth-token": "hidden" }, + { "client-secret-mode": "hidden" }, + { "X-API-KEY": "hidden" }, + { "x-api_key": "ordinary-value" }, + { "x-apikey": "ordinary-value" }, + { "x-authorization": "ordinary-value" }, + { "X_AuThOrIzAtIoN": "ordinary-value" }, + { "x-region": "line\rbreak" }, + { "x-region": "line\nbreak" }, + { "x-region": "value\0break" }, + { "x-region": "value\x7fbreak" } + ]) { + assert.throws( + () => validateProviderInput(validInput({ extraHeaders })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); + } + + const sensitivePlaceholder = "sensitive-placeholder"; + let error; + try { + validateProviderInput(validInput({ + extraHeaders: { "x-client-secret": sensitivePlaceholder } + })); + } catch (caught) { + error = caught; + } + assert.ok(error instanceof CrpError); + assert.doesNotMatch( + JSON.stringify(toPublicError(error, "request-1")), + new RegExp(sensitivePlaceholder) + ); +}); + +test("allows an empty auth scheme and requires non-empty schemes to be HTTP tokens", () => { + assert.equal( + normalizeProvider(validInput({ authScheme: "" }), { + id: "provider-1", + now: FIXED_NOW + }).authScheme, + "" + ); + for (const authScheme of ["Bearer value", "Bearer/Token", "Bearer,Token", "Bearer\n"]) { + assert.throws( + () => validateProviderInput(validInput({ authScheme })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); + } +}); + +test("validates passthrough and override model modes", () => { + assert.doesNotThrow(() => validateProviderInput(validInput({ + modelMode: "passthrough", + modelOverride: null + }))); + assert.doesNotThrow(() => validateProviderInput(validInput({ + modelMode: "override", + modelOverride: " compatible-model " + }))); + assert.throws( + () => validateProviderInput(validInput({ modelMode: "unknown" })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); + assert.throws( + () => validateProviderInput(validInput({ + modelMode: "override", + modelOverride: " " + })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); + + assert.deepEqual(TEST_STATUSES, new Set(["untested", "passed", "failed"])); + assert.deepEqual( + normalizeProvider(validInput({ + authHeader: undefined, + authScheme: undefined, + extraHeaders: undefined, + modelMode: undefined, + modelOverride: undefined + }), { id: "provider-1", now: FIXED_NOW }), + { + id: "provider-1", + name: "OpenRouter", + baseUrl: "https://openrouter.ai/api/v1", + credentialRef: "provider-1", + authHeader: "authorization", + authScheme: "Bearer", + extraHeaders: {}, + modelMode: "passthrough", + modelOverride: null, + lastTestAt: null, + lastTestStatus: "untested", + lastTestCode: null, + createdAt: FIXED_NOW, + updatedAt: FIXED_NOW + } + ); +}); + +test("marks passed and failed tests and manages active deletion", (t) => { + const { registryPath } = makeTempRegistry(t); + const registry = new ProviderRegistry({ + path: registryPath, + createId: makeIds("provider-1", "provider-2"), + now: makeClock(FIXED_NOW, FIXED_NOW, LATER_NOW, LATER_NOW, LATER_NOW, LATER_NOW) + }); + + registry.create(validInput()); + registry.create(validInput({ + name: "Backup", + baseUrl: "https://backup.example/v1", + credentialRef: "provider-2" + })); + + const passed = registry.markTest("provider-1", { status: "passed", code: null }); + assert.equal(passed.lastTestAt, LATER_NOW); + assert.equal(passed.lastTestStatus, "passed"); + assert.equal(passed.lastTestCode, null); + + const failed = registry.markTest("provider-2", { + status: "failed", + code: "UPSTREAM_AUTH_FAILED" + }); + assert.equal(failed.lastTestStatus, "failed"); + assert.equal(failed.lastTestCode, "UPSTREAM_AUTH_FAILED"); + const reset = registry.markTest("provider-2", { status: "untested", code: null }); + assert.equal(reset.lastTestAt, null); + assert.equal(reset.lastTestStatus, "untested"); + assert.equal(reset.lastTestCode, null); + + const activated = registry.setActive("provider-1"); + assert.equal(activated.id, "provider-1"); + assert.equal(registry.getActive().id, "provider-1"); + assert.equal(registry.getDocument().activeProviderId, "provider-1"); + assert.throws( + () => registry.delete("provider-1"), + assertCrpError("PROVIDER_ACTIVE", 409) + ); + + const deleted = registry.delete("provider-2"); + assert.equal(deleted.id, "provider-2"); + assert.equal(registry.list().length, 1); +}); + +test("preserves tests for name edits and invalidates them for operational changes", (t) => { + const { registryPath } = makeTempRegistry(t, "crp-provider-test-invalidation-"); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + registry.create(validInput()); + registry.markTest("provider-1", { status: "passed", code: null }); + + const renamed = registry.update("provider-1", { name: "Primary" }); + assert.equal(renamed.lastTestAt, FIXED_NOW); + assert.equal(renamed.lastTestStatus, "passed"); + assert.equal(renamed.lastTestCode, null); + + const operationalPatches = [ + { baseUrl: "https://alternate.example/v1" }, + { authHeader: "x-provider-auth" }, + { authScheme: "Token" }, + { extraHeaders: { "x-region": "eu-west" } }, + { modelMode: "override", modelOverride: "compatible-model-a" }, + { modelOverride: "compatible-model-b" } + ]; + for (const patch of operationalPatches) { + registry.markTest("provider-1", { status: "passed", code: null }); + const updated = registry.update("provider-1", patch); + assert.equal(updated.lastTestAt, null); + assert.equal(updated.lastTestStatus, "untested"); + assert.equal(updated.lastTestCode, null); + } +}); + +test("reloads the complete persisted document", (t) => { + const { registryPath } = makeTempRegistry(t); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: makeClock(FIXED_NOW, LATER_NOW, LATER_NOW, LATER_NOW) + }); + + registry.create(validInput({ + extraHeaders: { "x-region": "eu-west" }, + modelMode: "override", + modelOverride: "provider-model" + })); + registry.markTest("provider-1", { status: "passed", code: null }); + registry.setActive("provider-1"); + const beforeReload = registry.getDocument(); + + const reloaded = new ProviderRegistry({ path: registryPath }); + assert.deepEqual(reloaded.getDocument(), beforeReload); + assert.deepEqual(reloaded.getActive(), beforeReload.providers[0]); +}); + +test("rejects malformed JSON and invalid schema-version-2 documents", (t) => { + const { tempDir } = makeTempRegistry(t, "crp-provider-invalid-"); + const documents = [ + "{ malformed", + `${JSON.stringify({ schemaVersion: 1, activeProviderId: null, providers: [], settings: DEFAULT_SETTINGS })}\n`, + `${JSON.stringify({ schemaVersion: 2, activeProviderId: null, providers: {}, settings: DEFAULT_SETTINGS })}\n`, + `${JSON.stringify({ schemaVersion: 2, activeProviderId: "missing", providers: [], settings: DEFAULT_SETTINGS })}\n`, + `${JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [], + settings: { ...DEFAULT_SETTINGS, proxyPort: 15102 } + })}\n`, + `${JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [{ + ...normalizeProvider(validInput(), { id: "provider-1", now: FIXED_NOW }), + lastTestStatus: "unknown" + }], + settings: DEFAULT_SETTINGS + })}\n`, + `${JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [{ + ...normalizeProvider(validInput(), { id: "provider-1", now: FIXED_NOW }), + lastTestAt: "2026-07-09T23:59:59.000Z", + lastTestStatus: "passed" + }], + settings: DEFAULT_SETTINGS + })}\n`, + `${JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [{ + ...normalizeProvider(validInput(), { id: "provider-1", now: FIXED_NOW }), + lastTestAt: LATER_NOW, + lastTestStatus: "passed" + }], + settings: DEFAULT_SETTINGS + })}\n` + ]; + + for (const [index, bytes] of documents.entries()) { + const registryPath = join(tempDir, `providers-${index}.json`); + writeFileSync(registryPath, bytes, "utf8"); + assert.throws( + () => new ProviderRegistry({ path: registryPath }), + assertCrpError("PROVIDER_REGISTRY_INVALID", 500) + ); + assert.equal(readFileSync(registryPath, "utf8"), bytes); + } +}); + +test("persists through a same-directory fsynced rename with mode 0600", (t) => { + const { tempDir, registryPath } = makeTempRegistry(t, "crp-provider-atomic-"); + const lockPath = `${registryPath}.crp.lock`; + const operations = []; + const fileOperations = { + ...realFileOperations, + openSync(path, flags, mode) { + operations.push(["open", path, flags, mode]); + return realFileOperations.openSync(path, flags, mode); + }, + fsyncSync(fd) { + operations.push(["fsync", fd]); + return realFileOperations.fsyncSync(fd); + }, + chmodSync(path, mode) { + operations.push(["chmod", path, mode]); + return realFileOperations.chmodSync(path, mode); + }, + renameSync(source, destination) { + operations.push(["rename", source, destination]); + return realFileOperations.renameSync(source, destination); + }, + rmSync(path, options) { + operations.push(["rm", path]); + return realFileOperations.rmSync(path, options); + } + }; + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW, + fileOperations + }); + + registry.create(validInput()); + + const lockOpenIndex = operations.findIndex(([name, path]) => ( + name === "open" && path === lockPath + )); + const tempOpenIndex = operations.findIndex(([name, path]) => ( + name === "open" && path !== lockPath + )); + const fsyncIndex = operations.findIndex(([name]) => name === "fsync"); + const renameIndex = operations.findIndex(([name]) => name === "rename"); + const lockRemovalIndex = operations.findIndex(([name, path]) => ( + name === "rm" && path === lockPath + )); + const lockOpen = operations[lockOpenIndex]; + const open = operations[tempOpenIndex]; + const rename = operations[renameIndex]; + assert.ok(lockOpenIndex > -1); + assert.equal(lockOpen[2], "wx"); + assert.equal(lockOpen[3], 0o600); + assert.equal(dirname(open[1]), tempDir); + assert.equal(open[2], "wx"); + assert.equal(open[3], 0o600); + assert.equal(dirname(rename[1]), tempDir); + assert.equal(rename[2], registryPath); + assert.ok(lockOpenIndex < tempOpenIndex); + assert.ok(tempOpenIndex < fsyncIndex && fsyncIndex < renameIndex); + assert.ok(renameIndex < lockRemovalIndex); + if (process.platform !== "win32") { + assert.equal(statSync(registryPath).mode & 0o777, 0o600); + } else { + assert.doesNotThrow(() => readFileSync(registryPath)); + } + assert.deepEqual(listTempFiles(tempDir, registryPath), []); + assert.equal(existsSync(lockPath), false); +}); + +test("returns a durable result after a one-shot registry lock close failure", (t) => { + const { registryPath } = makeTempRegistry(t, "crp-provider-lock-close-retry-"); + const lockPath = `${registryPath}.crp.lock`; + let lockFileDescriptor; + let lockCloseAttempts = 0; + let registryRenameCount = 0; + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW, + fileOperations: { + ...realFileOperations, + openSync(path, flags, mode) { + const fileDescriptor = realFileOperations.openSync(path, flags, mode); + if (path === lockPath) { + lockFileDescriptor = fileDescriptor; + } + return fileDescriptor; + }, + closeSync(fileDescriptor) { + if (fileDescriptor === lockFileDescriptor) { + lockCloseAttempts += 1; + if (lockCloseAttempts === 1) { + realFileOperations.closeSync(fileDescriptor); + throw makeFileError("forced one-shot lock close failure", "EIO"); + } + } + return realFileOperations.closeSync(fileDescriptor); + }, + renameSync(source, destination) { + if (destination === registryPath) { + registryRenameCount += 1; + } + return realFileOperations.renameSync(source, destination); + } + } + }); + + const created = registry.create(validInput()); + + assert.equal(created.id, "provider-1"); + assert.equal(lockCloseAttempts, 2); + assert.equal(registryRenameCount, 1); + assert.equal(registry.document.providers.length, 1); + assert.equal(JSON.parse(readFileSync(registryPath, "utf8")).providers.length, 1); + assert.equal(existsSync(lockPath), false); +}); + +test("returns a durable result after a one-shot registry lock removal failure", (t) => { + const { registryPath } = makeTempRegistry(t, "crp-provider-lock-rm-retry-"); + const lockPath = `${registryPath}.crp.lock`; + let lockRemovalAttempts = 0; + let registryRenameCount = 0; + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW, + fileOperations: { + ...realFileOperations, + rmSync(path, options) { + if (path === lockPath) { + lockRemovalAttempts += 1; + if (lockRemovalAttempts === 1) { + throw makeFileError("forced one-shot lock removal failure", "EBUSY"); + } + } + return realFileOperations.rmSync(path, options); + }, + renameSync(source, destination) { + if (destination === registryPath) { + registryRenameCount += 1; + } + return realFileOperations.renameSync(source, destination); + } + } + }); + + const created = registry.create(validInput()); + + assert.equal(created.id, "provider-1"); + assert.equal(lockRemovalAttempts, 2); + assert.equal(registryRenameCount, 1); + assert.equal(registry.document.providers.length, 1); + assert.equal(JSON.parse(readFileSync(registryPath, "utf8")).providers.length, 1); + assert.equal(existsSync(lockPath), false); +}); + +test("preserves a primary persistence error when lock cleanup transiently fails", (t) => { + const { tempDir, registryPath } = makeTempRegistry(t, "crp-provider-primary-error-"); + const lockPath = `${registryPath}.crp.lock`; + const initial = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + initial.create(validInput()); + const originalBytes = readFileSync(registryPath); + const renameError = makeFileError("forced primary rename failure", "EIO"); + let lockRemovalAttempts = 0; + const registry = new ProviderRegistry({ + path: registryPath, + now: () => LATER_NOW, + fileOperations: { + ...realFileOperations, + renameSync() { + throw renameError; + }, + rmSync(path, options) { + if (path === lockPath) { + lockRemovalAttempts += 1; + if (lockRemovalAttempts === 1) { + throw makeFileError("forced transient cleanup failure", "EBUSY"); + } + } + return realFileOperations.rmSync(path, options); + } + } + }); + const originalDocument = structuredClone(registry.document); + + assert.throws( + () => registry.update("provider-1", { name: "Updated" }), + (error) => error === renameError + ); + assert.equal(lockRemovalAttempts, 2); + assert.deepEqual(readFileSync(registryPath), originalBytes); + assert.deepEqual(registry.document, originalDocument); + assert.equal(existsSync(lockPath), false); + assert.deepEqual(listTempFiles(tempDir, registryPath), []); +}); + +test("reports a durable mutation with a permanently degraded owned lock", (t) => { + const { registryPath } = makeTempRegistry(t, "crp-provider-lock-degraded-"); + const lockPath = `${registryPath}.crp.lock`; + let lockRemovalAttempts = 0; + let lockReadAttempts = 0; + let rejectLockRemoval = true; + let lockOpenAttempts = 0; + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW, + fileOperations: { + ...realFileOperations, + openSync(path, flags, mode) { + if (path === lockPath) { + lockOpenAttempts += 1; + } + return realFileOperations.openSync(path, flags, mode); + }, + readFileSync(path, ...args) { + if (path === lockPath) { + lockReadAttempts += 1; + } + return realFileOperations.readFileSync(path, ...args); + }, + rmSync(path, options) { + if (path === lockPath) { + lockRemovalAttempts += 1; + if (rejectLockRemoval) { + throw makeFileError("forced permanent lock removal failure", "EACCES"); + } + } + return realFileOperations.rmSync(path, options); + } + } + }); + + assert.throws( + () => registry.create(validInput()), + (error) => { + assert.ok(error instanceof CrpError); + assert.equal(error.code, "PROVIDER_REGISTRY_COMMITTED_LOCK_DEGRADED"); + assert.equal(error.status, 500); + assert.deepEqual(error.details, { committed: true }); + assert.doesNotMatch(error.action, /retry|try again/i); + return true; + } + ); + assert.equal(registry.document.providers.length, 1); + assert.equal(JSON.parse(readFileSync(registryPath, "utf8")).providers.length, 1); + assert.equal(existsSync(lockPath), true); + + const foreignLockBytes = Buffer.from("foreign-owner\n", "utf8"); + writeFileSync(lockPath, foreignLockBytes); + rejectLockRemoval = false; + const lockReadAttemptsBeforeRetry = lockReadAttempts; + const lockRemovalAttemptsBeforeRetry = lockRemovalAttempts; + const lockOpenAttemptsBeforeRetry = lockOpenAttempts; + assert.throws( + () => registry.update("provider-1", { name: "Updated" }), + assertCrpError("PROVIDER_REGISTRY_LOCK_DEGRADED", 500) + ); + assert.equal(lockReadAttempts, lockReadAttemptsBeforeRetry); + assert.equal(lockRemovalAttempts, lockRemovalAttemptsBeforeRetry); + assert.equal(lockOpenAttempts, lockOpenAttemptsBeforeRetry); + assert.deepEqual(readFileSync(lockPath), foreignLockBytes); + assert.equal(lockOpenAttempts, 1); + assert.equal(registry.document.providers[0].name, "OpenRouter"); +}); + +test("validation failures preserve disk bytes and in-memory state", (t) => { + const { tempDir, registryPath } = makeTempRegistry(t, "crp-provider-validation-rollback-"); + const registry = new ProviderRegistry({ + path: registryPath, + createId: makeIds("provider-1", "provider-2"), + now: () => FIXED_NOW + }); + registry.create(validInput()); + const originalBytes = readFileSync(registryPath); + const originalDocument = registry.getDocument(); + const originalDocumentBytes = JSON.stringify(originalDocument); + + assert.throws( + () => registry.create(validInput({ + name: "OPENROUTER", + credentialRef: "provider-2" + })), + assertCrpError("PROVIDER_NAME_CONFLICT", 409) + ); + assert.deepEqual(readFileSync(registryPath), originalBytes); + assert.deepEqual(registry.getDocument(), originalDocument); + assert.equal(JSON.stringify(registry.getDocument()), originalDocumentBytes); + assert.deepEqual(listTempFiles(tempDir, registryPath), []); +}); + +test("rename failures preserve disk bytes and in-memory state and clean the temp file", (t) => { + const { tempDir, registryPath } = makeTempRegistry(t, "crp-provider-rename-rollback-"); + const initial = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + initial.create(validInput()); + const originalBytes = readFileSync(registryPath); + const renameError = new Error("forced provider registry rename failure"); + const registry = new ProviderRegistry({ + path: registryPath, + now: () => LATER_NOW, + fileOperations: { + ...realFileOperations, + renameSync() { + throw renameError; + } + } + }); + const originalDocument = registry.getDocument(); + + assert.throws( + () => registry.update("provider-1", { name: "Updated" }), + (error) => error === renameError + ); + assert.deepEqual(readFileSync(registryPath), originalBytes); + assert.deepEqual(registry.getDocument(), originalDocument); + assert.deepEqual(listTempFiles(tempDir, registryPath), []); +}); + +test("list, get, getActive, and getDocument return defensive copies", (t) => { + const { registryPath } = makeTempRegistry(t); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + registry.create(validInput({ extraHeaders: { "x-region": "original" } })); + registry.setActive("provider-1"); + + const listed = registry.list(); + listed[0].name = "Mutated"; + listed[0].extraHeaders["x-region"] = "mutated"; + listed.push({ id: "injected" }); + + const fetched = registry.get("provider-1"); + fetched.name = "Mutated again"; + fetched.extraHeaders.injected = "value"; + + const active = registry.getActive(); + active.name = "Mutated active"; + + const document = registry.getDocument(); + document.activeProviderId = null; + document.providers.length = 0; + document.settings.proxyPort = 9999; + + assert.equal(registry.list().length, 1); + assert.equal(registry.get("provider-1").name, "OpenRouter"); + assert.deepEqual(registry.get("provider-1").extraHeaders, { "x-region": "original" }); + assert.equal(registry.getActive().id, "provider-1"); + assert.equal(registry.getDocument().activeProviderId, "provider-1"); + assert.equal(registry.getDocument().settings.proxyPort, 15100); +}); + +test("toPublicProvider returns an exact allowlisted shape with a boolean credential flag", () => { + const credentialReference = "credential-reference"; + const futureSensitiveValue = "future-sensitive-placeholder"; + const profile = { + ...normalizeProvider(validInput({ credentialRef: credentialReference }), { + id: "provider-1", + now: FIXED_NOW + }), + futureSecretField: futureSensitiveValue, + futureInternalState: { value: futureSensitiveValue } + }; + + const publicProvider = toPublicProvider(profile, true); + const serialized = JSON.stringify(publicProvider); + assert.deepEqual(Object.keys(publicProvider), [ + "id", + "name", + "baseUrl", + "authHeader", + "authScheme", + "extraHeaders", + "modelMode", + "modelOverride", + "lastTestAt", + "lastTestStatus", + "lastTestCode", + "createdAt", + "updatedAt", + "credentialConfigured" + ]); + assert.equal(Object.hasOwn(publicProvider, "credentialRef"), false); + assert.equal(publicProvider.credentialConfigured, true); + assert.doesNotMatch(serialized, new RegExp(credentialReference)); + assert.doesNotMatch(serialized, new RegExp(futureSensitiveValue)); + assert.throws( + () => toPublicProvider(profile, "true"), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); +}); + +test("an absent registry file stays absent until the first successful mutation", (t) => { + const { registryPath } = makeTempRegistry(t, "crp-provider-lazy-create-"); + const registry = new ProviderRegistry({ + path: registryPath, + createId: () => "provider-1", + now: () => FIXED_NOW + }); + + assert.equal(realFileOperations.existsSync(registryPath), false); + assert.deepEqual(registry.list(), []); + assert.equal(realFileOperations.existsSync(registryPath), false); + + assert.throws( + () => registry.create(validInput({ baseUrl: "http://remote.example/v1" })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); + assert.equal(realFileOperations.existsSync(registryPath), false); + + registry.create(validInput()); + assert.equal(realFileOperations.existsSync(registryPath), true); +}); From 343ffdfb612892f8bd62d9c10f5353b4293dc3c9 Mon Sep 17 00:00:00 2001 From: cluic Date: Sun, 12 Jul 2026 21:25:02 +0800 Subject: [PATCH 07/44] feat: add secure credential stores --- docs/AGENT_COORDINATION.md | 4 +- docs/AI_HANDOFF.md | 16 +- docs/ARCHITECTURE.md | 13 +- docs/DATA_MODEL.md | 27 +- docs/PERMISSIONS.md | 10 + docs/STATUS.md | 8 +- docs/TESTING.md | 15 +- .../plans/2026-07-10-crp-v1-implementation.md | 58 +- node/src/credentials/credential-store.mjs | 71 + .../src/credentials/file-credential-store.mjs | 796 ++++++++++ node/src/credentials/native-keyring.mjs | 133 ++ node/test/credential-store.test.mjs | 1347 +++++++++++++++++ 12 files changed, 2455 insertions(+), 43 deletions(-) create mode 100644 node/src/credentials/credential-store.mjs create mode 100644 node/src/credentials/file-credential-store.mjs create mode 100644 node/src/credentials/native-keyring.mjs create mode 100644 node/test/credential-store.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 2ddb184..9e8f6af 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 3 owns `node/src/providers/`, `node/test/provider-registry.test.mjs`, and affected living docs | Task 3 complete on `codex/harness-product-design`; Task 4 next | Task 2 shared paths, errors, and Codex bootstrap | Every file outside the active task; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 4 owns `node/src/credentials/`, `node/test/credential-store.test.mjs`, and affected security/data/living docs | Task 4 complete on `codex/harness-product-design`; Task 5 next | Task 3 atomic provider registry | Every file outside the active task; no simultaneous writable agent | ## Shared Contracts @@ -29,4 +29,4 @@ ## Decisions Needed -Subagent-driven execution is selected. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. Task 4 requires a fresh file scope before writable work begins. +Subagent-driven execution is selected. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index 929c161..d608269 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,16 +6,16 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 3 have landed, including the atomic provider metadata registry; provider credentials and lifecycle orchestration have not landed. Task 4, native and explicit-fallback credential stores, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 4 have landed, including the atomic provider metadata registry and native plus explicit-consent file credential adapters; provider-service lifecycle orchestration has not landed. Task 5, snapshot-based proxy settings, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, and a lock-serialized atomic schema-version-2 provider registry with refreshed defensive reads. Target: credential stores, provider-service orchestration, long-lived supervisor control plane, and independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry with refreshed defensive reads, and credential adapters with explicit-only file fallback. Target: provider-service orchestration, long-lived supervisor control plane, and independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API - Non-secret profiles now live in the implemented schema-versioned registry. -- API keys will live in native credential stores with explicit `0600` fallback after Task 4 lands. +- API keys use the landed native adapter by default or the landed schema-version-1 `0600` file adapter only after explicit consent. - Local API contract is in `docs/API.md`; data contract is in `docs/DATA_MODEL.md`. ## Permissions @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 3 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 4 are complete. ## How To Run Current Code @@ -44,6 +44,7 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 baseline and Task 1 gate: `npm test` passes 12/12 tests and `npm audit --omit=dev` reports zero vulnerabilities. - Node 22.19 Task 2 gate: `node --test test/codex-config.test.mjs` passes 15/15, including deterministic rename failure, exclusive same-timestamp backup collision, busy lock, external source change, CRLF preservation, guide semantics, and all three start aliases; `npm test` passes 27/27, `npm run lint` syntax-checks 9 source files, and `npm audit --omit=dev` reports zero vulnerabilities. - Node 22.19 Task 3 gate: `node --test test/provider-registry.test.mjs` passes 23/23, including multi-instance lock serialization, strict schema and header validation, test-state invalidation, primary-error preservation, degraded lock cleanup, refreshed defensive copies, and public allowlisting; `npm test` passes 50/50, `npm run lint` syntax-checks 11 source files, and `npm audit --omit=dev` reports zero vulnerabilities. +- Node 22.19 Task 4 gate: `node --test test/credential-store.test.mjs` passes 41/41, the combined credential/provider focus passes 64/64, `npm test` passes 91/91, and `npm run lint` syntax-checks 14 source files. Coverage includes construction-only fallback without operation replay, explicit file-label restart continuity, descriptor identity, strict parent/file modes, degraded temp cleanup, canonical lock restoration, claim-before-delete gate release, foreign replacement preservation, and synchronous second-instance blocking while a gate claim is validated. Native tests inject the loader and never invoke the real addon loader or touch the OS credential store; real native verification remains L3 on every supported system, including Windows and Linux. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. @@ -65,3 +66,10 @@ Credential migration, localhost browser security, worker IPC, port release races - Atomic configuration writes must compare content first and preserve source file permissions. - Registry mutation must persist successfully before replacing in-memory state. - Registry mutations must reload while holding the registry lock before replacing state. +- File credential fallback must never be selected without explicit consent. +- Selected native operations must never replay into the file credential namespace. +- Secret credential files must be read through a validated descriptor, never through a post-check path read. +- Credential mutation must remain gate-protected, and the primary lock must cover gate claim validation. +- Canonical gate paths must never be deleted after a separate identity check; atomically claim them to a unique path first. +- Canonical primary locks must remain until gate ownership or replacement-blocker state is proven. +- Shell validation patterns must be individually quoted so the scan itself is deterministic. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 87c8206..4654405 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 and 3 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, and an atomic schema-version-2 provider registry. The supervisor, credential stores, provider service, Admin API, and independent worker remain target-state architecture. +Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 4 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, an atomic schema-version-2 provider registry, and native plus explicit-consent file credential adapters. The supervisor, provider service, Admin API, and independent worker remain target-state architecture. ## Target Overview @@ -40,18 +40,20 @@ Active OpenAI-compatible upstream ## Module Boundaries -Landed in Tasks 2 and 3: +Landed in Tasks 2 through 4: - `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. - `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. - `codex-config`: preserves custom providers and source line endings while idempotently bootstrapping the fixed OpenAI provider entry; a sidecar CRP lock serializes writers, changed files receive an exclusive timestamped adjacent backup, and the source is rechecked before atomic mode-preserving replacement. - `provider-schema`: canonicalizes safe URLs, validates auth and extra-header syntax plus complete stored profiles, and builds public provider objects from an explicit allowlist. - `provider-registry`: synchronously manages provider CRUD, test state, and the active provider ID through strict schema-version-2 documents, lock-serialized reload-before-mutate writes, refreshed defensive reads, and same-directory fsynced `0600` atomic replacement. +- `credential-store`: selects native storage by default and chooses the schema-version-1 file fallback only with explicit consent when native construction fails before any credential operation; selected native operations never replay into the independent file namespace. Both adapters expose asynchronous get/set/has/delete operations without enumeration. +- `native-keyring`: lazily loads synchronous `@napi-rs/keyring` entries during construction under service `org.cluic.codex-remote-proxy`, supports injected loaders/factories, preserves backend failures only as internal causes, and exposes safe stable errors. +- `file-credential-store`: validates a real private parent and regular private file, opens and identity-checks a descriptor before reading, validates exact secret-only documents, refreshes reads, and serializes clone-before-commit mutations with exclusive `0600` lock and temporary files. Remaining target-state boundaries: - `supervisor`: owns state transitions, admin server, activity records, and child-process lifecycle. -- `credential-store`: exposes get/set/delete by opaque credential reference; implements Keychain, Credential Manager, and file fallback adapters. - `provider-service`: coordinates compatibility tests, credentials, activation, and credential-aware deletion above the metadata registry. - `worker-protocol`: versioned IPC messages for configure, drain, shutdown, health, and events. - `proxy-worker`: forwards traffic from immutable provider snapshots; does not own persistent configuration. @@ -67,13 +69,14 @@ Remaining target-state boundaries: ## Storage and Deployment - Package remains distributed through npm. -- The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; the provider registry now owns `providers.json`, while the other new stores remain target state. +- The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; the provider registry owns `providers.json`, the explicit fallback adapter owns `secrets.json`, and the other new stores remain target state. - Provider metadata: `~/.codex-remote-proxy/providers.json`, atomically replaced with mode `0600` where supported after complete document validation. - Supervisor runtime metadata target: `~/.codex-remote-proxy/state.json`, mode `0600` where supported. -- Credentials: native OS store; `~/.codex-remote-proxy/secrets.json` only after explicit fallback consent. +- Credentials: native OS store by default; `~/.codex-remote-proxy/secrets.json` only after strict explicit fallback consent. Native loader/factory failure before selection is a safe backend-unavailable error unless that consent is present. Once native is selected, operation failures remain native and are never replayed. Construction fallback exposes a `file` label that must be explicitly reused across restart; no credential migration is implicit. - Activity events: bounded local JSONL or SQLite store without request/response bodies. - Codex configuration replacement holds an exclusive sidecar lock, compares bytes before writing, creates backups with exclusive-copy semantics, rechecks the source before rename, and preserves the source permission mode through same-directory temporary-file `fsync` and rename; unchanged content is neither rewritten nor backed up. - Provider-registry mutation holds an exclusive `0600` sidecar lock across disk reload, complete validation, same-directory temporary-file `fsync`, `chmod 0600`, rename, and in-memory replacement; validation or persistence failure leaves the prior in-memory document unchanged. Bounded cleanup preserves primary errors and distinguishes a durable committed/degraded result from a retryable failure; permanent residual locks require explicit repair and restart and are never auto-removed. +- File-credential mutation follows the same conservative durable-commit rules while using a strict schema-version-1 secret-only document. Reads validate the parent and path before opening, verify descriptor identity, and never read secret bytes by path. An exclusive gate covers mutation. Release atomically renames the canonical gate to a unique claim, verifies and removes only that claim, and never deletes the canonical path. The canonical primary lock remains present throughout gate claim validation, so a competing instance that acquires the empty gate still reports busy; foreign or uncertain gate state must prove a canonical blocker before primary release, otherwise the primary lock is retained. Foreign or permanent claimed primary locks likewise restore a nonempty canonical blocker. Permanent secret-temp cleanup failure records uncommitted degradation and stops later mutations before another lock opens. Public errors contain no reference, secret, path, or file bytes. - macOS and Windows receive UI support; Linux uses the same supervisor and CLI without an initial UI guarantee. ## Risks diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index 400927e..9812545 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -1,6 +1,6 @@ # Data Model -Task 3 has implemented strict schema-version-2 provider validation and atomic metadata-registry persistence. Credential adapters, provider-service orchestration, and migration remain target-state work. +Tasks 3 and 4 have implemented strict schema-version-2 provider metadata persistence plus native and explicit-consent file credential adapters. Provider-service orchestration and migration remain target-state work. ## ProviderProfile @@ -45,6 +45,29 @@ Lock close and removal receive bounded cleanup retries. Cleanup failure never ma Changing `baseUrl`, authentication fields, extra headers, or model policy resets the compatibility-test state to `untested`; changing only the display name preserves it. Credential replacement can explicitly record the same reset. Public provider projection uses an explicit field allowlist and requires a boolean credential-configured flag. +## CredentialDocument + +The explicit fallback adapter persists exactly this schema and exposes only `set`, `get`, `has`, and `delete` operations by opaque reference: + +```json +{ + "schemaVersion": 1, + "credentials": { + "credential-ref": "secret-string" + } +} +``` + +Credential references are non-empty bounded ASCII identifiers, prototype-pollution keys are rejected, and every stored value is a non-empty string. The document contains no provider metadata and there is no credential enumeration API. + +Every file mutation acquires an exclusive `0600` sidecar lock, reloads and validates disk state under that lock, mutates a clone, writes a same-directory exclusive `0600` temporary file, fsyncs and closes it, reapplies mode `0600`, renames it over the destination, then replaces in-memory state. A failed write leaves disk and memory unchanged. Secret-temp removal receives bounded retries; a permanent residue reports `CREDENTIAL_STORE_TEMP_DEGRADED` with `committed: false`, records instance degradation, and blocks later mutations before any new lock or temporary file is opened. + +Reads refresh from disk only after validating a real private parent directory and the credential file metadata. On POSIX the parent must be exactly `0700` and the regular file exactly `0600`; the file is opened with `O_NOFOLLOW`, when available, checked with `fstat` against pre-open and post-open identity, read by descriptor, and always closed. Windows omits unsupported no-follow flags but retains descriptor and identity validation. + +Every mutation first acquires an exclusive same-directory protocol gate and a canonical primary lock before persistence. Release atomically renames the canonical gate directory to a unique claim path, verifies ownership there, and removes only a matching claim; the canonical gate path is never deleted after a separate identity check. The primary lock remains canonical throughout gate claim validation. A second instance may acquire the now-empty gate, but its primary acquisition reports busy and its own gate cleanup does not touch the first claim. A foreign gate claim is preserved, and exclusive canonical `mkdir` supplies a nonsecret blocker; `EEXIST` proves another blocker or owner already occupies the path. Primary release is permitted only after gate ownership or canonical blocker state is proven; otherwise the primary lock path is retained. Primary cleanup then claims the closed lock before reading its token. A foreign or permanent primary claim is preserved and linked back to the canonical path, or a nonsecret canonical blocker is created. Fresh instances therefore report busy instead of writing through degraded state. + +The native adapter lazily loads the addon during construction and stores the same opaque reference under service `org.cluic.codex-remote-proxy`. Backend selection defaults to native. Explicit file selection requires `fallbackConsent === true`; consent permits construction-time file selection only when native cannot be created before any credential operation. Once native is selected, backend-unavailable reads and mutations fail without file replay. A construction-time fallback exposes backend label `file`; callers must reuse that explicit label after restart to remain in the same namespace. Task 4 performs no automatic credential migration. + ## RuntimeState Runtime state includes supervisor PID, worker PID, worker status, snapshot generation, start timestamps, restart count, and the last sanitized error. It is observational and can be reconstructed. @@ -56,7 +79,7 @@ Activity events record timestamp, category, action, provider ID, result, stable ## Relationships - One registry has zero or one active provider. -- Each provider has exactly one credential reference after it is saved. +- Each provider has exactly one credential reference after it is saved; public provider projections omit that reference. - A worker snapshot references one provider and one resolved credential for its process lifetime. ## Lifecycle and Deletion diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md index d2c5b51..d0b8444 100644 --- a/docs/PERMISSIONS.md +++ b/docs/PERMISSIONS.md @@ -29,6 +29,12 @@ Record provider creation/update/deletion, test result, activation, Codex bootstr ## Security Requirements - Prefer native credential stores. +- Native credential entries use service `org.cluic.codex-remote-proxy` and opaque credential references; the addon loads only during adapter construction, and adapters expose no enumeration API. +- File credential fallback must never be selected without explicit consent. +- Explicit consent permits file selection only when native construction fails before any credential operation; once native is selected, operation failures are returned without replaying reads or mutations into file storage. +- File fallback requires a real `0700` parent and regular `0600` file on POSIX, opens with no-follow protection where supported, verifies descriptor identity, and reads secret bytes only through that descriptor. +- File fallback holds an exclusive protocol gate and canonical primary lock across the complete mutation. Release atomically claims the canonical gate to a unique path and deletes only an ownership-verified claim, never the canonical path. The primary lock remains canonical until gate ownership or an exclusive canonical `mkdir`/`EEXIST` blocker is proven; if neither can be proven, the primary path is retained. A competing instance may briefly own the canonical gate during validation, but its primary acquisition must report busy and its cleanup must not touch the original claim. +- A secret-bearing temporary file that cannot be removed after bounded retries degrades the instance, reports `CREDENTIAL_STORE_TEMP_DEGRADED`, and blocks later mutations before they open a new lock. - Set metadata, token, state, and fallback secret files to `0600` where supported. - Never accept sensitive keys inside `extraHeaders`; use the credential field. - Redact authorization, cookie, token, secret, and API-key headers and fields. @@ -36,6 +42,10 @@ Record provider creation/update/deletion, test result, activation, Codex bootstr - Validate URL scheme and block accidental credential forwarding to an unvalidated target. - Treat diagnostics and capture content as sensitive local data. +## Credential Verification Boundary + +Task 4 unit tests inject a native entry loader and never invoke the default addon loader or construct or query a real Keychain, Credential Manager, or Linux secret-service entry. Real native-backend verification remains an L3 platform gate on every supported system, including Windows and Linux. + ## Security Risks Credential migration, localhost browser attacks, log leakage, symlink/path attacks, and worker IPC spoofing make the first implementation an L3 merge-risk change. diff --git a/docs/STATUS.md b/docs/STATUS.md index dbbfb70..366ea48 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 3 are complete and Task 4 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 4 are complete and Task 5 is next. ## Done @@ -21,14 +21,15 @@ V1 implementation: provider lifecycle end to end. - Stabilized capture watcher tests with condition-based waits and assertion-safe cleanup on Node 22.19 and Node 24.2. - Completed Task 2 shared path and public-error contracts plus line-ending-preserving, lock-serialized, idempotent and atomic Codex bootstrap behavior, covered by 15/15 focused and 27/27 full Node 22.19 tests. - Completed Task 3 strict provider validation and lock-serialized atomic schema-version-2 registry behavior, covered by 23/23 focused and 50/50 full Node 22.19 tests. +- Completed Task 4 native and explicit-consent file credential adapters, including construction-only fallback without operation replay, explicit file-label restart continuity, descriptor-safe reads, degraded temp cleanup, canonical lock restoration, and primary-blocked claim-before-delete gate release, covered by 41/41 focused, 64/64 combined credential/provider, and 91/91 full Node 22.19 tests; syntax checking covers 14 source files. ## Blocked -- No current blocker is recorded for Task 4. +- No current blocker is recorded for Task 5. ## Next -1. Execute Task 4: add native and explicit-fallback credential stores. +1. Execute Task 5: make proxy settings snapshot-based. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks @@ -36,3 +37,4 @@ V1 implementation: provider lifecycle end to end. - Future V1 implementation is L3 because it handles credentials, local browser security, Codex configuration, and process lifecycle. - Cross-platform credential APIs and restart semantics require real macOS and Windows verification. - Provider-registry atomic rename and permission semantics remain unverified on real Windows and Linux hosts. +- Task 4 tests inject the native loader and never invoke the real addon loader or touch an OS credential store; native verification remains L3 on every supported system, including Windows and Linux. diff --git a/docs/TESTING.md b/docs/TESTING.md index c84e9e8..54a673c 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -11,7 +11,7 @@ ## Required and Mocked Services - Deterministic local mock upstreams for JSON responses, SSE, timeouts, TLS errors, 401, 404, compressed requests, and disconnects. -- Native credential stores are exercised in platform integration jobs; unit tests use an in-memory adapter. +- Native credential stores remain a platform integration target; Task 4 unit tests use injected entry loaders and an in-memory adapter without invoking the real addon loader or constructing a native entry. - No real API key is required in CI. ## Current Commands @@ -21,11 +21,13 @@ - Top-level unit suite: `cd node && npm run test:unit` - Task 2 focused suite: `cd node && node --test test/codex-config.test.mjs` - Task 3 focused suite: `cd node && node --test test/provider-registry.test.mjs` +- Task 4 focused suite: `cd node && node --test test/credential-store.test.mjs` +- Task 4 combined credential/provider suite: `cd node && node --test test/credential-store.test.mjs test/provider-registry.test.mjs` - Runtime audit: `cd node && npm audit --omit=dev` `npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. On Node 22.19, the Task 2 focused suite passes 15/15 tests. Its coverage verifies OpenAI provider creation and update, custom-provider and CRLF preservation, byte idempotency, one-time adjacent backup, exclusive same-timestamp backup collision handling, CRP lock contention, external source-change rejection, atomic mode-preserving replacement, deterministic rename-failure cleanup and original preservation, all nine injected-home paths, safe public error serialization, `start`/`install`/`setup` JSON and managed-state backup propagation, and accurate guide backup semantics. -The Node 22.19 Task 3 focused suite passes 23/23 tests and the current full suite passes 50/50 tests; `npm run lint` syntax-checks 11 source files. Task 3 coverage includes normalized CRUD, multi-instance lock serialization and read refresh, stale uniqueness rejection, immutable fields, canonical safe URLs, header and auth-scheme validation, model policy defaults, test-state invalidation/reset, timestamp ordering, activation and deletion, strict reload rejection, lazy first persistence, `0600` lock and file modes, same-directory fsync/rename ordering, validation and rename rollback, transient and permanent lock cleanup, primary-error preservation, foreign-lock and temporary-file preservation, defensive copies, and exact public projection. Real Windows and Linux rename and permission semantics remain unverified. +The Node 22.19 Task 3 focused suite passes 23/23 tests. The Task 4 credential suite passes 41/41, the combined credential/provider suite passes 64/64, and the current full suite passes 91/91; `npm run lint` syntax-checks 14 source files. Task 4 coverage includes the shared async adapter contract, lazy native-loader failure, construction-only fallback, no operation replay for native get/set/has/delete outages, explicit file-label restart continuity without migration, exact schema and string persistence, reload and refreshed reads, no enumeration API, two-instance lost-update prevention, strict parent/file modes, POSIX no-follow and simulated-Windows descriptor identity, symlink-swap rejection before byte reads, fsync/rename order, rollback cleanup, bounded secret-temp cleanup and permanent uncommitted degradation, gate-covered atomic lock claims, preexisting foreign-gate preservation, claim-before-delete gate release under an immediate foreign replacement, canonical blocker restoration, synchronous second-instance rejection during gate claim validation, normal claim cleanup and subsequent mutation, fresh-instance busy behavior, permanent committed lock degradation, input validation, and public provider masking. Tests never invoke the default native loader or construct or query a real native credential entry. Real native-backend verification remains L3 on every supported system, including Windows and Linux; file permission and rename semantics on Windows and Linux also remain unverified. `npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. @@ -37,6 +39,15 @@ The Node 22.19 Task 3 focused suite passes 23/23 tests and the current full suit - Registry mutations must reload while holding the registry lock before replacing state. - Lock cleanup must never mask a primary registry error or make a durable mutation appear retryable. - A recorded residual registry lock must never be inspected or removed automatically. +- File credential fallback must never be selected without explicit consent. +- Native operation failures must never be replayed into the independent file credential namespace. +- A construction-time file fallback label must be explicitly reused across restart; Task 4 performs no credential migration. +- Credential-store tests must inject native entry loaders and must never invoke the default loader or access a developer's real OS credential store. +- Secret files must be read from a validated descriptor, never by path after a metadata check. +- Secret-temp cleanup failure must degrade the instance instead of permitting another mutation. +- Credential mutation must remain gate-protected, and the canonical primary lock must cover gate claim validation. +- Gate release must atomically claim the canonical path and delete only an ownership-verified claim; it must never delete the canonical path after a separate identity check. +- Primary lock release must follow gate claim validation or proven blocker restoration; an uncertain gate state must retain the primary path. ## Test Matrix diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index 72a00b3..50c60da 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -398,22 +398,25 @@ git commit -m "feat: add atomic provider registry" - Create: `node/src/credentials/credential-store.mjs` - Create: `node/test/credential-store.test.mjs` -- [ ] **Step 1: Write one shared adapter contract test** +- [x] **Step 1: Write one shared adapter contract test** Run the same assertions against an in-memory test double and `FileCredentialStore`: ```js -await store.set("provider-1", "sk-test-secret"); -assert.equal(await store.has("provider-1"), true); -assert.equal(await store.get("provider-1"), "sk-test-secret"); -await store.delete("provider-1"); -assert.equal(await store.has("provider-1"), false); -await assert.rejects(() => store.get("provider-1"), /CREDENTIAL_NOT_FOUND/); +const ref = `provider-${randomUUID()}`; +const secret = ["test", "credential", randomUUID()].join("-"); +await store.set(ref, secret); +assert.equal(await store.has(ref), true); +assert.equal(await store.get(ref), secret); +assert.equal(await store.delete(ref), true); +assert.equal(await store.has(ref), false); +await assert.rejects(() => store.get(ref), (error) => error.code === "CREDENTIAL_NOT_FOUND"); +assert.equal(await store.delete(ref), false); ``` Assert the fallback file is `0600`, stores no provider metadata, survives reload, and never returns all secrets through a list method. -- [ ] **Step 2: Verify the contract tests fail** +- [x] **Step 2: Verify the contract tests fail** ```bash cd node @@ -422,36 +425,39 @@ node --test test/credential-store.test.mjs Expected: FAIL because credential modules do not exist. -- [ ] **Step 3: Implement the adapters** +- [x] **Step 3: Implement the adapters** -Use `@napi-rs/keyring` exactly as follows in `native-keyring.mjs`: +Load `@napi-rs/keyring` lazily during adapter construction, use service `org.cluic.codex-remote-proxy`, and allow an injected entry loader or factory for tests: ```js -import { Entry } from "@napi-rs/keyring"; +import { createRequire } from "node:module"; +const require = createRequire(import.meta.url); const SERVICE = "org.cluic.codex-remote-proxy"; +function loadEntry() { + return require("@napi-rs/keyring").Entry; +} + export class NativeKeyringStore { - async set(ref, secret) { new Entry(SERVICE, ref).setPassword(secret); } - async get(ref) { - const password = new Entry(SERVICE, ref).getPassword(); - if (!password) throw new Error("CREDENTIAL_NOT_FOUND"); - return password; + constructor({ entryLoader = loadEntry, entryFactory } = {}) { + const EntryClass = entryFactory === undefined ? entryLoader() : null; + this.entryFactory = entryFactory ?? ((service, ref) => new EntryClass(service, ref)); } async has(ref) { - try { return Boolean(await this.get(ref)); } catch { return false; } - } - async delete(ref) { - try { new Entry(SERVICE, ref).deletePassword(); } catch { return false; } - return true; + try { await this.get(ref); return true; } + catch (error) { + if (error.code === "CREDENTIAL_NOT_FOUND") return false; + throw error; + } } } ``` -`FileCredentialStore` persists `{ "schemaVersion": 1, "credentials": { "provider-1": "secret" } }` atomically with `0600` mode. `createCredentialStore({ backend, fallbackConsent, paths })` must never silently choose file storage; native failure returns `CREDENTIAL_BACKEND_UNAVAILABLE` unless `fallbackConsent === true`. +The native methods validate inputs, wrap synchronous entry calls in asynchronous methods, map missing passwords to `CREDENTIAL_NOT_FOUND`, map other native failures to `CREDENTIAL_BACKEND_UNAVAILABLE`, and keep causes internal. `FileCredentialStore` persists `{ "schemaVersion": 1, "credentials": { "provider-1": "secret" } }` atomically with `0600` mode, reads through a validated descriptor, degrades on permanent secret-temp cleanup failure, and holds a protocol gate plus canonical primary lock across mutation. Gate release atomically renames canonical state to a unique claim and deletes only a verified owned claim; the primary lock remains until ownership or a foreign-claim blocker is proven. `createCredentialStore({ backend, fallbackConsent, paths })` must never silently choose file storage; explicit consent permits construction-time fallback only before any credential operation. Selected native operations are never replayed, and a selected file label must be explicitly reused after restart without implicit migration. -- [ ] **Step 4: Verify no secret is exposed by public helpers** +- [x] **Step 4: Verify no secret is exposed by public helpers** -Add assertions that `JSON.stringify(toPublicProvider(profile, true))` contains neither `credentialRef` nor `sk-test-secret`, then run: +Add assertions that `JSON.stringify(toPublicProvider(profile, true))` contains neither `credentialRef` nor the runtime-generated secret, then run: ```bash cd node @@ -460,7 +466,9 @@ node --test test/credential-store.test.mjs test/provider-registry.test.mjs Expected: all focused tests pass. -- [ ] **Step 5: Commit** +Actual Node 22.19 Task 4 verification: the credential suite passes 41/41, the combined credential/provider suite passes 64/64, the full suite passes 91/91, and the portable syntax gate checks 14 source files. Native tests inject the entry loader and never invoke the default addon loader or access the real OS credential store. + +- [x] **Step 5: Commit** ```bash git add node/src/credentials node/test/credential-store.test.mjs diff --git a/node/src/credentials/credential-store.mjs b/node/src/credentials/credential-store.mjs new file mode 100644 index 0000000..b628d02 --- /dev/null +++ b/node/src/credentials/credential-store.mjs @@ -0,0 +1,71 @@ +import { FileCredentialStore } from "./file-credential-store.mjs"; +import { NativeKeyringStore } from "./native-keyring.mjs"; +import { getPaths } from "../shared/paths.mjs"; +import { CrpError } from "../shared/errors.mjs"; + +function backendInvalid() { + return new CrpError( + "CREDENTIAL_BACKEND_INVALID", + "The credential backend is invalid.", + "Choose the native or file credential backend.", + { status: 400 } + ); +} + +function fallbackConsentRequired() { + return new CrpError( + "CREDENTIAL_FALLBACK_CONSENT_REQUIRED", + "File credential storage requires explicit consent.", + "Confirm file fallback storage before trying again.", + { status: 400 } + ); +} + +function backendUnavailable(cause) { + return new CrpError( + "CREDENTIAL_BACKEND_UNAVAILABLE", + "The credential backend is unavailable.", + "Check the native credential service or explicitly consent to file fallback storage.", + { status: 500, cause } + ); +} + +function asBackendUnavailable(error) { + if (error instanceof CrpError && error.code === "CREDENTIAL_BACKEND_UNAVAILABLE") { + return error; + } + return backendUnavailable(error); +} + +function createFileStore(paths, fileStoreFactory) { + try { + return fileStoreFactory({ path: paths.secretFallbackPath }); + } catch (error) { + if (error instanceof CrpError) throw error; + throw backendUnavailable(error); + } +} + +export function createCredentialStore({ + backend = "native", + fallbackConsent = false, + paths = getPaths(), + nativeStoreFactory = () => new NativeKeyringStore(), + fileStoreFactory = (options) => new FileCredentialStore(options) +} = {}) { + if (backend !== "native" && backend !== "file") throw backendInvalid(); + + if (backend === "file") { + if (fallbackConsent !== true) throw fallbackConsentRequired(); + return createFileStore(paths, fileStoreFactory); + } + + let nativeStore; + try { + nativeStore = nativeStoreFactory(); + } catch (error) { + if (fallbackConsent !== true) throw asBackendUnavailable(error); + return createFileStore(paths, fileStoreFactory); + } + return nativeStore; +} diff --git a/node/src/credentials/file-credential-store.mjs b/node/src/credentials/file-credential-store.mjs new file mode 100644 index 0000000..48dee06 --- /dev/null +++ b/node/src/credentials/file-credential-store.mjs @@ -0,0 +1,796 @@ +import { randomUUID } from "node:crypto"; +import { + chmodSync, + closeSync, + constants as fsConstants, + existsSync, + fstatSync, + fsyncSync, + linkSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmdirSync, + rmSync, + writeFileSync +} from "node:fs"; +import { basename, dirname, join } from "node:path"; + +import { CrpError } from "../shared/errors.mjs"; + +const DOCUMENT_FIELDS = new Set(["schemaVersion", "credentials"]); +const FORBIDDEN_REFS = new Set(["__proto__", "constructor", "prototype"]); +const REF_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$/; +const LOCK_CLEANUP_ATTEMPTS = 2; +const DEFAULT_FILE_OPERATIONS = { + chmodSync, + closeSync, + existsSync, + fstatSync, + fsyncSync, + linkSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmdirSync, + rmSync, + writeFileSync +}; + +function emptyDocument() { + return { schemaVersion: 1, credentials: {} }; +} + +function clone(value) { + return structuredClone(value); +} + +function isPlainObject(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return false; + } + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function isValidRef(ref) { + return typeof ref === "string" + && REF_PATTERN.test(ref) + && !FORBIDDEN_REFS.has(ref); +} + +function credentialInputInvalid() { + return new CrpError( + "CREDENTIAL_INPUT_INVALID", + "The credential input is invalid.", + "Use a valid credential reference and a non-empty secret.", + { status: 400 } + ); +} + +function credentialNotFound() { + return new CrpError( + "CREDENTIAL_NOT_FOUND", + "The credential does not exist.", + "Save the provider credential and try again.", + { status: 404 } + ); +} + +function credentialFileInvalid(cause) { + return new CrpError( + "CREDENTIAL_FILE_INVALID", + "The credential fallback file is invalid.", + "Restore a valid credential fallback file or remove it after making a backup.", + { status: 500, cause } + ); +} + +function credentialFileInsecure(cause) { + return new CrpError( + "CREDENTIAL_FILE_INSECURE", + "The credential fallback file is not secure.", + "Replace it with a regular private file and try again.", + { status: 500, cause } + ); +} + +function backendUnavailable(cause) { + return new CrpError( + "CREDENTIAL_BACKEND_UNAVAILABLE", + "The credential backend is unavailable.", + "Check local file permissions and try again.", + { status: 500, cause } + ); +} + +function credentialStoreBusy(cause) { + return new CrpError( + "CREDENTIAL_STORE_BUSY", + "The credential fallback is already being updated.", + "Wait for the current credential update to finish and try again.", + { status: 409, cause } + ); +} + +function committedLockDegraded() { + return new CrpError( + "CREDENTIAL_STORE_COMMITTED_LOCK_DEGRADED", + "The credential change was saved, but its lock could not be fully released.", + "Stop CRP, explicitly repair the residual credential lock, then restart CRP.", + { status: 500, details: { committed: true } } + ); +} + +function credentialLockDegraded() { + return new CrpError( + "CREDENTIAL_STORE_LOCK_DEGRADED", + "The credential fallback lock could not be safely recovered.", + "Stop CRP, explicitly repair the residual credential lock, then restart CRP.", + { status: 500, details: { committed: false } } + ); +} + +function credentialTempDegraded(cause) { + return new CrpError( + "CREDENTIAL_STORE_TEMP_DEGRADED", + "A credential temporary file could not be safely removed.", + "Stop CRP, explicitly remove the residual credential temporary file, then restart CRP.", + { status: 500, details: { committed: false }, cause } + ); +} + +function assertRef(ref) { + if (!isValidRef(ref)) throw credentialInputInvalid(); +} + +function assertSecret(secret) { + if (typeof secret !== "string" || secret.length === 0) { + throw credentialInputInvalid(); + } +} + +function validateDocument(document) { + try { + if ( + !isPlainObject(document) + || Object.keys(document).length !== DOCUMENT_FIELDS.size + || !Object.keys(document).every((key) => DOCUMENT_FIELDS.has(key)) + || document.schemaVersion !== 1 + || !isPlainObject(document.credentials) + ) { + throw new Error("invalid credential document"); + } + for (const [ref, secret] of Object.entries(document.credentials)) { + if (!isValidRef(ref) || typeof secret !== "string" || secret.length === 0) { + throw new Error("invalid credential entry"); + } + } + return document; + } catch (error) { + if (error instanceof CrpError && error.code === "CREDENTIAL_FILE_INVALID") { + throw error; + } + throw credentialFileInvalid(error); + } +} + +function parseDocument(bytes) { + let document; + try { + document = JSON.parse(bytes); + } catch (error) { + throw credentialFileInvalid(error); + } + return validateDocument(document); +} + +function safeFileError(error) { + return error instanceof CrpError ? error : backendUnavailable(error); +} + +function sameFileIdentity(first, second) { + for (const field of ["dev", "ino"]) { + if (first[field] !== undefined && second[field] !== undefined) { + if (first[field] !== second[field]) return false; + } + } + return true; +} + +export class FileCredentialStore { + #document; + #degradedLock = null; + #degradedTemp = null; + + constructor({ path, fileOperations, platform = process.platform } = {}) { + if (typeof path !== "string" || path.length === 0) { + throw credentialInputInvalid(); + } + this.backend = "file"; + this.path = path; + this.lockPath = `${path}.crp.lock`; + this.gatePath = `${this.lockPath}.gate`; + this.platform = platform; + this.fileOperations = { ...DEFAULT_FILE_OPERATIONS, ...fileOperations }; + this.#document = this.#load(); + } + + #load() { + const parent = dirname(this.path); + const parentStats = this.#inspectParent(parent); + if (parentStats === null) return emptyDocument(); + + let pathStats; + try { + pathStats = this.fileOperations.lstatSync(this.path); + } catch (error) { + if (error?.code === "ENOENT") return emptyDocument(); + throw backendUnavailable(error); + } + this.#assertCredentialFile(pathStats); + + const noFollow = this.platform !== "win32" + && typeof fsConstants.O_NOFOLLOW === "number" + ? fsConstants.O_NOFOLLOW + : 0; + const flags = fsConstants.O_RDONLY | noFollow; + let fileDescriptor; + let bytes; + let primaryError; + try { + fileDescriptor = this.fileOperations.openSync(this.path, flags); + const openedStats = this.fileOperations.fstatSync(fileDescriptor); + this.#assertCredentialFile(openedStats); + if (!sameFileIdentity(pathStats, openedStats)) { + throw credentialFileInsecure(new Error("Credential file identity changed")); + } + const finalPathStats = this.fileOperations.lstatSync(this.path); + this.#assertCredentialFile(finalPathStats); + if (!sameFileIdentity(openedStats, finalPathStats)) { + throw credentialFileInsecure(new Error("Credential file identity changed")); + } + bytes = this.fileOperations.readFileSync(fileDescriptor, "utf8"); + } catch (error) { + primaryError = error?.code === "ELOOP" || error?.code === "ENOENT" + ? credentialFileInsecure(error) + : safeFileError(error); + } + let closeError; + if (fileDescriptor !== undefined) { + try { + this.fileOperations.closeSync(fileDescriptor); + } catch (error) { + closeError = error; + } + } + if (primaryError !== undefined) throw primaryError; + if (closeError !== undefined) throw backendUnavailable(closeError); + return parseDocument(bytes); + } + + #inspectParent(parent) { + let stats; + try { + stats = this.fileOperations.lstatSync(parent); + } catch (error) { + if (error?.code === "ENOENT") return null; + throw backendUnavailable(error); + } + if (stats.isSymbolicLink() || !stats.isDirectory()) { + throw credentialFileInsecure(new Error("Credential parent is not a real directory")); + } + if (this.platform !== "win32" && (stats.mode & 0o777) !== 0o700) { + throw credentialFileInsecure(new Error("Credential parent permissions are not private")); + } + return stats; + } + + #assertCredentialFile(stats) { + if (stats.isSymbolicLink?.() || !stats.isFile()) { + throw credentialFileInsecure(new Error("Credential path is not a regular file")); + } + if (this.platform !== "win32" && (stats.mode & 0o777) !== 0o600) { + throw credentialFileInsecure(new Error("Credential file permissions are not private")); + } + } + + #refresh() { + const document = this.#load(); + this.#document = document; + return document; + } + + #prepareParent() { + const parent = dirname(this.path); + if (this.#inspectParent(parent) !== null) return; + const created = this.fileOperations.mkdirSync(parent, { + recursive: true, + mode: 0o700 + }); + if (this.platform !== "win32" && created !== undefined) { + this.fileOperations.chmodSync(parent, 0o700); + } + this.#inspectParent(parent); + } + + #acquireGate() { + try { + this.fileOperations.mkdirSync(this.gatePath, { mode: 0o700 }); + } catch (error) { + if (error?.code === "EEXIST") throw credentialStoreBusy(error); + throw error; + } + + let stats; + try { + stats = this.fileOperations.lstatSync(this.gatePath); + } catch { + this.#degradedLock = { path: this.gatePath }; + throw credentialLockDegraded(); + } + if ( + stats.isSymbolicLink() + || !stats.isDirectory() + || (this.platform !== "win32" && (stats.mode & 0o777) !== 0o700) + ) { + this.#degradedLock = { path: this.gatePath }; + throw credentialLockDegraded(); + } + return { active: true, stats }; + } + + #releaseGate(gate) { + if (!gate?.active) { + return { + ok: true, + primaryReleaseSafe: true, + residualGate: false, + foreign: false, + path: null + }; + } + gate.active = false; + const claimPath = join( + dirname(this.gatePath), + `.${basename(this.gatePath)}.${process.pid}.${randomUUID()}.claim` + ); + try { + this.fileOperations.renameSync(this.gatePath, claimPath); + } catch (error) { + const blocked = this.#ensureGateBlocked(); + return { + ok: false, + primaryReleaseSafe: blocked, + residualGate: true, + foreign: false, + path: this.gatePath + }; + } + + let claimed; + try { + claimed = this.fileOperations.lstatSync(claimPath); + } catch { + const blocked = this.#ensureGateBlocked(); + return { + ok: false, + primaryReleaseSafe: blocked, + residualGate: true, + foreign: false, + path: blocked ? this.gatePath : claimPath + }; + } + if ( + claimed.isSymbolicLink() + || !claimed.isDirectory() + || !sameFileIdentity(gate.stats, claimed) + ) { + const blocked = this.#ensureGateBlocked(); + return { + ok: false, + primaryReleaseSafe: blocked, + residualGate: true, + foreign: true, + path: claimPath + }; + } + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + try { + this.fileOperations.rmdirSync(claimPath); + return { + ok: true, + primaryReleaseSafe: true, + residualGate: false, + foreign: false, + path: null + }; + } catch (error) { + if (error?.code === "ENOENT") { + return { + ok: true, + primaryReleaseSafe: true, + residualGate: false, + foreign: false, + path: null + }; + } + } + } + return { + ok: false, + primaryReleaseSafe: true, + residualGate: this.fileOperations.existsSync(claimPath), + foreign: false, + path: claimPath + }; + } + + #ensureGateBlocked() { + try { + this.fileOperations.mkdirSync(this.gatePath, { mode: 0o700 }); + return true; + } catch (error) { + return error?.code === "EEXIST"; + } + } + + #acquireLock() { + if (this.#degradedTemp !== null) { + throw credentialTempDegraded(); + } + if (this.#degradedLock !== null) { + throw credentialLockDegraded(); + } + this.#prepareParent(); + const gate = this.#acquireGate(); + const token = `${randomUUID()}\n`; + let fileDescriptor; + try { + fileDescriptor = this.fileOperations.openSync(this.lockPath, "wx", 0o600); + } catch (error) { + const gateCleanup = this.#releaseGate(gate); + if (!gateCleanup.ok) { + this.#degradedLock = { path: gateCleanup.path }; + } + if (error?.code === "EEXIST") throw credentialStoreBusy(error); + throw error; + } + + const lock = { fileDescriptor, token, gate, active: true }; + try { + this.fileOperations.writeFileSync(fileDescriptor, token, "utf8"); + return lock; + } catch (error) { + const cleanup = this.#releaseLock(lock); + if (cleanup.residualLock) this.#degradedLock = { token }; + throw error; + } + } + + #closeLock(fileDescriptor) { + let error = null; + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + try { + this.fileOperations.closeSync(fileDescriptor); + return { closed: true, error: null }; + } catch (caught) { + if (attempt > 0 && caught?.code === "EBADF") { + return { closed: true, error: null }; + } + error = caught; + } + } + return { closed: false, error }; + } + + #removeOwnedLock(token) { + let releasePath; + let error = null; + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + releasePath = join( + dirname(this.lockPath), + `.${basename(this.lockPath)}.${process.pid}.${randomUUID()}.release` + ); + try { + this.fileOperations.renameSync(this.lockPath, releasePath); + error = null; + break; + } catch (caught) { + if (caught?.code === "ENOENT") { + return { + removed: true, + residualLock: false, + foreign: false, + error: null, + residualPath: null + }; + } + error = caught; + } + } + if (error !== null) { + let residualLock = true; + try { + residualLock = this.fileOperations.existsSync(this.lockPath); + } catch { + // Uncertain claim state is treated as a permanent residual lock. + } + return { + removed: false, + residualLock, + foreign: false, + error, + residualPath: residualLock ? this.lockPath : null + }; + } + + let currentToken; + try { + currentToken = this.fileOperations.readFileSync(releasePath, "utf8"); + } catch (caught) { + return { + removed: false, + residualLock: true, + foreign: false, + error: caught, + residualPath: releasePath + }; + } + if (currentToken !== token) { + return { + removed: false, + residualLock: true, + foreign: true, + error: null, + residualPath: releasePath + }; + } + + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + try { + this.fileOperations.rmSync(releasePath, { force: true }); + return { + removed: true, + residualLock: false, + foreign: false, + error: null, + residualPath: null + }; + } catch (caught) { + if (caught?.code === "ENOENT") { + return { + removed: true, + residualLock: false, + foreign: false, + error: null, + residualPath: null + }; + } + error = caught; + } + } + let residualLock = true; + try { + residualLock = this.fileOperations.existsSync(releasePath); + } catch { + // Uncertain ownership state is treated as a permanent residual lock. + } + return { + removed: false, + residualLock, + foreign: false, + error, + residualPath: residualLock ? releasePath : null + }; + } + + #canonicalState() { + try { + const stats = this.fileOperations.lstatSync(this.lockPath); + return { + known: true, + blocking: stats.isFile() ? stats.size > 0 : true + }; + } catch (error) { + if (error?.code === "ENOENT") return { known: true, blocking: false }; + return { known: false, blocking: false }; + } + } + + #ensureCanonicalBlocked(residualPath) { + const initial = this.#canonicalState(); + if (initial.blocking) return true; + if (!initial.known) return false; + + if (residualPath !== null && residualPath !== this.lockPath) { + try { + this.fileOperations.linkSync(residualPath, this.lockPath); + return this.#canonicalState().blocking; + } catch (error) { + if (error?.code === "EEXIST") { + return this.#canonicalState().blocking; + } + } + } + + const marker = `blocked-${randomUUID()}\n`; + let fileDescriptor; + try { + fileDescriptor = this.fileOperations.openSync(this.lockPath, "wx", 0o600); + this.fileOperations.writeFileSync(fileDescriptor, marker, "utf8"); + this.fileOperations.fsyncSync(fileDescriptor); + this.fileOperations.closeSync(fileDescriptor); + fileDescriptor = undefined; + return this.#canonicalState().blocking; + } catch (error) { + if (fileDescriptor !== undefined) { + try { + this.fileOperations.closeSync(fileDescriptor); + } catch { + // Retain the gate when canonical occupancy cannot be proven. + } + } + if (error?.code === "EEXIST") return this.#canonicalState().blocking; + return false; + } + } + + #releaseLock(lock) { + if (!lock?.active) { + return { ok: true, residualLock: false, foreignLock: false }; + } + lock.active = false; + const gate = this.#releaseGate(lock.gate); + const close = this.#closeLock(lock.fileDescriptor); + const removal = gate.primaryReleaseSafe + ? this.#removeOwnedLock(lock.token) + : { + removed: false, + residualLock: true, + foreign: false, + error: null, + residualPath: this.lockPath + }; + const canonicalBlocked = !removal.residualLock + || this.#ensureCanonicalBlocked(removal.residualPath); + return { + ok: close.closed && removal.removed && canonicalBlocked && gate.ok, + closeError: close.error, + removalError: removal.error, + residualLock: removal.residualLock || gate.residualGate, + foreignLock: removal.foreign || gate.foreign, + residualPath: gate.residualGate ? gate.path : removal.residualPath + }; + } + + #recordCleanupState(lock, cleanup) { + this.#degradedLock = cleanup.residualLock + ? { token: lock.token, path: cleanup.residualPath } + : null; + } + + #persist(document) { + const bytes = `${JSON.stringify(document, null, 2)}\n`; + const parent = dirname(this.path); + const tempPath = join( + parent, + `.${basename(this.path)}.${process.pid}.${randomUUID()}.tmp` + ); + let fileDescriptor; + try { + fileDescriptor = this.fileOperations.openSync(tempPath, "wx", 0o600); + this.fileOperations.writeFileSync(fileDescriptor, bytes, "utf8"); + this.fileOperations.fsyncSync(fileDescriptor); + this.fileOperations.closeSync(fileDescriptor); + fileDescriptor = undefined; + this.fileOperations.chmodSync(tempPath, 0o600); + this.fileOperations.renameSync(tempPath, this.path); + } catch (error) { + if (fileDescriptor !== undefined) { + this.#closeTemporary(fileDescriptor); + } + const cleanup = this.#removeTemporary(tempPath); + if (!cleanup.removed) { + this.#degradedTemp = { path: tempPath }; + throw credentialTempDegraded(error); + } + throw error; + } + } + + #closeTemporary(fileDescriptor) { + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + try { + this.fileOperations.closeSync(fileDescriptor); + return true; + } catch (error) { + if (attempt > 0 && error?.code === "EBADF") return true; + } + } + return false; + } + + #removeTemporary(tempPath) { + for (let attempt = 0; attempt < LOCK_CLEANUP_ATTEMPTS; attempt += 1) { + try { + this.fileOperations.rmSync(tempPath, { force: true }); + return { removed: true, residual: false }; + } catch (error) { + if (error?.code === "ENOENT") { + return { removed: true, residual: false }; + } + } + } + let residual = true; + try { + residual = this.fileOperations.existsSync(tempPath); + } catch { + // Uncertain cleanup state is treated as a residual secret-bearing file. + } + return { removed: !residual, residual }; + } + + #commit(mutator) { + let lock; + let result; + let primaryError; + let committed = false; + try { + lock = this.#acquireLock(); + const candidate = clone(this.#load()); + result = mutator(candidate); + validateDocument(candidate); + this.#persist(candidate); + this.#document = candidate; + committed = true; + } catch (error) { + primaryError = error; + } + + let cleanup = { ok: true, residualLock: false }; + if (lock !== undefined) { + cleanup = this.#releaseLock(lock); + this.#recordCleanupState(lock, cleanup); + } + if (primaryError !== undefined) throw safeFileError(primaryError); + if (!cleanup.ok) { + if (committed) throw committedLockDegraded(); + throw credentialLockDegraded(); + } + return result; + } + + async set(ref, secret) { + assertRef(ref); + assertSecret(secret); + this.#commit((document) => { + document.credentials[ref] = secret; + }); + } + + async get(ref) { + assertRef(ref); + const document = this.#refresh(); + if (!Object.hasOwn(document.credentials, ref)) throw credentialNotFound(); + return document.credentials[ref]; + } + + async has(ref) { + assertRef(ref); + return Object.hasOwn(this.#refresh().credentials, ref); + } + + async delete(ref) { + assertRef(ref); + return this.#commit((document) => { + if (!Object.hasOwn(document.credentials, ref)) return false; + delete document.credentials[ref]; + return true; + }); + } +} diff --git a/node/src/credentials/native-keyring.mjs b/node/src/credentials/native-keyring.mjs new file mode 100644 index 0000000..da895fa --- /dev/null +++ b/node/src/credentials/native-keyring.mjs @@ -0,0 +1,133 @@ +import { createRequire } from "node:module"; + +import { CrpError } from "../shared/errors.mjs"; + +const SERVICE = "org.cluic.codex-remote-proxy"; +const FORBIDDEN_REFS = new Set(["__proto__", "constructor", "prototype"]); +const REF_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$/; +const require = createRequire(import.meta.url); + +function loadEntry() { + const { Entry } = require("@napi-rs/keyring"); + if (typeof Entry !== "function") { + throw new TypeError("Native credential Entry is unavailable"); + } + return Entry; +} + +function credentialInputInvalid() { + return new CrpError( + "CREDENTIAL_INPUT_INVALID", + "The credential input is invalid.", + "Use a valid credential reference and a non-empty secret.", + { status: 400 } + ); +} + +function credentialNotFound() { + return new CrpError( + "CREDENTIAL_NOT_FOUND", + "The credential does not exist.", + "Save the provider credential and try again.", + { status: 404 } + ); +} + +function backendUnavailable(cause) { + return new CrpError( + "CREDENTIAL_BACKEND_UNAVAILABLE", + "The credential backend is unavailable.", + "Check the operating-system credential service and try again.", + { status: 500, cause } + ); +} + +function assertRef(ref) { + if ( + typeof ref !== "string" + || !REF_PATTERN.test(ref) + || FORBIDDEN_REFS.has(ref) + ) { + throw credentialInputInvalid(); + } +} + +function assertSecret(secret) { + if (typeof secret !== "string" || secret.length === 0) { + throw credentialInputInvalid(); + } +} + +export class NativeKeyringStore { + constructor({ entryLoader = loadEntry, entryFactory } = {}) { + this.backend = "native"; + if (entryFactory !== undefined) { + if (typeof entryFactory !== "function") { + throw backendUnavailable(new TypeError("Native entry factory is invalid")); + } + this.entryFactory = entryFactory; + return; + } + try { + const EntryClass = entryLoader(); + if (typeof EntryClass !== "function") { + throw new TypeError("Native credential Entry is unavailable"); + } + this.entryFactory = (service, ref) => new EntryClass(service, ref); + } catch (error) { + throw backendUnavailable(error); + } + } + + #entry(ref) { + return this.entryFactory(SERVICE, ref); + } + + async set(ref, secret) { + assertRef(ref); + assertSecret(secret); + try { + this.#entry(ref).setPassword(secret); + } catch (error) { + throw backendUnavailable(error); + } + } + + async get(ref) { + assertRef(ref); + let password; + try { + password = this.#entry(ref).getPassword(); + } catch (error) { + throw backendUnavailable(error); + } + if (password === null || password === undefined || password === "") { + throw credentialNotFound(); + } + if (typeof password !== "string") { + throw backendUnavailable(new TypeError("Native credential was not a string")); + } + return password; + } + + async has(ref) { + try { + await this.get(ref); + return true; + } catch (error) { + if (error instanceof CrpError && error.code === "CREDENTIAL_NOT_FOUND") { + return false; + } + throw error; + } + } + + async delete(ref) { + assertRef(ref); + try { + return this.#entry(ref).deletePassword() !== false; + } catch (error) { + throw backendUnavailable(error); + } + } +} diff --git a/node/test/credential-store.test.mjs b/node/test/credential-store.test.mjs new file mode 100644 index 0000000..5b1fb0e --- /dev/null +++ b/node/test/credential-store.test.mjs @@ -0,0 +1,1347 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { randomUUID } from "node:crypto"; +import * as realFileOperations from "node:fs"; +import { + chmodSync, + existsSync, + lstatSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + symlinkSync, + writeFileSync +} from "node:fs"; +import os from "node:os"; +import { basename, dirname, join } from "node:path"; + +import { FileCredentialStore } from "../src/credentials/file-credential-store.mjs"; +import { NativeKeyringStore } from "../src/credentials/native-keyring.mjs"; +import { createCredentialStore } from "../src/credentials/credential-store.mjs"; +import { + normalizeProvider, + toPublicProvider +} from "../src/providers/provider-schema.mjs"; +import { CrpError, toPublicError } from "../src/shared/errors.mjs"; + +const NATIVE_SERVICE = "org.cluic.codex-remote-proxy"; + +function makeRef(label = "provider") { + return `${label}-${randomUUID()}`; +} + +function makeSecret() { + return ["test", "credential", randomUUID()].join("-"); +} + +function makeFileError(message, code) { + const error = new Error(message); + error.code = code; + return error; +} + +function assertCrpError(expectedCode, expectedStatus) { + return (error) => { + assert.ok(error instanceof CrpError); + assert.equal(error.code, expectedCode); + assert.equal(error.status, expectedStatus); + assert.equal(typeof error.action, "string"); + assert.notEqual(error.action.length, 0); + return true; + }; +} + +function assertSafePublicError(error, prohibited) { + const serialized = JSON.stringify(toPublicError(error, "request-test")); + for (const value of prohibited) { + assert.equal(serialized.includes(value), false); + } +} + +function makeTempCredentialPath(t, prefix = "crp-credentials-") { + const tempRoot = mkdtempSync(join(os.tmpdir(), prefix)); + const parent = join(tempRoot, "private"); + const path = join(parent, "secrets.json"); + t.after(() => rmSync(tempRoot, { recursive: true, force: true })); + return { tempRoot, parent, path, lockPath: `${path}.crp.lock` }; +} + +function writeCredentialBytes(path, bytes, mode = 0o600) { + mkdirSync(dirname(path), { recursive: true, mode: 0o700 }); + writeFileSync(path, bytes, { mode }); + chmodSync(path, mode); +} + +function credentialDocument(credentials = {}) { + return { schemaVersion: 1, credentials }; +} + +function listTempFiles(path) { + if (!existsSync(dirname(path))) return []; + const fileName = basename(path); + return readdirSync(dirname(path)).filter((name) => ( + name.startsWith(`.${fileName}.`) && name.endsWith(".tmp") + )); +} + +function isLockReleasePath(filePath, lockPath) { + if (typeof filePath !== "string" || dirname(filePath) !== dirname(lockPath)) { + return false; + } + const name = basename(filePath); + return name.startsWith(`.${basename(lockPath)}.`) && name.endsWith(".release"); +} + +function listLockReleaseFiles(lockPath) { + if (!existsSync(dirname(lockPath))) return []; + return readdirSync(dirname(lockPath)) + .map((name) => join(dirname(lockPath), name)) + .filter((filePath) => isLockReleasePath(filePath, lockPath)); +} + +function isGateClaimPath(filePath, gatePath) { + if (typeof filePath !== "string" || dirname(filePath) !== dirname(gatePath)) { + return false; + } + const name = basename(filePath); + return name.startsWith(`.${basename(gatePath)}.`) && name.endsWith(".claim"); +} + +function listGateClaimPaths(gatePath) { + if (!existsSync(dirname(gatePath))) return []; + return readdirSync(dirname(gatePath)) + .map((name) => join(dirname(gatePath), name)) + .filter((filePath) => isGateClaimPath(filePath, gatePath)); +} + +class MemoryCredentialStore { + constructor() { + this.values = new Map(); + this.backend = "memory"; + } + + async set(ref, secret) { + this.values.set(ref, secret); + } + + async get(ref) { + if (!this.values.has(ref)) { + throw new CrpError( + "CREDENTIAL_NOT_FOUND", + "The credential does not exist.", + "Save the provider credential and try again.", + { status: 404 } + ); + } + return this.values.get(ref); + } + + async has(ref) { + return this.values.has(ref); + } + + async delete(ref) { + return this.values.delete(ref); + } +} + +async function assertCredentialContract(store, { afterSet } = {}) { + const ref = makeRef(); + const secret = makeSecret(); + + assert.equal(typeof store.list, "undefined"); + await store.set(ref, secret); + assert.equal(await store.has(ref), true); + assert.equal(await store.get(ref), secret); + await afterSet?.({ ref, secret }); + assert.equal(await store.delete(ref), true); + assert.equal(await store.has(ref), false); + await assert.rejects( + () => store.get(ref), + assertCrpError("CREDENTIAL_NOT_FOUND", 404) + ); + assert.equal(await store.delete(ref), false); +} + +function createFakeEntryClass() { + const values = new Map(); + const constructions = []; + class FakeEntry { + constructor(service, ref) { + constructions.push({ service, ref }); + this.ref = ref; + } + + setPassword(secret) { + values.set(this.ref, secret); + } + + getPassword() { + return values.get(this.ref) ?? null; + } + + deletePassword() { + return values.delete(this.ref); + } + } + return { FakeEntry, values, constructions }; +} + +test("shared async credential contract works for an in-memory test double", async () => { + await assertCredentialContract(new MemoryCredentialStore()); +}); + +test("shared async credential contract persists the exact file schema and reloads", async (t) => { + const { parent, path } = makeTempCredentialPath(t); + const store = new FileCredentialStore({ path }); + + await assertCredentialContract(store, { + afterSet: async ({ ref, secret }) => { + assert.deepEqual( + JSON.parse(readFileSync(path, "utf8")), + credentialDocument({ [ref]: secret }) + ); + assert.equal(readFileSync(path, "utf8").includes("providerMetadata"), false); + assert.equal(readFileSync(path, "utf8").includes("baseUrl"), false); + const reloaded = new FileCredentialStore({ path }); + assert.equal(await reloaded.get(ref), secret); + assert.equal(typeof reloaded.list, "undefined"); + } + }); + + if (process.platform !== "win32") { + assert.equal(lstatSync(parent).mode & 0o777, 0o700); + assert.equal(lstatSync(path).mode & 0o777, 0o600); + } +}); + +test("file mutations avoid lost updates and reads refresh committed disk state", async (t) => { + const { path } = makeTempCredentialPath(t, "crp-credentials-multi-"); + const first = new FileCredentialStore({ path }); + const second = new FileCredentialStore({ path }); + const firstRef = makeRef("first"); + const secondRef = makeRef("second"); + const firstSecret = makeSecret(); + const secondSecret = makeSecret(); + + await first.set(firstRef, firstSecret); + await second.set(secondRef, secondSecret); + + assert.equal(await first.get(secondRef), secondSecret); + assert.equal(await second.get(firstRef), firstSecret); + assert.deepEqual( + JSON.parse(readFileSync(path, "utf8")), + credentialDocument({ + [firstRef]: firstSecret, + [secondRef]: secondSecret + }) + ); + + await first.delete(firstRef); + assert.equal(await second.has(firstRef), false); + assert.equal(await second.get(secondRef), secondSecret); +}); + +test("file persistence uses exclusive 0600 lock and temp files in atomic order", async (t) => { + const { parent, path, lockPath } = makeTempCredentialPath(t, "crp-credentials-atomic-"); + const gatePath = `${lockPath}.gate`; + const events = []; + const descriptorPaths = new Map(); + const fileOperations = { + ...realFileOperations, + openSync(filePath, flags, mode) { + const descriptor = realFileOperations.openSync(filePath, flags, mode); + descriptorPaths.set(descriptor, filePath); + events.push({ operation: "open", path: filePath, flags, mode }); + return descriptor; + }, + fsyncSync(descriptor) { + events.push({ operation: "fsync", path: descriptorPaths.get(descriptor) }); + return realFileOperations.fsyncSync(descriptor); + }, + closeSync(descriptor) { + events.push({ operation: "close", path: descriptorPaths.get(descriptor) }); + return realFileOperations.closeSync(descriptor); + }, + chmodSync(filePath, mode) { + events.push({ operation: "chmod", path: filePath, mode }); + return realFileOperations.chmodSync(filePath, mode); + }, + renameSync(source, target) { + events.push({ operation: "rename", source, target }); + return realFileOperations.renameSync(source, target); + } + }; + const store = new FileCredentialStore({ path, fileOperations }); + + await store.set(makeRef(), makeSecret()); + + const lockOpen = events.find((event) => event.operation === "open" && event.path === lockPath); + const tempOpen = events.find((event) => ( + event.operation === "open" + && event.path !== lockPath + && event.path.endsWith(".tmp") + )); + assert.deepEqual( + { flags: lockOpen.flags, mode: lockOpen.mode }, + { flags: "wx", mode: 0o600 } + ); + assert.deepEqual( + { flags: tempOpen.flags, mode: tempOpen.mode }, + { flags: "wx", mode: 0o600 } + ); + assert.equal(dirname(tempOpen.path), dirname(path)); + + const fsyncIndex = events.findIndex((event) => ( + event.operation === "fsync" && event.path === tempOpen.path + )); + const closeIndex = events.findIndex((event) => ( + event.operation === "close" && event.path === tempOpen.path + )); + const chmodIndex = events.findIndex((event) => ( + event.operation === "chmod" && event.path === tempOpen.path && event.mode === 0o600 + )); + const renameIndex = events.findIndex((event) => ( + event.operation === "rename" + && event.source === tempOpen.path + && event.target === path + )); + assert.ok(fsyncIndex < closeIndex); + assert.ok(closeIndex < chmodIndex); + assert.ok(chmodIndex < renameIndex); + assert.equal(existsSync(lockPath), false); + assert.equal(existsSync(gatePath), false); + assert.deepEqual(listGateClaimPaths(gatePath), []); + assert.deepEqual(listTempFiles(path), []); + + await store.set(makeRef("later"), makeSecret()); + assert.equal(existsSync(lockPath), false); + assert.equal(existsSync(gatePath), false); + assert.deepEqual(listGateClaimPaths(gatePath), []); + + if (process.platform !== "win32") { + assert.equal(lstatSync(parent).mode & 0o777, 0o700); + assert.equal(lstatSync(path).mode & 0o777, 0o600); + } +}); + +test("malformed and schema-invalid files fail closed without overwrite", async (t) => { + const secret = makeSecret(); + const ref = makeRef(); + const cases = [ + ["malformed JSON", `{${secret}`], + ["unsupported schema", JSON.stringify({ schemaVersion: 2, credentials: {} })], + ["extra metadata", JSON.stringify({ + schemaVersion: 1, + credentials: {}, + providerMetadata: { ref } + })], + ["array credentials", JSON.stringify({ schemaVersion: 1, credentials: [] })], + ["unsafe reference", JSON.stringify(credentialDocument({ ["__proto__"]: secret }))], + ["empty secret", JSON.stringify(credentialDocument({ [ref]: "" }))], + ["non-string secret", JSON.stringify(credentialDocument({ [ref]: 42 }))] + ]; + + for (const [label, bytes] of cases) { + await t.test(label, async (nested) => { + const { path, lockPath } = makeTempCredentialPath(nested, "crp-credentials-invalid-"); + const store = new FileCredentialStore({ path }); + writeCredentialBytes(path, bytes); + + let caught; + try { + await store.set(makeRef("new"), makeSecret()); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_FILE_INVALID", 500)(caught); + assert.equal(readFileSync(path, "utf8"), bytes); + assert.equal(existsSync(lockPath), false); + assert.deepEqual(listTempFiles(path), []); + assertSafePublicError(caught, [secret, ref, bytes]); + }); + } +}); + +test("symbolic-link credential files are rejected before reading", (t) => { + if (process.platform === "win32") { + t.skip("Windows symlink creation requires platform privileges"); + return; + } + const { parent, path } = makeTempCredentialPath(t, "crp-credentials-symlink-"); + const target = join(dirname(parent), "target.json"); + const secret = makeSecret(); + const bytes = `${JSON.stringify(credentialDocument({ [makeRef()]: secret }))}\n`; + writeCredentialBytes(target, bytes); + mkdirSync(parent, { recursive: true, mode: 0o700 }); + symlinkSync(target, path); + + let caught; + try { + new FileCredentialStore({ path }); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_FILE_INSECURE", 500)(caught); + assert.equal(readFileSync(target, "utf8"), bytes); + assertSafePublicError(caught, [secret, target, path]); +}); + +test("group or other accessible credential files are rejected before reading", (t) => { + if (process.platform === "win32") { + t.skip("POSIX permission bits are not authoritative on Windows"); + return; + } + const { path } = makeTempCredentialPath(t, "crp-credentials-mode-"); + const secret = makeSecret(); + const bytes = `${JSON.stringify(credentialDocument({ [makeRef()]: secret }))}\n`; + writeCredentialBytes(path, bytes, 0o644); + + let caught; + try { + new FileCredentialStore({ path }); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_FILE_INSECURE", 500)(caught); + assertSafePublicError(caught, [secret, bytes, path]); +}); + +test("credential reads fail closed when the path is swapped between lstat and open", async (t) => { + const platforms = process.platform === "win32" + ? ["win32"] + : [process.platform, "win32"]; + + for (const platform of platforms) { + await t.test(`fd identity on ${platform}`, (nested) => { + const { tempRoot, path } = makeTempCredentialPath( + nested, + `crp-credentials-swap-${platform}-` + ); + const originalPath = join(tempRoot, "original.json"); + const targetPath = join(tempRoot, "target.json"); + const originalSecret = makeSecret(); + const targetSecret = makeSecret(); + writeCredentialBytes( + path, + `${JSON.stringify(credentialDocument({ [makeRef("original")]: originalSecret }))}\n` + ); + writeCredentialBytes( + targetPath, + `${JSON.stringify(credentialDocument({ [makeRef("target")]: targetSecret }))}\n` + ); + let swapped = false; + let pathReadAttempts = 0; + const fileOperations = { + ...realFileOperations, + openSync(filePath, flags, mode) { + if (filePath === path && typeof flags === "number" && !swapped) { + realFileOperations.renameSync(path, originalPath); + realFileOperations.symlinkSync(targetPath, path); + swapped = true; + } + return realFileOperations.openSync(filePath, flags, mode); + }, + readFileSync(fileOrDescriptor, options) { + if (fileOrDescriptor === path || fileOrDescriptor === targetPath) { + pathReadAttempts += 1; + } + return realFileOperations.readFileSync(fileOrDescriptor, options); + } + }; + + let caught; + try { + new FileCredentialStore({ path, platform, fileOperations }); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_FILE_INSECURE", 500)(caught); + assert.equal(swapped, true); + assert.equal(pathReadAttempts, 0); + assertSafePublicError(caught, [ + originalSecret, + targetSecret, + originalPath, + targetPath, + path + ]); + }); + } +}); + +test("POSIX parent and file modes are validated before reading secret bytes", async (t) => { + if (process.platform === "win32") { + t.skip("POSIX permission bits are not authoritative on Windows"); + return; + } + const cases = [ + { + label: "parent broader than 0700", + setup({ parent, path }, bytes) { + writeCredentialBytes(path, bytes, 0o600); + chmodSync(parent, 0o750); + } + }, + { + label: "file mode is not exactly 0600", + setup({ path }, bytes) { + writeCredentialBytes(path, bytes, 0o400); + } + }, + { + label: "parent is a symbolic link", + setup({ tempRoot, parent }, bytes) { + const realParent = join(tempRoot, "real-private"); + const realPath = join(realParent, "secrets.json"); + writeCredentialBytes(realPath, bytes, 0o600); + symlinkSync(realParent, parent); + } + } + ]; + + for (const { label, setup } of cases) { + await t.test(label, (nested) => { + const paths = makeTempCredentialPath(nested, "crp-credentials-parent-mode-"); + const secret = makeSecret(); + const bytes = `${JSON.stringify(credentialDocument({ [makeRef()]: secret }))}\n`; + setup(paths, bytes); + let pathReadAttempts = 0; + const fileOperations = { + ...realFileOperations, + readFileSync(fileOrDescriptor, options) { + if (fileOrDescriptor === paths.path) pathReadAttempts += 1; + return realFileOperations.readFileSync(fileOrDescriptor, options); + } + }; + + let caught; + try { + new FileCredentialStore({ path: paths.path, fileOperations }); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_FILE_INSECURE", 500)(caught); + assert.equal(pathReadAttempts, 0); + assertSafePublicError(caught, [secret, bytes, paths.path, paths.parent]); + }); + } +}); + +test("rename failure preserves committed bytes and cleans temporary state", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-rename-"); + const oldRef = makeRef("old"); + const oldSecret = makeSecret(); + const newRef = makeRef("new"); + const newSecret = makeSecret(); + const original = new FileCredentialStore({ path }); + await original.set(oldRef, oldSecret); + const before = readFileSync(path, "utf8"); + const failure = makeFileError(`${newRef}:${newSecret}`, "EIO"); + const store = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + renameSync(source, target) { + if (target === path) throw failure; + return realFileOperations.renameSync(source, target); + } + } + }); + + let caught; + try { + await store.set(newRef, newSecret); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_BACKEND_UNAVAILABLE", 500)(caught); + assert.equal(caught.cause, failure); + assert.equal(readFileSync(path, "utf8"), before); + assert.equal(await store.get(oldRef), oldSecret); + assert.equal(await store.has(newRef), false); + assert.equal(existsSync(lockPath), false); + assert.deepEqual(listTempFiles(path), []); + assertSafePublicError(caught, [newRef, newSecret, failure.message, before]); +}); + +test("permanent secret-temp cleanup failure degrades the instance before later opens", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-temp-degraded-"); + const ref = makeRef(); + const secret = makeSecret(); + const persistenceFailure = makeFileError(`${ref}:${secret}`, "EIO"); + const descriptorPaths = new Map(); + let openCalls = 0; + let tempRemovalAttempts = 0; + const store = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + openSync(filePath, flags, mode) { + openCalls += 1; + const descriptor = realFileOperations.openSync(filePath, flags, mode); + descriptorPaths.set(descriptor, filePath); + return descriptor; + }, + fsyncSync(descriptor) { + if (descriptorPaths.get(descriptor)?.endsWith(".tmp")) { + throw persistenceFailure; + } + return realFileOperations.fsyncSync(descriptor); + }, + rmSync(filePath, options) { + if (filePath.endsWith(".tmp")) { + tempRemovalAttempts += 1; + throw makeFileError("permanent temp removal", "EPERM"); + } + return realFileOperations.rmSync(filePath, options); + } + } + }); + + let caught; + try { + await store.set(ref, secret); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_STORE_TEMP_DEGRADED", 500)(caught); + assert.deepEqual(caught.details, { committed: false }); + assert.equal(caught.cause, persistenceFailure); + assert.match(caught.action, /explicitly remove/i); + assert.equal(tempRemovalAttempts, 2); + assert.equal(listTempFiles(path).length, 1); + assert.equal(existsSync(lockPath), false); + assertSafePublicError(caught, [ref, secret, persistenceFailure.message, path]); + const opensAfterFailure = openCalls; + + await assert.rejects( + () => store.set(makeRef("blocked"), makeSecret()), + assertCrpError("CREDENTIAL_STORE_TEMP_DEGRADED", 500) + ); + assert.equal(openCalls, opensAfterFailure); + assert.equal(tempRemovalAttempts, 2); + assert.equal(listTempFiles(path).length, 1); +}); + +test("one-shot secret-temp removal failure is retried without degrading later writes", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-temp-retry-"); + const failure = makeFileError("forced pre-rename failure", "EIO"); + const descriptorPaths = new Map(); + let persistenceFailures = 0; + let tempRemovalAttempts = 0; + const store = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + openSync(filePath, flags, mode) { + const descriptor = realFileOperations.openSync(filePath, flags, mode); + descriptorPaths.set(descriptor, filePath); + return descriptor; + }, + fsyncSync(descriptor) { + if ( + descriptorPaths.get(descriptor)?.endsWith(".tmp") + && persistenceFailures === 0 + ) { + persistenceFailures += 1; + throw failure; + } + return realFileOperations.fsyncSync(descriptor); + }, + rmSync(filePath, options) { + if (filePath.endsWith(".tmp")) { + tempRemovalAttempts += 1; + if (tempRemovalAttempts === 1) { + throw makeFileError("transient temp removal", "EBUSY"); + } + } + return realFileOperations.rmSync(filePath, options); + } + } + }); + + await assert.rejects( + () => store.set(makeRef("first"), makeSecret()), + (error) => { + assertCrpError("CREDENTIAL_BACKEND_UNAVAILABLE", 500)(error); + assert.equal(error.cause, failure); + return true; + } + ); + assert.equal(tempRemovalAttempts, 2); + assert.deepEqual(listTempFiles(path), []); + assert.equal(existsSync(lockPath), false); + + const ref = makeRef("second"); + const secret = makeSecret(); + await store.set(ref, secret); + assert.equal(await store.get(ref), secret); + assert.deepEqual(listTempFiles(path), []); +}); + +test("transient lock close and removal failures are retried", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-retry-"); + const descriptorPaths = new Map(); + let closeFailures = 0; + let removalFailures = 0; + const store = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + openSync(filePath, flags, mode) { + const descriptor = realFileOperations.openSync(filePath, flags, mode); + descriptorPaths.set(descriptor, filePath); + return descriptor; + }, + closeSync(descriptor) { + if (descriptorPaths.get(descriptor) === lockPath && closeFailures === 0) { + closeFailures += 1; + throw makeFileError("transient close", "EINTR"); + } + return realFileOperations.closeSync(descriptor); + }, + rmSync(filePath, options) { + if (isLockReleasePath(filePath, lockPath) && removalFailures === 0) { + removalFailures += 1; + throw makeFileError("transient removal", "EBUSY"); + } + return realFileOperations.rmSync(filePath, options); + } + } + }); + + await store.set(makeRef(), makeSecret()); + + assert.equal(closeFailures, 1); + assert.equal(removalFailures, 1); + assert.equal(existsSync(lockPath), false); + assert.deepEqual(listLockReleaseFiles(lockPath), []); +}); + +test("permanent residual lock reports committed degradation and is never auto-removed", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-degraded-"); + const gatePath = `${lockPath}.gate`; + let lockReads = 0; + let lockRemovals = 0; + const ref = makeRef(); + const secret = makeSecret(); + const store = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + readFileSync(filePath, options) { + if (isLockReleasePath(filePath, lockPath)) lockReads += 1; + return realFileOperations.readFileSync(filePath, options); + }, + rmSync(filePath, options) { + if (isLockReleasePath(filePath, lockPath)) { + lockRemovals += 1; + throw makeFileError("permanent removal", "EPERM"); + } + return realFileOperations.rmSync(filePath, options); + } + } + }); + + await assert.rejects( + () => store.set(ref, secret), + (error) => { + assertCrpError("CREDENTIAL_STORE_COMMITTED_LOCK_DEGRADED", 500)(error); + assert.deepEqual(error.details, { committed: true }); + return true; + } + ); + assert.equal(await store.get(ref), secret); + assert.equal(existsSync(lockPath), true); + assert.notEqual(readFileSync(lockPath, "utf8").length, 0); + assert.equal(listLockReleaseFiles(lockPath).length, 1); + assert.equal(existsSync(gatePath), false); + const readsAfterCommit = lockReads; + const removalsAfterCommit = lockRemovals; + + await assert.rejects( + () => store.set(makeRef("blocked"), makeSecret()), + assertCrpError("CREDENTIAL_STORE_LOCK_DEGRADED", 500) + ); + assert.equal(lockReads, readsAfterCommit); + assert.equal(lockRemovals, removalsAfterCommit); + assert.equal(existsSync(lockPath), true); + assert.equal(listLockReleaseFiles(lockPath).length, 1); + + const second = new FileCredentialStore({ path }); + await assert.rejects( + () => second.set(makeRef("blocked-fresh"), makeSecret()), + assertCrpError("CREDENTIAL_STORE_BUSY", 409) + ); + assert.equal(existsSync(lockPath), true); + assert.equal(existsSync(gatePath), false); +}); + +test("a preexisting foreign lock remains in place and reports busy", async (t) => { + const { parent, path, lockPath } = makeTempCredentialPath( + t, + "crp-credentials-preexisting-lock-" + ); + const foreignToken = `${randomUUID()}\n`; + const store = new FileCredentialStore({ path }); + mkdirSync(parent, { recursive: true, mode: 0o700 }); + chmodSync(parent, 0o700); + writeFileSync(lockPath, foreignToken, { mode: 0o600 }); + + await assert.rejects( + () => store.set(makeRef(), makeSecret()), + assertCrpError("CREDENTIAL_STORE_BUSY", 409) + ); + assert.equal(readFileSync(lockPath, "utf8"), foreignToken); + assert.deepEqual(listLockReleaseFiles(lockPath), []); +}); + +test("a preexisting foreign gate remains intact and blocks mutation", async (t) => { + const { parent, path, lockPath } = makeTempCredentialPath( + t, + "crp-credentials-preexisting-gate-" + ); + const gatePath = `${lockPath}.gate`; + const markerPath = join(gatePath, "foreign-marker"); + const marker = `${randomUUID()}\n`; + const store = new FileCredentialStore({ path }); + mkdirSync(parent, { recursive: true, mode: 0o700 }); + chmodSync(parent, 0o700); + mkdirSync(gatePath, { mode: 0o700 }); + writeFileSync(markerPath, marker, { mode: 0o600 }); + + await assert.rejects( + () => store.set(makeRef(), makeSecret()), + assertCrpError("CREDENTIAL_STORE_BUSY", 409) + ); + assert.equal(existsSync(path), false); + assert.equal(readFileSync(markerPath, "utf8"), marker); + assert.equal(existsSync(gatePath), true); +}); + +test("gate release claims before delete and preserves an immediate foreign replacement", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-gate-claim-"); + const gatePath = `${lockPath}.gate`; + const displacedOwnedGatePath = `${gatePath}.${randomUUID()}.displaced`; + const ref = makeRef(); + const secret = makeSecret(); + let injected = false; + let foreignDeletedByCanonicalRmdir = false; + let foreignClaimPath = null; + + function replaceCanonicalGate(targetPath = null) { + realFileOperations.renameSync(gatePath, displacedOwnedGatePath); + realFileOperations.mkdirSync(gatePath, { mode: 0o700 }); + if (isGateClaimPath(targetPath, gatePath)) foreignClaimPath = targetPath; + injected = true; + } + + const store = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + renameSync(source, target) { + if (!injected && source === gatePath) replaceCanonicalGate(target); + return realFileOperations.renameSync(source, target); + }, + rmdirSync(directoryPath) { + if (!injected && directoryPath === gatePath) replaceCanonicalGate(); + const result = realFileOperations.rmdirSync(directoryPath); + if (directoryPath === gatePath && injected) { + foreignDeletedByCanonicalRmdir = !existsSync(gatePath); + } + return result; + } + } + }); + + await assert.rejects( + () => store.set(ref, secret), + assertCrpError("CREDENTIAL_STORE_COMMITTED_LOCK_DEGRADED", 500) + ); + assert.equal(injected, true); + assert.equal(foreignDeletedByCanonicalRmdir, false); + assert.equal(await store.get(ref), secret); + assert.equal(typeof foreignClaimPath, "string"); + assert.equal(existsSync(foreignClaimPath), true); + assert.deepEqual(listGateClaimPaths(gatePath), [foreignClaimPath]); + assert.equal(existsSync(gatePath), true); + assert.equal(lstatSync(gatePath).isDirectory(), true); + + const committedBytes = readFileSync(path, "utf8"); + const second = new FileCredentialStore({ path }); + await assert.rejects( + () => second.set(makeRef("blocked-after-gate-swap"), makeSecret()), + assertCrpError("CREDENTIAL_STORE_BUSY", 409) + ); + assert.equal(readFileSync(path, "utf8"), committedBytes); + assert.equal(existsSync(foreignClaimPath), true); + assert.equal(existsSync(gatePath), true); +}); + +test("primary lock blocks a synchronous second mutation during gate claim validation", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-gate-gap-"); + const gatePath = `${lockPath}.gate`; + const firstRef = makeRef("first-gap"); + const firstSecret = makeSecret(); + const secondRef = makeRef("second-gap"); + const secondSecret = makeSecret(); + const second = new FileCredentialStore({ path }); + let injected = false; + let bytesBeforeSecond; + let secondAttempt; + const first = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + lstatSync(filePath) { + if (!injected && isGateClaimPath(filePath, gatePath)) { + injected = true; + bytesBeforeSecond = realFileOperations.readFileSync(path, "utf8"); + secondAttempt = second.set(secondRef, secondSecret); + secondAttempt.catch(() => {}); + } + return realFileOperations.lstatSync(filePath); + } + } + }); + + await first.set(firstRef, firstSecret); + + assert.equal(injected, true); + assert.ok(secondAttempt instanceof Promise); + await assert.rejects( + secondAttempt, + assertCrpError("CREDENTIAL_STORE_BUSY", 409) + ); + assert.equal(readFileSync(path, "utf8"), bytesBeforeSecond); + assert.equal(await first.get(firstRef), firstSecret); + assert.equal(await first.has(secondRef), false); + assert.equal(existsSync(lockPath), false); + assert.equal(existsSync(gatePath), false); + assert.deepEqual(listGateClaimPaths(gatePath), []); +}); + +test("an immediate pre-claim lock swap preserves foreign bytes", async (t) => { + const { path, lockPath } = makeTempCredentialPath(t, "crp-credentials-foreign-"); + const gatePath = `${lockPath}.gate`; + const foreignToken = `${randomUUID()}\n`; + const displacedPath = `${lockPath}.${randomUUID()}.displaced`; + const ref = makeRef(); + const secret = makeSecret(); + let swapTriggered = false; + const store = new FileCredentialStore({ + path, + fileOperations: { + ...realFileOperations, + renameSync(source, target) { + if (source === lockPath && isLockReleasePath(target, lockPath)) { + realFileOperations.renameSync(lockPath, displacedPath); + realFileOperations.writeFileSync(lockPath, foreignToken, { mode: 0o600 }); + swapTriggered = true; + } + return realFileOperations.renameSync(source, target); + } + } + }); + + await assert.rejects( + () => store.set(ref, secret), + assertCrpError("CREDENTIAL_STORE_COMMITTED_LOCK_DEGRADED", 500) + ); + assert.equal(swapTriggered, true); + assert.equal(await store.get(ref), secret); + const survivingPaths = [lockPath, ...listLockReleaseFiles(lockPath)] + .filter((filePath) => existsSync(filePath)); + assert.equal( + survivingPaths.some((filePath) => readFileSync(filePath, "utf8") === foreignToken), + true + ); + assert.equal(existsSync(lockPath), true); + assert.notEqual(readFileSync(lockPath, "utf8").length, 0); + assert.equal(existsSync(gatePath), false); + + const before = readFileSync(path, "utf8"); + const second = new FileCredentialStore({ path }); + await assert.rejects( + () => second.set(makeRef("blocked-second"), makeSecret()), + assertCrpError("CREDENTIAL_STORE_BUSY", 409) + ); + assert.equal(readFileSync(path, "utf8"), before); + assert.equal(existsSync(lockPath), true); + assert.equal(existsSync(gatePath), false); +}); + +test("native adapter uses the injected Entry class and exact service contract", async () => { + const { FakeEntry, constructions } = createFakeEntryClass(); + const store = new NativeKeyringStore({ entryLoader: () => FakeEntry }); + + await assertCredentialContract(store); + + assert.equal(store.backend, "native"); + assert.equal(typeof store.list, "undefined"); + assert.ok(constructions.length > 0); + assert.equal(constructions.every(({ service }) => service === NATIVE_SERVICE), true); +}); + +test("native adapter distinguishes missing credentials from backend outages", async () => { + const ref = makeRef(); + const secret = makeSecret(); + const backendFailure = new Error(`${ref}:${secret}`); + class MissingEntry { + getPassword() { return null; } + deletePassword() { return false; } + } + const missing = new NativeKeyringStore({ entryLoader: () => MissingEntry }); + await assert.rejects( + () => missing.get(ref), + assertCrpError("CREDENTIAL_NOT_FOUND", 404) + ); + assert.equal(await missing.has(ref), false); + assert.equal(await missing.delete(ref), false); + + class FailingEntry { + setPassword() { throw backendFailure; } + getPassword() { throw backendFailure; } + deletePassword() { throw backendFailure; } + } + const failing = new NativeKeyringStore({ entryLoader: () => FailingEntry }); + for (const operation of [ + () => failing.set(ref, secret), + () => failing.get(ref), + () => failing.has(ref), + () => failing.delete(ref) + ]) { + let caught; + try { + await operation(); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_BACKEND_UNAVAILABLE", 500)(caught); + assert.equal(caught.cause, backendFailure); + assertSafePublicError(caught, [ref, secret, backendFailure.message]); + } +}); + +test("credential adapters reject unsafe references and empty secrets", async (t) => { + const { path } = makeTempCredentialPath(t, "crp-credentials-input-"); + const { FakeEntry } = createFakeEntryClass(); + const stores = [ + new FileCredentialStore({ path }), + new NativeKeyringStore({ entryLoader: () => FakeEntry }) + ]; + const unsafeReferences = ["", " ", "__proto__", "constructor", "prototype", null, {}]; + + for (const store of stores) { + for (const ref of unsafeReferences) { + await assert.rejects( + () => store.get(ref), + assertCrpError("CREDENTIAL_INPUT_INVALID", 400) + ); + } + await assert.rejects( + () => store.set(makeRef(), ""), + assertCrpError("CREDENTIAL_INPUT_INVALID", 400) + ); + await assert.rejects( + () => store.set(makeRef(), null), + assertCrpError("CREDENTIAL_INPUT_INVALID", 400) + ); + } +}); + +test("credential store selection requires explicit file fallback consent", () => { + const nativeMarker = { backend: "native-marker" }; + const fileMarker = { backend: "file-marker" }; + const paths = { secretFallbackPath: "/unused/test/secrets.json" }; + let nativeCalls = 0; + let fileCalls = 0; + const nativeStoreFactory = () => { + nativeCalls += 1; + return nativeMarker; + }; + const fileStoreFactory = () => { + fileCalls += 1; + return fileMarker; + }; + + assert.equal(createCredentialStore({ paths, nativeStoreFactory, fileStoreFactory }), nativeMarker); + assert.equal(nativeCalls, 1); + assert.equal(fileCalls, 0); + + assert.throws( + () => createCredentialStore({ + backend: "file", + fallbackConsent: false, + paths, + nativeStoreFactory, + fileStoreFactory + }), + assertCrpError("CREDENTIAL_FALLBACK_CONSENT_REQUIRED", 400) + ); + assert.equal(fileCalls, 0); + + assert.equal( + createCredentialStore({ + backend: "file", + fallbackConsent: true, + paths, + nativeStoreFactory, + fileStoreFactory + }), + fileMarker + ); + assert.equal(nativeCalls, 1); + assert.equal(fileCalls, 1); + + assert.throws( + () => createCredentialStore({ + backend: "unknown", + fallbackConsent: true, + paths, + nativeStoreFactory, + fileStoreFactory + }), + assertCrpError("CREDENTIAL_BACKEND_INVALID", 400) + ); + assert.equal(nativeCalls, 1); + assert.equal(fileCalls, 1); +}); + +test("native factory failure is safe and falls back only with consent", () => { + const ref = makeRef(); + const secret = makeSecret(); + const factoryFailure = new Error(`${ref}:${secret}`); + const fileMarker = { backend: "file-marker" }; + let fileCalls = 0; + const options = { + paths: { secretFallbackPath: "/unused/test/secrets.json" }, + nativeStoreFactory: () => { throw factoryFailure; }, + fileStoreFactory: () => { + fileCalls += 1; + return fileMarker; + } + }; + + let caught; + try { + createCredentialStore(options); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_BACKEND_UNAVAILABLE", 500)(caught); + assert.equal(caught.cause, factoryFailure); + assert.equal(fileCalls, 0); + assertSafePublicError(caught, [ref, secret, factoryFailure.message]); + + assert.equal( + createCredentialStore({ ...options, fallbackConsent: true }), + fileMarker + ); + assert.equal(fileCalls, 1); +}); + +test("native addon loader failure is caught before any Entry construction", () => { + const ref = makeRef(); + const secret = makeSecret(); + const loaderFailure = new Error(`${ref}:${secret}`); + const fileMarker = { backend: "file" }; + let fileCalls = 0; + let entryConstructions = 0; + class NeverConstructedEntry { + constructor() { + entryConstructions += 1; + } + } + const nativeStoreFactory = () => new NativeKeyringStore({ + entryLoader: () => { + void NeverConstructedEntry; + throw loaderFailure; + } + }); + const fileStoreFactory = () => { + fileCalls += 1; + return fileMarker; + }; + const options = { + paths: { secretFallbackPath: "/unused/test/secrets.json" }, + nativeStoreFactory, + fileStoreFactory + }; + + let caught; + try { + createCredentialStore(options); + } catch (error) { + caught = error; + } + assertCrpError("CREDENTIAL_BACKEND_UNAVAILABLE", 500)(caught); + assert.equal(caught.cause, loaderFailure); + assert.equal(fileCalls, 0); + assert.equal(entryConstructions, 0); + assertSafePublicError(caught, [ref, secret, loaderFailure.message]); + + assert.equal( + createCredentialStore({ ...options, fallbackConsent: true }), + fileMarker + ); + assert.equal(fileCalls, 1); + assert.equal(entryConstructions, 0); +}); + +test("native operation outages never replay into file after native selection", async () => { + const ref = makeRef(); + const secret = makeSecret(); + const backendFailure = new CrpError( + "CREDENTIAL_BACKEND_UNAVAILABLE", + "The credential backend is unavailable.", + "Check the credential backend and try again.", + { status: 500 } + ); + const nativeCalls = { set: 0, get: 0, has: 0, delete: 0 }; + const fileCalls = { factory: 0, set: 0, get: 0, has: 0, delete: 0 }; + const nativeStore = { + backend: "native", + async set() { nativeCalls.set += 1; throw backendFailure; }, + async get() { nativeCalls.get += 1; throw backendFailure; }, + async has() { nativeCalls.has += 1; throw backendFailure; }, + async delete() { nativeCalls.delete += 1; throw backendFailure; } + }; + const fileStore = { + backend: "file", + async set() { fileCalls.set += 1; }, + async get() { fileCalls.get += 1; }, + async has() { fileCalls.has += 1; }, + async delete() { fileCalls.delete += 1; } + }; + const store = createCredentialStore({ + fallbackConsent: true, + paths: { secretFallbackPath: "/unused/test/secrets.json" }, + nativeStoreFactory: () => nativeStore, + fileStoreFactory: () => { + fileCalls.factory += 1; + return fileStore; + } + }); + + assert.equal(store.backend, "native"); + for (const operation of [ + () => store.set(ref, secret), + () => store.get(ref), + () => store.has(ref), + () => store.delete(ref) + ]) { + await assert.rejects(operation, (error) => error === backendFailure); + } + assert.equal(store.backend, "native"); + assert.deepEqual(nativeCalls, { set: 1, get: 1, has: 1, delete: 1 }); + assert.deepEqual(fileCalls, { factory: 0, set: 0, get: 0, has: 0, delete: 0 }); +}); + +test("construction fallback stays explicit across restart without credential migration", async () => { + const ref = makeRef(); + const secret = makeSecret(); + const values = new Map(); + let nativeFactoryCalls = 0; + let fileFactoryCalls = 0; + const fileStoreFactory = () => { + fileFactoryCalls += 1; + return { + backend: "file", + async set(key, value) { values.set(key, value); }, + async get(key) { return values.get(key); }, + async has(key) { return values.has(key); }, + async delete(key) { return values.delete(key); } + }; + }; + const first = createCredentialStore({ + fallbackConsent: true, + paths: { secretFallbackPath: "/unused/test/secrets.json" }, + nativeStoreFactory: () => { + nativeFactoryCalls += 1; + throw new Error("native unavailable before selection"); + }, + fileStoreFactory + }); + await first.set(ref, secret); + assert.equal(first.backend, "file"); + + const restarted = createCredentialStore({ + backend: first.backend, + fallbackConsent: true, + paths: { secretFallbackPath: "/unused/test/secrets.json" }, + nativeStoreFactory: () => { + nativeFactoryCalls += 1; + return { backend: "native" }; + }, + fileStoreFactory + }); + assert.equal(restarted.backend, "file"); + assert.equal(await restarted.get(ref), secret); + assert.equal(nativeFactoryCalls, 1); + assert.equal(fileFactoryCalls, 2); +}); + +test("consented native selection does not fall back for input or not-found errors", async () => { + const errors = [ + new CrpError( + "CREDENTIAL_INPUT_INVALID", + "The credential input is invalid.", + "Use a valid credential input.", + { status: 400 } + ), + new CrpError( + "CREDENTIAL_NOT_FOUND", + "The credential does not exist.", + "Save the credential and try again.", + { status: 404 } + ) + ]; + + for (const expected of errors) { + let fileCalls = 0; + const store = createCredentialStore({ + fallbackConsent: true, + paths: { secretFallbackPath: "/unused/test/secrets.json" }, + nativeStoreFactory: () => ({ + backend: "native", + async get() { throw expected; } + }), + fileStoreFactory: () => { + fileCalls += 1; + return { backend: "file" }; + } + }); + + await assert.rejects(() => store.get(makeRef()), (error) => error === expected); + assert.equal(store.backend, "native"); + assert.equal(fileCalls, 0); + } +}); + +test("public provider projection never exposes the credential reference or secret", async (t) => { + const { path } = makeTempCredentialPath(t, "crp-credentials-public-"); + const store = new FileCredentialStore({ path }); + const ref = makeRef(); + const secret = makeSecret(); + await store.set(ref, secret); + const profile = normalizeProvider({ + name: "Public Test", + baseUrl: "https://public-test.example/v1", + credentialRef: ref + }, { + id: makeRef("id"), + now: "2026-07-12T00:00:00.000Z" + }); + + const publicProfile = toPublicProvider(profile, true); + const serialized = JSON.stringify(publicProfile); + assert.equal(Object.hasOwn(publicProfile, "credentialRef"), false); + assert.equal(serialized.includes(ref), false); + assert.equal(serialized.includes(secret), false); + assert.equal(publicProfile.credentialConfigured, true); +}); From 1f5135b490a3bb2c81e30378c3226cbeb49619f7 Mon Sep 17 00:00:00 2001 From: cluic Date: Sun, 12 Jul 2026 23:59:32 +0800 Subject: [PATCH 08/44] feat: switch proxy settings atomically --- docs/AGENT_COORDINATION.md | 2 +- docs/AI_HANDOFF.md | 10 +- docs/ARCHITECTURE.md | 8 +- docs/DATA_MODEL.md | 10 +- docs/PERMISSIONS.md | 3 +- docs/STATUS.md | 7 +- docs/TESTING.md | 7 + .../plans/2026-07-10-crp-v1-implementation.md | 14 +- node/src/server.mjs | 121 +++- node/src/worker/runtime-settings.mjs | 98 ++++ node/test/runtime-settings.test.mjs | 154 +++++ node/test/server.test.mjs | 555 +++++++++++++++++- 12 files changed, 936 insertions(+), 53 deletions(-) create mode 100644 node/src/worker/runtime-settings.mjs create mode 100644 node/test/runtime-settings.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 9e8f6af..bff0482 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus one active task agent | Execute the approved plan one task at a time with read-only spec and quality reviews | Current task files only; Task 4 owns `node/src/credentials/`, `node/test/credential-store.test.mjs`, and affected security/data/living docs | Task 4 complete on `codex/harness-product-design`; Task 5 next | Task 3 atomic provider registry | Every file outside the active task; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus Task 5 implementation agent | Resolve the Task 5 response-log review finding with one writer and read-only re-review | Task 5 owns `node/src/worker/runtime-settings.mjs`, `node/src/server.mjs`, `node/test/runtime-settings.test.mjs`, `node/test/server.test.mjs`, and affected living docs | Task 5 P1 response-log fix implemented; re-review next | Task 4 credential adapters | Credentials, providers, Codex bootstrap, CLI, IPC, UI, and every other module; no simultaneous writable agent | ## Shared Contracts diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index d608269..b4c8b77 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,11 +6,11 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 4 have landed, including the atomic provider metadata registry and native plus explicit-consent file credential adapters; provider-service lifecycle orchestration has not landed. Task 5, snapshot-based proxy settings, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 5 have landed, including atomic provider metadata, secure credential adapters, and snapshot-based proxy settings; worker IPC and provider-service lifecycle orchestration have not landed. Task 6, the worker protocol and entrypoint, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry with refreshed defensive reads, and credential adapters with explicit-only file fallback. Target: provider-service orchestration, long-lived supervisor control plane, and independent proxy worker. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry, credential adapters with explicit-only file fallback, and monotonic immutable runtime snapshots captured once per proxied request. Target: worker IPC, provider-service orchestration, and the long-lived supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 4 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 5 are complete. ## How To Run Current Code @@ -45,6 +45,7 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 Task 2 gate: `node --test test/codex-config.test.mjs` passes 15/15, including deterministic rename failure, exclusive same-timestamp backup collision, busy lock, external source change, CRLF preservation, guide semantics, and all three start aliases; `npm test` passes 27/27, `npm run lint` syntax-checks 9 source files, and `npm audit --omit=dev` reports zero vulnerabilities. - Node 22.19 Task 3 gate: `node --test test/provider-registry.test.mjs` passes 23/23, including multi-instance lock serialization, strict schema and header validation, test-state invalidation, primary-error preservation, degraded lock cleanup, refreshed defensive copies, and public allowlisting; `npm test` passes 50/50, `npm run lint` syntax-checks 11 source files, and `npm audit --omit=dev` reports zero vulnerabilities. - Node 22.19 Task 4 gate: `node --test test/credential-store.test.mjs` passes 41/41, the combined credential/provider focus passes 64/64, `npm test` passes 91/91, and `npm run lint` syntax-checks 14 source files. Coverage includes construction-only fallback without operation replay, explicit file-label restart continuity, descriptor identity, strict parent/file modes, degraded temp cleanup, canonical lock restoration, claim-before-delete gate release, foreign replacement preservation, and synchronous second-instance blocking while a gate claim is validated. Native tests inject the loader and never invoke the real addon loader or touch the OS credential store; real native verification remains L3 on every supported system, including Windows and Linux. +- Node 22.19 Task 5 gate: `node --test test/runtime-settings.test.mjs test/server.test.mjs` passes 13/13, `npm test` passes 102/102, and `npm run lint` syntax-checks 15 source files. Coverage includes strict generations, clone-before-freeze replacement, public-state allowlisting, one snapshot read per request before body listeners, delayed A versus immediate B switching, transport/TLS and timeout pinning, unconfigured-source rejection, health secret scans, request/response dynamic auth-header log masking, and bidirectional custom-auth capture redaction. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. @@ -73,3 +74,6 @@ Credential migration, localhost browser security, worker IPC, port release races - Canonical gate paths must never be deleted after a separate identity check; atomically claim them to a unique path first. - Canonical primary locks must remain until gate ownership or replacement-blocker state is proven. - Shell validation patterns must be individually quoted so the scan itself is deterministic. +- Runtime settings must be cloned and deeply frozen before one atomic reference replacement. +- Every proxied request must capture one runtime snapshot before body listeners are registered. +- The active authentication header must be masked from debug logs and capture records, including short values and nonstandard header names. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 4654405..985a77b 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 4 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, an atomic schema-version-2 provider registry, and native plus explicit-consent file credential adapters. The supervisor, provider service, Admin API, and independent worker remain target-state architecture. +Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 5 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, atomic provider and credential persistence, and immutable request-level runtime settings. The supervisor, provider service, Admin API, and IPC-hosted worker entrypoint remain target-state architecture. ## Target Overview @@ -40,7 +40,7 @@ Active OpenAI-compatible upstream ## Module Boundaries -Landed in Tasks 2 through 4: +Landed in Tasks 2 through 5: - `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. - `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. @@ -50,13 +50,15 @@ Landed in Tasks 2 through 4: - `credential-store`: selects native storage by default and chooses the schema-version-1 file fallback only with explicit consent when native construction fails before any credential operation; selected native operations never replay into the independent file namespace. Both adapters expose asynchronous get/set/has/delete operations without enumeration. - `native-keyring`: lazily loads synchronous `@napi-rs/keyring` entries during construction under service `org.cluic.codex-remote-proxy`, supports injected loaders/factories, preserves backend failures only as internal causes, and exposes safe stable errors. - `file-credential-store`: validates a real private parent and regular private file, opens and identity-checks a descriptor before reading, validates exact secret-only documents, refreshes reads, and serializes clone-before-commit mutations with exclusive `0600` lock and temporary files. +- `runtime-settings`: accepts only positive, strictly increasing safe-integer generations, clone-and-freezes plain settings before replacing one active reference, and exposes only configured state plus generation publicly. +- `proxy forwarding`: optionally captures exactly one runtime snapshot before request-body listeners and pins target, transport, request ID, authentication, extra headers, TLS, timeout, capture context, and logs for that request. Static configuration remains the generation-zero compatibility path. Dynamic health exposes only runtime and capture public state; an unconfigured source never falls back to the static upstream. Remaining target-state boundaries: - `supervisor`: owns state transitions, admin server, activity records, and child-process lifecycle. - `provider-service`: coordinates compatibility tests, credentials, activation, and credential-aware deletion above the metadata registry. - `worker-protocol`: versioned IPC messages for configure, drain, shutdown, health, and events. -- `proxy-worker`: forwards traffic from immutable provider snapshots; does not own persistent configuration. +- `proxy-worker`: hosts the landed forwarding server behind the future IPC entrypoint and does not own persistent configuration. - `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. - `web-ui`: static local app with onboarding and daily management views. diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index 9812545..c4b7a2a 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -1,6 +1,6 @@ # Data Model -Tasks 3 and 4 have implemented strict schema-version-2 provider metadata persistence plus native and explicit-consent file credential adapters. Provider-service orchestration and migration remain target-state work. +Tasks 3 through 5 have implemented strict schema-version-2 provider metadata persistence, native and explicit-consent file credential adapters, and immutable in-memory runtime snapshots. Provider-service orchestration and migration remain target-state work. ## ProviderProfile @@ -68,6 +68,12 @@ Every mutation first acquires an exclusive same-directory protocol gate and a ca The native adapter lazily loads the addon during construction and stores the same opaque reference under service `org.cluic.codex-remote-proxy`. Backend selection defaults to native. Explicit file selection requires `fallbackConsent === true`; consent permits construction-time file selection only when native cannot be created before any credential operation. Once native is selected, backend-unavailable reads and mutations fail without file replay. A construction-time fallback exposes backend label `file`; callers must reuse that explicit label after restart to remain in the same namespace. Task 4 performs no automatic credential migration. +## RuntimeSettingsSnapshot + +A runtime snapshot contains exactly one positive safe-integer `generation` and one plain-data `settings` graph. Applying a snapshot validates its generation, clones its settings, deeply freezes the clone, and replaces the active reference only after all prior work succeeds. Equal or lower generations are stale; invalid, unclonable, or non-plain settings leave the prior reference unchanged. + +`current()` returns the frozen active reference. `publicState()` returns only `{ configured, generation }`, both before and after configuration. A proxied request captures `current()` exactly once before body listeners and retains that reference for its target, authentication, headers, TLS, timeout, capture context, and logging lifetime. + ## RuntimeState Runtime state includes supervisor PID, worker PID, worker status, snapshot generation, start timestamps, restart count, and the last sanitized error. It is observational and can be reconstructed. @@ -80,7 +86,7 @@ Activity events record timestamp, category, action, provider ID, result, stable - One registry has zero or one active provider. - Each provider has exactly one credential reference after it is saved; public provider projections omit that reference. -- A worker snapshot references one provider and one resolved credential for its process lifetime. +- A runtime snapshot references one provider and one resolved credential; each request retains its starting snapshot for the request lifetime. ## Lifecycle and Deletion diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md index d0b8444..db3caa7 100644 --- a/docs/PERMISSIONS.md +++ b/docs/PERMISSIONS.md @@ -19,7 +19,7 @@ The MVP has one role: the local operating-system user. It has no accounts, remot | --- | --- | --- | --- | | Authenticated local UI session | Manage providers and worker | Read complete keys; bind remote interfaces | CSRF and origin checks required | | Local CLI with control token | Same management actions | Read complete keys | Designed for automation | -| Proxy worker | Use active snapshot | Read registry or other provider keys | Receives least-privilege configuration | +| Proxy worker | Use the active snapshot captured once per request | Read registry or other provider keys | Receives least-privilege configuration | | Remote network client | Nothing | Access UI, API, or proxy management | Admin server is loopback-only | ## Audit Requirements @@ -38,6 +38,7 @@ Record provider creation/update/deletion, test result, activation, Codex bootstr - Set metadata, token, state, and fallback secret files to `0600` where supported. - Never accept sensitive keys inside `extraHeaders`; use the credential field. - Redact authorization, cookie, token, secret, and API-key headers and fields. +- Redact the exact active authentication header before debug logging or capture persistence; short secrets must never be emitted unchanged. - Reject non-loopback admin binds. - Validate URL scheme and block accidental credential forwarding to an unvalidated target. - Treat diagnostics and capture content as sensitive local data. diff --git a/docs/STATUS.md b/docs/STATUS.md index 366ea48..eeba00e 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 4 are complete and Task 5 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 5 are complete and Task 6 is next. ## Done @@ -22,14 +22,15 @@ V1 implementation: provider lifecycle end to end. - Completed Task 2 shared path and public-error contracts plus line-ending-preserving, lock-serialized, idempotent and atomic Codex bootstrap behavior, covered by 15/15 focused and 27/27 full Node 22.19 tests. - Completed Task 3 strict provider validation and lock-serialized atomic schema-version-2 registry behavior, covered by 23/23 focused and 50/50 full Node 22.19 tests. - Completed Task 4 native and explicit-consent file credential adapters, including construction-only fallback without operation replay, explicit file-label restart continuity, descriptor-safe reads, degraded temp cleanup, canonical lock restoration, and primary-blocked claim-before-delete gate release, covered by 41/41 focused, 64/64 combined credential/provider, and 91/91 full Node 22.19 tests; syntax checking covers 14 source files. +- Completed Task 5 monotonic immutable runtime settings and request-start snapshot capture. In-flight A remains internally consistent while new traffic switches to B; dynamic health, transport and timeout pinning, request/response short and custom authentication log masking, and bidirectional custom-auth capture redaction are covered by 13/13 focused and 102/102 full Node 22.19 tests; syntax checking covers 15 source files. ## Blocked -- No current blocker is recorded for Task 5. +- No current blocker is recorded for Task 6. ## Next -1. Execute Task 5: make proxy settings snapshot-based. +1. Execute Task 6: define worker IPC and build the worker entrypoint. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/TESTING.md b/docs/TESTING.md index 54a673c..662e8e5 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -23,12 +23,15 @@ - Task 3 focused suite: `cd node && node --test test/provider-registry.test.mjs` - Task 4 focused suite: `cd node && node --test test/credential-store.test.mjs` - Task 4 combined credential/provider suite: `cd node && node --test test/credential-store.test.mjs test/provider-registry.test.mjs` +- Task 5 focused suite: `cd node && node --test test/runtime-settings.test.mjs test/server.test.mjs` - Runtime audit: `cd node && npm audit --omit=dev` `npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. On Node 22.19, the Task 2 focused suite passes 15/15 tests. Its coverage verifies OpenAI provider creation and update, custom-provider and CRLF preservation, byte idempotency, one-time adjacent backup, exclusive same-timestamp backup collision handling, CRP lock contention, external source-change rejection, atomic mode-preserving replacement, deterministic rename-failure cleanup and original preservation, all nine injected-home paths, safe public error serialization, `start`/`install`/`setup` JSON and managed-state backup propagation, and accurate guide backup semantics. The Node 22.19 Task 3 focused suite passes 23/23 tests. The Task 4 credential suite passes 41/41, the combined credential/provider suite passes 64/64, and the current full suite passes 91/91; `npm run lint` syntax-checks 14 source files. Task 4 coverage includes the shared async adapter contract, lazy native-loader failure, construction-only fallback, no operation replay for native get/set/has/delete outages, explicit file-label restart continuity without migration, exact schema and string persistence, reload and refreshed reads, no enumeration API, two-instance lost-update prevention, strict parent/file modes, POSIX no-follow and simulated-Windows descriptor identity, symlink-swap rejection before byte reads, fsync/rename order, rollback cleanup, bounded secret-temp cleanup and permanent uncommitted degradation, gate-covered atomic lock claims, preexisting foreign-gate preservation, claim-before-delete gate release under an immediate foreign replacement, canonical blocker restoration, synchronous second-instance rejection during gate claim validation, normal claim cleanup and subsequent mutation, fresh-instance busy behavior, permanent committed lock degradation, input validation, and public provider masking. Tests never invoke the default native loader or construct or query a real native credential entry. Real native-backend verification remains L3 on every supported system, including Windows and Linux; file permission and rename semantics on Windows and Linux also remain unverified. +The Node 22.19 Task 5 focused suite passes 13/13 and the current full suite passes 102/102; `npm run lint` syntax-checks 15 source files. Coverage verifies generation validation and failure atomicity, deep clone/freeze behavior, public allowlisting, exactly one request-start snapshot capture, delayed A versus immediate B switching, a transport-option spy for TLS pinning before body arrival, pinned authentication, headers, timeout, capture context and request IDs, static compatibility, unconfigured-source rejection, dynamic health secret scans, request/response short and custom authentication debug masking, and bidirectional custom-auth capture redaction. + `npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. ## Test Authoring Rules @@ -48,6 +51,10 @@ The Node 22.19 Task 3 focused suite passes 23/23 tests. The Task 4 credential su - Credential mutation must remain gate-protected, and the canonical primary lock must cover gate claim validation. - Gate release must atomically claim the canonical path and delete only an ownership-verified claim; it must never delete the canonical path after a separate identity check. - Primary lock release must follow gate claim validation or proven blocker restoration; an uncertain gate state must retain the primary path. +- Runtime snapshots must be cloned and deeply frozen before the active reference is replaced. +- Proxy tests must prove one snapshot read before body listeners and pin every request-level upstream decision to it. +- Snapshot-switch tests must make every pre-fix routing path terminate deterministically. +- Request and response debug/capture tests must use the active custom authentication header and scan for complete short and long secret values. ## Test Matrix diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index 50c60da..61a8921 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -483,7 +483,7 @@ git commit -m "feat: add secure credential stores" - Modify: `node/test/server.test.mjs` - Create: `node/test/runtime-settings.test.mjs` -- [ ] **Step 1: Write failing snapshot-generation tests** +- [x] **Step 1: Write failing snapshot-generation tests** ```js const source = new RuntimeSettingsSource(); @@ -496,11 +496,11 @@ assert.equal(source.current().settings.upstream.baseUrl, settingsB.upstream.base Assert returned snapshots are deeply frozen and callers cannot mutate headers. -- [ ] **Step 2: Add an in-flight switch integration test to `server.test.mjs`** +- [x] **Step 2: Add an in-flight switch integration test to `server.test.mjs`** Start delayed upstream A and immediate upstream B. Send request 1, wait until A receives it, apply generation 2, send request 2, release A, then assert response 1 came from A and response 2 came from B. -- [ ] **Step 3: Run both tests and verify failure** +- [x] **Step 3: Run both tests and verify failure** ```bash cd node @@ -509,7 +509,7 @@ node --test test/runtime-settings.test.mjs test/server.test.mjs Expected: runtime module is missing and the static server cannot switch settings. -- [ ] **Step 4: Implement snapshot capture at request start** +- [x] **Step 4: Implement snapshot capture at request start** `RuntimeSettingsSource` exposes `apply(snapshot)`, `current()`, and `publicState()`. Change `createServer` to accept an optional `settingsSource`; for every non-health request, obtain exactly one snapshot before reading the body: @@ -520,7 +520,7 @@ const requestSettings = active.settings; Use `requestSettings` for target URL, headers, timeout, SSL, capture context, and logs. Health uses `settingsSource.publicState()` and must not include credentials. -- [ ] **Step 5: Verify snapshot behavior and regressions** +- [x] **Step 5: Verify snapshot behavior and regressions** ```bash cd node @@ -530,7 +530,9 @@ npm test Expected: in-flight/new-request assertions pass and existing proxy/capture behavior remains green. -- [ ] **Step 6: Commit** +Actual Node 22.19 Task 5 verification: the runtime/server focus passes 13/13, the full suite passes 102/102, and the portable syntax gate checks 15 source files. Coverage additionally verifies validation failure atomicity, an unconfigured source without static fallback, pinned transport/TLS and timeout behavior, allowlisted health state, request/response dynamic authentication-header masking, and bidirectional custom-auth capture redaction. + +- [x] **Step 6: Commit** ```bash git add node/src/worker/runtime-settings.mjs node/src/server.mjs node/test/runtime-settings.test.mjs node/test/server.test.mjs diff --git a/node/src/server.mjs b/node/src/server.mjs index 6ae6e20..d909d9b 100644 --- a/node/src/server.mjs +++ b/node/src/server.mjs @@ -32,8 +32,6 @@ const HOP_BY_HOP_HEADERS = new Set([ "upgrade" ]); -let DEBUG_ENABLED = false; - export function resolveConfigPath() { return process.env[CONFIG_ENV_VAR] ? resolve(process.env[CONFIG_ENV_VAR]) : DEFAULT_CONFIG_PATH; } @@ -98,7 +96,7 @@ function maskSecret(value) { return "(empty)"; } if (value.length <= 8) { - return value; + return "[REDACTED]"; } return `${value.slice(0, 4)}...${value.slice(-4)}`; } @@ -109,8 +107,8 @@ export function log(level, message, fields = {}) { console.log(`${new Date().toISOString()} ${level.toUpperCase()} ${message}${suffix}`); } -function debugLog(label, data) { - if (!DEBUG_ENABLED) return; +function debugLog(label, data, enabled) { + if (!enabled) return; const timestamp = new Date().toISOString(); const json = typeof data === "string" ? data : JSON.stringify(data, null, 2); for (const line of json.split("\n")) { @@ -140,6 +138,14 @@ function formatAuthorization(upstream) { } const CONTENT_HEADERS = new Set(["content-encoding", "content-length"]); +const DEBUG_SENSITIVE_HEADER_NAMES = new Set([ + "authorization", + "proxy-authorization", + "cookie", + "set-cookie", + "x-api-key" +]); +const DEBUG_SENSITIVE_HEADER_PARTS = ["token", "secret", "api-key"]; function decompressBody(buffer, encoding) { const enc = encoding.toLowerCase().trim(); @@ -164,10 +170,26 @@ function autoDecompress(buffer) { try { return zlib.brotliDecompressSync(buffer); } catch { return null; } } -function sanitizeHeadersForDebug(headersObject) { +function sanitizeHeadersForDebug(headersObject, authHeader = "authorization") { const result = {}; + const activeAuthHeader = authHeader.toLowerCase(); for (const [key, value] of Object.entries(headersObject)) { - result[key] = key.toLowerCase() === "authorization" ? maskSecret(String(value)) : value; + const loweredKey = key.toLowerCase(); + const sensitive = loweredKey === activeAuthHeader + || DEBUG_SENSITIVE_HEADER_NAMES.has(loweredKey) + || DEBUG_SENSITIVE_HEADER_PARTS.some((part) => loweredKey.includes(part)); + result[key] = sensitive ? maskSecret(String(value)) : value; + } + return result; +} + +function sanitizeHeadersForCapture(headersInput, authHeader) { + const result = headersToObject(headersInput); + const activeAuthHeader = authHeader.toLowerCase(); + for (const key of Object.keys(result)) { + if (key.toLowerCase() === activeAuthHeader) { + result[key] = "[REDACTED]"; + } } return result; } @@ -239,7 +261,14 @@ function isEventStream(contentType = "") { return contentType.split(";", 1)[0].trim().toLowerCase() === "text/event-stream"; } -function buildHealthPayload(settings, captureManager) { +function buildHealthPayload(settings, captureManager, settingsSource) { + if (settingsSource) { + return { + ok: true, + ...settingsSource.publicState(), + ...captureManager.getPublicState() + }; + } return { ok: true, configPath: settings.configPath, @@ -265,6 +294,8 @@ function buildRequestContext({ req, settings, targetUrl, requestId, requestHeade } } + const captureHeaders = sanitizeHeadersForCapture(requestHeaders, settings.upstream.authHeader); + return { requestId, sessionId: typeof req.headers["session-id"] === "string" @@ -276,7 +307,7 @@ function buildRequestContext({ req, settings, targetUrl, requestId, requestHeade method: req.method || "GET", incomingUrl: new URL(req.url, `http://${settings.server.host}:${settings.server.port}`).href, targetUrl: targetUrl.href, - requestHeaders: headersToObject(requestHeaders), + requestHeaders: captureHeaders, requestBody, startedAt: new Date(startedAt).toISOString(), captureHandle @@ -309,7 +340,11 @@ function saveCaptureRecord(captureContext, fields) { }); } -export function createServer(settings, { captureManager = createCaptureManager({ configPath: settings.configPath, capture: settings.capture, log }).start(), logFn = log } = {}) { +export function createServer(settings, { + captureManager = createCaptureManager({ configPath: settings.configPath, capture: settings.capture, log }).start(), + logFn = log, + settingsSource +} = {}) { return http.createServer((req, res) => { if (!req.url) { writeJson(res, 400, { error: { message: "Missing request URL", type: "proxy_bad_request" } }); @@ -317,12 +352,27 @@ export function createServer(settings, { captureManager = createCaptureManager({ } if (req.url === HEALTH_PATH) { - writeJson(res, 200, buildHealthPayload(settings, captureManager)); + writeJson(res, 200, buildHealthPayload(settings, captureManager, settingsSource)); return; } - const requestId = req.headers[settings.proxy.requestIdHeader] || req.headers["x-request-id"] || "-"; - const targetUrl = buildTargetUrl(settings.upstream.baseUrl, req.url); + let active; + try { + active = settingsSource ? settingsSource.current() : { generation: 0, settings }; + } catch (error) { + const unavailable = error?.code === "RUNTIME_SETTINGS_UNAVAILABLE"; + writeJson(res, unavailable ? 503 : 500, { + error: { + code: unavailable ? "RUNTIME_SETTINGS_UNAVAILABLE" : "RUNTIME_SETTINGS_ERROR", + message: unavailable ? "Proxy settings are not configured." : "Proxy settings could not be loaded." + } + }); + return; + } + const requestSettings = active.settings; + const requestDebugEnabled = requestSettings.server.logLevel.toLowerCase() === "debug"; + const requestId = req.headers[requestSettings.proxy.requestIdHeader] || req.headers["x-request-id"] || "-"; + const targetUrl = buildTargetUrl(requestSettings.upstream.baseUrl, req.url); const transport = targetUrl.protocol === "https:" ? https : http; const startedAt = Date.now(); @@ -349,13 +399,13 @@ export function createServer(settings, { captureManager = createCaptureManager({ originalSize: body.length, decompressedSize: decompressed.length, magicBytes: `0x${body[0].toString(16).padStart(2, "0")} 0x${body[1].toString(16).padStart(2, "0")}` - }); + }, requestDebugEnabled); body = decompressed; bodyTransformed = true; } } - const headers = buildUpstreamHeaders(req, settings, targetUrl, { + const headers = buildUpstreamHeaders(req, requestSettings, targetUrl, { stripContentHeaders: bodyTransformed }); if (bodyTransformed && body.length) { @@ -365,7 +415,7 @@ export function createServer(settings, { captureManager = createCaptureManager({ const captureHandle = captureManager.beginRecord() ?? createNoopCaptureHandle(); const captureContext = buildRequestContext({ req, - settings, + settings: requestSettings, targetUrl, requestId, requestHeaders: headers, @@ -388,10 +438,16 @@ export function createServer(settings, { captureManager = createCaptureManager({ method: req.method, path: req.url, targetUrl: targetUrl.href, - incomingHeaders: sanitizeHeadersForDebug(Object.fromEntries(Object.entries(req.headers))), - upstreamHeaders: Object.fromEntries(headers.map(([k, v]) => [k, k.toLowerCase() === "authorization" ? maskSecret(v) : v])), + incomingHeaders: sanitizeHeadersForDebug( + Object.fromEntries(Object.entries(req.headers)), + requestSettings.upstream.authHeader + ), + upstreamHeaders: sanitizeHeadersForDebug( + Object.fromEntries(headers), + requestSettings.upstream.authHeader + ), body: safeBodyPreview(body) - }); + }, requestDebugEnabled); const upstreamRequest = transport.request( { @@ -401,16 +457,22 @@ export function createServer(settings, { captureManager = createCaptureManager({ port: targetUrl.port || undefined, path: `${targetUrl.pathname}${targetUrl.search}`, headers, - rejectUnauthorized: settings.upstream.verifySsl + rejectUnauthorized: requestSettings.upstream.verifySsl }, (upstreamResponse) => { const stream = isEventStream(upstreamResponse.headers["content-type"]); debugLog("RESPONSE HEADERS", { status: upstreamResponse.statusCode, - headers: upstreamResponse.headers - }); - - const responseHeaders = headersToObject(upstreamResponse.rawHeaders); + headers: sanitizeHeadersForDebug( + upstreamResponse.headers, + requestSettings.upstream.authHeader + ) + }, requestDebugEnabled); + + const responseHeaders = sanitizeHeadersForCapture( + upstreamResponse.headers, + requestSettings.upstream.authHeader + ); const respChunks = []; upstreamResponse.on("data", (chunk) => { respChunks.push(chunk); @@ -426,7 +488,7 @@ export function createServer(settings, { captureManager = createCaptureManager({ debugLog("RESPONSE BODY", { status: upstreamResponse.statusCode, body: safeBodyPreview(responseBody) - }); + }, requestDebugEnabled); } finalizeCapture({ responseStatus: upstreamResponse.statusCode || 502, @@ -447,7 +509,7 @@ export function createServer(settings, { captureManager = createCaptureManager({ } ); - upstreamRequest.setTimeout(settings.upstream.timeoutMs, () => { + upstreamRequest.setTimeout(requestSettings.upstream.timeoutMs, () => { upstreamRequest.destroy(new Error("upstream timeout")); }); @@ -471,7 +533,7 @@ export function createServer(settings, { captureManager = createCaptureManager({ error: error.message, code: error.code || "(none)", stack: error.stack - }); + }, requestDebugEnabled); if (!res.headersSent) { writeJson(res, statusCode, payload); } else { @@ -515,8 +577,7 @@ export function createServer(settings, { captureManager = createCaptureManager({ }); } -export function createApp(settings = loadConfig()) { - DEBUG_ENABLED = settings.server.logLevel.toLowerCase() === "debug"; +export function createApp(settings = loadConfig(), { settingsSource } = {}) { const captureManager = createCaptureManager({ configPath: settings.configPath, capture: settings.capture, @@ -533,7 +594,7 @@ export function createApp(settings = loadConfig()) { capture_db_path: settings.capture.dbPath }); - const server = createServer(settings, { captureManager, logFn: log }); + const server = createServer(settings, { captureManager, logFn: log, settingsSource }); server.on("close", () => { captureManager.close(); }); diff --git a/node/src/worker/runtime-settings.mjs b/node/src/worker/runtime-settings.mjs new file mode 100644 index 0000000..9d02783 --- /dev/null +++ b/node/src/worker/runtime-settings.mjs @@ -0,0 +1,98 @@ +function runtimeSettingsError(code, message) { + const error = new Error(`${code}: ${message}`); + error.name = "RuntimeSettingsError"; + error.code = code; + return error; +} + +function isPlainObject(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return false; + } + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function assertPlainData(value, seen = new WeakSet()) { + if (value === null || ["string", "boolean"].includes(typeof value)) { + return; + } + if (typeof value === "number") { + if (!Number.isFinite(value)) { + throw runtimeSettingsError("RUNTIME_SETTINGS_INVALID", "Settings must contain only finite numbers."); + } + return; + } + if (typeof value !== "object" || (!Array.isArray(value) && !isPlainObject(value))) { + throw runtimeSettingsError("RUNTIME_SETTINGS_INVALID", "Settings must contain only plain data."); + } + if (seen.has(value)) { + return; + } + seen.add(value); + for (const item of Array.isArray(value) ? value : Object.values(value)) { + assertPlainData(item, seen); + } +} + +function deepFreeze(value, seen = new WeakSet()) { + if (value === null || typeof value !== "object" || seen.has(value)) { + return value; + } + seen.add(value); + for (const item of Array.isArray(value) ? value : Object.values(value)) { + deepFreeze(item, seen); + } + return Object.freeze(value); +} + +function cloneAndFreezeSettings(settings) { + let cloned; + try { + cloned = structuredClone(settings); + } catch { + throw runtimeSettingsError("RUNTIME_SETTINGS_INVALID", "Settings could not be cloned."); + } + assertPlainData(cloned); + return deepFreeze(cloned); +} + +export class RuntimeSettingsSource { + #active = null; + + apply(snapshot) { + if (!isPlainObject(snapshot) || !Number.isSafeInteger(snapshot.generation) || snapshot.generation <= 0) { + throw runtimeSettingsError( + "RUNTIME_SETTINGS_INVALID", + "Snapshot generation must be a positive safe integer." + ); + } + if (!isPlainObject(snapshot.settings)) { + throw runtimeSettingsError("RUNTIME_SETTINGS_INVALID", "Snapshot settings must be an object."); + } + if (this.#active && snapshot.generation <= this.#active.generation) { + throw runtimeSettingsError("STALE_SNAPSHOT", "Snapshot generation must increase."); + } + + const next = Object.freeze({ + generation: snapshot.generation, + settings: cloneAndFreezeSettings(snapshot.settings) + }); + this.#active = next; + return next; + } + + current() { + if (!this.#active) { + throw runtimeSettingsError("RUNTIME_SETTINGS_UNAVAILABLE", "Runtime settings have not been configured."); + } + return this.#active; + } + + publicState() { + return Object.freeze({ + configured: this.#active !== null, + generation: this.#active?.generation ?? 0 + }); + } +} diff --git a/node/test/runtime-settings.test.mjs b/node/test/runtime-settings.test.mjs new file mode 100644 index 0000000..ea6c9e1 --- /dev/null +++ b/node/test/runtime-settings.test.mjs @@ -0,0 +1,154 @@ +import test from "node:test"; +import assert from "node:assert/strict"; + +import { RuntimeSettingsSource } from "../src/worker/runtime-settings.mjs"; + +function makeSettings(label = "a") { + return { + configPath: `/private/${label}/proxy-config.json`, + server: { + host: "127.0.0.1", + port: 15100, + logLevel: "info" + }, + upstream: { + baseUrl: `https://${label}.example.test/v1`, + apiKey: `${label}-super-secret-api-key`, + timeoutMs: 5000, + verifySsl: true, + authHeader: "x-provider-api-key", + authScheme: "Bearer", + extraHeaders: { + "x-provider-region": `${label}-region` + } + }, + proxy: { + overrideAuthorization: true, + requestIdHeader: "x-client-request-id" + }, + capture: { + enabled: false, + dbPath: `/private/${label}/traffic.sqlite3`, + ignoredPaths: ["/_proxy/health"] + } + }; +} + +test("RuntimeSettingsSource reports an allowlisted unconfigured public state", () => { + const source = new RuntimeSettingsSource(); + + assert.deepEqual(source.publicState(), { + configured: false, + generation: 0 + }); + assert.throws( + () => source.current(), + (error) => error?.code === "RUNTIME_SETTINGS_UNAVAILABLE" + ); +}); + +test("RuntimeSettingsSource accepts only strictly increasing generations", () => { + const source = new RuntimeSettingsSource(); + const settingsA = makeSettings("a"); + const settingsB = makeSettings("b"); + + source.apply({ generation: 2, settings: settingsA }); + const first = source.current(); + assert.equal(first.generation, 2); + + for (const generation of [2, 1]) { + assert.throws( + () => source.apply({ generation, settings: settingsB }), + (error) => error?.code === "STALE_SNAPSHOT" + ); + assert.strictEqual(source.current(), first); + } + + source.apply({ generation: 3, settings: settingsB }); + assert.equal(source.current().generation, 3); + assert.equal(source.current().settings.upstream.baseUrl, settingsB.upstream.baseUrl); +}); + +test("RuntimeSettingsSource rejects invalid input without replacing the current snapshot", () => { + const source = new RuntimeSettingsSource(); + source.apply({ generation: 3, settings: makeSettings("stable") }); + const stable = source.current(); + + const invalidSnapshots = [ + null, + {}, + { generation: -1, settings: makeSettings("negative") }, + { generation: 0, settings: makeSettings("zero") }, + { generation: 1.5, settings: makeSettings("fraction") }, + { generation: Number.NaN, settings: makeSettings("nan") }, + { generation: Number.POSITIVE_INFINITY, settings: makeSettings("infinity") }, + { generation: Number.MAX_SAFE_INTEGER + 1, settings: makeSettings("unsafe") }, + { generation: 4 }, + { generation: 4, settings: null }, + { generation: 4, settings: [] }, + { generation: 4, settings: { nested: () => "not cloneable" } }, + { generation: 4, settings: { nested: Number.POSITIVE_INFINITY } } + ]; + + for (const snapshot of invalidSnapshots) { + assert.throws( + () => source.apply(snapshot), + (error) => error?.code === "RUNTIME_SETTINGS_INVALID" + ); + assert.strictEqual(source.current(), stable); + } +}); + +test("RuntimeSettingsSource clones input and deeply freezes the active snapshot", () => { + const source = new RuntimeSettingsSource(); + const input = { + generation: 1, + settings: makeSettings("clone") + }; + + source.apply(input); + const active = source.current(); + + input.generation = 99; + input.settings.upstream.baseUrl = "https://mutated.example.test"; + input.settings.upstream.extraHeaders["x-provider-region"] = "mutated"; + input.settings.capture.ignoredPaths.push("/mutated"); + + assert.equal(active.generation, 1); + assert.equal(active.settings.upstream.baseUrl, "https://clone.example.test/v1"); + assert.equal(active.settings.upstream.extraHeaders["x-provider-region"], "clone-region"); + assert.deepEqual(active.settings.capture.ignoredPaths, ["/_proxy/health"]); + assert.equal(Object.isFrozen(active), true); + assert.equal(Object.isFrozen(active.settings), true); + assert.equal(Object.isFrozen(active.settings.upstream), true); + assert.equal(Object.isFrozen(active.settings.upstream.extraHeaders), true); + assert.equal(Object.isFrozen(active.settings.capture.ignoredPaths), true); + assert.throws(() => { + active.settings.upstream.extraHeaders["x-provider-region"] = "changed"; + }, TypeError); +}); + +test("RuntimeSettingsSource public state never projects settings or secret values", () => { + const source = new RuntimeSettingsSource(); + const settings = makeSettings("public-state-sentinel"); + source.apply({ generation: 7, settings }); + + const publicState = source.publicState(); + assert.deepEqual(publicState, { + configured: true, + generation: 7 + }); + assert.equal(Object.isFrozen(publicState), true); + + const serialized = JSON.stringify(publicState); + for (const forbidden of [ + "settings", + "apiKey", + "authHeader", + settings.upstream.apiKey, + settings.upstream.authHeader, + settings.upstream.extraHeaders["x-provider-region"] + ]) { + assert.equal(serialized.includes(forbidden), false, `public state leaked ${forbidden}`); + } +}); diff --git a/node/test/server.test.mjs b/node/test/server.test.mjs index 82147cf..8ef8ad2 100644 --- a/node/test/server.test.mjs +++ b/node/test/server.test.mjs @@ -1,13 +1,16 @@ import test from "node:test"; import assert from "node:assert/strict"; import http from "node:http"; +import https from "node:https"; +import { Readable } from "node:stream"; import { mkdirSync, rmSync, writeFileSync } from "node:fs"; import os from "node:os"; import { join } from "node:path"; -import { once } from "node:events"; +import { EventEmitter, once } from "node:events"; import { DatabaseSync } from "node:sqlite"; -import { createApp, isDirectExecution } from "../src/server.mjs"; +import { createApp, createServer, isDirectExecution } from "../src/server.mjs"; +import { RuntimeSettingsSource } from "../src/worker/runtime-settings.mjs"; function makeTempDir(prefix) { return join(os.tmpdir(), `${prefix}-${Date.now()}-${Math.random().toString(16).slice(2)}`); @@ -23,6 +26,104 @@ function listen(server, host = "127.0.0.1") { }); } +async function closeServer(server) { + if (!server.listening) { + return; + } + const closed = once(server, "close"); + server.close(); + await closed; +} + +function createGate() { + let release; + const promise = new Promise((resolvePromise) => { + release = resolvePromise; + }); + return { promise, release }; +} + +function createSignal() { + let resolve; + const promise = new Promise((resolvePromise) => { + resolve = resolvePromise; + }); + return { promise, resolve }; +} + +function makeSettings({ + baseUrl, + apiKey = "test-api-key", + authHeader = "authorization", + authScheme = "Bearer", + extraHeaders = {}, + timeoutMs = 300000, + verifySsl = true, + requestIdHeader = "x-client-request-id", + logLevel = "info", + captureEnabled = true +}) { + return { + configPath: "/tmp/crp-task5-proxy-config.json", + server: { + host: "127.0.0.1", + port: 0, + logLevel + }, + upstream: { + baseUrl, + apiKey, + timeoutMs, + verifySsl, + authHeader, + authScheme, + extraHeaders + }, + proxy: { + overrideAuthorization: true, + requestIdHeader + }, + capture: { + enabled: captureEnabled, + dbPath: "/tmp/crp-task5-traffic.sqlite3" + } + }; +} + +function createMemoryCaptureManager(publicState = {}) { + const records = []; + return { + records, + beginRecord() { + let saved = false; + return { + save(record) { + if (!saved) { + saved = true; + records.push(record); + } + } + }; + }, + getPublicState() { + return { + captureConfigured: true, + captureActive: true, + ...publicState + }; + }, + close() {} + }; +} + +async function fetchJson(url, options) { + const response = await fetch(url, options); + return { + status: response.status, + body: await response.json() + }; +} + function requestJson(url, body) { return fetch(url, { method: "POST", @@ -48,6 +149,7 @@ test("server writes proxied request and response to sqlite", async () => { res.statusCode = 200; res.setHeader("content-type", "application/json"); res.setHeader("x-request-id", "upstream-test-1"); + res.setHeader("x-provider-auth", "response-upstream-secret"); res.end(JSON.stringify({ ok: true, echoed: JSON.parse(payload) })); }); }); @@ -66,7 +168,7 @@ test("server writes proxied request and response to sqlite", async () => { apiKey: "upstream-secret", timeoutMs: 300000, verifySsl: true, - authHeader: "authorization", + authHeader: "x-provider-auth", authScheme: "Bearer", extraHeaders: {} }, @@ -92,7 +194,7 @@ test("server writes proxied request and response to sqlite", async () => { apiKey: "upstream-secret", timeoutMs: 300000, verifySsl: true, - authHeader: "authorization", + authHeader: "x-provider-auth", authScheme: "Bearer", extraHeaders: {} }, @@ -129,6 +231,9 @@ test("server writes proxied request and response to sqlite", async () => { assert.equal(rows[0].thread_id, "thread-it-1"); assert.equal(rows[0].upstream_request_id, "upstream-test-1"); assert.match(rows[0].request_headers_json, /REDACTED/); + assert.doesNotMatch(rows[0].request_headers_json, /upstream-secret/); + assert.match(rows[0].response_headers_json, /REDACTED/); + assert.doesNotMatch(rows[0].response_headers_json, /response-upstream-secret/); assert.match(rows[0].response_body, /"ok":true/); rmSync(dir, { recursive: true, force: true }); @@ -152,3 +257,445 @@ test("isDirectExecution handles both POSIX and Windows paths", () => { false ); }); + +test("dynamic requests capture current settings exactly once before body listeners", () => { + const settings = makeSettings({ baseUrl: "http://127.0.0.1:9" }); + const runtime = new RuntimeSettingsSource(); + runtime.apply({ generation: 1, settings }); + const events = []; + const settingsSource = { + current() { + events.push("current"); + return runtime.current(); + }, + publicState() { + return runtime.publicState(); + } + }; + const server = createServer(settings, { + settingsSource, + captureManager: createMemoryCaptureManager(), + logFn() {} + }); + const req = new EventEmitter(); + const originalOn = req.on.bind(req); + req.on = (eventName, listener) => { + if (eventName === "data" || eventName === "end") { + events.push(`on:${eventName}`); + } + return originalOn(eventName, listener); + }; + Object.assign(req, { + url: "/responses", + method: "POST", + headers: {}, + rawHeaders: [] + }); + const res = { + statusCode: 200, + setHeader() {}, + end() {}, + on() {}, + appendHeader() {} + }; + + server.emit("request", req, res); + + assert.deepEqual(events, ["current", "on:data", "on:end"]); +}); + +test("TLS and timeout options stay pinned when settings change before the request body", async (t) => { + const settingsA = makeSettings({ + baseUrl: "https://a.example.test:4443", + timeoutMs: 1111, + verifySsl: false + }); + const settingsB = makeSettings({ + baseUrl: "https://b.example.test:5443", + timeoutMs: 2222, + verifySsl: true + }); + const runtime = new RuntimeSettingsSource(); + runtime.apply({ generation: 1, settings: settingsA }); + const captured = createSignal(); + const settingsSource = { + current() { + const active = runtime.current(); + captured.resolve(); + return active; + }, + publicState() { + return runtime.publicState(); + } + }; + const observed = {}; + const originalHttpsRequest = https.request; + t.after(() => { + https.request = originalHttpsRequest; + }); + https.request = (options, onResponse) => { + Object.assign(observed, options); + const request = new EventEmitter(); + request.setTimeout = (timeoutMs) => { + observed.timeoutMs = timeoutMs; + }; + request.destroy = (error) => request.emit("error", error); + request.end = () => { + const response = Readable.from([Buffer.from(JSON.stringify({ ok: true }))]); + response.statusCode = 200; + response.headers = { "content-type": "application/json" }; + response.rawHeaders = ["content-type", "application/json"]; + queueMicrotask(() => onResponse(response)); + }; + return request; + }; + + const proxy = createServer(settingsB, { + settingsSource, + captureManager: createMemoryCaptureManager(), + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const responsePromise = new Promise((resolvePromise, rejectPromise) => { + const clientRequest = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: "/responses", + method: "POST" + }, (response) => { + const chunks = []; + response.on("data", (chunk) => chunks.push(chunk)); + response.on("end", () => resolvePromise({ + status: response.statusCode, + body: JSON.parse(Buffer.concat(chunks).toString("utf8")) + })); + }); + clientRequest.on("error", rejectPromise); + clientRequest.flushHeaders(); + captured.promise.then(() => { + runtime.apply({ generation: 2, settings: settingsB }); + clientRequest.end("{}"); + }, rejectPromise); + }); + + const response = await responsePromise; + assert.deepEqual(response, { status: 200, body: { ok: true } }); + assert.equal(observed.hostname, "a.example.test"); + assert.equal(observed.port, "4443"); + assert.equal(observed.rejectUnauthorized, false); + assert.equal(observed.timeoutMs, 1111); +}); + +test("in-flight request keeps A target, credential, headers, capture, and logs while new request uses B", async (t) => { + const releaseA = createGate(); + t.after(() => releaseA.release()); + const receivedA = createSignal(); + const observedA = []; + const observedB = []; + + const upstreamA = http.createServer((req, res) => { + const chunks = []; + req.on("data", (chunk) => chunks.push(chunk)); + req.on("end", () => { + observedA.push({ headers: req.headers, body: Buffer.concat(chunks).toString("utf8") }); + if (observedA.length === 1) { + receivedA.resolve(); + releaseA.promise.then(() => { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ upstream: "A" })); + }); + } else { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ upstream: "A-unexpected-repeat" })); + } + }); + }); + const portA = await listen(upstreamA); + t.after(async () => { + releaseA.release(); + await closeServer(upstreamA); + }); + + const upstreamB = http.createServer((req, res) => { + const chunks = []; + req.on("data", (chunk) => chunks.push(chunk)); + req.on("end", () => { + observedB.push({ headers: req.headers, body: Buffer.concat(chunks).toString("utf8") }); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ upstream: "B" })); + }); + }); + const portB = await listen(upstreamB); + t.after(() => closeServer(upstreamB)); + + const settingsA = makeSettings({ + baseUrl: `http://127.0.0.1:${portA}`, + apiKey: "a-api-key-sentinel", + authHeader: "x-provider-a-auth", + authScheme: "Token", + extraHeaders: { "x-snapshot-route": "A" }, + timeoutMs: 5000, + verifySsl: false, + requestIdHeader: "x-a-request-id" + }); + const settingsB = makeSettings({ + baseUrl: `http://127.0.0.1:${portB}`, + apiKey: "b-api-key-sentinel", + authHeader: "x-provider-b-auth", + authScheme: "", + extraHeaders: { "x-snapshot-route": "B" }, + timeoutMs: 1000, + verifySsl: true, + requestIdHeader: "x-b-request-id" + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 1, settings: settingsA }); + const captureManager = createMemoryCaptureManager(); + const logs = []; + const proxy = createServer(settingsA, { + settingsSource: source, + captureManager, + logFn(level, message, fields) { + logs.push({ level, message, fields }); + } + }); + const proxyPort = await listen(proxy); + t.after(async () => { + releaseA.release(); + await closeServer(proxy); + }); + + const responseAPromise = fetchJson(`http://127.0.0.1:${proxyPort}/responses?request=A`, { + method: "POST", + headers: { + "content-type": "application/json", + "x-a-request-id": "request-a", + "x-provider-a-auth": "client-a-value" + }, + body: JSON.stringify({ request: "A" }) + }); + await Promise.race([ + receivedA.promise, + responseAPromise.then((response) => { + throw new Error(`request A completed before reaching upstream A (${response.status})`); + }) + ]); + + source.apply({ generation: 2, settings: settingsB }); + const responseB = await fetchJson(`http://127.0.0.1:${proxyPort}/responses?request=B`, { + method: "POST", + headers: { + "content-type": "application/json", + "x-b-request-id": "request-b", + "x-provider-b-auth": "client-b-value" + }, + body: JSON.stringify({ request: "B" }) + }); + releaseA.release(); + const responseA = await responseAPromise; + + assert.deepEqual(responseA, { status: 200, body: { upstream: "A" } }); + assert.deepEqual(responseB, { status: 200, body: { upstream: "B" } }); + assert.equal(observedA.length, 1); + assert.equal(observedA[0].headers["x-provider-a-auth"], "Token a-api-key-sentinel"); + assert.equal(observedA[0].headers["x-snapshot-route"], "A"); + assert.equal(observedB.length, 1); + assert.equal(observedB[0].headers["x-provider-b-auth"], "b-api-key-sentinel"); + assert.equal(observedB[0].headers["x-snapshot-route"], "B"); + assert.deepEqual(captureManager.records.map((record) => new URL(record.targetUrl).host), [ + `127.0.0.1:${portB}`, + `127.0.0.1:${portA}` + ]); + assert.deepEqual( + logs.filter((entry) => entry.message === "Proxied request").map((entry) => entry.fields.request_id).sort(), + ["request-a", "request-b"] + ); +}); + +test("an in-flight A request retains its longer timeout after B is applied", async (t) => { + const releaseA = createGate(); + t.after(() => releaseA.release()); + const receivedA = createSignal(); + + const upstreamA = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + receivedA.resolve(); + releaseA.promise.then(() => res.end(JSON.stringify({ upstream: "A" }))); + }); + }); + const portA = await listen(upstreamA); + t.after(async () => { + releaseA.release(); + await closeServer(upstreamA); + }); + + const upstreamB = http.createServer((req) => { + req.resume(); + }); + const portB = await listen(upstreamB); + t.after(() => closeServer(upstreamB)); + + const settingsA = makeSettings({ baseUrl: `http://127.0.0.1:${portA}`, timeoutMs: 5000 }); + const settingsB = makeSettings({ baseUrl: `http://127.0.0.1:${portB}`, timeoutMs: 75 }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 1, settings: settingsA }); + const proxy = createServer(settingsB, { + settingsSource: source, + captureManager: createMemoryCaptureManager(), + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(async () => { + releaseA.release(); + await closeServer(proxy); + }); + + const responseAPromise = fetchJson(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + body: "A" + }); + await Promise.race([ + receivedA.promise, + responseAPromise.then((response) => { + throw new Error(`request A completed before reaching upstream A (${response.status})`); + }) + ]); + source.apply({ generation: 2, settings: settingsB }); + + const responseB = await fetchJson(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + body: "B" + }); + assert.equal(responseB.status, 504); + assert.equal(responseB.body.error.type, "proxy_timeout"); + + releaseA.release(); + const responseA = await responseAPromise; + assert.deepEqual(responseA, { status: 200, body: { upstream: "A" } }); +}); + +test("dynamic health is allowlisted and an unconfigured source never falls back to static settings", async (t) => { + let staticUpstreamHits = 0; + const staticUpstream = http.createServer((req, res) => { + staticUpstreamHits += 1; + req.resume(); + req.on("end", () => res.end(JSON.stringify({ unexpected: true }))); + }); + const upstreamPort = await listen(staticUpstream); + t.after(() => closeServer(staticUpstream)); + + const staticSettings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + apiKey: "static-health-secret", + authHeader: "x-static-health-auth", + extraHeaders: { "x-static-health-extra": "static-extra-secret" } + }); + const source = new RuntimeSettingsSource(); + const captureManager = createMemoryCaptureManager({ failedWriteCount: 0 }); + const proxy = createServer(staticSettings, { settingsSource: source, captureManager, logFn() {} }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const health = await fetchJson(`http://127.0.0.1:${proxyPort}/_proxy/health`); + assert.equal(health.status, 200); + assert.deepEqual(health.body, { + ok: true, + configured: false, + generation: 0, + captureConfigured: true, + captureActive: true, + failedWriteCount: 0 + }); + + const serializedHealth = JSON.stringify(health.body); + for (const forbidden of [ + "settings", + staticSettings.upstream.apiKey, + staticSettings.upstream.authHeader, + staticSettings.upstream.extraHeaders["x-static-health-extra"] + ]) { + assert.equal(serializedHealth.includes(forbidden), false, `health leaked ${forbidden}`); + } + + const unavailable = await fetchJson(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + body: "{}" + }); + assert.equal(unavailable.status, 503); + assert.equal(unavailable.body.error.code, "RUNTIME_SETTINGS_UNAVAILABLE"); + assert.equal(staticUpstreamHits, 0); +}); + +test("debug and startup logs mask short keys and the active custom auth header", async (t) => { + let observedAuthHeader = null; + const upstream = http.createServer((req, res) => { + observedAuthHeader = req.headers["x-provider-auth"] ?? null; + req.resume(); + req.on("end", () => { + res.setHeader("x-provider-auth", "response-custom-auth-secret"); + res.setHeader("set-cookie", "session-cookie-secret"); + res.setHeader("x-api-key", "tiny"); + res.setHeader("x-diagnostic", "trace-visible"); + res.end(JSON.stringify({ ok: true })); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + + const staticSettings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + apiKey: "static-api-key-sentinel", + logLevel: "debug" + }); + const activeSettings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + apiKey: "k3y", + authHeader: "x-provider-auth", + authScheme: "", + logLevel: "debug" + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 1, settings: activeSettings }); + const lines = []; + const originalConsoleLog = console.log; + console.log = (...args) => lines.push(args.join(" ")); + t.after(() => { + console.log = originalConsoleLog; + }); + + let app; + try { + app = createApp(staticSettings, { settingsSource: source }); + const proxyPort = await listen(app.server); + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { + "x-provider-auth": "client-custom-auth-sentinel" + }, + body: "{}" + }); + assert.equal(response.status, 200); + await response.text(); + } finally { + if (app) { + await closeServer(app.server); + app.captureManager.close(); + } + console.log = originalConsoleLog; + } + + const output = lines.join("\n"); + assert.match(output, /DEBUG \[REQUEST\]/); + assert.equal(output.includes("k3y"), false); + assert.equal(output.includes("client-custom-auth-sentinel"), false); + assert.equal(output.includes("response-custom-auth-secret"), false); + assert.equal(output.includes("session-cookie-secret"), false); + assert.equal(output.includes('"x-api-key": "tiny"'), false); + assert.match(output, /"x-api-key": "\[REDACTED\]"/); + assert.match(output, /"x-diagnostic": "trace-visible"/); + assert.equal(output.includes(JSON.stringify(source.current())), false); + assert.equal(observedAuthHeader, "k3y"); +}); From bc5a2f18e59759564417fb63c01e91a9254c7dab Mon Sep 17 00:00:00 2001 From: cluic Date: Mon, 13 Jul 2026 01:05:39 +0800 Subject: [PATCH 09/44] feat: add proxy worker protocol --- docs/AGENT_COORDINATION.md | 2 +- docs/AI_HANDOFF.md | 15 +- docs/ARCHITECTURE.md | 8 +- docs/DATA_MODEL.md | 8 +- docs/PERMISSIONS.md | 1 + docs/STATUS.md | 7 +- docs/TESTING.md | 14 +- .../plans/2026-07-10-crp-v1-implementation.md | 18 +- node/package.json | 2 +- node/scripts/run-tests.mjs | 11 + node/src/worker/protocol.mjs | 351 +++++++ node/src/worker/worker-entry.mjs | 317 ++++++ node/test/integration/worker-entry.test.mjs | 953 ++++++++++++++++++ node/test/worker-protocol.test.mjs | 390 +++++++ 14 files changed, 2075 insertions(+), 22 deletions(-) create mode 100644 node/scripts/run-tests.mjs create mode 100644 node/src/worker/protocol.mjs create mode 100644 node/src/worker/worker-entry.mjs create mode 100644 node/test/integration/worker-entry.test.mjs create mode 100644 node/test/worker-protocol.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index bff0482..7bc1eeb 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus Task 5 implementation agent | Resolve the Task 5 response-log review finding with one writer and read-only re-review | Task 5 owns `node/src/worker/runtime-settings.mjs`, `node/src/server.mjs`, `node/test/runtime-settings.test.mjs`, `node/test/server.test.mjs`, and affected living docs | Task 5 P1 response-log fix implemented; re-review next | Task 4 credential adapters | Credentials, providers, Codex bootstrap, CLI, IPC, UI, and every other module; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus Task 6 implementation agent | Resolve the final Task 6 shutdown/disconnect escalation review finding with one writer | Task 6 owns `node/src/worker/worker-entry.mjs`, `node/test/integration/worker-entry.test.mjs`, and affected living docs; the completed protocol and grouped test runner remain unchanged | Task 6 bounded shutdown/disconnect escalation fix complete on `codex/harness-product-design`; Task 7 is next | Task 6 amended commit | Task 5 server/runtime settings and capture source/tests, test runner/package, supervisor/manager, providers, credentials, Codex, CLI/bin, Admin API, UI, and every other module; no simultaneous writable agent | ## Shared Contracts diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index b4c8b77..c1ea738 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,11 +6,11 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 5 have landed, including atomic provider metadata, secure credential adapters, and snapshot-based proxy settings; worker IPC and provider-service lifecycle orchestration have not landed. Task 6, the worker protocol and entrypoint, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 6 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, and the versioned proxy-worker IPC entrypoint. Task 7, reliable worker lifecycle management, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry, credential adapters with explicit-only file fallback, and monotonic immutable runtime snapshots captured once per proxied request. Target: worker IPC, provider-service orchestration, and the long-lived supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry, credential adapters with explicit-only file fallback, monotonic immutable runtime snapshots captured once per proxied request, and a strict version-1 child-process protocol whose worker listens only after valid configuration. Target: worker-manager orchestration, provider-service orchestration, and the long-lived supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 5 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 6 are complete. ## How To Run Current Code @@ -46,12 +46,13 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 Task 3 gate: `node --test test/provider-registry.test.mjs` passes 23/23, including multi-instance lock serialization, strict schema and header validation, test-state invalidation, primary-error preservation, degraded lock cleanup, refreshed defensive copies, and public allowlisting; `npm test` passes 50/50, `npm run lint` syntax-checks 11 source files, and `npm audit --omit=dev` reports zero vulnerabilities. - Node 22.19 Task 4 gate: `node --test test/credential-store.test.mjs` passes 41/41, the combined credential/provider focus passes 64/64, `npm test` passes 91/91, and `npm run lint` syntax-checks 14 source files. Coverage includes construction-only fallback without operation replay, explicit file-label restart continuity, descriptor identity, strict parent/file modes, degraded temp cleanup, canonical lock restoration, claim-before-delete gate release, foreign replacement preservation, and synchronous second-instance blocking while a gate claim is validated. Native tests inject the loader and never invoke the real addon loader or touch the OS credential store; real native verification remains L3 on every supported system, including Windows and Linux. - Node 22.19 Task 5 gate: `node --test test/runtime-settings.test.mjs test/server.test.mjs` passes 13/13, `npm test` passes 102/102, and `npm run lint` syntax-checks 15 source files. Coverage includes strict generations, clone-before-freeze replacement, public-state allowlisting, one snapshot read per request before body listeners, delayed A versus immediate B switching, transport/TLS and timeout pinning, unconfigured-source rejection, health secret scans, request/response dynamic auth-header log masking, and bidirectional custom-auth capture redaction. +- Node 22.19 Task 6 gate: `node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` passes 21/21, `npm run test:integration` passes 11/11, and `npm test` passes a nonduplicated 112/112 top-level group followed by 11/11 integration tests; `npm run lint` syntax-checks 18 source files. Coverage includes exact version-1 directional schemas, HTTPS-or-loopback upstreams, HTTP-token authentication fields, final authentication-value validation, sensitive and conflicting header rejection, parent-only configure secrets, fixed invalid-message fatal IDs, configure-before-listen, monotonic reconfiguration, configure rejection after drain begins, retained in-flight drain acknowledgement, idempotent duplicate drain, health/status, idle keep-alive closure, clean shutdown, bounded parent-disconnect cleanup with a hanging upstream before or during shutdown, safe startup failures, stale generations, port conflicts, port release, and child-process cleanup without fixed sleeps. Real-fork integration runs after the ordinary group so it cannot race watcher readiness in another test process. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. ## Known Risks -Credential migration, localhost browser security, worker IPC, port release races, in-flight activation semantics, secret leakage, and cross-platform atomic rename and permission semantics. +Credential migration, localhost browser security, worker-manager crash recovery and restart escalation, in-flight activation semantics, secret leakage, and cross-platform atomic rename and permission semantics. ## Recent Decisions @@ -77,3 +78,9 @@ Credential migration, localhost browser security, worker IPC, port release races - Runtime settings must be cloned and deeply frozen before one atomic reference replacement. - Every proxied request must capture one runtime snapshot before body listeners are registered. - The active authentication header must be masked from debug logs and capture records, including short values and nonstandard header names. +- Worker IPC must use exact versioned directional schemas and provider-equivalent URL/header security; resolved settings may appear only in parent `configure` messages, while child messages and sanitized diagnostics expose allowlisted state and static errors only. Invalid messages always use the fixed `worker-fatal` correlation ID. +- A worker must not listen before its first valid configuration, and drain acknowledgement requires listener closure plus zero tracked in-flight requests. +- A worker must reject configuration after drain begins without replacing the active generation or losing the pending drain acknowledgement. +- Worker configure must validate the exact authentication header value that forwarding will send, without logging or returning the credential. +- Repeated drain commands must reuse the same completion and acknowledge each request without moving a drained worker back to draining. +- Parent IPC disconnect cleanup must start or reuse bounded escalation even when shutdown is already waiting on an upstream request. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 985a77b..6358b0c 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 5 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, atomic provider and credential persistence, and immutable request-level runtime settings. The supervisor, provider service, Admin API, and IPC-hosted worker entrypoint remain target-state architecture. +Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 6 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, atomic provider and credential persistence, immutable request-level runtime settings, and an IPC-hosted worker entrypoint. The supervisor, provider service, and Admin API remain target-state architecture. ## Target Overview @@ -40,7 +40,7 @@ Active OpenAI-compatible upstream ## Module Boundaries -Landed in Tasks 2 through 5: +Landed in Tasks 2 through 6: - `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. - `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. @@ -52,13 +52,13 @@ Landed in Tasks 2 through 5: - `file-credential-store`: validates a real private parent and regular private file, opens and identity-checks a descriptor before reading, validates exact secret-only documents, refreshes reads, and serializes clone-before-commit mutations with exclusive `0600` lock and temporary files. - `runtime-settings`: accepts only positive, strictly increasing safe-integer generations, clone-and-freezes plain settings before replacing one active reference, and exposes only configured state plus generation publicly. - `proxy forwarding`: optionally captures exactly one runtime snapshot before request-body listeners and pins target, transport, request ID, authentication, extra headers, TLS, timeout, capture context, and logs for that request. Static configuration remains the generation-zero compatibility path. Dynamic health exposes only runtime and capture public state; an unconfigured source never falls back to the static upstream. +- `worker-protocol`: validates exact version-1 parent messages (`configure`, `drain`, `shutdown`, `status`) and child messages (`ready`, `configured`, `drained`, `status`, `fatal`). Configure accepts HTTPS or explicit loopback HTTP only, requires HTTP-token authentication fields, validates the exact final authentication value with Node header rules, and rejects sensitive or authentication-conflicting extra headers. Resolved settings are accepted only inside parent configure messages; child output and sanitized projections contain only allowlisted lifecycle state and stable public errors, and invalid input receives the fixed `worker-fatal` correlation ID. +- `proxy-worker`: creates one runtime settings source and forwarding app after the first valid configure, binds only then, applies strictly increasing generations while ready or running, rejects configure once drain begins, tracks in-flight requests independently of later phase changes, closes the listener and idle keep-alive connections during drain, acknowledges repeated drain requests without leaving the drained phase, closes capture and IPC resources during shutdown, and bounds forced cleanup after parent disconnect even when graceful shutdown is already waiting so a hanging upstream cannot orphan the worker. Remaining target-state boundaries: - `supervisor`: owns state transitions, admin server, activity records, and child-process lifecycle. - `provider-service`: coordinates compatibility tests, credentials, activation, and credential-aware deletion above the metadata registry. -- `worker-protocol`: versioned IPC messages for configure, drain, shutdown, health, and events. -- `proxy-worker`: hosts the landed forwarding server behind the future IPC entrypoint and does not own persistent configuration. - `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. - `web-ui`: static local app with onboarding and daily management views. diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index c4b7a2a..f77c8e5 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -1,6 +1,6 @@ # Data Model -Tasks 3 through 5 have implemented strict schema-version-2 provider metadata persistence, native and explicit-consent file credential adapters, and immutable in-memory runtime snapshots. Provider-service orchestration and migration remain target-state work. +Tasks 3 through 6 have implemented strict schema-version-2 provider metadata persistence, native and explicit-consent file credential adapters, immutable in-memory runtime snapshots, and a strict version-1 worker protocol. Provider-service orchestration and migration remain target-state work. ## ProviderProfile @@ -74,6 +74,12 @@ A runtime snapshot contains exactly one positive safe-integer `generation` and o `current()` returns the frozen active reference. `publicState()` returns only `{ configured, generation }`, both before and after configuration. A proxied request captures `current()` exactly once before body listeners and retains that reference for its target, authentication, headers, TLS, timeout, capture context, and logging lifetime. +## WorkerProtocolMessage + +Every IPC message has exactly `{ version: 1, type, requestId, ...typeFields }`. Parent messages are `configure`, `drain`, `shutdown`, and `status`; only `configure` adds `{ generation, settings }`, and it is the only message allowed to carry resolved credentials. Child messages are `ready`, `configured`, `drained`, `status`, and `fatal`. Lifecycle messages expose only an exact public worker state `{ phase, configured, generation, listening, listenHost, listenPort, inFlight }`; fatal messages expose a stable code and static public message without causes or input payloads. + +Configure settings use the exact runtime graph required by the forwarding server and reject unknown or malformed fields before any bind. Upstream URLs require HTTPS except for explicit loopback HTTP; authentication names and schemes are HTTP tokens, the exact scheme-plus-key value must satisfy Node header-value rules, and extra headers cannot be sensitive or collide case-insensitively with the configured authentication header. A worker accepts only increasing positive safe-integer generations through `RuntimeSettingsSource` while ready or running and rejects configure after drain begins. Drain completion is idempotent: repeated request IDs receive drained acknowledgements without changing the drained state. Protocol sanitization removes configure settings, projects lifecycle state through the public allowlist, maps fatal output to safe static errors, and uses `worker-fatal` instead of an unvalidated input request ID. + ## RuntimeState Runtime state includes supervisor PID, worker PID, worker status, snapshot generation, start timestamps, restart count, and the last sanitized error. It is observational and can be reconstructed. diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md index db3caa7..f4b2502 100644 --- a/docs/PERMISSIONS.md +++ b/docs/PERMISSIONS.md @@ -39,6 +39,7 @@ Record provider creation/update/deletion, test result, activation, Codex bootstr - Never accept sensitive keys inside `extraHeaders`; use the credential field. - Redact authorization, cookie, token, secret, and API-key headers and fields. - Redact the exact active authentication header before debug logging or capture persistence; short secrets must never be emitted unchanged. +- Validate parent-child IPC with exact directional version-1 schemas, HTTPS-or-loopback URL rules, HTTP-token authentication fields, Node-compatible final authentication values, and case-insensitive sensitive/authentication-header collision rejection. Resolved credentials may cross the process boundary only inside a parent `configure` message; child lifecycle/fatal messages, sanitizers, stdout, and stderr must never echo settings, causes, complete secrets, or an unvalidated request ID. - Reject non-loopback admin binds. - Validate URL scheme and block accidental credential forwarding to an unvalidated target. - Treat diagnostics and capture content as sensitive local data. diff --git a/docs/STATUS.md b/docs/STATUS.md index eeba00e..e166ef1 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 5 are complete and Task 6 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 6 are complete and Task 7 is next. ## Done @@ -23,14 +23,15 @@ V1 implementation: provider lifecycle end to end. - Completed Task 3 strict provider validation and lock-serialized atomic schema-version-2 registry behavior, covered by 23/23 focused and 50/50 full Node 22.19 tests. - Completed Task 4 native and explicit-consent file credential adapters, including construction-only fallback without operation replay, explicit file-label restart continuity, descriptor-safe reads, degraded temp cleanup, canonical lock restoration, and primary-blocked claim-before-delete gate release, covered by 41/41 focused, 64/64 combined credential/provider, and 91/91 full Node 22.19 tests; syntax checking covers 14 source files. - Completed Task 5 monotonic immutable runtime settings and request-start snapshot capture. In-flight A remains internally consistent while new traffic switches to B; dynamic health, transport and timeout pinning, request/response short and custom authentication log masking, and bidirectional custom-auth capture redaction are covered by 13/13 focused and 102/102 full Node 22.19 tests; syntax checking covers 15 source files. +- Completed Task 6 strict version-1 worker IPC and the proxy-worker child entrypoint. Configure enforces HTTPS-or-loopback upstreams, HTTP-token authentication fields, Node-compatible final authentication values, and sensitive/header-collision rejection; the worker remains unbound until valid configuration, rejects configure after drain begins without losing the pending drain acknowledgement, keeps repeated drain acknowledgements and status drained, bounds parent-disconnect cleanup when an upstream hangs before or during shutdown, uses a fixed correlation ID for invalid-message fatals, releases its port on shutdown, and emits only sanitized child messages. Coverage passes 21/21 focused and 11/11 integration tests; the full gate passes 112/112 top-level tests followed by 11/11 integration tests without duplication. Syntax checking covers 18 source files, including the sequential group runner that isolates real-fork load from watcher tests. ## Blocked -- No current blocker is recorded for Task 6. +- No current blocker is recorded for Task 7. ## Next -1. Execute Task 6: define worker IPC and build the worker entrypoint. +1. Execute Task 7: implement reliable worker lifecycle management. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/TESTING.md b/docs/TESTING.md index 662e8e5..9c58c21 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -24,6 +24,8 @@ - Task 4 focused suite: `cd node && node --test test/credential-store.test.mjs` - Task 4 combined credential/provider suite: `cd node && node --test test/credential-store.test.mjs test/provider-registry.test.mjs` - Task 5 focused suite: `cd node && node --test test/runtime-settings.test.mjs test/server.test.mjs` +- Task 6 focused suite: `cd node && node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` +- Worker integration suite: `cd node && npm run test:integration` - Runtime audit: `cd node && npm audit --omit=dev` `npm run lint` recursively checks `.mjs` and `.js` files under `bin`, `src`, `scripts`, and `ui`, skipping source roots that have not landed. On Node 22.19, the Task 2 focused suite passes 15/15 tests. Its coverage verifies OpenAI provider creation and update, custom-provider and CRLF preservation, byte idempotency, one-time adjacent backup, exclusive same-timestamp backup collision handling, CRP lock contention, external source-change rejection, atomic mode-preserving replacement, deterministic rename-failure cleanup and original preservation, all nine injected-home paths, safe public error serialization, `start`/`install`/`setup` JSON and managed-state backup propagation, and accurate guide backup semantics. @@ -32,7 +34,9 @@ The Node 22.19 Task 3 focused suite passes 23/23 tests. The Task 4 credential su The Node 22.19 Task 5 focused suite passes 13/13 and the current full suite passes 102/102; `npm run lint` syntax-checks 15 source files. Coverage verifies generation validation and failure atomicity, deep clone/freeze behavior, public allowlisting, exactly one request-start snapshot capture, delayed A versus immediate B switching, a transport-option spy for TLS pinning before body arrival, pinned authentication, headers, timeout, capture context and request IDs, static compatibility, unconfigured-source rejection, dynamic health secret scans, request/response short and custom authentication debug masking, and bidirectional custom-auth capture redaction. -`npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner is present and recursively discovers `test/integration/**/*.test.mjs`, but no integration tests exist yet, so it intentionally fails with an explicit no-files error and is not part of the current runnable gate. `test:e2e` and the combined `test:all` command are also not current gates until the UI, Playwright configuration, and E2E specs land. +The Node 22.19 Task 6 focused suite passes 21/21, `npm run test:integration` passes 11/11, and `npm test` passes 112/112 top-level tests followed by 11/11 integration tests without duplication; `npm run lint` syntax-checks 18 source files. Coverage verifies exact directional schemas, HTTPS-or-loopback URL enforcement, HTTP-token authentication fields, Node-compatible final authentication values, sensitive and authentication-conflicting header rejection, secret sanitization, fixed invalid-message fatal IDs, and real child-process ready/configure/reconfigure/proxy/status/drain/shutdown flows. Synchronized tests prove configure is rejected after drain begins without replacing generation or losing its acknowledgement, duplicate drain requests keep both acknowledgements and later status drained, and parent disconnect terminates within a deadline despite a hanging upstream both before and after shutdown starts waiting. Invalid authentication values, configuration, stale generation, occupied-port startup, port release, request settlement, stderr/stdout scans, and deterministic deadline-driven cleanup remain covered. + +`npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner recursively discovers `test/integration/**/*.test.mjs`; the current worker-entry suite passes 11/11. `npm test` invokes those two nonoverlapping groups in sequence, retaining concurrency inside the top-level group while ensuring real child-process integration load cannot race watcher readiness. `test:e2e` and the combined `test:all` command are not current gates until the UI, Playwright configuration, and E2E specs land. ## Test Authoring Rules @@ -55,6 +59,14 @@ The Node 22.19 Task 5 focused suite passes 13/13 and the current full suite pass - Proxy tests must prove one snapshot read before body listeners and pin every request-level upstream decision to it. - Snapshot-switch tests must make every pre-fix routing path terminate deterministically. - Request and response debug/capture tests must use the active custom authentication header and scan for complete short and long secret values. +- Worker protocol tests must reject unknown fields and versions, remote plaintext upstreams, non-token authentication fields, and sensitive or authentication-conflicting extra headers, and prove that child messages and sanitized projections contain no configure settings, complete secrets, or unvalidated request IDs. +- Child-process integration tests must use observable events and bounded deadlines instead of fixed sleeps, register cleanup before assertions, and leave no worker or test process behind. +- Worker drain must close idle keep-alive sockets after the final in-flight response before acknowledging. +- Worker configuration must be rejected once drain begins without replacing generation or losing the pending drain acknowledgement. +- Duplicate drain requests must keep every acknowledgement and subsequent status in the drained phase. +- Configure tests must validate the exact authentication header value and prove invalid keys are absent from child IPC, stdout, and stderr. +- Parent IPC disconnect cleanup must start or reuse bounded escalation even when shutdown is already waiting on an upstream request. +- Real-fork integration tests must run after watcher-bearing top-level tests, never concurrently with that group. ## Test Matrix diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index 61a8921..be82de5 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -546,16 +546,18 @@ git commit -m "feat: switch proxy settings atomically" - Create: `node/src/worker/worker-entry.mjs` - Create: `node/test/worker-protocol.test.mjs` - Create: `node/test/integration/worker-entry.test.mjs` +- Create: `node/scripts/run-tests.mjs` +- Modify: `node/package.json` -- [ ] **Step 1: Write failing protocol validation tests** +- [x] **Step 1: Write failing protocol validation tests** Cover parent messages `configure`, `drain`, `shutdown`, `status` and child messages `ready`, `configured`, `drained`, `status`, `fatal`. Assert secrets are permitted only inside the parent-only configure payload and are removed by `sanitizeProtocolMessage()`. -- [ ] **Step 2: Write a child-process integration test** +- [x] **Step 2: Write a child-process integration test** Fork `worker-entry.mjs` with IPC, wait for `ready`, send generation 1 settings on port 0, wait for `configured`, issue a proxied request, request status, then send shutdown and assert exit code 0. -- [ ] **Step 3: Run tests and verify failure** +- [x] **Step 3: Run tests and verify failure** ```bash cd node @@ -564,11 +566,11 @@ node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs Expected: FAIL because protocol and entrypoint modules do not exist. -- [ ] **Step 4: Implement the versioned protocol** +- [x] **Step 4: Implement the versioned protocol** Use `PROTOCOL_VERSION = 1`. Every message must contain `{ version: 1, type, requestId }`; configure additionally contains `{ generation, settings }`. Unknown versions/types return a `fatal` child message with stable code `WORKER_PROTOCOL_INVALID` and never echo the input payload. -- [ ] **Step 5: Implement graceful worker behavior** +- [x] **Step 5: Implement graceful worker behavior** `worker-entry.mjs` must: @@ -578,18 +580,20 @@ Use `PROTOCOL_VERSION = 1`. Every message must contain `{ version: 1, type, requ 4. Close capture resources and exit 0 on shutdown. 5. Emit sanitized fatal messages for uncaught startup errors. -- [ ] **Step 6: Verify and commit** +- [x] **Step 6: Verify and commit** ```bash cd node node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs npm test -git add node/src/worker node/test/worker-protocol.test.mjs node/test/integration/worker-entry.test.mjs +git add node/src/worker node/test/worker-protocol.test.mjs node/test/integration/worker-entry.test.mjs node/scripts/run-tests.mjs node/package.json git commit -m "feat: add proxy worker protocol" ``` Expected: focused and full suites pass before commit. +Actual Node 22.19 Task 6 verification: the focused protocol/entrypoint suite passes 21/21, the integration runner passes 11/11, and the full gate passes a nonduplicated 112/112 top-level group followed by 11/11 integration tests. The portable syntax gate checks 18 source files. Coverage additionally verifies provider-equivalent URL/header security, Node-compatible final authentication values, child-message secret exclusion, fixed invalid-message fatal IDs, configure-before-listen, monotonic reconfiguration, configure rejection after drain begins, retained tracked in-flight drain acknowledgement with idle keep-alive closure, idempotent duplicate drain, safe invalid/stale/port-conflict fatal exits, bounded parent-disconnect cleanup for a hanging upstream before or during shutdown, and released ports without fixed sleeps. The full runner executes real-fork integration only after watcher-bearing top-level tests complete. + ## Task 7: Implement Reliable Worker Lifecycle Management **Files:** diff --git a/node/package.json b/node/package.json index fa8a5b8..9a53307 100644 --- a/node/package.json +++ b/node/package.json @@ -27,7 +27,7 @@ "status": "node bin/crp.mjs status", "guide": "node bin/crp.mjs guide", "lint": "node scripts/check-source.mjs", - "test": "node --test", + "test": "node scripts/run-tests.mjs", "test:unit": "node scripts/run-test-group.mjs unit", "test:integration": "node scripts/run-test-group.mjs integration", "test:e2e": "playwright test", diff --git a/node/scripts/run-tests.mjs b/node/scripts/run-tests.mjs new file mode 100644 index 0000000..1935ec4 --- /dev/null +++ b/node/scripts/run-tests.mjs @@ -0,0 +1,11 @@ +import { spawnSync } from "node:child_process"; +import { fileURLToPath } from "node:url"; + +const groupRunner = fileURLToPath(new URL("./run-test-group.mjs", import.meta.url)); + +for (const group of ["unit", "integration"]) { + const result = spawnSync(process.execPath, [groupRunner, group], { stdio: "inherit" }); + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} diff --git a/node/src/worker/protocol.mjs b/node/src/worker/protocol.mjs new file mode 100644 index 0000000..20272db --- /dev/null +++ b/node/src/worker/protocol.mjs @@ -0,0 +1,351 @@ +import { validateHeaderValue } from "node:http"; + +export const PROTOCOL_VERSION = 1; + +const PARENT_TYPES = new Set(["configure", "drain", "shutdown", "status"]); +const CHILD_STATE_TYPES = new Set(["ready", "configured", "drained", "status"]); +const CHILD_TYPES = new Set([...CHILD_STATE_TYPES, "fatal"]); +const BASE_FIELDS = new Set(["version", "type", "requestId"]); +const CONFIGURE_FIELDS = new Set([...BASE_FIELDS, "generation", "settings"]); +const CHILD_STATE_FIELDS = new Set([...BASE_FIELDS, "state"]); +const FATAL_FIELDS = new Set([...BASE_FIELDS, "error"]); +const STATE_FIELDS = new Set([ + "phase", + "configured", + "generation", + "listening", + "listenHost", + "listenPort", + "inFlight" +]); +const ERROR_FIELDS = new Set(["code", "message"]); +const SETTINGS_FIELDS = new Set(["configPath", "server", "upstream", "proxy", "capture"]); +const SERVER_FIELDS = new Set(["host", "port", "logLevel"]); +const UPSTREAM_FIELDS = new Set([ + "baseUrl", + "apiKey", + "timeoutMs", + "verifySsl", + "authHeader", + "authScheme", + "extraHeaders" +]); +const PROXY_FIELDS = new Set(["overrideAuthorization", "requestIdHeader"]); +const CAPTURE_FIELDS = new Set(["enabled", "dbPath"]); +const WORKER_PHASES = new Set(["ready", "running", "draining", "drained", "stopping", "failed"]); +const REQUEST_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/; +const HEADER_NAME_PATTERN = /^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/; +const CONTROL_CHARACTER_PATTERN = /[\u0000-\u001f\u007f]/; +const SENSITIVE_HEADER_TERMS = [ + "authorization", + "cookie", + "token", + "secret", + "apikey" +]; + +const FATAL_MESSAGES = new Map([ + ["WORKER_PROTOCOL_INVALID", "Worker protocol message is invalid."], + ["WORKER_CONFIGURE_FAILED", "Worker configuration failed."], + ["WORKER_START_FAILED", "Worker failed to start."], + ["WORKER_RUNTIME_FAILED", "Worker runtime failed."], + ["STALE_SNAPSHOT", "Worker rejected a stale settings snapshot."], + ["RUNTIME_SETTINGS_INVALID", "Worker settings are invalid."] +]); + +function protocolError() { + const error = new Error("Worker protocol message is invalid."); + error.name = "WorkerProtocolError"; + error.code = "WORKER_PROTOCOL_INVALID"; + return error; +} + +function isPlainObject(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) { + return false; + } + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function hasExactFields(value, fields) { + const keys = Object.keys(value); + return keys.length === fields.size && keys.every((key) => fields.has(key)); +} + +function isRequestId(value) { + return typeof value === "string" && REQUEST_ID_PATTERN.test(value); +} + +function isNonEmptyString(value) { + return typeof value === "string" + && value.length > 0 + && !CONTROL_CHARACTER_PATTERN.test(value); +} + +function isValidHeaderName(value) { + return isNonEmptyString(value) && HEADER_NAME_PATTERN.test(value); +} + +function isSensitiveHeaderName(name) { + const compact = name.toLowerCase().replace(/[^a-z0-9]/g, ""); + return SENSITIVE_HEADER_TERMS.some((term) => compact.includes(term)); +} + +function isValidExtraHeaders(value, authHeader) { + if (!isPlainObject(value)) { + return false; + } + for (const [name, headerValue] of Object.entries(value)) { + if (!isValidHeaderName(name) + || isSensitiveHeaderName(name) + || name.toLowerCase() === authHeader.toLowerCase() + || typeof headerValue !== "string") { + return false; + } + try { + validateHeaderValue(name, headerValue); + } catch { + return false; + } + } + return true; +} + +function isValidAuthenticationHeader(upstream) { + const scheme = upstream.authScheme.trim(); + const value = scheme ? `${scheme} ${upstream.apiKey}` : upstream.apiKey; + try { + validateHeaderValue(upstream.authHeader, value); + return true; + } catch { + return false; + } +} + +function isValidBaseUrl(value) { + if (!isNonEmptyString(value)) { + return false; + } + try { + const parsed = new URL(value); + return (parsed.protocol === "http:" || parsed.protocol === "https:") + && parsed.username === "" + && parsed.password === "" + && !authorityContainsUserInfo(value) + && (parsed.protocol === "https:" || isLoopbackHostname(parsed.hostname)); + } catch { + return false; + } +} + +function authorityContainsUserInfo(value) { + const authorityStart = value.indexOf("://"); + if (authorityStart === -1) { + return false; + } + const remainder = value.slice(authorityStart + 3); + const authorityEnd = remainder.search(/[/?#]/); + const authority = authorityEnd === -1 ? remainder : remainder.slice(0, authorityEnd); + return authority.includes("@"); +} + +function isLoopbackHostname(hostname) { + const lower = hostname.toLowerCase(); + if (lower === "localhost" || lower === "::1" || lower === "[::1]") { + return true; + } + const octets = lower.split("."); + if (octets.length !== 4 || octets.some((octet) => !/^\d{1,3}$/.test(octet))) { + return false; + } + const numbers = octets.map(Number); + return numbers[0] === 127 && numbers.every((octet) => octet >= 0 && octet <= 255); +} + +function validateRuntimeSettings(settings) { + if (!isPlainObject(settings) + || !hasExactFields(settings, SETTINGS_FIELDS) + || !isNonEmptyString(settings.configPath) + || !isPlainObject(settings.server) + || !hasExactFields(settings.server, SERVER_FIELDS) + || settings.server.host !== "127.0.0.1" + || !Number.isInteger(settings.server.port) + || settings.server.port < 0 + || settings.server.port > 65535 + || !isNonEmptyString(settings.server.logLevel) + || !isPlainObject(settings.upstream) + || !hasExactFields(settings.upstream, UPSTREAM_FIELDS) + || !isValidBaseUrl(settings.upstream.baseUrl) + || typeof settings.upstream.apiKey !== "string" + || !Number.isFinite(settings.upstream.timeoutMs) + || settings.upstream.timeoutMs <= 0 + || typeof settings.upstream.verifySsl !== "boolean" + || !isValidHeaderName(settings.upstream.authHeader) + || typeof settings.upstream.authScheme !== "string" + || CONTROL_CHARACTER_PATTERN.test(settings.upstream.authScheme) + || (settings.upstream.authScheme !== "" + && !HEADER_NAME_PATTERN.test(settings.upstream.authScheme)) + || !isValidExtraHeaders(settings.upstream.extraHeaders, settings.upstream.authHeader) + || !isPlainObject(settings.proxy) + || !hasExactFields(settings.proxy, PROXY_FIELDS) + || typeof settings.proxy.overrideAuthorization !== "boolean" + || !isValidHeaderName(settings.proxy.requestIdHeader) + || (settings.proxy.overrideAuthorization + && !isValidAuthenticationHeader(settings.upstream)) + || (settings.proxy.overrideAuthorization && settings.upstream.apiKey.length === 0) + || !isPlainObject(settings.capture) + || !hasExactFields(settings.capture, CAPTURE_FIELDS) + || typeof settings.capture.enabled !== "boolean" + || !isNonEmptyString(settings.capture.dbPath)) { + throw protocolError(); + } + return settings; +} + +function validateBase(message, allowedTypes) { + if (!isPlainObject(message) + || message.version !== PROTOCOL_VERSION + || !allowedTypes.has(message.type) + || !isRequestId(message.requestId)) { + throw protocolError(); + } +} + +function isListenHost(value) { + return value === null + || (typeof value === "string" + && value.length > 0 + && value.length <= 255 + && !CONTROL_CHARACTER_PATTERN.test(value)); +} + +function isListenPort(value) { + return value === null || (Number.isInteger(value) && value >= 0 && value <= 65535); +} + +function validateState(state, { exact = true } = {}) { + if (!isPlainObject(state) + || (exact && !hasExactFields(state, STATE_FIELDS)) + || !WORKER_PHASES.has(state.phase) + || typeof state.configured !== "boolean" + || !Number.isSafeInteger(state.generation) + || state.generation < 0 + || typeof state.listening !== "boolean" + || !isListenHost(state.listenHost) + || !isListenPort(state.listenPort) + || !Number.isSafeInteger(state.inFlight) + || state.inFlight < 0) { + throw protocolError(); + } + return state; +} + +function validateFatalError(error) { + if (!isPlainObject(error) || !hasExactFields(error, ERROR_FIELDS)) { + throw protocolError(); + } + const expectedMessage = FATAL_MESSAGES.get(error.code); + if (!expectedMessage || error.message !== expectedMessage) { + throw protocolError(); + } + return error; +} + +function projectState(state) { + try { + validateState(state, { exact: false }); + } catch { + return undefined; + } + return { + phase: state.phase, + configured: state.configured, + generation: state.generation, + listening: state.listening, + listenHost: state.listenHost, + listenPort: state.listenPort, + inFlight: state.inFlight + }; +} + +export function validateParentMessage(message) { + validateBase(message, PARENT_TYPES); + if (message.type === "configure") { + if (!hasExactFields(message, CONFIGURE_FIELDS) + || !Number.isSafeInteger(message.generation) + || message.generation <= 0 + || !isPlainObject(message.settings)) { + throw protocolError(); + } + validateRuntimeSettings(message.settings); + return message; + } + if (!hasExactFields(message, BASE_FIELDS)) { + throw protocolError(); + } + return message; +} + +export function validateChildMessage(message) { + validateBase(message, CHILD_TYPES); + if (CHILD_STATE_TYPES.has(message.type)) { + if (!hasExactFields(message, CHILD_STATE_FIELDS)) { + throw protocolError(); + } + validateState(message.state); + return message; + } + if (!hasExactFields(message, FATAL_FIELDS)) { + throw protocolError(); + } + validateFatalError(message.error); + return message; +} + +export function sanitizeProtocolMessage(message) { + if (!isPlainObject(message) + || message.version !== PROTOCOL_VERSION + || (!PARENT_TYPES.has(message.type) && !CHILD_TYPES.has(message.type)) + || !isRequestId(message.requestId)) { + return {}; + } + + const result = { + version: PROTOCOL_VERSION, + type: message.type, + requestId: message.requestId + }; + if (message.type === "configure" && Number.isSafeInteger(message.generation) && message.generation > 0) { + result.generation = message.generation; + } + if (CHILD_STATE_TYPES.has(message.type)) { + const state = projectState(message.state); + if (state) { + result.state = state; + } + } + if (message.type === "fatal" && isPlainObject(message.error)) { + const messageText = FATAL_MESSAGES.get(message.error.code); + if (messageText) { + result.error = { + code: message.error.code, + message: messageText + }; + } + } + return result; +} + +export function createFatalMessage({ requestId, code }) { + const safeCode = FATAL_MESSAGES.has(code) ? code : "WORKER_RUNTIME_FAILED"; + return { + version: PROTOCOL_VERSION, + type: "fatal", + requestId: isRequestId(requestId) ? requestId : "worker-fatal", + error: { + code: safeCode, + message: FATAL_MESSAGES.get(safeCode) + } + }; +} diff --git a/node/src/worker/worker-entry.mjs b/node/src/worker/worker-entry.mjs new file mode 100644 index 0000000..4e32ff0 --- /dev/null +++ b/node/src/worker/worker-entry.mjs @@ -0,0 +1,317 @@ +import { createApp } from "../server.mjs"; +import { + PROTOCOL_VERSION, + createFatalMessage, + validateChildMessage, + validateParentMessage +} from "./protocol.mjs"; +import { RuntimeSettingsSource } from "./runtime-settings.mjs"; + +const PARENT_DISCONNECT_GRACE_MS = 250; + +const runtimeSettings = new RuntimeSettingsSource(); + +let app = null; +let phase = "ready"; +let listenHost = null; +let listenPort = null; +let inFlight = 0; +let stopping = false; +let intendedExitCode = 0; +let operation = Promise.resolve(); +let drainPromise = null; +let resourceClosePromise = null; +let parentDisconnectCleanupPromise = null; + +function getPublicState() { + const runtime = runtimeSettings.publicState(); + return { + phase, + configured: runtime.configured, + generation: runtime.generation, + listening: app?.server.listening === true, + listenHost: app?.server.listening === true ? listenHost : null, + listenPort: app?.server.listening === true ? listenPort : null, + inFlight + }; +} + +function sendChildMessage(message) { + validateChildMessage(message); + if (typeof process.send !== "function" || !process.connected) { + return Promise.reject(new Error("Worker IPC channel is unavailable.")); + } + return new Promise((resolvePromise, rejectPromise) => { + process.send(message, (error) => { + if (error) { + rejectPromise(error); + } else { + resolvePromise(); + } + }); + }); +} + +function lifecycleMessage(type, requestId) { + return { + version: PROTOCOL_VERSION, + type, + requestId, + state: getPublicState() + }; +} + +function trackRequests(server) { + server.prependListener("request", (_req, res) => { + inFlight += 1; + let completed = false; + const complete = () => { + if (completed) { + return; + } + completed = true; + inFlight -= 1; + if (drainPromise && inFlight === 0) { + setImmediate(() => { + if (drainPromise && inFlight === 0) { + app?.server.closeIdleConnections?.(); + } + }); + } + }; + res.once("finish", complete); + res.once("close", complete); + }); +} + +function listen(server, settings) { + return new Promise((resolvePromise, rejectPromise) => { + const onError = (error) => { + server.off("listening", onListening); + rejectPromise(error); + }; + const onListening = () => { + server.off("error", onError); + const address = server.address(); + listenHost = typeof address === "object" && address ? address.address : settings.server.host; + listenPort = typeof address === "object" && address ? address.port : settings.server.port; + resolvePromise(); + }; + server.once("error", onError); + server.once("listening", onListening); + server.listen(settings.server.port, settings.server.host); + }); +} + +async function closeResources() { + if (!app) { + return; + } + if (drainPromise) { + await drainPromise; + return; + } + if (!resourceClosePromise) { + if (app.server.listening) { + resourceClosePromise = new Promise((resolvePromise) => app.server.close(resolvePromise)); + } else { + app.captureManager.close(); + resourceClosePromise = Promise.resolve(); + } + } + await resourceClosePromise; +} + +async function finishProcess(exitCode) { + intendedExitCode = exitCode; + await closeResources(); + process.exitCode = exitCode; + if (process.connected) { + process.disconnect(); + } +} + +async function failWorker({ requestId, code }) { + if (stopping) { + return; + } + stopping = true; + phase = "failed"; + intendedExitCode = 1; + try { + await sendChildMessage(createFatalMessage({ requestId, code })); + } catch { + // The parent may already be gone; cleanup still has to complete. + } + await finishProcess(1); +} + +async function configure(message) { + if (stopping || (phase !== "ready" && phase !== "running")) { + const error = new Error("Worker configuration is not allowed in the current phase."); + error.code = "WORKER_CONFIGURE_FAILED"; + throw error; + } + runtimeSettings.apply({ + generation: message.generation, + settings: message.settings + }); + if (!app) { + app = createApp(message.settings, { settingsSource: runtimeSettings }); + trackRequests(app.server); + try { + await listen(app.server, message.settings); + } catch (error) { + error.workerFatalCode = "WORKER_START_FAILED"; + throw error; + } + } + phase = "running"; + await sendChildMessage(lifecycleMessage("configured", message.requestId)); +} + +async function shutdown() { + if (stopping) { + return; + } + stopping = true; + phase = "stopping"; + intendedExitCode = 0; + await finishProcess(0); +} + +function forceCloseResources() { + if (!app) { + return; + } + app.server.closeIdleConnections?.(); + app.server.closeAllConnections?.(); + app.captureManager.close(); +} + +async function performParentDisconnectCleanup() { + const exitCode = intendedExitCode; + if (!stopping) { + stopping = true; + phase = "stopping"; + } + + let deadline; + const timedOut = new Promise((resolvePromise) => { + deadline = setTimeout(() => resolvePromise(false), PARENT_DISCONNECT_GRACE_MS); + }); + const closedGracefully = closeResources() + .then(() => true) + .catch(() => false); + const outcome = await Promise.race([closedGracefully, timedOut]); + clearTimeout(deadline); + + if (outcome === true) { + process.exitCode = exitCode; + return; + } + try { + forceCloseResources(); + } finally { + process.exit(exitCode); + } +} + +function shutdownAfterParentDisconnect() { + if (!parentDisconnectCleanupPromise) { + parentDisconnectCleanupPromise = performParentDisconnectCleanup(); + } + return parentDisconnectCleanupPromise; +} + +function closeForDrain() { + if (!app?.server.listening) { + return Promise.resolve(); + } + return new Promise((resolvePromise, rejectPromise) => { + app.server.close((error) => { + if (error) { + rejectPromise(error); + } else { + resolvePromise(); + } + }); + }); +} + +function beginDrain(requestId) { + if (!drainPromise) { + phase = "draining"; + drainPromise = closeForDrain().then(() => { + phase = "drained"; + }); + } + void drainPromise + .then(() => { + phase = "drained"; + return sendChildMessage(lifecycleMessage("drained", requestId)); + }) + .catch(() => failWorker({ requestId, code: "WORKER_RUNTIME_FAILED" })); +} + +async function handleParentMessage(rawMessage) { + let message; + try { + message = validateParentMessage(rawMessage); + } catch (error) { + await failWorker({ + requestId: "worker-fatal", + code: error.code === "WORKER_PROTOCOL_INVALID" ? error.code : "WORKER_RUNTIME_FAILED" + }); + return; + } + + try { + if (message.type === "configure") { + await configure(message); + return; + } + if (message.type === "status") { + await sendChildMessage(lifecycleMessage("status", message.requestId)); + return; + } + if (message.type === "shutdown") { + await shutdown(); + return; + } + beginDrain(message.requestId); + } catch (error) { + await failWorker({ + requestId: message.requestId, + code: error.workerFatalCode ?? error.code ?? "WORKER_RUNTIME_FAILED" + }); + } +} + +function scheduleMessage(message) { + operation = operation + .then(() => handleParentMessage(message)) + .catch(() => failWorker({ requestId: "worker-fatal", code: "WORKER_RUNTIME_FAILED" })); +} + +async function startWorker() { + if (typeof process.send !== "function") { + throw new Error("Worker requires an IPC channel."); + } + process.on("message", scheduleMessage); + process.once("disconnect", () => { + void shutdownAfterParentDisconnect(); + }); + await sendChildMessage(lifecycleMessage("ready", "worker-ready")); +} + +process.once("uncaughtException", () => { + void failWorker({ requestId: "worker-fatal", code: "WORKER_RUNTIME_FAILED" }); +}); +process.once("unhandledRejection", () => { + void failWorker({ requestId: "worker-fatal", code: "WORKER_RUNTIME_FAILED" }); +}); + +void startWorker().catch(() => { + void failWorker({ requestId: "worker-fatal", code: "WORKER_START_FAILED" }); +}); diff --git a/node/test/integration/worker-entry.test.mjs b/node/test/integration/worker-entry.test.mjs new file mode 100644 index 0000000..ed6051e --- /dev/null +++ b/node/test/integration/worker-entry.test.mjs @@ -0,0 +1,953 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { fork } from "node:child_process"; +import { once } from "node:events"; +import { mkdirSync, rmSync, writeFileSync } from "node:fs"; +import http from "node:http"; +import net from "node:net"; +import os from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { validateChildMessage } from "../../src/worker/protocol.mjs"; + +const WORKER_ENTRY_PATH = fileURLToPath(new URL("../../src/worker/worker-entry.mjs", import.meta.url)); +const EVENT_DEADLINE_MS = 3000; + +function makeTempDir(prefix) { + return join(os.tmpdir(), `${prefix}-${process.pid}-${Date.now()}-${Math.random().toString(16).slice(2)}`); +} + +function createGate() { + let release; + const promise = new Promise((resolvePromise) => { + release = resolvePromise; + }); + return { promise, release }; +} + +function createSignal() { + let resolve; + const promise = new Promise((resolvePromise) => { + resolve = resolvePromise; + }); + return { promise, resolve }; +} + +function listen(server, host = "127.0.0.1") { + return new Promise((resolvePromise, rejectPromise) => { + server.once("error", rejectPromise); + server.listen(0, host, () => { + server.off("error", rejectPromise); + resolvePromise(server.address().port); + }); + }); +} + +async function closeServer(server) { + if (!server.listening) { + return; + } + const closed = once(server, "close"); + server.close(); + await closed; +} + +function withDeadline(promise, description, timeoutMs = EVENT_DEADLINE_MS) { + let timeout; + const deadline = new Promise((_, rejectPromise) => { + timeout = setTimeout(() => rejectPromise(new Error(`Timed out waiting for ${description}`)), timeoutMs); + }); + return Promise.race([promise, deadline]).finally(() => clearTimeout(timeout)); +} + +async function waitForListenerClose(port, timeoutMs = EVENT_DEADLINE_MS) { + const startedAt = Date.now(); + while (Date.now() - startedAt < timeoutMs) { + const connected = await new Promise((resolvePromise) => { + const socket = net.createConnection({ host: "127.0.0.1", port }); + socket.once("connect", () => { + socket.destroy(); + resolvePromise(true); + }); + socket.once("error", () => resolvePromise(false)); + }); + if (!connected) { + return; + } + await new Promise((resolvePromise) => setTimeout(resolvePromise, 10)); + } + throw new Error(`Timed out waiting for listener on port ${port} to close`); +} + +function waitForExit(child, timeoutMs = EVENT_DEADLINE_MS) { + if (child.exitCode !== null || child.signalCode !== null) { + return Promise.resolve({ code: child.exitCode, signal: child.signalCode }); + } + return withDeadline( + once(child, "exit").then(([code, signal]) => ({ code, signal })), + "worker exit", + timeoutMs + ); +} + +function sendMessage(child, message) { + return new Promise((resolvePromise, rejectPromise) => { + child.send(message, (error) => { + if (error) { + rejectPromise(error); + } else { + resolvePromise(); + } + }); + }); +} + +async function cleanupChild(child) { + if (child.exitCode !== null || child.signalCode !== null) { + return; + } + if (child.connected) { + try { + await sendMessage(child, { version: 1, type: "shutdown", requestId: "test-cleanup" }); + await waitForExit(child, 500); + return; + } catch { + // Escalate below. + } + } + child.kill("SIGTERM"); + try { + await waitForExit(child, 500); + } catch { + child.kill("SIGKILL"); + await waitForExit(child, 1000).catch(() => {}); + } +} + +function spawnWorker(t) { + const child = fork(WORKER_ENTRY_PATH, [], { + execPath: process.execPath, + stdio: ["ignore", "pipe", "pipe", "ipc"] + }); + const messages = []; + const waiters = new Set(); + let stdout = ""; + let stderr = ""; + + child.stdout.setEncoding("utf8"); + child.stderr.setEncoding("utf8"); + child.stdout.on("data", (chunk) => { stdout += chunk; }); + child.stderr.on("data", (chunk) => { stderr += chunk; }); + child.on("message", (message) => { + messages.push(message); + for (const waiter of [...waiters]) { + if (waiter.predicate(message)) { + waiters.delete(waiter); + waiter.resolve(message); + } + } + }); + child.on("exit", (code, signal) => { + for (const waiter of [...waiters]) { + waiters.delete(waiter); + waiter.reject(new Error(`Worker exited before ${waiter.description}: code=${code} signal=${signal}`)); + } + }); + + t.after(() => cleanupChild(child)); + + function waitForMessage(predicate, description) { + const existing = messages.find(predicate); + if (existing) { + return Promise.resolve(existing); + } + let waiter; + const pending = new Promise((resolvePromise, rejectPromise) => { + waiter = { predicate, resolve: resolvePromise, reject: rejectPromise, description }; + waiters.add(waiter); + }); + return withDeadline(pending, description).finally(() => waiters.delete(waiter)); + } + + return { + child, + messages: () => [...messages], + output: () => `${stdout}\n${stderr}`, + waitForMessage + }; +} + +function makeSettings({ baseUrl, configPath, port = 0, apiKey = "worker-integration-secret" }) { + return { + configPath, + server: { + host: "127.0.0.1", + port, + logLevel: "info" + }, + upstream: { + baseUrl, + apiKey, + timeoutMs: 5000, + verifySsl: true, + authHeader: "x-provider-auth", + authScheme: "Bearer", + extraHeaders: {} + }, + proxy: { + overrideAuthorization: true, + requestIdHeader: "x-client-request-id" + }, + capture: { + enabled: false, + dbPath: join(configPath, "..", "traffic.sqlite3") + } + }; +} + +test("worker configures once, proxies traffic, reports public state, and shuts down cleanly", async (t) => { + const dir = makeTempDir("crp-worker-entry"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + let observedAuthorization = null; + const upstream = http.createServer((req, res) => { + observedAuthorization = req.headers["x-provider-auth"] ?? null; + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ upstream: true })); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + + const configPath = join(dir, "proxy-config.json"); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + configPath + }); + writeFileSync(configPath, `${JSON.stringify(settings, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + const ready = await worker.waitForMessage( + (message) => message?.type === "ready", + "worker ready" + ); + validateChildMessage(ready); + assert.deepEqual(ready.state, { + phase: "ready", + configured: false, + generation: 0, + listening: false, + listenHost: null, + listenPort: null, + inFlight: 0 + }); + + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-1", + generation: 1, + settings + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" && message.requestId === "configure-1", + "worker configured" + ); + validateChildMessage(configured); + assert.equal(configured.state.phase, "running"); + assert.equal(configured.state.generation, 1); + assert.equal(configured.state.listening, true); + assert.equal(configured.state.listenHost, "127.0.0.1"); + assert.ok(configured.state.listenPort > 0); + + const proxyUrl = `http://127.0.0.1:${configured.state.listenPort}`; + const proxyResponse = await fetch(`${proxyUrl}/responses`, { + method: "POST", + headers: { + "content-type": "application/json", + "x-client-request-id": "proxy-request-1" + }, + body: JSON.stringify({ hello: "worker" }) + }); + assert.equal(proxyResponse.status, 200); + assert.deepEqual(await proxyResponse.json(), { upstream: true }); + assert.equal(observedAuthorization, "Bearer worker-integration-secret"); + + const healthResponse = await fetch(`${proxyUrl}/_proxy/health`); + assert.equal(healthResponse.status, 200); + const health = await healthResponse.json(); + assert.equal(health.configured, true); + assert.equal(health.generation, 1); + assert.equal(JSON.stringify(health).includes(settings.upstream.apiKey), false); + + await sendMessage(worker.child, { version: 1, type: "status", requestId: "status-1" }); + const status = await worker.waitForMessage( + (message) => message?.type === "status" && message.requestId === "status-1", + "worker status" + ); + validateChildMessage(status); + assert.deepEqual(status.state, configured.state); + assert.equal(JSON.stringify(status).includes(settings.upstream.apiKey), false); + + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-2", + generation: 2, + settings + }); + const reconfigured = await worker.waitForMessage( + (message) => message?.type === "configured" && message.requestId === "configure-2", + "worker reconfigured" + ); + assert.equal(reconfigured.state.generation, 2); + assert.equal(reconfigured.state.listenPort, configured.state.listenPort); + const reconfiguredHealth = await fetch(`${proxyUrl}/_proxy/health`).then((response) => response.json()); + assert.equal(reconfiguredHealth.generation, 2); + assert.equal(JSON.stringify(reconfiguredHealth).includes(settings.upstream.apiKey), false); + + await sendMessage(worker.child, { version: 1, type: "shutdown", requestId: "shutdown-1" }); + const exit = await waitForExit(worker.child); + assert.deepEqual(exit, { code: 0, signal: null }); + assert.equal(worker.output().includes(settings.upstream.apiKey), false); +}); + +test("worker drain rejects new traffic and waits for an in-flight request before acknowledging", async (t) => { + const dir = makeTempDir("crp-worker-drain"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const releaseUpstream = createGate(); + const receivedUpstream = createSignal(); + t.after(() => releaseUpstream.release()); + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + receivedUpstream.resolve(); + releaseUpstream.promise.then(() => { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ drained: true })); + }); + }); + }); + const upstreamPort = await listen(upstream); + t.after(async () => { + releaseUpstream.release(); + await closeServer(upstream); + }); + + const configPath = join(dir, "proxy-config.json"); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + configPath, + apiKey: "worker-drain-secret" + }); + writeFileSync(configPath, `${JSON.stringify(settings, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-drain", + generation: 1, + settings + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" && message.requestId === "configure-drain", + "worker configured" + ); + const proxyUrl = `http://127.0.0.1:${configured.state.listenPort}`; + + const firstRequest = fetch(`${proxyUrl}/responses`, { + method: "POST", + body: "in-flight" + }); + t.after(() => firstRequest.catch(() => {})); + await withDeadline(receivedUpstream.promise, "upstream receiving in-flight request"); + + await sendMessage(worker.child, { version: 1, type: "drain", requestId: "drain-1" }); + await sendMessage(worker.child, { version: 1, type: "status", requestId: "status-draining" }); + const draining = await worker.waitForMessage( + (message) => message?.type === "status" && message.requestId === "status-draining", + "draining worker status" + ); + assert.equal(draining.state.phase, "draining"); + assert.equal(draining.state.inFlight, 1); + assert.equal(draining.state.listening, false); + + await assert.rejects(fetch(`${proxyUrl}/responses`, { + method: "POST", + body: "must-not-be-accepted" + })); + + releaseUpstream.release(); + const firstResponse = await firstRequest; + assert.equal(firstResponse.status, 200); + assert.deepEqual(await firstResponse.json(), { drained: true }); + + await sendMessage(worker.child, { version: 1, type: "status", requestId: "status-after-response" }); + const afterResponse = await worker.waitForMessage( + (message) => message?.type === "status" && message.requestId === "status-after-response", + "post-response worker status" + ); + assert.equal(afterResponse.state.phase, "drained"); + assert.equal(afterResponse.state.inFlight, 0); + + const drained = await worker.waitForMessage( + (message) => message?.type === "drained" && message.requestId === "drain-1", + "worker drained" + ); + validateChildMessage(drained); + assert.equal(drained.state.phase, "drained"); + assert.equal(drained.state.inFlight, 0); + assert.equal(drained.state.listening, false); + + await sendMessage(worker.child, { version: 1, type: "shutdown", requestId: "shutdown-drained" }); + assert.deepEqual(await waitForExit(worker.child), { code: 0, signal: null }); + assert.equal(worker.output().includes(settings.upstream.apiKey), false); +}); + +test("worker keeps duplicate drain acknowledgements and status drained", async (t) => { + const dir = makeTempDir("crp-worker-duplicate-drain"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const configPath = join(dir, "proxy-config.json"); + const settings = makeSettings({ + baseUrl: "http://127.0.0.1:9", + configPath, + apiKey: "worker-duplicate-drain-secret" + }); + writeFileSync(configPath, `${JSON.stringify(settings, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-duplicate-drain", + generation: 1, + settings + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" + && message.requestId === "configure-duplicate-drain", + "worker configured" + ); + + await sendMessage(worker.child, { version: 1, type: "drain", requestId: "drain-first" }); + const firstDrained = await worker.waitForMessage( + (message) => message?.type === "drained" && message.requestId === "drain-first", + "first drain acknowledgement" + ); + assert.equal(firstDrained.state.phase, "drained"); + assert.equal(firstDrained.state.listening, false); + + await sendMessage(worker.child, { version: 1, type: "drain", requestId: "drain-second" }); + const secondDrained = await worker.waitForMessage( + (message) => message?.type === "drained" && message.requestId === "drain-second", + "second drain acknowledgement" + ); + assert.equal(secondDrained.state.phase, "drained"); + assert.equal(secondDrained.state.listening, false); + + await sendMessage(worker.child, { version: 1, type: "status", requestId: "status-after-drains" }); + const status = await worker.waitForMessage( + (message) => message?.type === "status" && message.requestId === "status-after-drains", + "status after duplicate drain" + ); + assert.equal(status.state.phase, "drained"); + assert.equal(status.state.listening, false); + + await sendMessage(worker.child, { version: 1, type: "shutdown", requestId: "shutdown-duplicate-drain" }); + assert.deepEqual(await waitForExit(worker.child), { code: 0, signal: null }); + assert.equal(worker.output().includes(settings.upstream.apiKey), false); + + const portProbe = http.createServer(); + await new Promise((resolvePromise, rejectPromise) => { + portProbe.once("error", rejectPromise); + portProbe.listen(configured.state.listenPort, "127.0.0.1", resolvePromise); + }); + await closeServer(portProbe); +}); + +test("worker rejects configure during drain without losing the in-flight drain", async (t) => { + const dir = makeTempDir("crp-worker-drain-configure"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const releaseUpstream = createGate(); + const receivedUpstream = createSignal(); + t.after(() => releaseUpstream.release()); + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + receivedUpstream.resolve(); + releaseUpstream.promise.then(() => { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ drained: true })); + }); + }); + }); + const upstreamPort = await listen(upstream); + t.after(async () => { + releaseUpstream.release(); + await closeServer(upstream); + }); + + const configPath = join(dir, "proxy-config.json"); + const settingsA = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + configPath, + apiKey: "worker-drain-race-a-secret" + }); + const settingsB = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + configPath, + apiKey: "worker-drain-race-b-secret" + }); + writeFileSync(configPath, `${JSON.stringify(settingsA, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-drain-race-a", + generation: 1, + settings: settingsA + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" && message.requestId === "configure-drain-race-a", + "worker configured" + ); + const proxyUrl = `http://127.0.0.1:${configured.state.listenPort}`; + + const firstRequest = fetch(`${proxyUrl}/responses`, { + method: "POST", + body: "in-flight" + }); + t.after(() => firstRequest.catch(() => {})); + await withDeadline(receivedUpstream.promise, "upstream receiving drain-race request"); + + await sendMessage(worker.child, { version: 1, type: "drain", requestId: "drain-race" }); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-during-drain", + generation: 2, + settings: settingsB + }); + const outcome = await worker.waitForMessage( + (message) => message?.requestId === "configure-during-drain" + && (message.type === "fatal" || message.type === "configured"), + "configure-during-drain outcome" + ); + releaseUpstream.release(); + + assert.equal(outcome.type, "fatal"); + validateChildMessage(outcome); + assert.deepEqual(outcome.error, { + code: "WORKER_CONFIGURE_FAILED", + message: "Worker configuration failed." + }); + + const firstResponse = await firstRequest; + assert.equal(firstResponse.status, 200); + assert.deepEqual(await firstResponse.json(), { drained: true }); + const drained = await worker.waitForMessage( + (message) => message?.type === "drained" && message.requestId === "drain-race", + "drain-race acknowledgement" + ); + assert.equal(drained.state.phase, "drained"); + assert.equal(drained.state.listening, false); + assert.equal(drained.state.inFlight, 0); + + assert.deepEqual(await waitForExit(worker.child), { code: 1, signal: null }); + assert.equal( + worker.messages().some((message) => ( + message?.type === "configured" && message.requestId === "configure-during-drain" + )), + false + ); + const output = `${JSON.stringify(worker.messages())}\n${worker.output()}`; + assert.equal(output.includes(settingsA.upstream.apiKey), false); + assert.equal(output.includes(settingsB.upstream.apiKey), false); + + const portProbe = http.createServer(); + await new Promise((resolvePromise, rejectPromise) => { + portProbe.once("error", rejectPromise); + portProbe.listen(configured.state.listenPort, "127.0.0.1", resolvePromise); + }); + await closeServer(portProbe); +}); + +test("worker rejects an invalid configure message before listening without echoing its payload", async (t) => { + const worker = spawnWorker(t); + const ready = await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + assert.equal(ready.state.listening, false); + + const reflectedRequestId = "sk-secret-sentinel"; + const settings = makeSettings({ + baseUrl: "http://127.0.0.1:9", + configPath: "/tmp/crp-invalid-worker-config.json", + apiKey: "invalid-configure-secret" + }); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: reflectedRequestId, + generation: 1, + settings, + unexpected: "invalid-payload-secret" + }); + + const fatal = await worker.waitForMessage( + (message) => message?.type === "fatal", + "invalid protocol fatal" + ); + validateChildMessage(fatal); + assert.deepEqual(fatal, { + version: 1, + type: "fatal", + requestId: "worker-fatal", + error: { + code: "WORKER_PROTOCOL_INVALID", + message: "Worker protocol message is invalid." + } + }); + const serialized = JSON.stringify(fatal); + assert.equal(serialized.includes("invalid-configure-secret"), false); + assert.equal(serialized.includes("invalid-payload-secret"), false); + assert.equal(serialized.includes(reflectedRequestId), false); + assert.deepEqual(await waitForExit(worker.child), { code: 1, signal: null }); + assert.equal(worker.output().includes("invalid-configure-secret"), false); + assert.equal(worker.output().includes("invalid-payload-secret"), false); + assert.equal(worker.output().includes(reflectedRequestId), false); +}); + +test("worker rejects an unsendable API key without leaking it", async (t) => { + const apiKey = "invalid-key\r\nsk-secret-api-key-sentinel"; + const settings = makeSettings({ + baseUrl: "http://127.0.0.1:9", + configPath: "/tmp/crp-invalid-worker-api-key.json", + apiKey + }); + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-invalid-api-key", + generation: 1, + settings + }); + + const outcome = await worker.waitForMessage( + (message) => message?.type === "fatal" || message?.type === "configured", + "invalid API key outcome" + ); + assert.equal(outcome.type, "fatal"); + validateChildMessage(outcome); + assert.deepEqual(outcome, { + version: 1, + type: "fatal", + requestId: "worker-fatal", + error: { + code: "WORKER_PROTOCOL_INVALID", + message: "Worker protocol message is invalid." + } + }); + assert.deepEqual(await waitForExit(worker.child), { code: 1, signal: null }); + const output = `${JSON.stringify(worker.messages())}\n${worker.output()}`; + assert.equal(output.includes(apiKey), false); + assert.equal(output.includes("sk-secret-api-key-sentinel"), false); +}); + +test("worker rejects a stale generation safely and releases its listening port", async (t) => { + const dir = makeTempDir("crp-worker-stale"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const configPath = join(dir, "proxy-config.json"); + const settingsA = makeSettings({ + baseUrl: "http://127.0.0.1:9", + configPath, + apiKey: "worker-stale-a-secret" + }); + const settingsB = makeSettings({ + baseUrl: "http://127.0.0.1:10", + configPath, + apiKey: "worker-stale-b-secret" + }); + writeFileSync(configPath, `${JSON.stringify(settingsA, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-stale-a", + generation: 1, + settings: settingsA + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" && message.requestId === "configure-stale-a", + "worker configured" + ); + + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-stale-b", + generation: 1, + settings: settingsB + }); + const fatal = await worker.waitForMessage( + (message) => message?.type === "fatal" && message.requestId === "configure-stale-b", + "stale snapshot fatal" + ); + validateChildMessage(fatal); + assert.equal(fatal.error.code, "STALE_SNAPSHOT"); + assert.equal(JSON.stringify(fatal).includes(settingsA.upstream.apiKey), false); + assert.equal(JSON.stringify(fatal).includes(settingsB.upstream.apiKey), false); + assert.deepEqual(await waitForExit(worker.child), { code: 1, signal: null }); + assert.equal(worker.output().includes(settingsA.upstream.apiKey), false); + assert.equal(worker.output().includes(settingsB.upstream.apiKey), false); + + const portProbe = http.createServer(); + await new Promise((resolvePromise, rejectPromise) => { + portProbe.once("error", rejectPromise); + portProbe.listen(configured.state.listenPort, "127.0.0.1", resolvePromise); + }); + await closeServer(portProbe); +}); + +test("worker reports a sanitized startup fatal when the requested port is occupied", async (t) => { + const dir = makeTempDir("crp-worker-port-busy"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const blocker = http.createServer(); + const blockedPort = await listen(blocker); + t.after(() => closeServer(blocker)); + + const configPath = join(dir, "proxy-config.json"); + const settings = makeSettings({ + baseUrl: "http://127.0.0.1:9", + configPath, + port: blockedPort, + apiKey: "worker-port-busy-secret" + }); + writeFileSync(configPath, `${JSON.stringify(settings, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-port-busy", + generation: 1, + settings + }); + const fatal = await worker.waitForMessage( + (message) => message?.type === "fatal" && message.requestId === "configure-port-busy", + "port conflict fatal" + ); + validateChildMessage(fatal); + assert.deepEqual(fatal.error, { + code: "WORKER_START_FAILED", + message: "Worker failed to start." + }); + assert.equal(JSON.stringify(fatal).includes(settings.upstream.apiKey), false); + assert.deepEqual(await waitForExit(worker.child), { code: 1, signal: null }); + assert.equal(worker.output().includes(settings.upstream.apiKey), false); + assert.equal(blocker.listening, true); +}); + +test("worker closes resources when the parent IPC channel disconnects", async (t) => { + const dir = makeTempDir("crp-worker-disconnect"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const configPath = join(dir, "proxy-config.json"); + const settings = makeSettings({ + baseUrl: "http://127.0.0.1:9", + configPath, + apiKey: "worker-disconnect-secret" + }); + writeFileSync(configPath, `${JSON.stringify(settings, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-disconnect", + generation: 1, + settings + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" && message.requestId === "configure-disconnect", + "worker configured" + ); + + worker.child.disconnect(); + assert.deepEqual(await waitForExit(worker.child), { code: 0, signal: null }); + assert.equal(worker.output().includes(settings.upstream.apiKey), false); + + const portProbe = http.createServer(); + await new Promise((resolvePromise, rejectPromise) => { + portProbe.once("error", rejectPromise); + portProbe.listen(configured.state.listenPort, "127.0.0.1", resolvePromise); + }); + await closeServer(portProbe); +}); + +test("worker bounds parent disconnect cleanup with a hanging upstream request", async (t) => { + const dir = makeTempDir("crp-worker-disconnect-hanging"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const receivedUpstream = createSignal(); + const upstream = http.createServer((req) => { + req.resume(); + req.on("end", () => receivedUpstream.resolve()); + }); + const upstreamPort = await listen(upstream); + t.after(async () => { + upstream.closeAllConnections?.(); + await closeServer(upstream); + }); + + const configPath = join(dir, "proxy-config.json"); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + configPath, + apiKey: "worker-disconnect-hanging-secret" + }); + settings.upstream.timeoutMs = 30_000; + writeFileSync(configPath, `${JSON.stringify(settings, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-disconnect-hanging", + generation: 1, + settings + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" + && message.requestId === "configure-disconnect-hanging", + "worker configured" + ); + const proxyRequestOutcome = fetch(`http://127.0.0.1:${configured.state.listenPort}/responses`, { + method: "POST", + body: "hang-until-parent-disconnect" + }).then( + () => ({ status: "resolved" }), + () => ({ status: "rejected" }) + ); + await withDeadline(receivedUpstream.promise, "upstream receiving disconnect request"); + + worker.child.disconnect(); + assert.deepEqual(await waitForExit(worker.child, 1500), { code: 0, signal: null }); + const requestOutcome = await withDeadline( + proxyRequestOutcome, + "proxy request settling after parent disconnect", + 1500 + ); + assert.equal(requestOutcome.status, "rejected"); + + const output = `${JSON.stringify(worker.messages())}\n${worker.output()}`; + assert.equal(output.includes(settings.upstream.apiKey), false); + + const portProbe = http.createServer(); + await new Promise((resolvePromise, rejectPromise) => { + portProbe.once("error", rejectPromise); + portProbe.listen(configured.state.listenPort, "127.0.0.1", resolvePromise); + }); + await closeServer(portProbe); +}); + +test("worker bounds parent disconnect cleanup after shutdown is already waiting", async (t) => { + const dir = makeTempDir("crp-worker-shutdown-disconnect-hanging"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const receivedUpstream = createSignal(); + const upstream = http.createServer((req) => { + req.resume(); + req.on("end", () => receivedUpstream.resolve()); + }); + const upstreamPort = await listen(upstream); + t.after(async () => { + upstream.closeAllConnections?.(); + await closeServer(upstream); + }); + + const configPath = join(dir, "proxy-config.json"); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + configPath, + apiKey: "worker-shutdown-disconnect-hanging-secret" + }); + settings.upstream.timeoutMs = 30_000; + writeFileSync(configPath, `${JSON.stringify(settings, null, 2)}\n`, "utf8"); + + const worker = spawnWorker(t); + await worker.waitForMessage((message) => message?.type === "ready", "worker ready"); + await sendMessage(worker.child, { + version: 1, + type: "configure", + requestId: "configure-shutdown-disconnect-hanging", + generation: 1, + settings + }); + const configured = await worker.waitForMessage( + (message) => message?.type === "configured" + && message.requestId === "configure-shutdown-disconnect-hanging", + "worker configured" + ); + const proxyRequestOutcome = fetch(`http://127.0.0.1:${configured.state.listenPort}/responses`, { + method: "POST", + body: "hang-through-shutdown-until-parent-disconnect" + }).then( + () => ({ status: "resolved" }), + () => ({ status: "rejected" }) + ); + await withDeadline(receivedUpstream.promise, "upstream receiving shutdown-disconnect request"); + + await sendMessage(worker.child, { + version: 1, + type: "shutdown", + requestId: "shutdown-before-parent-disconnect" + }); + await waitForListenerClose(configured.state.listenPort); + worker.child.disconnect(); + + assert.deepEqual(await waitForExit(worker.child, 1500), { code: 0, signal: null }); + const requestOutcome = await withDeadline( + proxyRequestOutcome, + "proxy request settling after shutdown and parent disconnect", + 1500 + ); + assert.equal(requestOutcome.status, "rejected"); + + const output = `${JSON.stringify(worker.messages())}\n${worker.output()}`; + assert.equal(output.includes(settings.upstream.apiKey), false); + + const portProbe = http.createServer(); + await new Promise((resolvePromise, rejectPromise) => { + portProbe.once("error", rejectPromise); + portProbe.listen(configured.state.listenPort, "127.0.0.1", resolvePromise); + }); + await closeServer(portProbe); +}); diff --git a/node/test/worker-protocol.test.mjs b/node/test/worker-protocol.test.mjs new file mode 100644 index 0000000..9491784 --- /dev/null +++ b/node/test/worker-protocol.test.mjs @@ -0,0 +1,390 @@ +import test from "node:test"; +import assert from "node:assert/strict"; + +import { + PROTOCOL_VERSION, + createFatalMessage, + sanitizeProtocolMessage, + validateChildMessage, + validateParentMessage +} from "../src/worker/protocol.mjs"; + +function makeSettings() { + return { + configPath: "/tmp/crp-worker/proxy-config.json", + server: { + host: "127.0.0.1", + port: 0, + logLevel: "info" + }, + upstream: { + baseUrl: "http://127.0.0.1:41001", + apiKey: "protocol-test-secret", + timeoutMs: 5000, + verifySsl: true, + authHeader: "x-provider-auth", + authScheme: "Bearer", + extraHeaders: { + "x-region": "test" + } + }, + proxy: { + overrideAuthorization: true, + requestIdHeader: "x-client-request-id" + }, + capture: { + enabled: false, + dbPath: "/tmp/crp-worker/traffic.sqlite3" + } + }; +} + +function makeState(overrides = {}) { + return { + phase: "running", + configured: true, + generation: 1, + listening: true, + listenHost: "127.0.0.1", + listenPort: 15100, + inFlight: 0, + ...overrides + }; +} + +test("parent protocol accepts exact configure, drain, shutdown, and status messages", () => { + assert.equal(PROTOCOL_VERSION, 1); + const messages = [ + { + version: 1, + type: "configure", + requestId: "configure-1", + generation: 1, + settings: makeSettings() + }, + { version: 1, type: "drain", requestId: "drain-1" }, + { version: 1, type: "shutdown", requestId: "shutdown-1" }, + { version: 1, type: "status", requestId: "status-1" } + ]; + + for (const message of messages) { + assert.deepEqual(validateParentMessage(message), message); + } +}); + +test("parent protocol rejects unknown, malformed, and secret-bearing non-configure messages", () => { + const invalidMessages = [ + null, + [], + { version: 2, type: "status", requestId: "status-1" }, + { version: 1, type: "unknown", requestId: "unknown-1" }, + { version: 1, type: "status", requestId: "" }, + { version: 1, type: "status", requestId: "contains spaces" }, + { version: 1, type: "status", requestId: "status-1", extra: true }, + { version: 1, type: "drain", requestId: "drain-1", apiKey: "must-not-pass" }, + { version: 1, type: "configure", requestId: "configure-1", generation: 0, settings: makeSettings() }, + { version: 1, type: "configure", requestId: "configure-1", generation: 1 }, + { version: 1, type: "configure", requestId: "configure-1", generation: 1, settings: [] } + ]; + + for (const message of invalidMessages) { + assert.throws( + () => validateParentMessage(message), + (error) => error?.code === "WORKER_PROTOCOL_INVALID" + && !String(error.message).includes("must-not-pass") + ); + } +}); + +test("configure rejects incomplete, extra, and invalid runtime settings before worker startup", () => { + const missingServer = makeSettings(); + delete missingServer.server; + const extraRootField = { ...makeSettings(), credentialRef: "must-not-pass" }; + const extraNestedField = makeSettings(); + extraNestedField.upstream.secret = "must-not-pass"; + const invalidPort = makeSettings(); + invalidPort.server.port = -1; + const invalidTimeout = makeSettings(); + invalidTimeout.upstream.timeoutMs = Number.POSITIVE_INFINITY; + const emptyApiKey = makeSettings(); + emptyApiKey.upstream.apiKey = ""; + const invalidExtraHeaders = makeSettings(); + invalidExtraHeaders.upstream.extraHeaders = { "x-region": 123 }; + + for (const settings of [ + missingServer, + extraRootField, + extraNestedField, + invalidPort, + invalidTimeout, + emptyApiKey, + invalidExtraHeaders + ]) { + assert.throws( + () => validateParentMessage({ + version: 1, + type: "configure", + requestId: "configure-invalid", + generation: 1, + settings + }), + (error) => error?.code === "WORKER_PROTOCOL_INVALID" + && !String(error.message).includes("must-not-pass") + ); + } +}); + +test("configure enforces provider URL and header security contracts", () => { + for (const baseUrl of [ + "https://api.example.com/v1", + "http://localhost:41001/v1", + "http://127.42.0.9:41001/v1", + "http://[::1]:41001/v1" + ]) { + const settings = makeSettings(); + settings.upstream.baseUrl = baseUrl; + assert.doesNotThrow(() => validateParentMessage({ + version: 1, + type: "configure", + requestId: "configure-valid-security", + generation: 1, + settings + })); + } + + const remotePlaintext = makeSettings(); + remotePlaintext.upstream.baseUrl = "http://api.example.com/v1"; + const spacedAuthScheme = makeSettings(); + spacedAuthScheme.upstream.authScheme = "Bearer token"; + const controlAuthScheme = makeSettings(); + controlAuthScheme.upstream.authScheme = "Bearer\n"; + const invalidAuthHeader = makeSettings(); + invalidAuthHeader.upstream.authHeader = "x provider auth"; + const unsafeHeaderValue = makeSettings(); + unsafeHeaderValue.upstream.extraHeaders = { "x-region": "safe\r\nmust-not-pass" }; + + const invalidSettings = [ + remotePlaintext, + spacedAuthScheme, + controlAuthScheme, + invalidAuthHeader, + unsafeHeaderValue + ]; + for (const name of [ + "Authorization", + "Proxy-Authorization", + "Cookie", + "Set-Cookie", + "X-API-Key", + "X-Service-Secret", + "X-PROVIDER-AUTH" + ]) { + const settings = makeSettings(); + settings.upstream.extraHeaders = { [name]: "must-not-pass" }; + invalidSettings.push(settings); + } + + for (const settings of invalidSettings) { + assert.throws( + () => validateParentMessage({ + version: 1, + type: "configure", + requestId: "configure-invalid-security", + generation: 1, + settings + }), + (error) => error?.code === "WORKER_PROTOCOL_INVALID" + && error.message === "Worker protocol message is invalid." + && !String(error.message).includes("must-not-pass") + ); + } +}); + +test("configure rejects API keys that cannot form an HTTP authentication header", () => { + const validSettings = makeSettings(); + validSettings.upstream.apiKey = "sk-test_123.abc-XYZ"; + assert.doesNotThrow(() => validateParentMessage({ + version: 1, + type: "configure", + requestId: "configure-valid-api-key", + generation: 1, + settings: validSettings + })); + + for (const apiKey of [ + "line-one\r\nx-injected: must-not-pass", + "control-\u0000must-not-pass", + "unicode-\u{1f512}-must-not-pass" + ]) { + const settings = makeSettings(); + settings.upstream.apiKey = apiKey; + assert.throws( + () => validateParentMessage({ + version: 1, + type: "configure", + requestId: "configure-invalid-api-key", + generation: 1, + settings + }), + (error) => error?.code === "WORKER_PROTOCOL_INVALID" + && error.message === "Worker protocol message is invalid." + && !String(error.message).includes(apiKey) + ); + } +}); + +test("child protocol accepts exact lifecycle acknowledgements, status, and fatal messages", () => { + const messages = [ + { version: 1, type: "ready", requestId: "worker-ready", state: makeState({ + phase: "ready", + configured: false, + generation: 0, + listening: false, + listenHost: null, + listenPort: null + }) }, + { version: 1, type: "configured", requestId: "configure-1", state: makeState() }, + { version: 1, type: "drained", requestId: "drain-1", state: makeState({ + phase: "drained", + listening: false, + listenHost: null, + listenPort: null + }) }, + { version: 1, type: "status", requestId: "status-1", state: makeState() }, + { + version: 1, + type: "fatal", + requestId: "configure-1", + error: { + code: "WORKER_CONFIGURE_FAILED", + message: "Worker configuration failed." + } + } + ]; + + for (const message of messages) { + assert.deepEqual(validateChildMessage(message), message); + } +}); + +test("child protocol rejects invalid state, extra fields, and secret-bearing errors", () => { + const invalidMessages = [ + { version: 1, type: "ready", requestId: "worker-ready" }, + { version: 1, type: "configured", requestId: "configure-1", state: makeState({ generation: -1 }) }, + { version: 1, type: "status", requestId: "status-1", state: makeState({ inFlight: 1.5 }) }, + { version: 1, type: "drained", requestId: "drain-1", state: { ...makeState(), apiKey: "must-not-pass" } }, + { version: 1, type: "fatal", requestId: "fatal-1", error: { code: "bad-code", message: "bad" } }, + { + version: 1, + type: "fatal", + requestId: "fatal-1", + error: { + code: "WORKER_START_FAILED", + message: "Worker failed.", + secret: "must-not-pass" + } + } + ]; + + for (const message of invalidMessages) { + assert.throws( + () => validateChildMessage(message), + (error) => error?.code === "WORKER_PROTOCOL_INVALID" + && !String(error.message).includes("must-not-pass") + ); + } +}); + +test("sanitizeProtocolMessage removes configure settings and projects only safe fields", () => { + const message = { + version: 1, + type: "configure", + requestId: "configure-1", + generation: 7, + settings: makeSettings(), + authorization: "complete-authorization-secret", + nested: { + cookie: "complete-cookie-secret" + } + }; + + const sanitized = sanitizeProtocolMessage(message); + assert.deepEqual(sanitized, { + version: 1, + type: "configure", + requestId: "configure-1", + generation: 7 + }); + const serialized = JSON.stringify(sanitized); + for (const forbidden of [ + "settings", + "apiKey", + "authorization", + "cookie", + "protocol-test-secret", + "complete-authorization-secret", + "complete-cookie-secret" + ]) { + assert.equal(serialized.includes(forbidden), false, `sanitized message leaked ${forbidden}`); + } +}); + +test("sanitizeProtocolMessage allowlists child state and replaces arbitrary fatal details", () => { + const stateMessage = { + version: 1, + type: "status", + requestId: "status-1", + state: { + ...makeState(), + apiKey: "state-secret", + settings: makeSettings() + } + }; + assert.deepEqual(sanitizeProtocolMessage(stateMessage), { + version: 1, + type: "status", + requestId: "status-1", + state: makeState() + }); + + const fatal = sanitizeProtocolMessage({ + version: 1, + type: "fatal", + requestId: "fatal-1", + error: { + code: "WORKER_START_FAILED", + message: "backend included complete-secret-value", + stack: "complete-secret-value" + } + }); + assert.deepEqual(fatal, { + version: 1, + type: "fatal", + requestId: "fatal-1", + error: { + code: "WORKER_START_FAILED", + message: "Worker failed to start." + } + }); + assert.equal(JSON.stringify(fatal).includes("complete-secret-value"), false); +}); + +test("createFatalMessage uses a safe fallback request ID and never echoes causes", () => { + const message = createFatalMessage({ + requestId: "unsafe request id complete-secret-value", + code: "WORKER_PROTOCOL_INVALID", + message: "Protocol rejected complete-secret-value", + cause: new Error("complete-secret-value") + }); + + assert.deepEqual(message, { + version: 1, + type: "fatal", + requestId: "worker-fatal", + error: { + code: "WORKER_PROTOCOL_INVALID", + message: "Worker protocol message is invalid." + } + }); + assert.equal(JSON.stringify(message).includes("complete-secret-value"), false); +}); From 0ebb18e53058a397126a1ddc73fafa7b40bcd56a Mon Sep 17 00:00:00 2001 From: cluic Date: Mon, 13 Jul 2026 09:34:58 +0800 Subject: [PATCH 10/44] feat: manage proxy worker lifecycle --- docs/AGENT_COORDINATION.md | 2 +- docs/AI_HANDOFF.md | 16 +- docs/ARCHITECTURE.md | 9 +- docs/DATA_MODEL.md | 2 +- docs/PERMISSIONS.md | 1 + docs/STATUS.md | 7 +- docs/TESTING.md | 10 +- .../plans/2026-07-10-crp-v1-implementation.md | 12 +- node/scripts/run-test-group.mjs | 10 +- node/scripts/run-tests.mjs | 2 +- node/src/supervisor/worker-manager.mjs | 787 ++++++++++++++++++ node/test/fixtures/fake-worker.mjs | 103 +++ node/test/integration/worker-restart.test.mjs | 111 +++ node/test/worker-manager.test.mjs | 732 ++++++++++++++++ 14 files changed, 1783 insertions(+), 21 deletions(-) create mode 100644 node/src/supervisor/worker-manager.mjs create mode 100644 node/test/fixtures/fake-worker.mjs create mode 100644 node/test/integration/worker-restart.test.mjs create mode 100644 node/test/worker-manager.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 7bc1eeb..d823005 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus Task 6 implementation agent | Resolve the final Task 6 shutdown/disconnect escalation review finding with one writer | Task 6 owns `node/src/worker/worker-entry.mjs`, `node/test/integration/worker-entry.test.mjs`, and affected living docs; the completed protocol and grouped test runner remain unchanged | Task 6 bounded shutdown/disconnect escalation fix complete on `codex/harness-product-design`; Task 7 is next | Task 6 amended commit | Task 5 server/runtime settings and capture source/tests, test runner/package, supervisor/manager, providers, credentials, Codex, CLI/bin, Admin API, UI, and every other module; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus Task 7 implementation agent | Resolve Task 7 lifecycle review findings with one writer | Task 7 owns `node/src/supervisor/worker-manager.mjs`, `node/test/fixtures/fake-worker.mjs`, `node/test/worker-manager.test.mjs`, `node/test/integration/worker-restart.test.mjs`, `node/scripts/run-tests.mjs`, the minimal group selection in `node/scripts/run-test-group.mjs`, and affected living docs | Task 7 review fixes complete and deterministic gates pass; pending re-review | Task 7 commit `9129df281a3fb58724aeb5866d0bb08aabf2747d` | Task 6 protocol/worker entry, Task 5 server/runtime settings and capture source/tests, provider and credential stores, Codex, CLI/bin, Admin API, UI, package scripts, and every other module; no simultaneous writable agent | ## Shared Contracts diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index c1ea738..d37a56c 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,11 +6,11 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 6 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, and the versioned proxy-worker IPC entrypoint. Task 7, reliable worker lifecycle management, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 7 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, the versioned proxy-worker IPC entrypoint, and reliable fixed-port worker lifecycle management. Task 8, activity, migration, and provider orchestration, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry, credential adapters with explicit-only file fallback, monotonic immutable runtime snapshots captured once per proxied request, and a strict version-1 child-process protocol whose worker listens only after valid configuration. Target: worker-manager orchestration, provider-service orchestration, and the long-lived supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry, credential adapters with explicit-only file fallback, monotonic immutable runtime snapshots captured once per proxied request, a strict version-1 child-process protocol whose worker listens only after valid configuration, and a worker manager with acknowledged configuration, bounded drain/termination, fixed-port release confirmation, health verification, epoch isolation, and bounded crash recovery. Target: provider-service orchestration and the long-lived supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 6 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 7 are complete. ## How To Run Current Code @@ -47,12 +47,13 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 Task 4 gate: `node --test test/credential-store.test.mjs` passes 41/41, the combined credential/provider focus passes 64/64, `npm test` passes 91/91, and `npm run lint` syntax-checks 14 source files. Coverage includes construction-only fallback without operation replay, explicit file-label restart continuity, descriptor identity, strict parent/file modes, degraded temp cleanup, canonical lock restoration, claim-before-delete gate release, foreign replacement preservation, and synchronous second-instance blocking while a gate claim is validated. Native tests inject the loader and never invoke the real addon loader or touch the OS credential store; real native verification remains L3 on every supported system, including Windows and Linux. - Node 22.19 Task 5 gate: `node --test test/runtime-settings.test.mjs test/server.test.mjs` passes 13/13, `npm test` passes 102/102, and `npm run lint` syntax-checks 15 source files. Coverage includes strict generations, clone-before-freeze replacement, public-state allowlisting, one snapshot read per request before body listeners, delayed A versus immediate B switching, transport/TLS and timeout pinning, unconfigured-source rejection, health secret scans, request/response dynamic auth-header log masking, and bidirectional custom-auth capture redaction. - Node 22.19 Task 6 gate: `node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` passes 21/21, `npm run test:integration` passes 11/11, and `npm test` passes a nonduplicated 112/112 top-level group followed by 11/11 integration tests; `npm run lint` syntax-checks 18 source files. Coverage includes exact version-1 directional schemas, HTTPS-or-loopback upstreams, HTTP-token authentication fields, final authentication-value validation, sensitive and conflicting header rejection, parent-only configure secrets, fixed invalid-message fatal IDs, configure-before-listen, monotonic reconfiguration, configure rejection after drain begins, retained in-flight drain acknowledgement, idempotent duplicate drain, health/status, idle keep-alive closure, clean shutdown, bounded parent-disconnect cleanup with a hanging upstream before or during shutdown, safe startup failures, stale generations, port conflicts, port release, and child-process cleanup without fixed sleeps. Real-fork integration runs after the ordinary group so it cannot race watcher readiness in another test process. +- Node 22.19 Task 7 gate: strict-unhandled `node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` passes 22/22; exact `npm test` passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests without duplication; `npm run test:unit` retains all-top-level coverage and passes 133/133. `npm run lint` syntax-checks 19 source files, and `npm audit --omit=dev` reports zero vulnerabilities. Coverage includes ready/configure/health startup, acknowledgement-atomic snapshot application, concurrent restart reuse before inspecting its snapshot, restart prevalidation before drain when no operation exists, immediate waiter observation and send-failure cancellation, graceful drain/shutdown, TERM/KILL escalation, retained child control after termination timeout, same-port real-worker restart with a changed PID, partial-start and port-release cleanup, correlated fatal and malformed-message sanitization, old-epoch isolation, cancellable injected-clock crash backoff, immediate failure on the fifth crash in 60 seconds, and idempotent retryable close without child, timer, listener, or port residue. The exact runner isolates the preexisting polling capture watcher between the core unit and real-fork integration groups so its registration baseline cannot race unrelated unit load. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. ## Known Risks -Credential migration, localhost browser security, worker-manager crash recovery and restart escalation, in-flight activation semantics, secret leakage, and cross-platform atomic rename and permission semantics. +Credential migration, localhost browser security, in-flight activation semantics, secret leakage, cross-platform worker signal/port-release semantics, and cross-platform atomic rename and permission semantics. ## Recent Decisions @@ -84,3 +85,10 @@ Credential migration, localhost browser security, worker-manager crash recovery - Worker configure must validate the exact authentication header value that forwarding will send, without logging or returning the credential. - Repeated drain commands must reuse the same completion and acknowledge each request without moving a drained worker back to draining. - Parent IPC disconnect cleanup must start or reuse bounded escalation even when shutdown is already waiting on an upstream request. +- Worker lifecycle operations must share the current operation, and messages or exits from an older child epoch must never alter the replacement state. +- Lifecycle entrypoints must return the active shared operation before inspecting new call arguments. +- Restart must confirm the fixed port is exclusively bindable after bounded drain, TERM, and KILL handling before spawning and health-checking a replacement. +- The fifth crash inside a rolling 60-second window must enter `failed` immediately; the 250/500/1000/2000 ms delays are therefore used for the first four crashes, while 4000 ms remains the exponential cap rather than a fifth in-window retry. +- Restart snapshots must pass complete configure validation before the current worker is drained or its state changes. +- IPC acknowledgement promises must be observed at registration and cancelled synchronously when send fails. +- A child reference and its listeners must remain owned after TERM/KILL timeout until a later lifecycle attempt confirms exit and fixed-port release. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 6358b0c..6c27af6 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 6 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, atomic provider and credential persistence, immutable request-level runtime settings, and an IPC-hosted worker entrypoint. The supervisor, provider service, and Admin API remain target-state architecture. +Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 7 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, atomic provider and credential persistence, immutable request-level runtime settings, an IPC-hosted worker entrypoint, and the worker lifecycle manager. The composed supervisor, provider service, and Admin API remain target-state architecture. ## Target Overview @@ -40,7 +40,7 @@ Active OpenAI-compatible upstream ## Module Boundaries -Landed in Tasks 2 through 6: +Landed in Tasks 2 through 7: - `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. - `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. @@ -54,10 +54,11 @@ Landed in Tasks 2 through 6: - `proxy forwarding`: optionally captures exactly one runtime snapshot before request-body listeners and pins target, transport, request ID, authentication, extra headers, TLS, timeout, capture context, and logs for that request. Static configuration remains the generation-zero compatibility path. Dynamic health exposes only runtime and capture public state; an unconfigured source never falls back to the static upstream. - `worker-protocol`: validates exact version-1 parent messages (`configure`, `drain`, `shutdown`, `status`) and child messages (`ready`, `configured`, `drained`, `status`, `fatal`). Configure accepts HTTPS or explicit loopback HTTP only, requires HTTP-token authentication fields, validates the exact final authentication value with Node header rules, and rejects sensitive or authentication-conflicting extra headers. Resolved settings are accepted only inside parent configure messages; child output and sanitized projections contain only allowlisted lifecycle state and stable public errors, and invalid input receives the fixed `worker-fatal` correlation ID. - `proxy-worker`: creates one runtime settings source and forwarding app after the first valid configure, binds only then, applies strictly increasing generations while ready or running, rejects configure once drain begins, tracks in-flight requests independently of later phase changes, closes the listener and idle keep-alive connections during drain, acknowledges repeated drain requests without leaving the drained phase, closes capture and IPC resources during shutdown, and bounds forced cleanup after parent disconnect even when graceful shutdown is already waiting so a hanging upstream cannot orphan the worker. +- `worker-manager`: serializes lifecycle operations, validates restart snapshots before disturbing the current worker, validates every child message before use, correlates immediately observed acknowledgements by request ID and child epoch, cancels waiters on send failure, confirms configuration health before running, drains and escalates termination within deadlines, retains child control after termination timeout, proves the fixed port free before replacement, and recovers unexpected exits with a cancellable rolling crash window and sanitized public state. Remaining target-state boundaries: -- `supervisor`: owns state transitions, admin server, activity records, and child-process lifecycle. +- `supervisor`: composes the worker manager with the future admin server, provider service, and activity records. - `provider-service`: coordinates compatibility tests, credentials, activation, and credential-aware deletion above the metadata registry. - `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. - `web-ui`: static local app with onboarding and daily management views. @@ -66,7 +67,7 @@ Remaining target-state boundaries: - Activate: validate profile → resolve secret → optionally re-test → persist `activeProviderId` → send snapshot generation N+1 → wait for worker acknowledgement. - Restart: mark restarting → ask worker to drain → enforce timeout → wait for exit and port release → spawn → send active snapshot → require health success. -- Crash: record sanitized error → restart with capped exponential backoff → stop looping after the configured threshold. +- Crash: record sanitized error → back off 250/500/1000/2000 ms for the first four crashes in a rolling 60-second window → enter `failed` immediately on the fifth crash. The exponential schedule remains capped at 4000 ms, but that fifth in-window delay is not executed. ## Storage and Deployment diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index f77c8e5..5cb4584 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -82,7 +82,7 @@ Configure settings use the exact runtime graph required by the forwarding server ## RuntimeState -Runtime state includes supervisor PID, worker PID, worker status, snapshot generation, start timestamps, restart count, and the last sanitized error. It is observational and can be reconstructed. +Runtime state includes supervisor PID plus the worker manager's exact public projection `{ phase, pid, generation, state, restartCount, startedAt, error }`. Nested `state` is the child protocol's public lifecycle state, and `error` is either null or a stable `{ code, message }`; snapshots, credentials, raw causes, argv, environment, stdout, and stderr are excluded. The state is observational and can be reconstructed. ## ActivityEvent diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md index f4b2502..81c675a 100644 --- a/docs/PERMISSIONS.md +++ b/docs/PERMISSIONS.md @@ -40,6 +40,7 @@ Record provider creation/update/deletion, test result, activation, Codex bootstr - Redact authorization, cookie, token, secret, and API-key headers and fields. - Redact the exact active authentication header before debug logging or capture persistence; short secrets must never be emitted unchanged. - Validate parent-child IPC with exact directional version-1 schemas, HTTPS-or-loopback URL rules, HTTP-token authentication fields, Node-compatible final authentication values, and case-insensitive sensitive/authentication-header collision rejection. Resolved credentials may cross the process boundary only inside a parent `configure` message; child lifecycle/fatal messages, sanitizers, stdout, and stderr must never echo settings, causes, complete secrets, or an unvalidated request ID. +- Validate every child message before lifecycle use, correlate acknowledgements to the current request and child epoch, and expose worker-manager status only through its positive allowlist. - Reject non-loopback admin binds. - Validate URL scheme and block accidental credential forwarding to an unvalidated target. - Treat diagnostics and capture content as sensitive local data. diff --git a/docs/STATUS.md b/docs/STATUS.md index e166ef1..abf12ef 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 6 are complete and Task 7 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 7 are complete and Task 8 is next. ## Done @@ -24,14 +24,15 @@ V1 implementation: provider lifecycle end to end. - Completed Task 4 native and explicit-consent file credential adapters, including construction-only fallback without operation replay, explicit file-label restart continuity, descriptor-safe reads, degraded temp cleanup, canonical lock restoration, and primary-blocked claim-before-delete gate release, covered by 41/41 focused, 64/64 combined credential/provider, and 91/91 full Node 22.19 tests; syntax checking covers 14 source files. - Completed Task 5 monotonic immutable runtime settings and request-start snapshot capture. In-flight A remains internally consistent while new traffic switches to B; dynamic health, transport and timeout pinning, request/response short and custom authentication log masking, and bidirectional custom-auth capture redaction are covered by 13/13 focused and 102/102 full Node 22.19 tests; syntax checking covers 15 source files. - Completed Task 6 strict version-1 worker IPC and the proxy-worker child entrypoint. Configure enforces HTTPS-or-loopback upstreams, HTTP-token authentication fields, Node-compatible final authentication values, and sensitive/header-collision rejection; the worker remains unbound until valid configuration, rejects configure after drain begins without losing the pending drain acknowledgement, keeps repeated drain acknowledgements and status drained, bounds parent-disconnect cleanup when an upstream hangs before or during shutdown, uses a fixed correlation ID for invalid-message fatals, releases its port on shutdown, and emits only sanitized child messages. Coverage passes 21/21 focused and 11/11 integration tests; the full gate passes 112/112 top-level tests followed by 11/11 integration tests without duplication. Syntax checking covers 18 source files, including the sequential group runner that isolates real-fork load from watcher tests. +- Completed Task 7 reliable worker lifecycle management and its review fixes. Startup requires ready, correlated configure acknowledgement, and matching health; snapshot generations advance only after matching acknowledgement; concurrent restart calls reuse the current operation before inspecting a new snapshot, while an unshared restart validates the complete snapshot before drain; send failures synchronously cancel observed acknowledgement waiters; stop and restart bound drain, TERM, and KILL, confirm exclusive fixed-port release, retain control after termination timeout, and isolate old child epochs. Unexpected exits recover with cancellable injected-clock backoff and fail immediately on the fifth crash in 60 seconds. Strict-unhandled coverage passes 22/22 focused tests; the exact full gate passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests; the all-top-level compatibility command passes 133/133. Syntax checking covers 19 source files, and the runtime audit reports zero vulnerabilities. ## Blocked -- No current blocker is recorded for Task 7. +- No current blocker is recorded for Task 8. ## Next -1. Execute Task 7: implement reliable worker lifecycle management. +1. Execute Task 8: add activity, migration, and provider orchestration. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/TESTING.md b/docs/TESTING.md index 9c58c21..971f959 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -25,6 +25,7 @@ - Task 4 combined credential/provider suite: `cd node && node --test test/credential-store.test.mjs test/provider-registry.test.mjs` - Task 5 focused suite: `cd node && node --test test/runtime-settings.test.mjs test/server.test.mjs` - Task 6 focused suite: `cd node && node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` +- Task 7 focused suite: `cd node && node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` - Worker integration suite: `cd node && npm run test:integration` - Runtime audit: `cd node && npm audit --omit=dev` @@ -36,7 +37,9 @@ The Node 22.19 Task 5 focused suite passes 13/13 and the current full suite pass The Node 22.19 Task 6 focused suite passes 21/21, `npm run test:integration` passes 11/11, and `npm test` passes 112/112 top-level tests followed by 11/11 integration tests without duplication; `npm run lint` syntax-checks 18 source files. Coverage verifies exact directional schemas, HTTPS-or-loopback URL enforcement, HTTP-token authentication fields, Node-compatible final authentication values, sensitive and authentication-conflicting header rejection, secret sanitization, fixed invalid-message fatal IDs, and real child-process ready/configure/reconfigure/proxy/status/drain/shutdown flows. Synchronized tests prove configure is rejected after drain begins without replacing generation or losing its acknowledgement, duplicate drain requests keep both acknowledgements and later status drained, and parent disconnect terminates within a deadline despite a hanging upstream both before and after shutdown starts waiting. Invalid authentication values, configuration, stale generation, occupied-port startup, port release, request settlement, stderr/stdout scans, and deterministic deadline-driven cleanup remain covered. -`npm run test:unit` runs only top-level `test/*.test.mjs` files. The `test:integration` runner recursively discovers `test/integration/**/*.test.mjs`; the current worker-entry suite passes 11/11. `npm test` invokes those two nonoverlapping groups in sequence, retaining concurrency inside the top-level group while ensuring real child-process integration load cannot race watcher readiness. `test:e2e` and the combined `test:all` command are not current gates until the UI, Playwright configuration, and E2E specs land. +The Node 22.19 Task 7 strict-unhandled focused suite passes 22/22. Exact `npm test` passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run test:unit` passes all 133/133 top-level assertions; `npm run lint` syntax-checks 19 source files. Coverage verifies ready/configure/health startup, strict request correlation, acknowledgement-atomic generation changes, concurrent restart Promise identity and zero snapshot inspection before shared-operation reuse, restart prevalidation without drain side effects when no operation exists, send-failure waiter cancellation without unhandled rejections, early acknowledgement retention, graceful drain/shutdown, TERM/KILL escalation, retained retryable control after termination timeout, partial-start error preservation, fixed-port cleanup, same-port real-worker PID replacement, sanitized fatal and malformed messages, old-epoch isolation, cancellable injected-clock crash recovery, immediate failure on the fifth crash in 60 seconds, and idempotent close with no child, timer, listener, process, or port residue. The fifth in-window crash does not schedule the 4000 ms capped delay; only the first four 250/500/1000/2000 ms delays execute before the threshold. + +`npm run test:unit` retains its public behavior and runs all top-level `test/*.test.mjs` files. The exact `npm test` gate runs the same top-level set without duplication as two sequential groups: every non-capture unit file first, then `capture-store.test.mjs` alone, followed by recursively discovered `test/integration/**/*.test.mjs`. This isolates the polling watcher registration baseline from unrelated unit load and keeps real child-process integration after watcher cleanup. `test:e2e` and the combined `test:all` command are not current gates until the UI, Playwright configuration, and E2E specs land. ## Test Authoring Rules @@ -67,6 +70,11 @@ The Node 22.19 Task 6 focused suite passes 21/21, `npm run test:integration` pas - Configure tests must validate the exact authentication header value and prove invalid keys are absent from child IPC, stdout, and stderr. - Parent IPC disconnect cleanup must start or reuse bounded escalation even when shutdown is already waiting on an upstream request. - Real-fork integration tests must run after watcher-bearing top-level tests, never concurrently with that group. +- Worker lifecycle tests must drive injected clocks and observable process, IPC, HTTP, and port events; fixed sleeps are prohibited. +- Polling watcher suites without an observable ready event must run in an isolated sequential group. +- Restart validation must complete before any drain message, phase change, or current-worker mutation. +- IPC acknowledgement waiters must be observed immediately and cancelled on send failure. +- Lifecycle code must retain child control after termination timeout until exit and port release are confirmed. ## Test Matrix diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index be82de5..e755cc4 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -602,7 +602,7 @@ Actual Node 22.19 Task 6 verification: the focused protocol/entrypoint suite pas - Create: `node/test/worker-manager.test.mjs` - Create: `node/test/integration/worker-restart.test.mjs` -- [ ] **Step 1: Write failing state-machine tests** +- [x] **Step 1: Write failing state-machine tests** Assert transitions: @@ -616,11 +616,11 @@ backoff -> failed after the configured crash threshold Use injected `forkWorker`, `clock`, and `waitForPortFree` functions so unit tests contain no sleeps. -- [ ] **Step 2: Write the fixed-port restart test** +- [x] **Step 2: Write the fixed-port restart test** Start a real worker on a chosen free port, call `restart(snapshot)`, assert the PID changes, the supervisor-side manager remains available, the port is rebound, and health returns generation 1. -- [ ] **Step 3: Run and verify failures** +- [x] **Step 3: Run and verify failures** ```bash cd node @@ -629,7 +629,7 @@ node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mj Expected: FAIL because `WorkerManager` does not exist. -- [ ] **Step 4: Implement the lifecycle contract** +- [x] **Step 4: Implement the lifecycle contract** Export a `WorkerManager` with: @@ -644,7 +644,7 @@ close(); Restart must wait for `drained`, escalate to `SIGTERM`, then `SIGKILL` after a bounded timeout, confirm port release by attempting an exclusive listen, spawn, configure, and require health before reporting success. Crash recovery uses delays 250ms, 500ms, 1000ms, 2000ms, 4000ms and enters `failed` after five crashes in 60 seconds. -- [ ] **Step 5: Verify and commit** +- [x] **Step 5: Verify and commit** ```bash cd node @@ -654,6 +654,8 @@ git add node/src/supervisor/worker-manager.mjs node/test/worker-manager.test.mjs git commit -m "feat: manage proxy worker lifecycle" ``` +Actual Node 22.19 Task 7 verification after review fixes: the strict-unhandled focused manager/restart suite passes 22/22, the exact full gate passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests without duplication, and the all-top-level compatibility command passes 133/133. The portable syntax gate checks 19 source files, and the runtime audit reports zero vulnerabilities. Unit tests use injected process, clock, health, and port-probe boundaries without fixed sleeps. Coverage includes acknowledgement-atomic generations, shared-operation restart identity before any new snapshot inspection, restart prevalidation before drain when no operation exists, immediate waiter observation and send-failure cancellation, early acknowledgement retention, bounded partial-start cleanup with original-error preservation, graceful and forced termination, retained retryable control after termination timeout, fixed-port release failures, correlated and malformed child-message sanitization, epoch isolation, idempotent close, cancellable backoff stop, and immediate `failed` on the fifth crash in a rolling 60-second window. Because that threshold is immediate, the first four in-window delays are 250/500/1000/2000 ms and the 4000 ms exponential cap is not scheduled as a fifth retry. The real-worker integration changes PID, rebinds the same port, verifies generation-matched health, and proves final port release. The exact runner executes non-capture unit, isolated polling capture, and real-fork integration groups sequentially while preserving `npm run test:unit` as the all-top-level command. + ## Task 8: Add Activity, Migration, and Provider Orchestration **Files:** diff --git a/node/scripts/run-test-group.mjs b/node/scripts/run-test-group.mjs index c6df129..cddd282 100644 --- a/node/scripts/run-test-group.mjs +++ b/node/scripts/run-test-group.mjs @@ -3,6 +3,8 @@ import { join, resolve } from "node:path"; import { spawnSync } from "node:child_process"; const group = process.argv[2]; +const supportedGroups = new Set(["unit", "unit-core", "capture", "integration"]); +if (!supportedGroups.has(group)) throw new Error(`Unknown test group: ${group}`); const testRoot = resolve("test"); const selectedRoot = group === "integration" ? join(testRoot, "integration") : testRoot; const recursive = group === "integration"; @@ -17,9 +19,15 @@ function collect(dir, descend) { return files; } -const files = statSync(selectedRoot, { throwIfNoEntry: false })?.isDirectory() +let files = statSync(selectedRoot, { throwIfNoEntry: false })?.isDirectory() ? collect(selectedRoot, recursive).sort() : []; +if (group === "unit-core") { + files = files.filter((file) => file !== join(testRoot, "capture-store.test.mjs")); +} +if (group === "capture") { + files = files.filter((file) => file === join(testRoot, "capture-store.test.mjs")); +} if (files.length === 0) throw new Error(`No ${group} test files found`); const result = spawnSync(process.execPath, ["--test", ...files], { stdio: "inherit" }); process.exit(result.status ?? 1); diff --git a/node/scripts/run-tests.mjs b/node/scripts/run-tests.mjs index 1935ec4..6ed91c4 100644 --- a/node/scripts/run-tests.mjs +++ b/node/scripts/run-tests.mjs @@ -3,7 +3,7 @@ import { fileURLToPath } from "node:url"; const groupRunner = fileURLToPath(new URL("./run-test-group.mjs", import.meta.url)); -for (const group of ["unit", "integration"]) { +for (const group of ["unit-core", "capture", "integration"]) { const result = spawnSync(process.execPath, [groupRunner, group], { stdio: "inherit" }); if (result.status !== 0) { process.exit(result.status ?? 1); diff --git a/node/src/supervisor/worker-manager.mjs b/node/src/supervisor/worker-manager.mjs new file mode 100644 index 0000000..531bc85 --- /dev/null +++ b/node/src/supervisor/worker-manager.mjs @@ -0,0 +1,787 @@ +import { fork } from "node:child_process"; +import { createServer } from "node:net"; +import { fileURLToPath } from "node:url"; + +import { + PROTOCOL_VERSION, + validateChildMessage, + validateParentMessage +} from "../worker/protocol.mjs"; + +const WORKER_ENTRY_PATH = fileURLToPath(new URL("../worker/worker-entry.mjs", import.meta.url)); +const CRASH_WINDOW_MS = 60_000; +const CRASH_BACKOFF_MS = Object.freeze([250, 500, 1_000, 2_000, 4_000]); + +const REAL_CLOCK = Object.freeze({ + now: () => Date.now(), + setTimeout: (callback, delay) => setTimeout(callback, delay), + clearTimeout: (timer) => clearTimeout(timer) +}); + +const ERROR_MESSAGES = Object.freeze({ + WORKER_MANAGER_CLOSED: "Worker manager is closed.", + WORKER_READY_TIMEOUT: "Worker did not become ready in time.", + WORKER_ACK_TIMEOUT: "Worker did not acknowledge the request in time.", + WORKER_HEALTH_FAILED: "Worker health verification failed.", + WORKER_PROTOCOL_INVALID: "Worker protocol message is invalid.", + WORKER_START_FAILED: "Worker failed to start.", + WORKER_IPC_SEND_FAILED: "Worker IPC send failed.", + WORKER_NOT_RUNNING: "Worker is not running.", + WORKER_SNAPSHOT_INVALID: "Worker settings snapshot is invalid.", + WORKER_EXIT_TIMEOUT: "Worker did not exit in time.", + WORKER_STOP_FAILED: "Worker could not be stopped.", + WORKER_PORT_BUSY: "The fixed proxy port is still in use.", + WORKER_EXITED: "Proxy worker exited unexpectedly.", + WORKER_CONFIGURE_FAILED: "Worker configuration failed.", + WORKER_RUNTIME_FAILED: "Worker runtime failed.", + STALE_SNAPSHOT: "Worker rejected a stale settings snapshot.", + RUNTIME_SETTINGS_INVALID: "Worker settings are invalid." +}); + +function managerError(code) { + const error = new Error(ERROR_MESSAGES[code] ?? "Worker lifecycle operation failed."); + error.name = "WorkerManagerError"; + error.code = code; + return error; +} + +function defaultForkWorker() { + return fork(WORKER_ENTRY_PATH, [], { + execPath: process.execPath, + stdio: ["ignore", "ignore", "ignore", "ipc"] + }); +} + +function defaultWaitForPortFree(host, port) { + return new Promise((resolvePromise, rejectPromise) => { + const probe = createServer(); + probe.unref(); + probe.once("error", () => rejectPromise(managerError("WORKER_PORT_BUSY"))); + probe.listen({ host, port, exclusive: true }, () => { + probe.close((error) => { + if (error) rejectPromise(managerError("WORKER_PORT_BUSY")); + else resolvePromise(); + }); + }); + }); +} + +function publicStateCopy(state) { + return state ? { + phase: state.phase, + configured: state.configured, + generation: state.generation, + listening: state.listening, + listenHost: state.listenHost, + listenPort: state.listenPort, + inFlight: state.inFlight + } : null; +} + +export class WorkerManager { + #host; + #port; + #clock; + #forkWorker; + #fetch; + #readyTimeoutMs; + #ackTimeoutMs; + #healthTimeoutMs; + #terminateTimeoutMs; + #killTimeoutMs; + #waitForPortFree; + #phase = "stopped"; + #child = null; + #epoch = 0; + #generation = 0; + #workerState = null; + #restartCount = 0; + #startedAt = null; + #lastError = null; + #lastSnapshot = null; + #requestSequence = 0; + #waiters = new Set(); + #exits = new Map(); + #expectedEpochs = new Set(); + #exitErrors = new Map(); + #crashTimes = []; + #recoveryTimer = null; + #recoveryVersion = 0; + #operation = null; + #closed = false; + #closePromise = null; + + constructor({ + host = "127.0.0.1", + port = 15100, + clock = REAL_CLOCK, + forkWorker = defaultForkWorker, + fetchImpl = globalThis.fetch, + readyTimeoutMs = 5_000, + ackTimeoutMs = 5_000, + healthTimeoutMs = 5_000, + terminateTimeoutMs = 1_000, + killTimeoutMs = 1_000, + waitForPortFree = defaultWaitForPortFree + } = {}) { + this.#host = host; + this.#port = port; + this.#clock = clock; + this.#forkWorker = forkWorker; + this.#fetch = fetchImpl; + this.#readyTimeoutMs = readyTimeoutMs; + this.#ackTimeoutMs = ackTimeoutMs; + this.#healthTimeoutMs = healthTimeoutMs; + this.#terminateTimeoutMs = terminateTimeoutMs; + this.#killTimeoutMs = killTimeoutMs; + this.#waitForPortFree = waitForPortFree; + } + + getPublicState() { + return { + phase: this.#phase, + pid: this.#child?.pid ?? null, + generation: this.#generation, + state: publicStateCopy(this.#workerState), + restartCount: this.#restartCount, + startedAt: this.#startedAt, + error: this.#lastError ? { ...this.#lastError } : null + }; + } + + start(snapshot) { + if (this.#closed) { + return Promise.reject(managerError("WORKER_MANAGER_CLOSED")); + } + if (this.#operation) { + return this.#operation; + } + if (this.#phase === "running") { + return Promise.resolve(this.getPublicState()); + } + if (this.#child) { + return Promise.reject(managerError("WORKER_STOP_FAILED")); + } + return this.#trackOperation(this.#performStart(snapshot)); + } + + applySnapshot(snapshot) { + if (this.#closed) { + return Promise.reject(managerError("WORKER_MANAGER_CLOSED")); + } + if (this.#operation) { + return this.#operation; + } + if (this.#phase !== "running" || !this.#child) { + return Promise.reject(managerError("WORKER_NOT_RUNNING")); + } + return this.#trackOperation(this.#performApplySnapshot(snapshot)); + } + + stop({ drainTimeoutMs = 5_000 } = {}) { + if (this.#closed) { + return Promise.reject(managerError("WORKER_MANAGER_CLOSED")); + } + if (this.#operation) return this.#operation; + if (!this.#child || this.#phase === "stopped") { + this.#cancelRecovery(); + this.#phase = "stopped"; + this.#workerState = null; + return Promise.resolve(this.getPublicState()); + } + return this.#trackOperation(this.#performStop({ drainTimeoutMs })); + } + + restart(snapshot, { drainTimeoutMs = 5_000 } = {}) { + if (this.#closed) { + return Promise.reject(managerError("WORKER_MANAGER_CLOSED")); + } + if (this.#operation) return this.#operation; + let validatedSnapshot; + try { + validatedSnapshot = this.#validateSnapshot(snapshot); + } catch (error) { + return Promise.reject(error); + } + return this.#trackOperation(this.#performRestart(validatedSnapshot, { drainTimeoutMs })); + } + + close() { + if (this.#closePromise) return this.#closePromise; + this.#closed = true; + this.#cancelRecovery(); + const attempt = (this.#operation ?? Promise.resolve()) + .catch(() => {}) + .then(() => this.#performClose()); + this.#closePromise = attempt; + void attempt.catch(() => { + if (this.#closePromise === attempt && this.#child) this.#closePromise = null; + }); + return attempt; + } + + async #performClose() { + const child = this.#child; + const epoch = this.#epoch; + this.#rejectWaiters(epoch, managerError("WORKER_MANAGER_CLOSED")); + let failure = null; + if (child) { + this.#expectedEpochs.add(epoch); + try { + await this.#terminateChild(child, epoch); + await this.#waitForPortFree(this.#host, this.#port); + } catch (error) { + const code = error?.code === "WORKER_PORT_BUSY" ? "WORKER_PORT_BUSY" : "WORKER_STOP_FAILED"; + failure = managerError(code); + } finally { + if (this.#hasExited(child)) this.#releaseChild(child, epoch); + } + } else { + for (const staleEpoch of this.#exits.keys()) { + this.#detachChild(staleEpoch); + this.#exits.delete(staleEpoch); + } + } + this.#workerState = null; + if (failure) { + this.#phase = "failed"; + this.#lastError = { code: failure.code, message: failure.message }; + throw failure; + } + this.#phase = "stopped"; + return this.getPublicState(); + } + + async #performStart(snapshot) { + const requestId = this.#nextRequestId("configure"); + const configureMessage = { + version: PROTOCOL_VERSION, + type: "configure", + requestId, + generation: snapshot?.generation, + settings: snapshot?.settings + }; + validateParentMessage(configureMessage); + if (snapshot.settings.server.host !== this.#host || snapshot.settings.server.port !== this.#port) { + throw managerError("WORKER_START_FAILED"); + } + + this.#phase = "starting"; + let child; + try { + child = this.#forkWorker(); + } catch { + const error = managerError("WORKER_START_FAILED"); + this.#lastError = { code: error.code, message: error.message }; + this.#phase = "stopped"; + throw error; + } + const epoch = ++this.#epoch; + this.#child = child; + this.#attachChild(child, epoch); + + try { + const ready = await this.#waitForMessage({ + epoch, + requestId: "worker-ready", + type: "ready", + timeoutMs: this.#readyTimeoutMs, + timeoutCode: "WORKER_READY_TIMEOUT" + }); + if (ready.state.phase !== "ready" || ready.state.configured || ready.state.generation !== 0) { + throw managerError("WORKER_PROTOCOL_INVALID"); + } + + const configured = await this.#sendAndWait(child, configureMessage, { + epoch, + requestId, + type: "configured", + timeoutMs: this.#ackTimeoutMs, + timeoutCode: "WORKER_ACK_TIMEOUT" + }); + if (configured.state.phase !== "running" + || configured.state.generation !== snapshot.generation + || configured.state.listenHost !== this.#host + || configured.state.listenPort !== this.#port) { + throw managerError("WORKER_PROTOCOL_INVALID"); + } + + await this.#verifyHealth(snapshot.generation); + if (epoch !== this.#epoch || child !== this.#child) { + throw managerError("WORKER_START_FAILED"); + } + this.#workerState = publicStateCopy(configured.state); + this.#generation = snapshot.generation; + this.#lastSnapshot = structuredClone(snapshot); + this.#phase = "running"; + this.#startedAt = new Date(this.#clock.now()).toISOString(); + return this.getPublicState(); + } catch (error) { + this.#lastError = { + code: error?.code && ERROR_MESSAGES[error.code] ? error.code : "WORKER_START_FAILED", + message: ERROR_MESSAGES[error?.code] ?? ERROR_MESSAGES.WORKER_START_FAILED + }; + this.#expectedEpochs.add(epoch); + let cleanupError = null; + try { + await this.#terminateChild(child, epoch); + await this.#waitForPortFree(this.#host, this.#port); + } catch (cleanupFailure) { + cleanupError = cleanupFailure; + } + this.#workerState = null; + if (this.#hasExited(child)) { + this.#releaseChild(child, epoch); + if (cleanupError) { + const code = cleanupError?.code === "WORKER_PORT_BUSY" + ? "WORKER_PORT_BUSY" + : "WORKER_STOP_FAILED"; + const publicCleanupError = managerError(code); + this.#lastError = { + code: publicCleanupError.code, + message: publicCleanupError.message + }; + this.#phase = "failed"; + } else { + this.#phase = "stopped"; + } + } else { + this.#phase = "failed"; + } + throw error?.code ? error : managerError("WORKER_START_FAILED"); + } + } + + async #performApplySnapshot(snapshot) { + const child = this.#child; + const epoch = this.#epoch; + const requestId = this.#nextRequestId("configure"); + const message = { + version: PROTOCOL_VERSION, + type: "configure", + requestId, + generation: snapshot?.generation, + settings: snapshot?.settings + }; + try { + validateParentMessage(message); + } catch { + throw managerError("WORKER_SNAPSHOT_INVALID"); + } + if (snapshot.generation <= this.#generation + || snapshot.settings.server.host !== this.#host + || snapshot.settings.server.port !== this.#port) { + throw managerError("WORKER_SNAPSHOT_INVALID"); + } + const configured = await this.#sendAndWait(child, message, { + epoch, + requestId, + type: "configured", + timeoutMs: this.#ackTimeoutMs, + timeoutCode: "WORKER_ACK_TIMEOUT" + }); + if (epoch !== this.#epoch + || child !== this.#child + || configured.state.phase !== "running" + || configured.state.generation !== snapshot.generation + || configured.state.listenHost !== this.#host + || configured.state.listenPort !== this.#port) { + throw managerError("WORKER_PROTOCOL_INVALID"); + } + this.#workerState = publicStateCopy(configured.state); + this.#generation = snapshot.generation; + this.#lastSnapshot = structuredClone(snapshot); + return this.getPublicState(); + } + + async #performStop({ drainTimeoutMs }) { + const child = this.#child; + const epoch = this.#epoch; + this.#expectedEpochs.add(epoch); + this.#phase = "draining"; + + let drained = false; + let drainFailure = null; + const requestId = this.#nextRequestId("drain"); + try { + const message = await this.#sendAndWait( + child, + { version: PROTOCOL_VERSION, type: "drain", requestId }, + { + epoch, + requestId, + type: "drained", + timeoutMs: drainTimeoutMs, + timeoutCode: "WORKER_ACK_TIMEOUT" + } + ); + if (message.state.phase !== "drained" + || message.state.generation !== this.#generation + || message.state.listening) { + throw managerError("WORKER_PROTOCOL_INVALID"); + } + this.#workerState = publicStateCopy(message.state); + drained = true; + } catch (error) { + if (error?.code === "WORKER_IPC_SEND_FAILED") drainFailure = error; + // Escalation below owns cleanup after a bounded drain failure. + } + + let failure = null; + try { + if (drained) { + const shutdownRequestId = this.#nextRequestId("shutdown"); + try { + await this.#send(child, { + version: PROTOCOL_VERSION, + type: "shutdown", + requestId: shutdownRequestId + }); + await this.#waitForExit(epoch, this.#terminateTimeoutMs); + } catch { + await this.#terminateChild(child, epoch); + } + } else { + await this.#terminateChild(child, epoch); + } + await this.#waitForPortFree(this.#host, this.#port); + } catch (error) { + const code = error?.code === "WORKER_PORT_BUSY" ? "WORKER_PORT_BUSY" : "WORKER_STOP_FAILED"; + failure = managerError(code); + this.#lastError = { code: failure.code, message: failure.message }; + } finally { + if (this.#hasExited(child)) { + this.#releaseChild(child, epoch); + this.#workerState = null; + } + } + + if (failure) { + this.#phase = "failed"; + throw failure; + } + this.#phase = "stopped"; + if (drainFailure) { + this.#lastError = { code: drainFailure.code, message: drainFailure.message }; + throw drainFailure; + } + return this.getPublicState(); + } + + async #performRestart(snapshot, { drainTimeoutMs }) { + this.#cancelRecovery(); + if (this.#child && this.#phase !== "stopped") { + await this.#performStop({ drainTimeoutMs }); + this.#restartCount += 1; + } + return this.#performStart(snapshot); + } + + #validateSnapshot(snapshot) { + let cloned; + try { + cloned = structuredClone(snapshot); + validateParentMessage({ + version: PROTOCOL_VERSION, + type: "configure", + requestId: "snapshot-validation", + generation: cloned?.generation, + settings: cloned?.settings + }); + } catch { + throw managerError("WORKER_SNAPSHOT_INVALID"); + } + if (cloned.settings.server.host !== this.#host || cloned.settings.server.port !== this.#port) { + throw managerError("WORKER_SNAPSHOT_INVALID"); + } + return cloned; + } + + async #terminateChild(child, epoch) { + if (child.exitCode !== null || child.signalCode !== null) return; + child.kill("SIGTERM"); + try { + await this.#waitForExit(epoch, this.#terminateTimeoutMs); + return; + } catch { + // A stuck worker receives one final bounded escalation. + } + child.kill("SIGKILL"); + try { + await this.#waitForExit(epoch, this.#killTimeoutMs); + } catch { + throw managerError("WORKER_STOP_FAILED"); + } + } + + #hasExited(child) { + return child.exitCode !== null || child.signalCode !== null; + } + + #releaseChild(child, epoch) { + if (this.#child === child) this.#child = null; + this.#detachChild(epoch); + this.#exits.delete(epoch); + this.#expectedEpochs.delete(epoch); + this.#exitErrors.delete(epoch); + } + + #attachChild(child, epoch) { + let resolveExit; + const exitPromise = new Promise((resolvePromise) => { + resolveExit = resolvePromise; + }); + const onMessage = (rawMessage) => { + if (epoch !== this.#epoch || child !== this.#child) return; + let message; + try { + message = validateChildMessage(rawMessage); + } catch { + const error = managerError("WORKER_PROTOCOL_INVALID"); + this.#lastError = { code: error.code, message: error.message }; + this.#rejectWaiters(epoch, error); + if (this.#phase === "running") { + this.#exitErrors.set(epoch, this.#lastError); + void this.#terminateChild(child, epoch).catch(() => { + if (epoch !== this.#epoch || child !== this.#child) return; + const stopError = managerError("WORKER_STOP_FAILED"); + this.#lastError = { code: stopError.code, message: stopError.message }; + this.#phase = "failed"; + }); + } + return; + } + if (message.type === "fatal") { + const error = managerError(message.error.code); + this.#lastError = { code: error.code, message: error.message }; + for (const waiter of [...this.#waiters]) { + if (waiter.epoch === epoch + && (waiter.requestId === message.requestId || message.requestId === "worker-fatal")) { + waiter.reject(error); + } + } + return; + } + for (const waiter of [...this.#waiters]) { + if (waiter.epoch === epoch + && waiter.requestId === message.requestId + && waiter.type === message.type) { + waiter.resolve(message); + } + } + }; + const onError = () => { + this.#rejectWaiters(epoch, managerError("WORKER_START_FAILED")); + }; + const onExit = (code, signal) => { + const exit = this.#exits.get(epoch); + if (exit && !exit.settled) { + exit.settled = true; + exit.result = { code, signal }; + resolveExit(exit.result); + } + this.#rejectWaiters(epoch, managerError("WORKER_START_FAILED")); + if (epoch === this.#epoch + && child === this.#child + && !this.#expectedEpochs.has(epoch) + && !this.#closed + && this.#phase === "running") { + this.#child = null; + this.#workerState = null; + const exitError = this.#exitErrors.get(epoch); + this.#exitErrors.delete(epoch); + this.#scheduleRecovery(exitError); + this.#detachChild(epoch); + this.#exits.delete(epoch); + } + }; + this.#exits.set(epoch, { + promise: exitPromise, + settled: false, + child, + onMessage, + onError, + onExit + }); + child.on("message", onMessage); + child.once("error", onError); + child.once("exit", onExit); + } + + #detachChild(epoch) { + const context = this.#exits.get(epoch); + if (!context) return; + context.child.off("message", context.onMessage); + context.child.off("error", context.onError); + context.child.off("exit", context.onExit); + } + + #scheduleRecovery(exitError = null) { + const now = this.#clock.now(); + this.#phase = "crashed"; + this.#lastError = exitError ? { ...exitError } : { + code: "WORKER_EXITED", + message: ERROR_MESSAGES.WORKER_EXITED + }; + this.#crashTimes = this.#crashTimes + .filter((crashedAt) => now - crashedAt < CRASH_WINDOW_MS); + this.#crashTimes.push(now); + if (this.#crashTimes.length >= CRASH_BACKOFF_MS.length) { + this.#phase = "failed"; + return; + } + + const delay = CRASH_BACKOFF_MS[this.#crashTimes.length - 1]; + const snapshot = structuredClone(this.#lastSnapshot); + this.#phase = "backoff"; + this.#restartCount += 1; + const recoveryVersion = ++this.#recoveryVersion; + this.#recoveryTimer = this.#clock.setTimeout(() => { + if (recoveryVersion !== this.#recoveryVersion) return; + this.#recoveryTimer = null; + if (this.#closed || this.#phase !== "backoff") return; + const recovery = Promise.resolve() + .then(() => this.#waitForPortFree(this.#host, this.#port)) + .then(() => this.#performStart(snapshot)); + this.#trackOperation(recovery); + void recovery.catch(() => { + if (!this.#closed) this.#scheduleRecovery(); + }); + }, delay); + } + + #cancelRecovery() { + this.#recoveryVersion += 1; + if (this.#recoveryTimer !== null) { + this.#clock.clearTimeout(this.#recoveryTimer); + this.#recoveryTimer = null; + } + } + + #waitForExit(epoch, timeoutMs) { + const exit = this.#exits.get(epoch); + if (!exit) return Promise.reject(managerError("WORKER_STOP_FAILED")); + if (exit.settled) return Promise.resolve(exit.result); + return this.#withTimeout(exit.promise, timeoutMs, "WORKER_EXIT_TIMEOUT"); + } + + #waitForMessage({ epoch, requestId, type, timeoutMs, timeoutCode }) { + return this.#registerMessageWaiter({ + epoch, + requestId, + type, + timeoutMs, + timeoutCode + }).promise; + } + + #registerMessageWaiter({ epoch, requestId, type, timeoutMs, timeoutCode }) { + const waiter = { + epoch, + requestId, + type, + timer: null, + settled: false, + resolve: null, + reject: null + }; + const promise = new Promise((resolvePromise, rejectPromise) => { + const settle = (callback, value) => { + if (waiter.settled) return; + waiter.settled = true; + this.#waiters.delete(waiter); + this.#clock.clearTimeout(waiter.timer); + callback(value); + }; + waiter.resolve = (value) => settle(resolvePromise, value); + waiter.reject = (error) => settle(rejectPromise, error); + }); + waiter.timer = this.#clock.setTimeout( + () => waiter.reject(managerError(timeoutCode)), + timeoutMs + ); + this.#waiters.add(waiter); + return { + promise, + cancel: (error) => waiter.reject(error) + }; + } + + async #sendAndWait(child, message, waitFor) { + const waiter = this.#registerMessageWaiter(waitFor); + const observed = waiter.promise.then( + (value) => ({ value }), + (error) => ({ error }) + ); + try { + await this.#send(child, message); + } catch (error) { + waiter.cancel(error); + await observed; + throw error; + } + const result = await observed; + if (result.error) throw result.error; + return result.value; + } + + #rejectWaiters(epoch, error) { + for (const waiter of [...this.#waiters]) { + if (waiter.epoch !== epoch) continue; + waiter.reject(error); + } + } + + #send(child, message) { + validateParentMessage(message); + return new Promise((resolvePromise, rejectPromise) => { + if (!child.connected) { + rejectPromise(managerError("WORKER_IPC_SEND_FAILED")); + return; + } + child.send(message, (error) => { + if (error) rejectPromise(managerError("WORKER_IPC_SEND_FAILED")); + else resolvePromise(); + }); + }); + } + + async #verifyHealth(generation) { + const healthPromise = Promise.resolve() + .then(() => this.#fetch(`http://${this.#host}:${this.#port}/_proxy/health`)) + .then(async (response) => { + if (!response?.ok) throw managerError("WORKER_HEALTH_FAILED"); + const health = await response.json(); + if (health?.configured !== true || health?.generation !== generation) { + throw managerError("WORKER_HEALTH_FAILED"); + } + }); + await this.#withTimeout(healthPromise, this.#healthTimeoutMs, "WORKER_HEALTH_FAILED"); + } + + #withTimeout(promise, timeoutMs, code) { + let timer; + const timeout = new Promise((_, rejectPromise) => { + timer = this.#clock.setTimeout(() => rejectPromise(managerError(code)), timeoutMs); + }); + return Promise.race([promise, timeout]).finally(() => this.#clock.clearTimeout(timer)); + } + + #nextRequestId(prefix) { + this.#requestSequence += 1; + return `${prefix}-${this.#requestSequence}`; + } + + #trackOperation(operation) { + this.#operation = operation; + void operation.then( + () => { + if (this.#operation === operation) this.#operation = null; + }, + () => { + if (this.#operation === operation) this.#operation = null; + } + ); + return operation; + } +} diff --git a/node/test/fixtures/fake-worker.mjs b/node/test/fixtures/fake-worker.mjs new file mode 100644 index 0000000..ed96d94 --- /dev/null +++ b/node/test/fixtures/fake-worker.mjs @@ -0,0 +1,103 @@ +import { + PROTOCOL_VERSION, + validateParentMessage +} from "../../src/worker/protocol.mjs"; + +const modes = new Set((process.env.CRP_FAKE_WORKER_MODES ?? "") + .split(",") + .map((mode) => mode.trim()) + .filter(Boolean)); + +let generation = 0; +let phase = "ready"; +let listening = false; +let listenHost = null; +let listenPort = null; + +function state() { + return { + phase, + configured: generation > 0, + generation, + listening, + listenHost, + listenPort, + inFlight: 0 + }; +} + +function send(message) { + if (process.connected) process.send(message); +} + +function lifecycle(type, requestId) { + send({ version: PROTOCOL_VERSION, type, requestId, state: state() }); +} + +function handleMessage(rawMessage) { + let message; + try { + message = validateParentMessage(rawMessage); + } catch { + send({ + version: PROTOCOL_VERSION, + type: "fatal", + requestId: "worker-fatal", + error: { + code: "WORKER_PROTOCOL_INVALID", + message: "Worker protocol message is invalid." + } + }); + return; + } + + if (message.type === "configure") { + if (modes.has("no-configure")) return; + generation = message.generation; + phase = "running"; + listening = true; + listenHost = message.settings.server.host; + listenPort = message.settings.server.port; + lifecycle("configured", message.requestId); + if (modes.has("exit-after-configure")) process.exit(23); + return; + } + if (message.type === "status") { + lifecycle("status", message.requestId); + return; + } + if (message.type === "drain") { + if (modes.has("no-drain")) return; + phase = "drained"; + listening = false; + listenHost = null; + listenPort = null; + lifecycle("drained", message.requestId); + return; + } + if (!modes.has("no-shutdown")) process.exit(0); +} + +process.on("message", handleMessage); +process.on("SIGTERM", () => { + if (!modes.has("ignore-term")) process.exit(0); +}); + +if (!modes.has("no-ready")) lifecycle("ready", "worker-ready"); + +if (modes.has("malformed-secret")) { + setImmediate(() => { + send({ + version: PROTOCOL_VERSION, + type: "status", + requestId: "malformed-secret", + state: { ...state(), apiKey: "fixture-secret-must-not-pass" } + }); + }); +} + +if (modes.has("late-message")) { + process.once("disconnect", () => { + lifecycle("status", "late-after-disconnect"); + }); +} diff --git a/node/test/integration/worker-restart.test.mjs b/node/test/integration/worker-restart.test.mjs new file mode 100644 index 0000000..352f820 --- /dev/null +++ b/node/test/integration/worker-restart.test.mjs @@ -0,0 +1,111 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import http from "node:http"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { WorkerManager } from "../../src/supervisor/worker-manager.mjs"; + +const SECRET = "worker-restart-integration-secret"; + +function listen(server, port = 0) { + return new Promise((resolvePromise, rejectPromise) => { + server.once("error", rejectPromise); + server.listen(port, "127.0.0.1", () => { + server.off("error", rejectPromise); + resolvePromise(server.address().port); + }); + }); +} + +function closeServer(server) { + if (!server.listening) return Promise.resolve(); + return new Promise((resolvePromise, rejectPromise) => { + server.close((error) => { + if (error) rejectPromise(error); + else resolvePromise(); + }); + }); +} + +async function reservePort() { + const probe = http.createServer(); + const port = await listen(probe); + await closeServer(probe); + return port; +} + +function makeSnapshot({ generation, port, upstreamPort, dir }) { + return { + generation, + settings: { + configPath: join(dir, "proxy-config.json"), + server: { host: "127.0.0.1", port, logLevel: "info" }, + upstream: { + baseUrl: `http://127.0.0.1:${upstreamPort}`, + apiKey: SECRET, + timeoutMs: 5_000, + verifySsl: true, + authHeader: "x-provider-auth", + authScheme: "Bearer", + extraHeaders: {} + }, + proxy: { overrideAuthorization: true, requestIdHeader: "x-client-request-id" }, + capture: { enabled: false, dbPath: join(dir, "traffic.sqlite3") } + } + }; +} + +test("real worker restart changes PID and restores matching health on the same fixed port", { + timeout: 15_000 +}, async (t) => { + const dir = mkdtempSync(join(tmpdir(), "crp-worker-restart-")); + t.after(() => rmSync(dir, { recursive: true, force: true })); + + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ ok: true })); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + + const port = await reservePort(); + const manager = new WorkerManager({ + host: "127.0.0.1", + port, + readyTimeoutMs: 3_000, + ackTimeoutMs: 3_000, + healthTimeoutMs: 3_000, + terminateTimeoutMs: 1_000, + killTimeoutMs: 1_000 + }); + t.after(() => manager.close()); + const snapshot = makeSnapshot({ generation: 1, port, upstreamPort, dir }); + + const started = await manager.start(snapshot); + const oldPid = started.pid; + assert.equal(started.phase, "running"); + assert.equal(started.state.listenPort, port); + + const restarted = await manager.restart(snapshot, { drainTimeoutMs: 2_000 }); + assert.equal(restarted.phase, "running"); + assert.notEqual(restarted.pid, oldPid); + assert.equal(restarted.state.listenPort, port); + assert.equal(manager.getPublicState().phase, "running"); + + const healthResponse = await fetch(`http://127.0.0.1:${port}/_proxy/health`); + assert.equal(healthResponse.status, 200); + const health = await healthResponse.json(); + assert.equal(health.configured, true); + assert.equal(health.generation, 1); + assert.equal(JSON.stringify(health).includes(SECRET), false); + + await manager.stop({ drainTimeoutMs: 2_000 }); + const exclusiveProbe = http.createServer(); + await listen(exclusiveProbe, port); + await closeServer(exclusiveProbe); +}); diff --git a/node/test/worker-manager.test.mjs b/node/test/worker-manager.test.mjs new file mode 100644 index 0000000..6404b6c --- /dev/null +++ b/node/test/worker-manager.test.mjs @@ -0,0 +1,732 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { EventEmitter } from "node:events"; + +import { WorkerManager } from "../src/supervisor/worker-manager.mjs"; + +const SECRET = "manager-unit-secret"; + +function makeSnapshot(generation = 1, port = 15100) { + return { + generation, + settings: { + configPath: "/tmp/crp-worker-manager/proxy-config.json", + server: { host: "127.0.0.1", port, logLevel: "info" }, + upstream: { + baseUrl: "http://127.0.0.1:41001", + apiKey: SECRET, + timeoutMs: 5000, + verifySsl: true, + authHeader: "x-provider-auth", + authScheme: "Bearer", + extraHeaders: {} + }, + proxy: { overrideAuthorization: true, requestIdHeader: "x-client-request-id" }, + capture: { enabled: false, dbPath: "/tmp/crp-worker-manager/traffic.sqlite3" } + } + }; +} + +function state(overrides = {}) { + return { + phase: "running", + configured: true, + generation: 1, + listening: true, + listenHost: "127.0.0.1", + listenPort: 15100, + inFlight: 0, + ...overrides + }; +} + +function childMessage(type, requestId, overrides = {}) { + return { version: 1, type, requestId, state: state(overrides) }; +} + +function createClock() { + let current = 1_000; + let sequence = 0; + const timers = new Map(); + return { + now: () => current, + setTimeout(callback, delay) { + const id = ++sequence; + timers.set(id, { id, at: current + delay, callback }); + return id; + }, + clearTimeout(id) { + timers.delete(id); + }, + nextDelay() { + if (timers.size === 0) return null; + return Math.min(...[...timers.values()].map((timer) => timer.at - current)); + }, + async advance(delay) { + current += delay; + const due = [...timers.values()] + .filter((timer) => timer.at <= current) + .sort((left, right) => left.at - right.at || left.id - right.id); + for (const timer of due) { + timers.delete(timer.id); + timer.callback(); + await Promise.resolve(); + } + }, + pending: () => timers.size + }; +} + +let nextPid = 9_000; + +class FakeChild extends EventEmitter { + constructor(script = {}) { + super(); + this.pid = ++nextPid; + this.connected = true; + this.exitCode = null; + this.signalCode = null; + this.sent = []; + this.killed = []; + this.script = script; + this.sendCounts = new Map(); + queueMicrotask(() => { + if (script.ready !== false) { + this.emit("message", childMessage("ready", "worker-ready", { + phase: "ready", + configured: false, + generation: 0, + listening: false, + listenHost: null, + listenPort: null + })); + } + }); + } + + send(message, callback) { + this.sent.push(structuredClone(message)); + const sendCount = (this.sendCounts.get(message.type) ?? 0) + 1; + this.sendCounts.set(message.type, sendCount); + queueMicrotask(() => { + if (this.script.failSend?.[message.type]?.includes(sendCount)) { + callback?.(new Error("sensitive send failure must not pass")); + return; + } + if (message.type === "configure" + && this.script.ackBeforeCallback + && this.script.configure !== false) { + const requestId = this.script.configureRequestId ?? message.requestId; + this.emit("message", childMessage("configured", requestId, { + generation: message.generation, + listenPort: message.settings.server.port + })); + callback?.(null); + return; + } + callback?.(null); + if (message.type === "configure" && this.script.configure !== false) { + const requestId = this.script.configureRequestId ?? message.requestId; + this.emit("message", childMessage("configured", requestId, { + generation: message.generation, + listenPort: message.settings.server.port + })); + } + if (message.type === "drain" && this.script.drain !== false) { + this.emit("message", childMessage("drained", message.requestId, { + phase: "drained", + generation: this.script.generation ?? 1, + listening: false, + listenHost: null, + listenPort: null + })); + } + if (message.type === "shutdown" && this.script.shutdown !== false) { + this.exit(0, null); + } + }); + } + + kill(signal = "SIGTERM") { + this.killed.push(signal); + if (signal === "SIGTERM" && this.script.ignoreTerm) return true; + if (signal === "SIGKILL" && this.script.ignoreKill) return true; + queueMicrotask(() => this.exit(null, signal)); + return true; + } + + disconnect() { + this.connected = false; + } + + exit(code = 1, signal = null) { + if (this.exitCode !== null || this.signalCode !== null) return; + this.connected = false; + this.exitCode = code; + this.signalCode = signal; + this.emit("exit", code, signal); + } +} + +function createHarness(scripts = [], { + healthOk = true, + forkError = false, + portError = false +} = {}) { + const clock = createClock(); + const children = []; + const healthCalls = []; + const portChecks = []; + const manager = new WorkerManager({ + host: "127.0.0.1", + port: 15100, + clock, + readyTimeoutMs: 100, + ackTimeoutMs: 100, + healthTimeoutMs: 100, + terminateTimeoutMs: 100, + killTimeoutMs: 100, + forkWorker() { + if (forkError) throw new Error("sensitive fork cause must not pass"); + const child = new FakeChild(scripts[children.length] ?? {}); + children.push(child); + return child; + }, + async fetchImpl(url) { + healthCalls.push(url); + const generation = children.at(-1)?.sent.findLast((message) => message.type === "configure")?.generation; + return { ok: healthOk, json: async () => ({ configured: healthOk, generation }) }; + }, + async waitForPortFree(host, port) { + portChecks.push({ host, port }); + if (portError) { + const error = new Error("sensitive port probe cause must not pass"); + error.code = "WORKER_PORT_BUSY"; + throw error; + } + } + }); + return { manager, clock, children, healthCalls, portChecks }; +} + +async function settle(promise, clock, { maxSteps = 30 } = {}) { + let settled = false; + let result; + let failure; + promise.then((value) => { + settled = true; + result = value; + }, (error) => { + settled = true; + failure = error; + }); + for (let step = 0; step < maxSteps && !settled; step += 1) { + for (let turn = 0; turn < 12 && !settled; turn += 1) { + await Promise.resolve(); + } + if (!settled && clock.nextDelay() !== null) { + await clock.advance(clock.nextDelay()); + } + } + if (!settled) throw new Error("Test promise did not settle."); + if (failure) throw failure; + return result; +} + +async function flushUntil(predicate, description, maxTurns = 40) { + for (let turn = 0; turn < maxTurns; turn += 1) { + if (predicate()) return; + await Promise.resolve(); + } + throw new Error(`Timed out waiting for ${description}`); +} + +function collectUnhandledRejections(t) { + const reasons = []; + const listener = (reason) => reasons.push(reason); + process.on("unhandledRejection", listener); + t.after(() => process.off("unhandledRejection", listener)); + return reasons; +} + +test("start waits for ready, correlated configure, and matching health before running", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + + assert.equal(harness.manager.getPublicState().phase, "stopped"); + const started = await settle(harness.manager.start(makeSnapshot()), harness.clock); + + assert.equal(started.phase, "running"); + assert.equal(started.pid, harness.children[0].pid); + assert.equal(started.generation, 1); + assert.equal(started.state.phase, "running"); + assert.equal(started.restartCount, 0); + assert.equal(typeof started.startedAt, "string"); + assert.deepEqual(Object.keys(started).sort(), [ + "error", "generation", "phase", "pid", "restartCount", "startedAt", "state" + ]); + assert.equal(JSON.stringify(started).includes(SECRET), false); + assert.equal(harness.healthCalls.length, 1); + assert.equal(harness.children[0].sent[0].type, "configure"); + assert.equal(harness.children[0].sent[0].settings.upstream.apiKey, SECRET); +}); + +test("applySnapshot updates the confirmed generation only after a matching acknowledgement", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + + const applied = await settle(harness.manager.applySnapshot(makeSnapshot(2)), harness.clock); + assert.equal(applied.generation, 2); + assert.equal(applied.state.generation, 2); + + harness.children[0].script.configureRequestId = "wrong-request"; + await assert.rejects( + settle(harness.manager.applySnapshot(makeSnapshot(3)), harness.clock), + (error) => error?.code === "WORKER_ACK_TIMEOUT" && !error.message.includes(SECRET) + ); + assert.equal(harness.manager.getPublicState().generation, 2); + assert.equal(harness.manager.getPublicState().state.generation, 2); +}); + +test("stop drains, shuts down, observes exit, and confirms fixed-port release", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + + const stopped = await settle(harness.manager.stop({ drainTimeoutMs: 100 }), harness.clock); + + assert.equal(stopped.phase, "stopped"); + assert.equal(stopped.pid, null); + assert.deepEqual(harness.children[0].sent.map((message) => message.type), [ + "configure", "drain", "shutdown" + ]); + assert.deepEqual(harness.portChecks, [{ host: "127.0.0.1", port: 15100 }]); +}); + +test("stop escalates a drain timeout through TERM and bounded KILL", async (t) => { + const harness = createHarness([{ drain: false, ignoreTerm: true }]); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + + await settle(harness.manager.stop({ drainTimeoutMs: 100 }), harness.clock); + + assert.deepEqual(harness.children[0].killed, ["SIGTERM", "SIGKILL"]); + assert.equal(harness.manager.getPublicState().phase, "stopped"); + assert.equal(harness.portChecks.length, 1); +}); + +test("termination timeout retains child control so a later close can retry cleanup", async () => { + const harness = createHarness([{ + drain: false, + ignoreTerm: true, + ignoreKill: true + }]); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const child = harness.children[0]; + + await assert.rejects( + settle(harness.manager.stop({ drainTimeoutMs: 100 }), harness.clock), + (error) => error?.code === "WORKER_STOP_FAILED" + ); + const failed = harness.manager.getPublicState(); + assert.equal(failed.phase, "failed"); + assert.equal(failed.pid, child.pid); + assert.equal(child.listenerCount("exit"), 1); + assert.deepEqual(child.killed, ["SIGTERM", "SIGKILL"]); + assert.equal(harness.children.length, 1); + await Promise.resolve(); + await assert.rejects( + harness.manager.start(makeSnapshot()), + (error) => error?.code === "WORKER_STOP_FAILED" + ); + assert.equal(harness.children.length, 1); + assert.equal(harness.manager.getPublicState().pid, child.pid); + + child.script.ignoreTerm = false; + child.script.ignoreKill = false; + const closed = await settle(harness.manager.close(), harness.clock); + assert.equal(closed.phase, "stopped"); + assert.equal(closed.pid, null); + assert.deepEqual(child.killed, ["SIGTERM", "SIGKILL", "SIGTERM"]); + assert.equal(harness.portChecks.length, 1); + assert.equal(child.listenerCount("message"), 0); + assert.equal(child.listenerCount("error"), 0); + assert.equal(child.listenerCount("exit"), 0); + assert.equal(harness.children.length, 1); +}); + +test("partial-start termination timeout preserves the startup error and retryable child control", async () => { + const harness = createHarness([{ + ignoreTerm: true, + ignoreKill: true + }], { healthOk: false }); + const childPromise = harness.manager.start(makeSnapshot()); + + await assert.rejects( + settle(childPromise, harness.clock), + (error) => error?.code === "WORKER_HEALTH_FAILED" + ); + const failed = harness.manager.getPublicState(); + const child = harness.children[0]; + assert.equal(failed.phase, "failed"); + assert.equal(failed.pid, child.pid); + assert.deepEqual(failed.error, { + code: "WORKER_HEALTH_FAILED", + message: "Worker health verification failed." + }); + assert.equal(child.listenerCount("exit"), 1); + + child.script.ignoreTerm = false; + child.script.ignoreKill = false; + const closed = await settle(harness.manager.close(), harness.clock); + assert.equal(closed.phase, "stopped"); + assert.equal(closed.pid, null); + assert.equal(harness.portChecks.length, 1); + assert.equal(harness.children.length, 1); +}); + +test("restart releases the fixed port, changes PID, and requires matching health", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const oldPid = harness.manager.getPublicState().pid; + + const restarted = await settle( + harness.manager.restart(makeSnapshot(2), { drainTimeoutMs: 100 }), + harness.clock + ); + + assert.equal(restarted.phase, "running"); + assert.notEqual(restarted.pid, oldPid); + assert.equal(restarted.generation, 2); + assert.equal(restarted.restartCount, 1); + assert.equal(harness.children.length, 2); + assert.equal(harness.portChecks.length, 1); + assert.equal(harness.healthCalls.length, 2); +}); + +test("restart shares an active lifecycle operation before inspecting its snapshot", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + const starting = harness.manager.start(makeSnapshot()); + let validationReads = 0; + const invalid = { + get generation() { + validationReads += 1; + return 0; + } + }; + + const concurrent = harness.manager.restart(invalid, { drainTimeoutMs: 100 }); + void concurrent.catch(() => {}); + + assert.equal(validationReads, 0); + assert.equal(concurrent, starting); + assert.deepEqual( + await settle(concurrent, harness.clock), + await starting + ); +}); + +test("restart rejects an invalid snapshot before draining or changing the running worker", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const before = harness.manager.getPublicState(); + const invalid = makeSnapshot(2); + invalid.settings.upstream.baseUrl = "http://remote.example.test"; + + await assert.rejects( + harness.manager.restart(invalid, { drainTimeoutMs: 100 }), + (error) => error?.code === "WORKER_SNAPSHOT_INVALID" + ); + + const after = harness.manager.getPublicState(); + assert.equal(after.phase, "running"); + assert.equal(after.pid, before.pid); + assert.equal(after.generation, before.generation); + assert.deepEqual(after.state, before.state); + assert.deepEqual(harness.children[0].sent.map((message) => message.type), ["configure"]); +}); + +test("an unexpected exit backs off 250 ms and ignores the old child epoch after recovery", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const oldChild = harness.children[0]; + + oldChild.exit(1, null); + await flushUntil( + () => harness.manager.getPublicState().phase === "backoff", + "worker backoff" + ); + assert.equal(harness.clock.nextDelay(), 250); + await harness.clock.advance(250); + await flushUntil( + () => harness.manager.getPublicState().phase === "running" + && harness.manager.getPublicState().pid !== oldChild.pid, + "worker recovery" + ); + const recovered = harness.manager.getPublicState(); + + oldChild.emit("message", childMessage("configured", "late-old", { generation: 99 })); + oldChild.emit("exit", 1, null); + await Promise.resolve(); + + assert.equal(harness.manager.getPublicState().pid, recovered.pid); + assert.equal(harness.manager.getPublicState().generation, 1); + assert.equal(harness.manager.getPublicState().phase, "running"); + assert.equal(harness.manager.getPublicState().restartCount, 1); +}); + +test("the fifth crash in 60 seconds enters failed without spawning again", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const observedDelays = []; + + for (let crash = 1; crash <= 5; crash += 1) { + harness.children.at(-1).exit(1, null); + if (crash === 5) { + await flushUntil( + () => harness.manager.getPublicState().phase === "failed", + "terminal failed state" + ); + break; + } + await flushUntil( + () => harness.manager.getPublicState().phase === "backoff", + `backoff ${crash}` + ); + observedDelays.push(harness.clock.nextDelay()); + await harness.clock.advance(harness.clock.nextDelay()); + await flushUntil( + () => harness.manager.getPublicState().phase === "running", + `recovery ${crash}` + ); + } + + assert.deepEqual(observedDelays, [250, 500, 1000, 2000]); + assert.equal(harness.manager.getPublicState().phase, "failed"); + assert.equal(harness.manager.getPublicState().restartCount, 4); + assert.equal(harness.children.length, 5); + assert.equal(harness.clock.pending(), 0); +}); + +test("close is idempotent, cancels backoff, and permanently disables recovery and start", async () => { + const harness = createHarness(); + const first = harness.manager.start(makeSnapshot()); + const concurrent = harness.manager.start(makeSnapshot()); + assert.equal(first, concurrent); + await settle(first, harness.clock); + + harness.children[0].exit(1, null); + await flushUntil( + () => harness.manager.getPublicState().phase === "backoff", + "worker backoff before close" + ); + assert.equal(harness.clock.pending(), 1); + + const closing = harness.manager.close(); + assert.equal(harness.manager.close(), closing); + await settle(closing, harness.clock); + + assert.equal(harness.clock.pending(), 0); + assert.equal(harness.manager.getPublicState().phase, "stopped"); + assert.equal(harness.children.length, 1); + await assert.rejects( + harness.manager.start(makeSnapshot()), + (error) => error?.code === "WORKER_MANAGER_CLOSED" + ); +}); + +test("stop cancels backoff and prevents the scheduled recovery from spawning", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + harness.children[0].exit(1, null); + await flushUntil( + () => harness.manager.getPublicState().phase === "backoff", + "backoff before stop" + ); + assert.equal(harness.clock.nextDelay(), 250); + + const stopped = await harness.manager.stop(); + assert.equal(stopped.phase, "stopped"); + assert.equal(stopped.pid, null); + assert.equal(harness.clock.pending(), 0); + assert.equal(harness.children.length, 1); +}); + +test("a correlated fatal rejects startup with its stable code instead of waiting for timeout", async (t) => { + const harness = createHarness([{ configure: false }]); + t.after(() => harness.manager.close()); + const starting = harness.manager.start(makeSnapshot()); + await flushUntil( + () => harness.children[0]?.sent.some((message) => message.type === "configure"), + "configure send before fatal" + ); + const requestId = harness.children[0].sent.find((message) => message.type === "configure").requestId; + + harness.children[0].emit("message", { + version: 1, + type: "fatal", + requestId, + error: { + code: "WORKER_CONFIGURE_FAILED", + message: "Worker configuration failed." + } + }); + + await assert.rejects( + settle(starting, harness.clock), + (error) => error?.code === "WORKER_CONFIGURE_FAILED" + && error.message === "Worker configuration failed." + && !error.message.includes(SECRET) + ); +}); + +test("send failures cancel and consume start, apply, and drain waiters without unhandled rejection", async (t) => { + const unhandled = collectUnhandledRejections(t); + + const starting = createHarness([{ failSend: { configure: [1] } }]); + t.after(() => starting.manager.close()); + await assert.rejects( + settle(starting.manager.start(makeSnapshot()), starting.clock), + (error) => error?.code === "WORKER_IPC_SEND_FAILED" + && !error.message.includes("sensitive send failure") + ); + assert.equal(starting.clock.pending(), 0); + + const applying = createHarness([{ failSend: { configure: [2] } }]); + t.after(() => applying.manager.close()); + await settle(applying.manager.start(makeSnapshot()), applying.clock); + await assert.rejects( + settle(applying.manager.applySnapshot(makeSnapshot(2)), applying.clock), + (error) => error?.code === "WORKER_IPC_SEND_FAILED" + ); + assert.equal(applying.clock.pending(), 0); + assert.equal(applying.manager.getPublicState().generation, 1); + + const stopping = createHarness([{ failSend: { drain: [1] } }]); + t.after(() => stopping.manager.close()); + await settle(stopping.manager.start(makeSnapshot()), stopping.clock); + await assert.rejects( + settle(stopping.manager.stop({ drainTimeoutMs: 100 }), stopping.clock), + (error) => error?.code === "WORKER_IPC_SEND_FAILED" + ); + assert.equal(stopping.clock.pending(), 0); + assert.equal(stopping.manager.getPublicState().phase, "stopped"); + + for (let turn = 0; turn < 8; turn += 1) await Promise.resolve(); + assert.deepEqual(unhandled, []); +}); + +test("configure acknowledgements arriving before the send callback are retained", async (t) => { + const harness = createHarness([{ ackBeforeCallback: true }]); + t.after(() => harness.manager.close()); + + const started = await settle(harness.manager.start(makeSnapshot()), harness.clock); + assert.equal(started.generation, 1); + const applied = await settle(harness.manager.applySnapshot(makeSnapshot(2)), harness.clock); + assert.equal(applied.generation, 2); + assert.equal(harness.clock.pending(), 0); +}); + +test("a failed startup terminates the partial child and confirms fixed-port release", async (t) => { + const harness = createHarness([], { healthOk: false }); + t.after(() => harness.manager.close()); + + await assert.rejects( + settle(harness.manager.start(makeSnapshot()), harness.clock), + (error) => error?.code === "WORKER_HEALTH_FAILED" + ); + + assert.deepEqual(harness.children[0].killed, ["SIGTERM"]); + assert.deepEqual(harness.portChecks, [{ host: "127.0.0.1", port: 15100 }]); + assert.equal(harness.manager.getPublicState().phase, "stopped"); + assert.equal(harness.manager.getPublicState().pid, null); +}); + +test("close terminates a live child, confirms port release, and removes child listeners", async () => { + const harness = createHarness(); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const child = harness.children[0]; + + await settle(harness.manager.close(), harness.clock); + + assert.deepEqual(child.killed, ["SIGTERM"]); + assert.deepEqual(harness.portChecks, [{ host: "127.0.0.1", port: 15100 }]); + assert.equal(child.listenerCount("message"), 0); + assert.equal(child.listenerCount("error"), 0); + assert.equal(child.listenerCount("exit"), 0); + assert.equal(harness.manager.getPublicState().phase, "stopped"); +}); + +test("a synchronous fork failure returns to stopped with a stable sanitized error", async (t) => { + const harness = createHarness([], { forkError: true }); + t.after(() => harness.manager.close()); + + await assert.rejects( + harness.manager.start(makeSnapshot()), + (error) => error?.code === "WORKER_START_FAILED" + && !error.message.includes("sensitive fork cause") + ); + + const current = harness.manager.getPublicState(); + assert.equal(current.phase, "stopped"); + assert.equal(current.pid, null); + assert.deepEqual(current.error, { + code: "WORKER_START_FAILED", + message: "Worker failed to start." + }); +}); + +test("a port-release failure cleans lifecycle resources and enters a stable failed state", async (t) => { + const harness = createHarness([], { portError: true }); + t.after(() => harness.manager.close().catch(() => {})); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const child = harness.children[0]; + + await assert.rejects( + settle(harness.manager.stop({ drainTimeoutMs: 100 }), harness.clock), + (error) => error?.code === "WORKER_PORT_BUSY" + && !error.message.includes("sensitive port probe cause") + ); + + const current = harness.manager.getPublicState(); + assert.equal(current.phase, "failed"); + assert.equal(current.pid, null); + assert.deepEqual(current.error, { + code: "WORKER_PORT_BUSY", + message: "The fixed proxy port is still in use." + }); + assert.equal(child.listenerCount("message"), 0); + assert.equal(child.listenerCount("error"), 0); + assert.equal(child.listenerCount("exit"), 0); +}); + +test("a malformed secret-bearing child message is sanitized and recovered as a protocol failure", async (t) => { + const harness = createHarness(); + t.after(() => harness.manager.close()); + await settle(harness.manager.start(makeSnapshot()), harness.clock); + const child = harness.children[0]; + + child.emit("message", { + version: 1, + type: "status", + requestId: "malformed-secret", + state: { ...state(), apiKey: SECRET } + }); + await flushUntil( + () => harness.manager.getPublicState().phase === "backoff", + "protocol-failure backoff" + ); + + const current = harness.manager.getPublicState(); + assert.deepEqual(child.killed, ["SIGTERM"]); + assert.equal(current.error.code, "WORKER_PROTOCOL_INVALID"); + assert.equal(current.error.message, "Worker protocol message is invalid."); + assert.equal(JSON.stringify(current).includes(SECRET), false); +}); From 776265c8113f39b26f2bb5cd1418cfc275c4156e Mon Sep 17 00:00:00 2001 From: cluic Date: Mon, 13 Jul 2026 11:25:27 +0800 Subject: [PATCH 11/44] feat: orchestrate provider lifecycle --- docs/AGENT_COORDINATION.md | 2 +- docs/AI_HANDOFF.md | 11 +- docs/ARCHITECTURE.md | 4 +- docs/DATA_MODEL.md | 8 +- docs/PERMISSIONS.md | 6 + docs/STATUS.md | 7 +- docs/TESTING.md | 9 + .../plans/2026-07-10-crp-v1-implementation.md | 14 +- node/src/supervisor/activity-store.mjs | 382 ++++++++ node/src/supervisor/migration.mjs | 674 ++++++++++++++ node/src/supervisor/provider-service.mjs | 725 +++++++++++++++ node/test/activity-store.test.mjs | 321 +++++++ node/test/migration.test.mjs | 631 +++++++++++++ node/test/provider-service.test.mjs | 857 ++++++++++++++++++ 14 files changed, 3633 insertions(+), 18 deletions(-) create mode 100644 node/src/supervisor/activity-store.mjs create mode 100644 node/src/supervisor/migration.mjs create mode 100644 node/src/supervisor/provider-service.mjs create mode 100644 node/test/activity-store.test.mjs create mode 100644 node/test/migration.test.mjs create mode 100644 node/test/provider-service.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index d823005..94a6507 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus Task 7 implementation agent | Resolve Task 7 lifecycle review findings with one writer | Task 7 owns `node/src/supervisor/worker-manager.mjs`, `node/test/fixtures/fake-worker.mjs`, `node/test/worker-manager.test.mjs`, `node/test/integration/worker-restart.test.mjs`, `node/scripts/run-tests.mjs`, the minimal group selection in `node/scripts/run-test-group.mjs`, and affected living docs | Task 7 review fixes complete and deterministic gates pass; pending re-review | Task 7 commit `9129df281a3fb58724aeb5866d0bb08aabf2747d` | Task 6 protocol/worker entry, Task 5 server/runtime settings and capture source/tests, provider and credential stores, Codex, CLI/bin, Admin API, UI, package scripts, and every other module; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus Task 8 implementation agent | Resolve Task 8 security-review findings with one writer | Task 8 owns `node/src/supervisor/activity-store.mjs`, `node/src/supervisor/migration.mjs`, `node/src/supervisor/provider-service.mjs`, their three focused tests, and affected living docs | Review fixes implemented; deterministic gates passing; re-review pending; real HOME/keyring/network migration remains prohibited | Task 8 commit `fa5e485468065c3f2eff54d2c9ebcc69fbcf8e35` | Existing provider registry, credential adapters, worker manager, server/runtime settings, capture, Codex, CLI/bin, Admin API, UI, runners, package scripts, and every other module; no simultaneous writable agent | ## Shared Contracts diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index d37a56c..dd826cd 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,11 +6,11 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 7 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, the versioned proxy-worker IPC entrypoint, and reliable fixed-port worker lifecycle management. Task 8, activity, migration, and provider orchestration, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 8 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, the versioned proxy-worker IPC entrypoint, reliable fixed-port worker lifecycle management, sanitized activity, transactional legacy migration, and provider orchestration. Task 9, the secured loopback Admin API, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, idempotent Codex bootstrap with source-EOL preservation and atomic replacement, strict provider-schema validation, a lock-serialized atomic schema-version-2 provider registry, credential adapters with explicit-only file fallback, monotonic immutable runtime snapshots captured once per proxied request, a strict version-1 child-process protocol whose worker listens only after valid configuration, and a worker manager with acknowledged configuration, bounded drain/termination, fixed-port release confirmation, health verification, epoch isolation, and bounded crash recovery. Target: provider-service orchestration and the long-lived supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap, strict provider storage, secure credential adapters, immutable request snapshots, strict worker IPC, reliable worker management, bounded sanitized activity, transactional v0.2.2 migration, and serialized provider CRUD/test/activation with compensating rollback. Target: the long-lived secured supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 7 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 8 are complete. ## How To Run Current Code @@ -48,6 +48,7 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 Task 5 gate: `node --test test/runtime-settings.test.mjs test/server.test.mjs` passes 13/13, `npm test` passes 102/102, and `npm run lint` syntax-checks 15 source files. Coverage includes strict generations, clone-before-freeze replacement, public-state allowlisting, one snapshot read per request before body listeners, delayed A versus immediate B switching, transport/TLS and timeout pinning, unconfigured-source rejection, health secret scans, request/response dynamic auth-header log masking, and bidirectional custom-auth capture redaction. - Node 22.19 Task 6 gate: `node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` passes 21/21, `npm run test:integration` passes 11/11, and `npm test` passes a nonduplicated 112/112 top-level group followed by 11/11 integration tests; `npm run lint` syntax-checks 18 source files. Coverage includes exact version-1 directional schemas, HTTPS-or-loopback upstreams, HTTP-token authentication fields, final authentication-value validation, sensitive and conflicting header rejection, parent-only configure secrets, fixed invalid-message fatal IDs, configure-before-listen, monotonic reconfiguration, configure rejection after drain begins, retained in-flight drain acknowledgement, idempotent duplicate drain, health/status, idle keep-alive closure, clean shutdown, bounded parent-disconnect cleanup with a hanging upstream before or during shutdown, safe startup failures, stale generations, port conflicts, port release, and child-process cleanup without fixed sleeps. Real-fork integration runs after the ordinary group so it cannot race watcher readiness in another test process. - Node 22.19 Task 7 gate: strict-unhandled `node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` passes 22/22; exact `npm test` passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests without duplication; `npm run test:unit` retains all-top-level coverage and passes 133/133. `npm run lint` syntax-checks 19 source files, and `npm audit --omit=dev` reports zero vulnerabilities. Coverage includes ready/configure/health startup, acknowledgement-atomic snapshot application, concurrent restart reuse before inspecting its snapshot, restart prevalidation before drain when no operation exists, immediate waiter observation and send-failure cancellation, graceful drain/shutdown, TERM/KILL escalation, retained child control after termination timeout, same-port real-worker restart with a changed PID, partial-start and port-release cleanup, correlated fatal and malformed-message sanitization, old-epoch isolation, cancellable injected-clock crash backoff, immediate failure on the fifth crash in 60 seconds, and idempotent retryable close without child, timer, listener, or port residue. The exact runner isolates the preexisting polling capture watcher between the core unit and real-fork integration groups so its registration baseline cannot race unrelated unit load. +- Node 22.19 Task 8 gate: `node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs` passes 42/42; exact `npm test` passes 168/168 core assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run lint` syntax-checks 22 source files. Tests use only temporary paths, injected credential adapters/fetch responses, fake worker boundaries, and loopback redirect servers. Coverage includes recursive lifecycle-field redaction, ownership-checked activity locks, descriptor-safe migration paths, exclusive final-path registry creation, symlink and foreign-replacement preservation, committed-state reconciliation, redirect refusal, active-update rejection, replacement-secret compensation, selected credentials, explicit operation serialization, and conservative activation rollback `1 -> 2 -> 3` after health or lost-ACK uncertainty. Real HOME migration, native keyrings, cross-platform filesystem semantics, and live upstreams remain prohibited until L3 platform review. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. @@ -92,3 +93,7 @@ Credential migration, localhost browser security, in-flight activation semantics - Restart snapshots must pass complete configure validation before the current worker is drained or its state changes. - IPC acknowledgement promises must be observed at registration and cancelled synchronously when send fails. - A child reference and its listeners must remain owned after TERM/KILL timeout until a later lifecycle attempt confirms exit and fixed-port release. +- Provider-service operations must use their own mutex; a worker-manager shared operation must never substitute for provider-level serialization. +- A post-ack activation rollback must advance to a new generation; never regress a worker generation to restore the confirmed provider. +- Compensation tests must run and fail before compensation code is written. +- Migration must create a missing registry exclusively at its final path and must never adopt, modify, or delete an `EEXIST` registry. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 6c27af6..8e1a0f5 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -58,7 +58,7 @@ Landed in Tasks 2 through 7: Remaining target-state boundaries: -- `supervisor`: composes the worker manager with the future admin server, provider service, and activity records. +- `supervisor`: now owns worker management, bounded activity, transactional migration, and provider orchestration; the Admin server remains the next composition layer. - `provider-service`: coordinates compatibility tests, credentials, activation, and credential-aware deletion above the metadata registry. - `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. - `web-ui`: static local app with onboarding and daily management views. @@ -89,5 +89,7 @@ Remaining target-state boundaries: - Port release races during restart. - Localhost CSRF and DNS-rebinding-style attacks. - Safe migration from the existing flat secret-bearing configuration. +- Provider compatibility tests never follow redirects; 3xx is a stable failure so custom authentication cannot be forwarded to another origin. Active profiles reject every update until another provider is activated or the proxy is stopped. +- Provider-service activation persists the candidate active ID, reconciles committed/degraded persistence from the registry, marks the worker attempt before sending, and applies one selected credential through a strictly increasing snapshot. A health failure or rejected acknowledgement is treated as an unknown worker commit and restores the prior profile with a newer confirmed generation (`1 -> 2 -> 3`); rollback uncertainty is degraded and requires explicit repair. These areas require integration tests and L3 review before release. diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index 5cb4584..a33ad5e 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -1,6 +1,6 @@ # Data Model -Tasks 3 through 6 have implemented strict schema-version-2 provider metadata persistence, native and explicit-consent file credential adapters, immutable in-memory runtime snapshots, and a strict version-1 worker protocol. Provider-service orchestration and migration remain target-state work. +Tasks 3 through 8 have implemented strict schema-version-2 provider metadata persistence, native and explicit-consent file credential adapters, immutable runtime snapshots, strict worker IPC, bounded activity, transactional migration, and provider-service orchestration. ## ProviderProfile @@ -86,7 +86,7 @@ Runtime state includes supervisor PID plus the worker manager's exact public pro ## ActivityEvent -Activity events record timestamp, category, action, provider ID, result, stable error code, and sanitized details. They never contain complete keys, authorization headers, cookies, or request/response bodies. +Activity events persist the exact allowlist `{ timestamp, category, action, providerId, result, errorCode, details }`. Details are recursively serialized with cycles and non-JSON values bounded. Error messages/causes/stacks are omitted, and compacted authorization, cookie, token, secret, API-key, credential-ref, request/response-body, cause, stack, headers, and backup-path field names are redacted. Atomic `0600` JSONL replacement retains the newest 10,000 events no older than 30 days; lock initialization and release claim canonical state and remove only identity-proven ownership. Uncertainty restores a canonical blocker and stops later mutations. ## Relationships @@ -96,7 +96,7 @@ Activity events record timestamp, category, action, provider ID, result, stable ## Lifecycle and Deletion -The following provider-service lifecycle behavior remains target state; Task 3 implements only profile test-state recording, active-ID persistence, active-profile delete rejection, and inactive-profile metadata deletion. +The provider service now implements the following lifecycle behavior: - A profile must pass a Responses compatibility test before first activation. - Deleting the active profile is rejected until another profile is activated or the proxy is stopped. @@ -105,4 +105,4 @@ The following provider-service lifecycle behavior remains target state; Task 3 i ## Migration -On first version-2 start, back up existing CRP files, convert the flat upstream to a provider named `Default`, move the API key into the selected credential backend, persist schema version 2, and verify the resulting profile before removing the old secret field. Migration must be transactional and recoverable from its backup. +On first version-2 start, migration transaction-locks injected legacy paths, rejects symbolic links, reads regular sources through lstat/open-no-follow/fstat identity validation, and writes collision-safe byte-exact `0600` backups. It converts the flat upstream to an inactive and untested provider named `Default`, moves the key to an opaque credential reference, validates schema version 2, and only then scrubs legacy secret fields. Failure restores source bytes in reverse order and removes registry state only when the current identity and bytes match this transaction. Foreign replacements are preserved, committed mutation errors are reconciled from disk, and lock uncertainty restores a canonical blocker. Existing valid schema version 2 is idempotently preserved. diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md index 81c675a..49f5fe9 100644 --- a/docs/PERMISSIONS.md +++ b/docs/PERMISSIONS.md @@ -44,6 +44,12 @@ Record provider creation/update/deletion, test result, activation, Codex bootstr - Reject non-loopback admin binds. - Validate URL scheme and block accidental credential forwarding to an unvalidated target. - Treat diagnostics and capture content as sensitive local data. +- Activity persistence must use only its exact public event allowlist and must omit Error messages, causes, stacks, credentials, headers, bodies, and backup paths. +- Migration tests and ordinary deterministic gates must inject temporary paths and adapters; real HOME, native-keyring, and upstream migration is reserved for L3 platform confirmation. +- Provider CRUD, testing, and activation must serialize through a provider-service mutex and resolve only the selected provider credential for test/activation. +- Provider tests must use a no-follow redirect policy; no 3xx target may receive the configured authentication header or credential. +- Migration and activity cleanup may delete only atomically claimed paths whose descriptor identity or ownership token matches; foreign replacements must remain canonical blockers. +- A `committed: true` persistence error must be reconciled from durable state before compensation; it must never trigger an inverse credential mutation by assumption. ## Credential Verification Boundary diff --git a/docs/STATUS.md b/docs/STATUS.md index abf12ef..0b1e631 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 7 are complete and Task 8 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 8 are complete and Task 9 is next. ## Done @@ -25,14 +25,15 @@ V1 implementation: provider lifecycle end to end. - Completed Task 5 monotonic immutable runtime settings and request-start snapshot capture. In-flight A remains internally consistent while new traffic switches to B; dynamic health, transport and timeout pinning, request/response short and custom authentication log masking, and bidirectional custom-auth capture redaction are covered by 13/13 focused and 102/102 full Node 22.19 tests; syntax checking covers 15 source files. - Completed Task 6 strict version-1 worker IPC and the proxy-worker child entrypoint. Configure enforces HTTPS-or-loopback upstreams, HTTP-token authentication fields, Node-compatible final authentication values, and sensitive/header-collision rejection; the worker remains unbound until valid configuration, rejects configure after drain begins without losing the pending drain acknowledgement, keeps repeated drain acknowledgements and status drained, bounds parent-disconnect cleanup when an upstream hangs before or during shutdown, uses a fixed correlation ID for invalid-message fatals, releases its port on shutdown, and emits only sanitized child messages. Coverage passes 21/21 focused and 11/11 integration tests; the full gate passes 112/112 top-level tests followed by 11/11 integration tests without duplication. Syntax checking covers 18 source files, including the sequential group runner that isolates real-fork load from watcher tests. - Completed Task 7 reliable worker lifecycle management and its review fixes. Startup requires ready, correlated configure acknowledgement, and matching health; snapshot generations advance only after matching acknowledgement; concurrent restart calls reuse the current operation before inspecting a new snapshot, while an unshared restart validates the complete snapshot before drain; send failures synchronously cancel observed acknowledgement waiters; stop and restart bound drain, TERM, and KILL, confirm exclusive fixed-port release, retain control after termination timeout, and isolate old child epochs. Unexpected exits recover with cancellable injected-clock backoff and fail immediately on the fifth crash in 60 seconds. Strict-unhandled coverage passes 22/22 focused tests; the exact full gate passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests; the all-top-level compatibility command passes 133/133. Syntax checking covers 19 source files, and the runtime audit reports zero vulnerabilities. +- Completed Task 8 sanitized activity, transactional legacy migration, and provider orchestration. Activity uses an exact event allowlist, expanded recursive denylist, bounded retention, atomic replacement, and ownership-checked degraded locks. Migration uses descriptor-safe source/registry reads, symlink rejection, byte-exact exclusive backups, transaction locking, exclusive final-path registry creation, identity-checked rollback, committed-state reconciliation, and canonical blockers. Provider CRUD/test/activation rejects active edits, refuses redirects, reconciles committed mutations, serializes selected-credential operations, and deterministically rolls an uncertain candidate back with a newer generation. Focused coverage passes 42/42; exact full coverage passes 168/168 core, 7/7 capture, and 12/12 integration assertions; syntax covers 22 source files. All tests use temporary paths, injected adapters, and loopback mocks; real HOME/keyring/live-network migration remains an L3 gate. ## Blocked -- No current blocker is recorded for Task 8. +- No current blocker is recorded for Task 9. ## Next -1. Execute Task 8: add activity, migration, and provider orchestration. +1. Execute Task 9: build the secured loopback Admin API. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/TESTING.md b/docs/TESTING.md index 971f959..c0e854e 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -26,6 +26,7 @@ - Task 5 focused suite: `cd node && node --test test/runtime-settings.test.mjs test/server.test.mjs` - Task 6 focused suite: `cd node && node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` - Task 7 focused suite: `cd node && node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` +- Task 8 focused suite: `cd node && node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs` - Worker integration suite: `cd node && npm run test:integration` - Runtime audit: `cd node && npm audit --omit=dev` @@ -39,6 +40,8 @@ The Node 22.19 Task 6 focused suite passes 21/21, `npm run test:integration` pas The Node 22.19 Task 7 strict-unhandled focused suite passes 22/22. Exact `npm test` passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run test:unit` passes all 133/133 top-level assertions; `npm run lint` syntax-checks 19 source files. Coverage verifies ready/configure/health startup, strict request correlation, acknowledgement-atomic generation changes, concurrent restart Promise identity and zero snapshot inspection before shared-operation reuse, restart prevalidation without drain side effects when no operation exists, send-failure waiter cancellation without unhandled rejections, early acknowledgement retention, graceful drain/shutdown, TERM/KILL escalation, retained retryable control after termination timeout, partial-start error preservation, fixed-port cleanup, same-port real-worker PID replacement, sanitized fatal and malformed messages, old-epoch isolation, cancellable injected-clock crash recovery, immediate failure on the fifth crash in 60 seconds, and idempotent close with no child, timer, listener, process, or port residue. The fifth in-window crash does not schedule the 4000 ms capped delay; only the first four 250/500/1000/2000 ms delays execute before the threshold. +The Node 22.19 Task 8 focused suite passes 42/42. Exact `npm test` passes 168/168 core assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run lint` syntax-checks 22 source files. Coverage verifies the expanded activity denylist and full-file secret scans, Error/cycle/non-JSON handling, retention, atomic failure preservation, foreign lock replacement, and canonical degraded blocking; descriptor-safe temp-only migration, symlink rejection, byte-exact backups, exclusive final-path registry creation, identity/byte-owned rollback, foreign registry preservation, transaction locks, committed schema reconciliation, and canonical release blockers; active-update rejection, no-follow redirect behavior across two loopback origins, CRUD/credential committed outcomes, replacement rollback safety, selected credentials, explicit serialization, and deterministic `1 -> 2 -> 3` rollback after health or acknowledgement uncertainty. No test accesses a real HOME, native credential backend, or live upstream network. + `npm run test:unit` retains its public behavior and runs all top-level `test/*.test.mjs` files. The exact `npm test` gate runs the same top-level set without duplication as two sequential groups: every non-capture unit file first, then `capture-store.test.mjs` alone, followed by recursively discovered `test/integration/**/*.test.mjs`. This isolates the polling watcher registration baseline from unrelated unit load and keeps real child-process integration after watcher cleanup. `test:e2e` and the combined `test:all` command are not current gates until the UI, Playwright configuration, and E2E specs land. ## Test Authoring Rules @@ -75,6 +78,12 @@ The Node 22.19 Task 7 strict-unhandled focused suite passes 22/22. Exact `npm te - Restart validation must complete before any drain message, phase change, or current-worker mutation. - IPC acknowledgement waiters must be observed immediately and cancelled on send failure. - Lifecycle code must retain child control after termination timeout until exit and port release are confirmed. +- Activity tests must scan the complete persisted file for every generated secret and cover retention, atomic-write failure, foreign locks, and committed lock degradation. +- Migration tests must use only injected temporary paths/adapters and cover byte-exact exclusive backups, schema-2 idempotency, reverse rollback, residual-lock degradation, and backup retention. +- Provider-service compensation tests must fail before implementation and cover create, credential replacement, delete, active-delete ordering, stable test classifications, selected credentials, explicit concurrency serialization, and activation rollback. +- Post-ack activation rollback must use a newer generation and confirm both acknowledgement and health before replacing the confirmed snapshot. +- A rejected worker start/apply Promise must be treated as an unknown commit once sending was attempted; prior state requires a higher-generation deterministic rollback, while no-prior state requires bounded stop. +- Committed/degraded mutation tests must reconcile registry and credential facts without inverse compensation, and replacement rollback must never restore `passed` unless the old secret was restored successfully. ## Test Matrix diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index e755cc4..0fc0918 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -666,19 +666,19 @@ Actual Node 22.19 Task 7 verification after review fixes: the strict-unhandled f - Create: `node/test/migration.test.mjs` - Create: `node/test/provider-service.test.mjs` -- [ ] **Step 1: Write failing activity redaction tests** +- [x] **Step 1: Write failing activity redaction tests** Append an event containing authorization, cookie, token, secret, API key, and nested error fields. Assert the persisted JSONL contains `[REDACTED]`, never the test values, and retention truncates to the newest 10,000 rows. -- [ ] **Step 2: Write transactional migration tests** +- [x] **Step 2: Write transactional migration tests** Given the current flat `config.json`, assert migration creates one `Default` provider, stores its key through the injected credential adapter, backs up original files, and removes the secret from provider metadata. Inject a credential-write failure and assert original bytes are restored and no schema-2 file remains. -- [ ] **Step 3: Write provider-service tests** +- [x] **Step 3: Write provider-service tests** Use two local mock upstreams and assert `testProvider(id, model)` classifies DNS/TLS/timeout/401/404/invalid-Responses errors, marks success, and that `activate(id)` rejects untested profiles, resolves only the selected credential, increments generation, persists active ID, and waits for worker acknowledgement. -- [ ] **Step 4: Run tests and verify failure** +- [x] **Step 4: Run tests and verify failure** ```bash cd node @@ -687,7 +687,7 @@ node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-s Expected: FAIL because the orchestration modules do not exist. -- [ ] **Step 5: Implement the three services** +- [x] **Step 5: Implement the three services** Use one safe recursive redactor shared by activity and errors. `ProviderService` exposes: @@ -703,7 +703,7 @@ getStatus(); `testProvider` sends a minimal `POST /responses` request with `stream: false`, `input: "Reply with OK."`, and the selected model. Store only stable result codes and timestamps. -- [ ] **Step 6: Verify and commit** +- [x] **Step 6: Verify and commit** ```bash cd node @@ -713,6 +713,8 @@ git add node/src/supervisor/activity-store.mjs node/src/supervisor/migration.mjs git commit -m "feat: orchestrate provider lifecycle" ``` +Actual Node 22.19 Task 8 verification after security review fixes: the focused activity/migration/provider-service suite passes 42/42; exact `npm test` passes 168/168 core assertions, 7/7 isolated capture assertions, and 12/12 integration tests; syntax checking covers 22 source files. Coverage includes redirect refusal, active-update rejection, expanded activity redaction, ownership-checked locks, descriptor-safe migration, exclusive final-path registry creation, symlink and foreign-state preservation, committed-state reconciliation, unknown worker-commit rollback, and replacement-secret rollback safety. All migration, credential, fetch, and worker boundaries are injected, temporary, or loopback-only. Real HOME migration, native keyrings, cross-platform permission/rename behavior, and live upstream activation remain L3 expert gates. + ## Task 9: Build the Secured Loopback Admin API **Files:** diff --git a/node/src/supervisor/activity-store.mjs b/node/src/supervisor/activity-store.mjs new file mode 100644 index 0000000..8581c02 --- /dev/null +++ b/node/src/supervisor/activity-store.mjs @@ -0,0 +1,382 @@ +import { + chmodSync, + closeSync, + existsSync, + fsyncSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +} from "node:fs"; +import { randomUUID } from "node:crypto"; +import { basename, dirname, join } from "node:path"; + +import { CrpError } from "../shared/errors.mjs"; + +const REDACTED = "[REDACTED]"; +const UNSERIALIZABLE = "[UNSERIALIZABLE]"; +const CIRCULAR = "[CIRCULAR]"; +const SENSITIVE_TERMS = [ + "authorization", + "cookie", + "token", + "secret", + "apikey", + "credentialref", + "requestbody", + "responsebody", + "cause", + "stack", + "headers", + "backuppath" +]; +const EVENT_FIELDS = new Set([ + "timestamp", + "category", + "action", + "providerId", + "result", + "errorCode", + "details" +]); +const DEFAULT_MAX_EVENTS = 10_000; +const DEFAULT_RETENTION_MS = 30 * 24 * 60 * 60 * 1_000; +const DEFAULT_FILE_OPERATIONS = { + chmodSync, + closeSync, + existsSync, + fsyncSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +}; + +function activityError(code, message, action, status = 500, details = {}) { + return new CrpError(code, message, action, { status, details }); +} + +function invalidActivity() { + return activityError( + "ACTIVITY_EVENT_INVALID", + "The activity event is invalid.", + "Record only supported lifecycle activity fields.", + 400 + ); +} + +function storeError(code = "ACTIVITY_STORE_WRITE_FAILED", details = {}) { + const messages = { + ACTIVITY_STORE_BUSY: [ + "The activity store is already being updated.", + "Wait for the current activity update to finish and try again." + ], + ACTIVITY_STORE_INVALID: [ + "The activity store is invalid.", + "Restore a valid activity file or remove it after making a backup." + ], + ACTIVITY_STORE_WRITE_FAILED: [ + "The activity event could not be saved.", + "Check local storage permissions and try again." + ], + ACTIVITY_STORE_COMMITTED_LOCK_DEGRADED: [ + "The activity event was saved, but its lock could not be fully released.", + "Stop CRP, explicitly repair the residual activity lock, then restart CRP." + ], + ACTIVITY_STORE_LOCK_DEGRADED: [ + "The activity store lock could not be safely recovered.", + "Stop CRP, explicitly repair the residual activity lock, then restart CRP." + ] + }; + const [message, action] = messages[code] ?? messages.ACTIVITY_STORE_WRITE_FAILED; + return activityError(code, message, action, code === "ACTIVITY_STORE_BUSY" ? 409 : 500, details); +} + +function isIsoTimestamp(value) { + if (typeof value !== "string") return false; + try { + return new Date(value).toISOString() === value; + } catch { + return false; + } +} + +function normalizeString(value, { nullable = false } = {}) { + if (nullable && value === null) return null; + if (typeof value !== "string" || value.length === 0) throw invalidActivity(); + return value; +} + +function isSensitiveKey(key) { + const compact = String(key).toLowerCase().replace(/[^a-z0-9]/g, ""); + return SENSITIVE_TERMS.some((term) => compact.includes(term)); +} + +function sanitizeValue(value, seen) { + if (value === null || typeof value === "boolean" || typeof value === "string") { + return value; + } + if (typeof value === "number") { + return Number.isFinite(value) ? value : UNSERIALIZABLE; + } + if (typeof value === "bigint") return value.toString(); + if (typeof value === "undefined" || typeof value === "function" || typeof value === "symbol") { + return UNSERIALIZABLE; + } + if (seen.has(value)) return CIRCULAR; + seen.add(value); + + try { + if (value instanceof Date) { + return Number.isNaN(value.getTime()) ? UNSERIALIZABLE : value.toISOString(); + } + if (Array.isArray(value)) { + return value.map((item) => sanitizeValue(item, seen)); + } + + const sanitized = {}; + if (value instanceof Error) sanitized.name = String(value.name || "Error"); + for (const key of Object.keys(value)) { + if (isSensitiveKey(key)) { + sanitized[key] = REDACTED; + continue; + } + let property; + try { + property = value[key]; + } catch { + property = UNSERIALIZABLE; + } + sanitized[key] = property === UNSERIALIZABLE + ? UNSERIALIZABLE + : sanitizeValue(property, seen); + } + return sanitized; + } finally { + seen.delete(value); + } +} + +export function sanitizeActivityValue(value) { + return sanitizeValue(value, new WeakSet()); +} + +function normalizeEvent(event, now) { + if (event === null || typeof event !== "object" || Array.isArray(event)) { + throw invalidActivity(); + } + const timestamp = event.timestamp ?? now(); + if (!isIsoTimestamp(timestamp)) throw invalidActivity(); + const errorCode = event.errorCode ?? null; + if (errorCode !== null && !/^[A-Z][A-Z0-9_]*$/.test(errorCode)) { + throw invalidActivity(); + } + return { + timestamp, + category: normalizeString(event.category), + action: normalizeString(event.action), + providerId: normalizeString(event.providerId ?? null, { nullable: true }), + result: normalizeString(event.result), + errorCode, + details: sanitizeActivityValue(event.details ?? {}) + }; +} + +function validateStoredEvent(event) { + if (event === null || typeof event !== "object" || Array.isArray(event) + || Object.keys(event).length !== EVENT_FIELDS.size + || Object.keys(event).some((key) => !EVENT_FIELDS.has(key))) { + throw storeError("ACTIVITY_STORE_INVALID"); + } + const normalized = normalizeEvent(event, () => event.timestamp); + if (JSON.stringify(normalized) !== JSON.stringify(event)) { + throw storeError("ACTIVITY_STORE_INVALID"); + } + return normalized; +} + +export class ActivityStore { + constructor({ + path, + now = () => new Date().toISOString(), + maxEvents = DEFAULT_MAX_EVENTS, + retentionMs = DEFAULT_RETENTION_MS, + fileOperations = DEFAULT_FILE_OPERATIONS, + createId = randomUUID + }) { + if (typeof path !== "string" || path.length === 0 + || !Number.isSafeInteger(maxEvents) || maxEvents < 1 + || !Number.isSafeInteger(retentionMs) || retentionMs < 1) { + throw invalidActivity(); + } + this.path = path; + this.lockPath = `${path}.crp.lock`; + this.now = now; + this.maxEvents = maxEvents; + this.retentionMs = retentionMs; + this.fileOperations = fileOperations; + this.createId = createId; + this.degraded = false; + } + + append(event) { + const normalized = normalizeEvent(event, this.now); + this.#ensureParent(); + if (this.degraded) throw storeError("ACTIVITY_STORE_LOCK_DEGRADED"); + const lock = this.#acquireLock(); + let primaryError; + let committed = false; + try { + const events = this.#load(); + events.push(normalized); + const nowMs = new Date(this.now()).getTime(); + const cutoff = nowMs - this.retentionMs; + const retained = events + .filter((entry) => new Date(entry.timestamp).getTime() >= cutoff) + .slice(-this.maxEvents); + this.#persist(retained); + committed = true; + } catch (error) { + primaryError = error instanceof CrpError + ? error + : storeError("ACTIVITY_STORE_WRITE_FAILED"); + } + + const released = this.#releaseLock(lock); + if (!released) this.degraded = true; + if (primaryError) throw primaryError; + if (!released) { + throw storeError("ACTIVITY_STORE_COMMITTED_LOCK_DEGRADED", { committed }); + } + return structuredClone(normalized); + } + + list({ limit = this.maxEvents } = {}) { + if (!Number.isSafeInteger(limit) || limit < 1 || limit > this.maxEvents) { + throw invalidActivity(); + } + this.#ensureParent(); + return this.#load().slice(-limit).reverse().map((event) => structuredClone(event)); + } + + #ensureParent() { + const parent = dirname(this.path); + this.fileOperations.mkdirSync(parent, { recursive: true, mode: 0o700 }); + try { + this.fileOperations.chmodSync(parent, 0o700); + } catch { + // Windows ACL verification remains an L3 platform gate. + } + } + + #load() { + if (!this.fileOperations.existsSync(this.path)) return []; + let text; + try { + text = this.fileOperations.readFileSync(this.path, "utf8"); + } catch { + throw storeError("ACTIVITY_STORE_INVALID"); + } + if (text.length === 0) return []; + try { + return text.trimEnd().split("\n").map((line) => validateStoredEvent(JSON.parse(line))); + } catch (error) { + if (error instanceof CrpError && error.code === "ACTIVITY_STORE_INVALID") throw error; + throw storeError("ACTIVITY_STORE_INVALID"); + } + } + + #acquireLock() { + const token = `${this.createId()}\n`; + let descriptor; + let owned = false; + let closed = false; + try { + descriptor = this.fileOperations.openSync(this.lockPath, "wx", 0o600); + owned = true; + this.fileOperations.writeFileSync(descriptor, token, "utf8"); + this.fileOperations.fsyncSync(descriptor); + this.fileOperations.closeSync(descriptor); + closed = true; + descriptor = undefined; + this.fileOperations.chmodSync(this.lockPath, 0o600); + return token; + } catch (error) { + if (!owned) { + if (error?.code === "EEXIST") throw storeError("ACTIVITY_STORE_BUSY"); + throw storeError("ACTIVITY_STORE_WRITE_FAILED"); + } + if (!closed && descriptor !== undefined) { + try { + this.fileOperations.closeSync(descriptor); + closed = true; + } catch {} + } + const cleaned = closed ? this.#releaseLock(token) : false; + if (!closed || !cleaned) { + this.degraded = true; + throw storeError("ACTIVITY_STORE_LOCK_DEGRADED", { committed: false }); + } + throw storeError("ACTIVITY_STORE_WRITE_FAILED"); + } + } + + #releaseLock(token) { + const claimPath = join( + dirname(this.lockPath), + `.${basename(this.lockPath)}.${this.createId()}.release` + ); + let claimed = false; + try { + this.fileOperations.renameSync(this.lockPath, claimPath); + claimed = true; + if (this.fileOperations.readFileSync(claimPath, "utf8") !== token) { + this.#restoreClaim(claimPath); + return false; + } + this.fileOperations.rmSync(claimPath); + return true; + } catch { + if (claimed) this.#restoreClaim(claimPath); + return false; + } + } + + #restoreClaim(claimPath) { + try { + this.fileOperations.renameSync(claimPath, this.lockPath); + } catch { + // A foreign canonical lock is already a blocker; never remove it. + } + } + + #persist(events) { + const tempPath = join( + dirname(this.path), + `.${basename(this.path)}.${this.createId()}.tmp` + ); + const bytes = events.length === 0 + ? "" + : `${events.map((event) => JSON.stringify(event)).join("\n")}\n`; + let descriptor; + try { + descriptor = this.fileOperations.openSync(tempPath, "wx", 0o600); + this.fileOperations.writeFileSync(descriptor, bytes, "utf8"); + this.fileOperations.fsyncSync(descriptor); + this.fileOperations.closeSync(descriptor); + descriptor = undefined; + this.fileOperations.chmodSync(tempPath, 0o600); + this.fileOperations.renameSync(tempPath, this.path); + } catch { + if (descriptor !== undefined) { + try { this.fileOperations.closeSync(descriptor); } catch {} + } + try { this.fileOperations.rmSync(tempPath, { force: true }); } catch {} + throw storeError("ACTIVITY_STORE_WRITE_FAILED"); + } + } +} diff --git a/node/src/supervisor/migration.mjs b/node/src/supervisor/migration.mjs new file mode 100644 index 0000000..ff59de8 --- /dev/null +++ b/node/src/supervisor/migration.mjs @@ -0,0 +1,674 @@ +import { randomUUID } from "node:crypto"; +import { + closeSync, + constants as FS_CONSTANTS, + fchmodSync, + fstatSync, + fsyncSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +} from "node:fs"; +import { basename, dirname, join } from "node:path"; + +import { ProviderRegistry } from "../providers/provider-registry.mjs"; +import { CrpError } from "../shared/errors.mjs"; + +const DEFAULT_FILE_OPERATIONS = { + closeSync, + fchmodSync, + fstatSync, + fsyncSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +}; +const BACKUP_ATTEMPTS = 8; + +function migrationError(code, cause, details = {}) { + const contracts = { + MIGRATION_BUSY: [ + "Provider migration is already running.", + "Wait for the current migration to finish and try again.", + 409 + ], + MIGRATION_INPUT_INVALID: [ + "The legacy provider configuration is invalid.", + "Restore a complete legacy provider URL and credential before migrating.", + 400 + ], + MIGRATION_REGISTRY_CONFLICT: [ + "The provider registry was created by another process during migration.", + "Review the current provider registry before retrying migration.", + 409 + ], + MIGRATION_FAILED: [ + "CRP could not migrate the legacy provider configuration.", + "Review local storage and credential-backend health, then retry migration.", + 500 + ], + MIGRATION_ROLLBACK_DEGRADED: [ + "Provider migration failed and rollback could not be completed safely.", + "Stop CRP and restore the retained migration backup before restarting.", + 500 + ], + MIGRATION_COMMITTED_LOCK_DEGRADED: [ + "Provider migration completed, but its transaction lock could not be fully released.", + "Stop CRP, explicitly repair the residual migration lock, then restart CRP.", + 500 + ], + MIGRATION_COMMITTED_DEGRADED: [ + "Provider migration completed, but persistence cleanup degraded.", + "Stop CRP and repair the retained migration state before restarting.", + 500 + ] + }; + const [message, action, status] = contracts[code] ?? contracts.MIGRATION_FAILED; + return new CrpError(code, message, action, { status, cause, details }); +} + +function isCommittedError(error) { + return error instanceof CrpError && error.details?.committed === true; +} + +function parseJson(bytes) { + try { + const value = JSON.parse(bytes); + if (value === null || typeof value !== "object" || Array.isArray(value)) { + throw new Error("not an object"); + } + return value; + } catch (error) { + throw migrationError("MIGRATION_INPUT_INVALID", error); + } +} + +function optionalString(...values) { + return values.find((value) => typeof value === "string" && value.trim().length > 0)?.trim() ?? null; +} + +function sourceValues(config, runtime) { + const runtimeUpstream = runtime?.upstream; + const runtimeObject = runtimeUpstream !== null && typeof runtimeUpstream === "object" + && !Array.isArray(runtimeUpstream) ? runtimeUpstream : {}; + const secretCandidates = [ + runtimeObject.apiKey, + runtime?.apiKey, + runtime?.upstreamApiKey, + runtime?.upstream_api_key, + config?.apiKey, + config?.upstreamApiKey, + config?.upstream_api_key + ].filter((value) => typeof value === "string" && value.length > 0); + if (new Set(secretCandidates).size > 1) { + throw migrationError("MIGRATION_INPUT_INVALID"); + } + return { + baseUrl: optionalString( + runtimeObject.baseUrl, + runtime?.baseUrl, + runtime?.upstreamBaseUrl, + runtime?.upstream_base_url, + config?.upstreamBaseUrl, + config?.upstream_base_url, + config?.baseUrl + ), + secret: secretCandidates[0] ?? null, + authHeader: optionalString(runtimeObject.authHeader, runtime?.authHeader) ?? "authorization", + authScheme: typeof runtimeObject.authScheme === "string" + ? runtimeObject.authScheme + : (typeof runtime?.authScheme === "string" ? runtime.authScheme : "Bearer"), + extraHeaders: runtimeObject.extraHeaders ?? runtime?.extraHeaders ?? {} + }; +} + +function scrubDocument(document) { + const next = structuredClone(document); + for (const key of ["apiKey", "upstreamApiKey", "upstream_api_key"]) delete next[key]; + if (next.upstream !== null && typeof next.upstream === "object" && !Array.isArray(next.upstream)) { + delete next.upstream.apiKey; + } + return next; +} + +function emptyRegistryBytes() { + return Buffer.from(`${JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [], + settings: { + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: 15101, + captureEnabled: false + } + }, null, 2)}\n`, "utf8"); +} + +function identityOf(stats) { + return { dev: stats.dev, ino: stats.ino }; +} + +function sameIdentity(left, right) { + return left !== null && right !== null + && left.dev === right.dev + && left.ino === right.ino; +} + +function lstatRegular(path, fileOperations, { missing = false } = {}) { + let stats; + try { + stats = fileOperations.lstatSync(path); + } catch (error) { + if (missing && error?.code === "ENOENT") return null; + throw migrationError("MIGRATION_INPUT_INVALID", error); + } + if (!stats.isFile() || stats.isSymbolicLink()) { + throw migrationError("MIGRATION_INPUT_INVALID"); + } + return { stats, identity: identityOf(stats) }; +} + +function readSafeFile(path, fileOperations, { missing = false } = {}) { + const before = lstatRegular(path, fileOperations, { missing }); + if (before === null) return null; + const noFollow = typeof FS_CONSTANTS.O_NOFOLLOW === "number" ? FS_CONSTANTS.O_NOFOLLOW : 0; + let descriptor; + try { + descriptor = fileOperations.openSync(path, FS_CONSTANTS.O_RDONLY | noFollow); + const descriptorStats = fileOperations.fstatSync(descriptor); + if (!descriptorStats.isFile() + || !sameIdentity(before.identity, identityOf(descriptorStats))) { + throw migrationError("MIGRATION_INPUT_INVALID"); + } + const bytes = fileOperations.readFileSync(descriptor); + fileOperations.closeSync(descriptor); + descriptor = undefined; + return { path, bytes, identity: before.identity }; + } catch (error) { + if (descriptor !== undefined) { + try { fileOperations.closeSync(descriptor); } catch {} + } + if (error instanceof CrpError) throw error; + throw migrationError("MIGRATION_INPUT_INVALID", error); + } +} + +function restoreClaim(claimPath, canonicalPath, fileOperations) { + try { + fileOperations.renameSync(claimPath, canonicalPath); + } catch { + // A foreign canonical path is already a blocker; never remove it. + } +} + +function ensureCanonicalBlocker(path, fileOperations) { + let descriptor; + try { + descriptor = fileOperations.openSync(path, "wx", 0o600); + fileOperations.writeFileSync(descriptor, "migration-degraded\n", "utf8"); + fileOperations.fsyncSync(descriptor); + fileOperations.closeSync(descriptor); + descriptor = undefined; + return true; + } catch (error) { + if (descriptor !== undefined) { + try { fileOperations.closeSync(descriptor); } catch {} + } + return error?.code === "EEXIST"; + } +} + +function claimOwnedPath(path, expectedIdentity, fileOperations, createId = randomUUID) { + const claimPath = join(dirname(path), `.${basename(path)}.${createId()}.claim`); + try { + fileOperations.renameSync(path, claimPath); + } catch { + return false; + } + let claim; + try { + claim = lstatRegular(claimPath, fileOperations); + } catch { + restoreClaim(claimPath, path, fileOperations); + return false; + } + if (!sameIdentity(claim.identity, expectedIdentity)) { + restoreClaim(claimPath, path, fileOperations); + return false; + } + try { + fileOperations.rmSync(claimPath); + return true; + } catch { + restoreClaim(claimPath, path, fileOperations); + return false; + } +} + +function createLock({ lockPath, fileOperations, createId }) { + fileOperations.mkdirSync(dirname(lockPath), { recursive: true, mode: 0o700 }); + const token = `${createId()}\n`; + let descriptor; + let identity = null; + let closed = false; + try { + descriptor = fileOperations.openSync(lockPath, "wx", 0o600); + identity = identityOf(fileOperations.fstatSync(descriptor)); + fileOperations.writeFileSync(descriptor, token, "utf8"); + fileOperations.fchmodSync(descriptor, 0o600); + fileOperations.fsyncSync(descriptor); + fileOperations.closeSync(descriptor); + closed = true; + descriptor = undefined; + return { token, identity }; + } catch (error) { + if (identity === null) { + if (error?.code === "EEXIST") throw migrationError("MIGRATION_BUSY", error); + throw migrationError("MIGRATION_FAILED", error); + } + if (!closed && descriptor !== undefined) { + try { + fileOperations.closeSync(descriptor); + closed = true; + } catch {} + } + const cleaned = closed && claimOwnedPath(lockPath, identity, fileOperations, createId); + if (!cleaned) { + ensureCanonicalBlocker(lockPath, fileOperations); + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", error, { + committed: false, + degraded: true + }); + } + throw migrationError("MIGRATION_FAILED", error); + } +} + +function releaseLock({ lockPath, lock, fileOperations, createId }) { + const claimPath = join( + dirname(lockPath), + `.${basename(lockPath)}.${createId()}.release` + ); + try { + fileOperations.renameSync(lockPath, claimPath); + } catch { + ensureCanonicalBlocker(lockPath, fileOperations); + return false; + } + try { + const claimed = readSafeFile(claimPath, fileOperations); + if (!sameIdentity(claimed.identity, lock.identity) + || claimed.bytes.toString("utf8") !== lock.token) { + restoreClaim(claimPath, lockPath, fileOperations); + ensureCanonicalBlocker(lockPath, fileOperations); + return false; + } + fileOperations.rmSync(claimPath); + return true; + } catch { + restoreClaim(claimPath, lockPath, fileOperations); + ensureCanonicalBlocker(lockPath, fileOperations); + return false; + } +} + +function writeExclusive(path, bytes, fileOperations, createId = randomUUID) { + let descriptor; + let identity = null; + let closed = false; + try { + descriptor = fileOperations.openSync(path, "wx", 0o600); + identity = identityOf(fileOperations.fstatSync(descriptor)); + fileOperations.writeFileSync(descriptor, bytes); + fileOperations.fchmodSync(descriptor, 0o600); + fileOperations.fsyncSync(descriptor); + fileOperations.closeSync(descriptor); + closed = true; + descriptor = undefined; + const committed = lstatRegular(path, fileOperations); + if (!sameIdentity(committed.identity, identity)) { + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", null, { + committed: false, + degraded: true + }); + } + return committed.identity; + } catch (error) { + if (identity === null) throw error; + if (!closed && descriptor !== undefined) { + try { + fileOperations.closeSync(descriptor); + closed = true; + } catch {} + } + const cleaned = closed && claimOwnedPath(path, identity, fileOperations, createId); + if (!cleaned) { + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", error, { + committed: false, + degraded: true + }); + } + throw error; + } +} + +function createBackup(source, fileOperations, createBackupId) { + for (let attempt = 0; attempt < BACKUP_ATTEMPTS; attempt += 1) { + const backupPath = `${source.path}.${createBackupId()}.bak`; + try { + writeExclusive(backupPath, source.bytes, fileOperations, createBackupId); + return backupPath; + } catch (error) { + if (error?.code !== "EEXIST") throw error; + } + } + throw migrationError("MIGRATION_FAILED"); +} + +function replaceFile(path, bytes, expectedIdentity, fileOperations, createId) { + const tempPath = join(dirname(path), `.${basename(path)}.${createId()}.tmp`); + let tempIdentity = null; + try { + const before = lstatRegular(path, fileOperations); + if (!sameIdentity(before.identity, expectedIdentity)) { + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", null, { + committed: false, + degraded: true + }); + } + tempIdentity = writeExclusive(tempPath, bytes, fileOperations, createId); + const current = lstatRegular(path, fileOperations); + if (!sameIdentity(current.identity, expectedIdentity)) { + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", null, { + committed: false, + degraded: true + }); + } + fileOperations.renameSync(tempPath, path); + const committed = lstatRegular(path, fileOperations); + if (!sameIdentity(committed.identity, tempIdentity)) { + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", null, { + committed: true, + degraded: true + }); + } + return committed.identity; + } catch (error) { + if (tempIdentity !== null) { + claimOwnedPath(tempPath, tempIdentity, fileOperations, createId); + } + throw error; + } +} + +function readSource(path, fileOperations) { + const source = readSafeFile(path, fileOperations, { missing: true }); + if (source === null) return null; + return { ...source, currentIdentity: source.identity, document: parseJson(source.bytes) }; +} + +function assertCurrentRegistry(path, fileOperations) { + const source = readSafeFile(path, fileOperations, { missing: true }); + if (source === null) return false; + const document = parseJson(source.bytes); + if (document.schemaVersion !== 2) throw migrationError("MIGRATION_INPUT_INVALID"); + new ProviderRegistry({ path, fileOperations }); + const after = lstatRegular(path, fileOperations); + if (!sameIdentity(after.identity, source.identity)) { + throw migrationError("MIGRATION_INPUT_INVALID"); + } + return true; +} + +export async function migrateLegacyConfiguration({ + paths, + credentialStore, + activityStore = null, + now = () => new Date().toISOString(), + createProviderId = randomUUID, + createCredentialRef = randomUUID, + createBackupId = randomUUID, + createLockId = randomUUID, + fileOperations: overrides = {} +}) { + if (!paths || typeof paths.globalHome !== "string" || typeof paths.registryPath !== "string" + || !credentialStore || typeof credentialStore.set !== "function" + || typeof credentialStore.delete !== "function") { + throw migrationError("MIGRATION_INPUT_INVALID"); + } + const fileOperations = { ...DEFAULT_FILE_OPERATIONS, ...overrides }; + const legacyConfigPath = paths.legacyConfigPath ?? join(paths.globalHome, "config.json"); + const runtimeConfigPath = paths.runtimeConfigPath + ?? join(paths.globalHome, "node", "proxy-config.json"); + const lockPath = `${paths.registryPath}.migration.lock`; + const lock = createLock({ + lockPath, + fileOperations, + createId: createLockId + }); + + let completed = false; + let providerId = null; + let credentialRef = null; + let credentialAttempted = false; + let registryOwned = null; + let commitWarning = null; + const sources = []; + const scrubbedSources = []; + let failure; + + try { + if (assertCurrentRegistry(paths.registryPath, fileOperations)) { + completed = true; + return { migrated: false, reason: "already-current" }; + } + + const configSource = readSource(legacyConfigPath, fileOperations); + const runtimeSource = readSource(runtimeConfigPath, fileOperations); + if (configSource) sources.push(configSource); + if (runtimeSource) sources.push(runtimeSource); + if (sources.length === 0) { + completed = true; + return { migrated: false, reason: "no-legacy-config" }; + } + + const values = sourceValues(configSource?.document, runtimeSource?.document); + if (!values.baseUrl || !values.secret) throw migrationError("MIGRATION_INPUT_INVALID"); + + fileOperations.mkdirSync(paths.globalHome, { recursive: true, mode: 0o700 }); + for (const source of sources) { + createBackup(source, fileOperations, createBackupId); + } + + providerId = createProviderId(); + credentialRef = createCredentialRef(); + credentialAttempted = true; + try { + await credentialStore.set(credentialRef, values.secret); + } catch (error) { + if (isCommittedError(error)) commitWarning = error; + else throw error; + } + + const initialRegistryBytes = emptyRegistryBytes(); + try { + registryOwned = { + identity: writeExclusive( + paths.registryPath, + initialRegistryBytes, + fileOperations, + createLockId + ), + bytes: Buffer.from(initialRegistryBytes) + }; + } catch (error) { + if (error?.code === "EEXIST") { + throw migrationError("MIGRATION_REGISTRY_CONFLICT", error); + } + throw error; + } + + const registry = new ProviderRegistry({ + path: paths.registryPath, + createId: () => providerId, + now, + fileOperations + }); + try { + registry.create({ + name: "Default", + baseUrl: values.baseUrl, + credentialRef, + authHeader: values.authHeader, + authScheme: values.authScheme, + extraHeaders: values.extraHeaders, + modelMode: "passthrough", + modelOverride: null + }); + } catch (error) { + if (isCommittedError(error)) commitWarning = error; + else throw error; + } + const committed = registry.getDocument(); + if (committed.schemaVersion !== 2 + || committed.activeProviderId !== null + || committed.providers.length !== 1 + || committed.providers[0].lastTestStatus !== "untested") { + throw migrationError("MIGRATION_FAILED"); + } + const committedRegistry = readSafeFile(paths.registryPath, fileOperations); + registryOwned = { + identity: committedRegistry.identity, + bytes: Buffer.from(committedRegistry.bytes) + }; + + for (const source of sources) { + const scrubbedBytes = Buffer.from( + `${JSON.stringify(scrubDocument(source.document), null, 2)}\n`, + "utf8" + ); + source.currentIdentity = replaceFile( + source.path, + scrubbedBytes, + source.currentIdentity, + fileOperations, + createLockId + ); + scrubbedSources.push(source); + } + + if (activityStore) { + await activityStore.append({ + category: "migration", + action: "legacy-config", + providerId, + result: commitWarning ? "degraded" : "success", + errorCode: commitWarning ? "MIGRATION_COMMITTED_DEGRADED" : null, + details: { sourceCount: sources.length } + }); + } + completed = true; + if (commitWarning) { + throw migrationError("MIGRATION_COMMITTED_DEGRADED", commitWarning, { + committed: true, + degraded: true + }); + } + return { migrated: true, providerId }; + } catch (error) { + if (completed && isCommittedError(error)) throw error; + failure = error; + let rollbackFailed = error?.code === "MIGRATION_ROLLBACK_DEGRADED" + || error?.details?.degraded === true; + for (const source of scrubbedSources.reverse()) { + try { + source.currentIdentity = replaceFile( + source.path, + source.bytes, + source.currentIdentity, + fileOperations, + createLockId + ); + } catch { + rollbackFailed = true; + } + } + if (registryOwned !== null) { + try { + const currentRegistry = readSafeFile(paths.registryPath, fileOperations, { missing: true }); + if (currentRegistry === null + || !sameIdentity(currentRegistry.identity, registryOwned.identity) + || !currentRegistry.bytes.equals(registryOwned.bytes) + || !claimOwnedPath( + paths.registryPath, + registryOwned.identity, + fileOperations, + createLockId + )) { + rollbackFailed = true; + } + } catch { + rollbackFailed = true; + } + } + if (credentialAttempted && credentialRef !== null) { + try { await credentialStore.delete(credentialRef); } catch { rollbackFailed = true; } + } + if (activityStore) { + try { + await activityStore.append({ + category: "migration", + action: "legacy-config", + providerId, + result: "failed", + errorCode: rollbackFailed ? "MIGRATION_ROLLBACK_DEGRADED" : "MIGRATION_FAILED", + details: { rollbackDegraded: rollbackFailed } + }); + } catch { + rollbackFailed = true; + } + } + if (rollbackFailed) { + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", failure, { + committed: false, + degraded: true + }); + } + if (error instanceof CrpError && ( + error.code === "MIGRATION_INPUT_INVALID" + || error.code === "MIGRATION_REGISTRY_CONFLICT" + )) { + throw error; + } + throw migrationError("MIGRATION_FAILED", failure, { committed: false }); + } finally { + const released = releaseLock({ + lockPath, + lock, + fileOperations, + createId: createLockId + }); + if (!released && completed) { + throw migrationError("MIGRATION_COMMITTED_LOCK_DEGRADED", null, { + committed: true, + degraded: true + }); + } + if (!released && !completed) { + throw migrationError("MIGRATION_ROLLBACK_DEGRADED", failure, { + committed: false, + degraded: true + }); + } + } +} diff --git a/node/src/supervisor/provider-service.mjs b/node/src/supervisor/provider-service.mjs new file mode 100644 index 0000000..5e73111 --- /dev/null +++ b/node/src/supervisor/provider-service.mjs @@ -0,0 +1,725 @@ +import { randomUUID } from "node:crypto"; + +import { toPublicProvider } from "../providers/provider-schema.mjs"; +import { CrpError } from "../shared/errors.mjs"; + +function serviceError(code, { status = 500, cause, details = {} } = {}) { + const contracts = { + PROVIDER_SECRET_INVALID: [ + "The provider credential is invalid.", + "Enter a non-empty provider credential and try again." + ], + PROVIDER_ACTIVE: [ + "The active provider cannot be deleted.", + "Activate another provider or stop the proxy first." + ], + PROVIDER_CREATE_FAILED: [ + "The provider could not be created.", + "Review the provider settings and try again." + ], + PROVIDER_CREATE_ROLLBACK_DEGRADED: [ + "Provider creation failed and its credential could not be removed safely.", + "Stop CRP and repair the credential entry before retrying." + ], + PROVIDER_CREATE_COMMITTED_DEGRADED: [ + "The provider was created, but its persistence cleanup degraded.", + "Stop CRP and repair the residual provider state before restarting." + ], + PROVIDER_UPDATE_ROLLBACK_DEGRADED: [ + "Provider update failed and its prior credential could not be restored safely.", + "Stop CRP and repair the provider credential before retrying." + ], + PROVIDER_UPDATE_COMMITTED_DEGRADED: [ + "The provider update was saved, but its persistence cleanup degraded.", + "Stop CRP and repair the residual provider state before restarting." + ], + PROVIDER_DELETE_FAILED: [ + "The provider could not be deleted.", + "Review Activity and try again." + ], + PROVIDER_DELETE_ROLLBACK_DEGRADED: [ + "Provider deletion failed and its credential could not be restored safely.", + "Stop CRP and repair the provider credential before retrying." + ], + PROVIDER_DELETE_COMMITTED_DEGRADED: [ + "The provider was deleted, but its persistence cleanup degraded.", + "Stop CRP and repair the residual provider state before restarting." + ], + PROVIDER_TEST_INPUT_INVALID: [ + "The provider test model is invalid.", + "Enter a non-empty model name and try again." + ], + PROVIDER_NOT_READY: [ + "The provider has not passed its compatibility test.", + "Test the provider successfully before activating it." + ], + PROVIDER_ACTIVATION_FAILED: [ + "The provider could not be activated.", + "Review worker health and try the activation again." + ], + PROVIDER_ACTIVATION_ROLLBACK_DEGRADED: [ + "Provider activation failed and the prior active provider could not be restored safely.", + "Stop CRP and repair the active-provider state before restarting." + ], + PROVIDER_ACTIVATION_COMMITTED_DEGRADED: [ + "The provider was activated, but its persistence cleanup degraded.", + "Stop CRP and repair the residual active-provider state before restarting." + ] + }; + const [message, action] = contracts[code] ?? [ + "The provider operation failed.", + "Review Activity and try again." + ]; + return new CrpError(code, message, action, { status, cause, details }); +} + +function assertSecret(secret) { + if (typeof secret !== "string" || secret.length === 0) { + throw serviceError("PROVIDER_SECRET_INVALID", { status: 400 }); + } +} + +function isCommittedError(error) { + return error instanceof CrpError && error.details?.committed === true; +} + +function committedServiceError(action, cause) { + return serviceError(`PROVIDER_${action.toUpperCase()}_COMMITTED_DEGRADED`, { + cause, + details: { committed: true, degraded: true } + }); +} + +export class ProviderService { + #operationTail = Promise.resolve(); + + constructor({ + registry, + credentialStore, + activityStore, + workerManager, + createCredentialRef = randomUUID, + now = () => new Date().toISOString(), + ...options + }) { + if (!registry || !credentialStore || !activityStore || !workerManager) { + throw new TypeError("ProviderService dependencies are required."); + } + this.registry = registry; + this.credentialStore = credentialStore; + this.activityStore = activityStore; + this.workerManager = workerManager; + this.createCredentialRef = createCredentialRef; + this.now = now; + this.options = options; + this.fetchImpl = options.fetchImpl ?? globalThis.fetch; + this.createTimeoutSignal = options.createTimeoutSignal + ?? ((timeoutMs) => AbortSignal.timeout(timeoutMs)); + this.testTimeoutMs = options.testTimeoutMs ?? 15_000; + this.verifyWorkerHealth = options.verifyWorkerHealth ?? (async (generation, state) => ( + state?.phase === "running" && state?.generation === generation + )); + this.paths = options.paths ?? {}; + const workerGeneration = workerManager.getPublicState()?.generation; + this.confirmedGeneration = Number.isSafeInteger(workerGeneration) && workerGeneration >= 0 + ? workerGeneration + : 0; + this.confirmedSnapshot = options.initialSnapshot + ? structuredClone(options.initialSnapshot) + : null; + } + + async listProviders() { + const profiles = this.registry.list(); + return await Promise.all(profiles.map((profile) => this.#toPublic(profile))); + } + + createProvider(input, secret, { fallbackConsent = false } = {}) { + return this.#runExclusive(async () => { + let credentialRef = null; + let credentialWritten = false; + let credentialCommitWarning = null; + let profile; + try { + assertSecret(secret); + credentialRef = this.createCredentialRef(); + try { + await this.credentialStore.set(credentialRef, secret, { fallbackConsent }); + credentialWritten = true; + } catch (error) { + if (isCommittedError(error)) { + credentialWritten = true; + credentialCommitWarning = error; + } else { + throw error; + } + } + profile = this.registry.create({ ...input, credentialRef }); + } catch (error) { + if (isCommittedError(error)) { + profile = this.registry.list().find((candidate) => ( + candidate.credentialRef === credentialRef + )); + if (profile) { + const committed = committedServiceError("create", error); + await this.#recordCommitted("create", profile.id, committed); + throw committed; + } + const degraded = serviceError("PROVIDER_CREATE_ROLLBACK_DEGRADED", { + cause: error, + details: { committed: false, degraded: true } + }); + await this.#safeRecordFailure("create", null, degraded); + throw degraded; + } + let failure = error; + if (credentialWritten) { + try { + await this.credentialStore.delete(credentialRef); + } catch (rollbackError) { + failure = serviceError("PROVIDER_CREATE_ROLLBACK_DEGRADED", { + cause: rollbackError, + details: { committed: false, degraded: true } + }); + } + } + if (!(failure instanceof CrpError)) { + failure = serviceError("PROVIDER_CREATE_FAILED", { cause: failure }); + } + await this.#safeRecordFailure("create", null, failure); + throw failure; + } + if (credentialCommitWarning) { + const committed = committedServiceError("create", credentialCommitWarning); + await this.#recordCommitted("create", profile.id, committed); + throw committed; + } + const publicProfile = toPublicProvider(profile, true); + await this.#record("create", profile.id, "success", null, {}); + return publicProfile; + }); + } + + updateProvider(id, patch, replacementSecret) { + return this.#runExclusive(async () => { + let profile; + let safeId = null; + try { + const current = this.registry.get(id); + safeId = current.id; + if (this.registry.getDocument().activeProviderId === current.id) { + throw serviceError("PROVIDER_ACTIVE", { status: 409 }); + } + if (replacementSecret !== undefined) assertSecret(replacementSecret); + profile = replacementSecret === undefined + ? this.registry.update(id, patch) + : await this.#updateWithReplacementSecret(current, patch, replacementSecret); + } catch (error) { + if (isCommittedError(error)) { + const committed = committedServiceError("update", error); + await this.#recordCommitted("update", safeId, committed); + throw committed; + } + await this.#safeRecordFailure("update", safeId, error); + throw error; + } + const publicProfile = await this.#toPublic(profile); + await this.#record( + "update", + profile.id, + "success", + null, + replacementSecret === undefined ? {} : { credentialReplaced: true } + ); + return publicProfile; + }); + } + + deleteProvider(id) { + return this.#runExclusive(async () => { + let profile; + let oldSecret; + let deleted; + let credentialDeleted = false; + let credentialCommitWarning = null; + try { + const document = this.registry.getDocument(); + if (document.activeProviderId === id) { + throw serviceError("PROVIDER_ACTIVE", { status: 409 }); + } + profile = this.registry.get(id); + oldSecret = await this.credentialStore.get(profile.credentialRef); + try { + credentialDeleted = await this.credentialStore.delete(profile.credentialRef); + } catch (error) { + if (isCommittedError(error)) { + credentialDeleted = true; + credentialCommitWarning = error; + } else { + throw error; + } + } + if (!credentialDeleted) throw serviceError("PROVIDER_DELETE_FAILED"); + deleted = this.registry.delete(id); + } catch (error) { + if (isCommittedError(error)) { + const committed = committedServiceError("delete", error); + await this.#recordCommitted("delete", profile?.id ?? id, committed); + throw committed; + } + if (credentialCommitWarning) { + const degraded = serviceError("PROVIDER_DELETE_ROLLBACK_DEGRADED", { + cause: error, + details: { committed: false, degraded: true } + }); + await this.#safeRecordFailure("delete", profile?.id ?? id, degraded); + throw degraded; + } + let failure = error; + if (credentialDeleted) { + try { + await this.credentialStore.set(profile.credentialRef, oldSecret); + } catch (rollbackError) { + failure = serviceError("PROVIDER_DELETE_ROLLBACK_DEGRADED", { + cause: rollbackError, + details: { committed: false, degraded: true } + }); + } + } + if (!(failure instanceof CrpError)) { + failure = serviceError("PROVIDER_DELETE_FAILED", { cause: failure }); + } + await this.#safeRecordFailure("delete", profile?.id ?? null, failure); + throw failure; + } + if (credentialCommitWarning) { + const committed = committedServiceError("delete", credentialCommitWarning); + await this.#recordCommitted("delete", deleted.id, committed); + throw committed; + } + await this.#record("delete", deleted.id, "success", null, {}); + return toPublicProvider(deleted, false); + }); + } + + testProvider(id, model) { + return this.#runExclusive(async () => { + if (typeof model !== "string" || model.trim().length === 0) { + throw serviceError("PROVIDER_TEST_INPUT_INVALID", { status: 400 }); + } + const profile = this.registry.get(id); + const secret = await this.credentialStore.get(profile.credentialRef); + let result; + let status = null; + try { + const base = profile.baseUrl.endsWith("/") ? profile.baseUrl : `${profile.baseUrl}/`; + const response = await this.fetchImpl(new URL("responses", base).toString(), { + method: "POST", + redirect: "manual", + headers: { + "content-type": "application/json", + ...profile.extraHeaders, + [profile.authHeader]: profile.authScheme + ? `${profile.authScheme} ${secret}` + : secret + }, + body: JSON.stringify({ + model: model.trim(), + stream: false, + input: "Reply with OK." + }), + signal: this.createTimeoutSignal(this.testTimeoutMs) + }); + status = Number.isInteger(response?.status) ? response.status : null; + if (!response?.ok) { + result = { ok: false, code: classifyHttpStatus(status) }; + } else { + let payload; + try { + payload = await response.json(); + } catch { + result = { ok: false, code: "PROVIDER_TEST_INVALID_JSON" }; + } + if (!result) { + result = isCompatibleResponsesPayload(payload) + ? { ok: true, code: null } + : { ok: false, code: "PROVIDER_TEST_INVALID_RESPONSES" }; + } + } + } catch (error) { + result = { ok: false, code: classifyFetchError(error) }; + } + + this.registry.markTest(id, result.ok + ? { status: "passed" } + : { status: "failed", code: result.code }); + await this.#record( + "test", + id, + result.ok ? "success" : "failed", + result.code, + status === null ? {} : { httpStatus: status } + ); + return result; + }); + } + + activate(id) { + return this.#runExclusive(async () => { + const profile = this.registry.get(id); + if (profile.lastTestStatus !== "passed") { + await this.#record( + "activate", + id, + "failed", + "PROVIDER_NOT_READY", + {} + ); + throw serviceError("PROVIDER_NOT_READY", { status: 409 }); + } + const secret = await this.credentialStore.get(profile.credentialRef); + const previousId = this.registry.getDocument().activeProviderId; + const generation = this.confirmedGeneration + 1; + if (!Number.isSafeInteger(generation) || generation < 1) { + throw serviceError("PROVIDER_ACTIVATION_FAILED"); + } + const snapshot = this.#buildSnapshot(profile, secret, generation); + let activePersisted = false; + let workerAttempted = false; + let activeCommitWarning = null; + let activationCompleted = false; + try { + try { + this.registry.setActive(id); + activePersisted = true; + } catch (error) { + if (isCommittedError(error) + && this.registry.getDocument().activeProviderId === id) { + activePersisted = true; + activeCommitWarning = error; + } else { + throw error; + } + } + const before = this.workerManager.getPublicState(); + workerAttempted = true; + const workerState = before.phase === "running" + ? await this.workerManager.applySnapshot(snapshot) + : await this.workerManager.start(snapshot); + if (!isConfirmedWorkerState(workerState, generation)) { + throw new Error("worker generation was not confirmed"); + } + const healthy = await this.verifyWorkerHealth(generation, workerState); + if (healthy !== true) throw new Error("worker health was not confirmed"); + this.confirmedGeneration = generation; + this.confirmedSnapshot = structuredClone(snapshot); + if (activeCommitWarning) { + const committed = committedServiceError("activation", activeCommitWarning); + await this.#recordCommitted("activate", id, committed, { generation }); + activationCompleted = true; + throw committed; + } + await this.#record("activate", id, "success", null, { generation }); + return { + activeProviderId: id, + activeProvider: toPublicProvider(profile, true), + generation, + worker: publicWorkerState(workerState) + }; + } catch (error) { + if (activationCompleted && isCommittedError(error)) throw error; + let rollbackFailure = null; + if (activePersisted) { + try { + this.registry.setActive(previousId); + } catch (rollbackError) { + rollbackFailure = rollbackError; + } + } + if (workerAttempted) { + try { + if (this.confirmedSnapshot) { + const observedGeneration = this.workerManager.getPublicState()?.generation; + const rollbackGeneration = Math.max( + generation, + this.confirmedGeneration, + Number.isSafeInteger(observedGeneration) ? observedGeneration : 0 + ) + 1; + if (!Number.isSafeInteger(rollbackGeneration)) { + throw new Error("worker rollback generation is invalid"); + } + const rollbackSnapshot = structuredClone(this.confirmedSnapshot); + rollbackSnapshot.generation = rollbackGeneration; + const rollbackState = this.workerManager.getPublicState(); + const restored = rollbackState?.phase === "running" + ? await this.workerManager.applySnapshot(rollbackSnapshot) + : await this.workerManager.restart(rollbackSnapshot); + if (!isConfirmedWorkerState(restored, rollbackGeneration)) { + throw new Error("worker rollback was not confirmed"); + } + const rollbackHealthy = await this.verifyWorkerHealth( + rollbackGeneration, + restored + ); + if (rollbackHealthy !== true) { + throw new Error("worker rollback health was not confirmed"); + } + this.confirmedGeneration = rollbackGeneration; + this.confirmedSnapshot = structuredClone(rollbackSnapshot); + } else { + const stopped = await this.workerManager.stop(); + if (stopped?.phase !== "stopped") { + throw new Error("unconfirmed worker did not stop"); + } + } + } catch (workerRollbackError) { + rollbackFailure ??= workerRollbackError; + } + } + if (rollbackFailure) { + const degraded = serviceError("PROVIDER_ACTIVATION_ROLLBACK_DEGRADED", { + cause: rollbackFailure, + details: { committed: false, degraded: true } + }); + await this.#safeRecordFailure("activate", id, degraded, { generation }); + throw degraded; + } + await this.#safeRecordFailure("activate", id, error, { generation }); + throw serviceError("PROVIDER_ACTIVATION_FAILED", { cause: error }); + } + }); + } + + async getStatus() { + const document = this.registry.getDocument(); + const activeProfile = document.activeProviderId === null + ? null + : document.providers.find((profile) => profile.id === document.activeProviderId) ?? null; + return { + activeProviderId: document.activeProviderId, + activeProvider: activeProfile === null ? null : await this.#toPublic(activeProfile), + generation: this.confirmedGeneration, + worker: publicWorkerState(this.workerManager.getPublicState()) + }; + } + + async #updateWithReplacementSecret(current, patch, replacementSecret) { + const oldSecret = await this.credentialStore.get(current.credentialRef); + let replacementWritten = false; + let replacementCommitWarning = null; + let testReset = false; + try { + try { + await this.credentialStore.set(current.credentialRef, replacementSecret); + replacementWritten = true; + } catch (error) { + if (isCommittedError(error)) { + replacementWritten = true; + replacementCommitWarning = error; + } else { + throw error; + } + } + this.registry.markTest(current.id, { status: "untested" }); + testReset = true; + this.registry.update(current.id, patch); + if (replacementCommitWarning) throw replacementCommitWarning; + return this.registry.get(current.id); + } catch (error) { + if (isCommittedError(error)) throw error; + let rollbackFailure = null; + let credentialRestored = false; + if (replacementWritten) { + try { + await this.credentialStore.set(current.credentialRef, oldSecret); + credentialRestored = true; + } catch (rollbackError) { + rollbackFailure = rollbackError; + } + } + if (testReset && credentialRestored) { + try { + this.registry.markTest(current.id, { + status: current.lastTestStatus, + code: current.lastTestCode + }); + } catch (rollbackError) { + rollbackFailure ??= rollbackError; + } + } + if (rollbackFailure) { + throw serviceError("PROVIDER_UPDATE_ROLLBACK_DEGRADED", { + cause: rollbackFailure, + details: { committed: false, degraded: true } + }); + } + throw error; + } + } + + async #toPublic(profile) { + const configured = await this.credentialStore.has(profile.credentialRef); + return toPublicProvider(profile, configured); + } + + #buildSnapshot(profile, secret, generation) { + const document = this.registry.getDocument(); + const runtimeConfigPath = this.paths.runtimeConfigPath; + const capturePath = this.paths.capturePath; + if (typeof runtimeConfigPath !== "string" || runtimeConfigPath.length === 0 + || typeof capturePath !== "string" || capturePath.length === 0) { + throw serviceError("PROVIDER_ACTIVATION_FAILED"); + } + return { + generation, + settings: { + configPath: runtimeConfigPath, + server: { + host: document.settings.proxyHost, + port: document.settings.proxyPort, + logLevel: "info" + }, + upstream: { + baseUrl: profile.baseUrl, + apiKey: secret, + timeoutMs: 300_000, + verifySsl: true, + authHeader: profile.authHeader, + authScheme: profile.authScheme, + extraHeaders: { ...profile.extraHeaders } + }, + proxy: { + overrideAuthorization: true, + requestIdHeader: "x-client-request-id" + }, + capture: { + enabled: document.settings.captureEnabled, + dbPath: capturePath + } + } + }; + } + + async #record(action, providerId, result, errorCode, details) { + await this.activityStore.append({ + category: "provider", + action, + providerId, + result, + errorCode, + details + }); + } + + async #safeRecordFailure(action, providerId, error, details = {}) { + const errorCode = stableErrorCode(error, `PROVIDER_${action.toUpperCase()}_FAILED`); + try { + await this.#record(action, providerId, "failed", errorCode, details); + } catch { + // Preserve the operation error when the audit store is also unavailable. + } + } + + async #recordCommitted(action, providerId, error, details = {}) { + await this.#record(action, providerId, "degraded", error.code, details); + } + + #runExclusive(operation) { + const run = this.#operationTail.then(operation, operation); + this.#operationTail = run.catch(() => {}); + return run; + } +} + +function isCompatibleResponsesPayload(payload) { + return payload !== null + && typeof payload === "object" + && !Array.isArray(payload) + && typeof payload.id === "string" + && payload.id.length > 0 + && payload.object === "response" + && Array.isArray(payload.output); +} + +function classifyHttpStatus(status) { + if (status >= 300 && status <= 399) return "PROVIDER_TEST_REDIRECT"; + if (status === 401 || status === 403) return "PROVIDER_TEST_AUTH"; + if (status === 404) return "PROVIDER_TEST_NOT_FOUND"; + return "PROVIDER_TEST_HTTP"; +} + +function collectErrorCodes(error) { + const codes = []; + const seen = new Set(); + let current = error; + while (current && typeof current === "object" && !seen.has(current)) { + seen.add(current); + if (typeof current.code === "string") codes.push(current.code.toUpperCase()); + current = current.cause; + } + return codes; +} + +function classifyFetchError(error) { + const codes = collectErrorCodes(error); + if (error?.name === "AbortError" || error?.name === "TimeoutError" + || codes.includes("ETIMEDOUT") || codes.includes("UND_ERR_CONNECT_TIMEOUT")) { + return "PROVIDER_TEST_TIMEOUT"; + } + if (codes.includes("ENOTFOUND") || codes.includes("EAI_AGAIN")) { + return "PROVIDER_TEST_DNS"; + } + if (codes.some((code) => code.startsWith("CERT_") + || code.startsWith("ERR_TLS_") + || code === "DEPTH_ZERO_SELF_SIGNED_CERT" + || code === "UNABLE_TO_VERIFY_LEAF_SIGNATURE")) { + return "PROVIDER_TEST_TLS"; + } + return "PROVIDER_TEST_NETWORK"; +} + +function stableErrorCode(error, fallback) { + return typeof error?.code === "string" && /^[A-Z][A-Z0-9_]*$/.test(error.code) + ? error.code + : fallback; +} + +function isConfirmedWorkerState(state, generation) { + return state !== null + && typeof state === "object" + && state.phase === "running" + && state.generation === generation + && state.state !== null + && typeof state.state === "object" + && state.state.phase === "running" + && state.state.configured === true + && state.state.generation === generation + && state.state.listening === true; +} + +function publicWorkerState(state) { + if (state === null || typeof state !== "object") return null; + const nested = state.state === null || typeof state.state !== "object" + ? null + : { + phase: state.state.phase, + configured: state.state.configured, + generation: state.state.generation, + listening: state.state.listening, + listenHost: state.state.listenHost, + listenPort: state.state.listenPort, + inFlight: state.state.inFlight + }; + const error = state.error === null || typeof state.error !== "object" + ? null + : { code: state.error.code, message: state.error.message }; + return { + phase: state.phase, + pid: state.pid, + generation: state.generation, + state: nested, + restartCount: state.restartCount, + startedAt: state.startedAt, + error + }; +} diff --git a/node/test/activity-store.test.mjs b/node/test/activity-store.test.mjs new file mode 100644 index 0000000..2d3f91d --- /dev/null +++ b/node/test/activity-store.test.mjs @@ -0,0 +1,321 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import * as realFileOperations from "node:fs"; +import { + existsSync, + lstatSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + writeFileSync +} from "node:fs"; +import os from "node:os"; +import { join } from "node:path"; + +import { ActivityStore } from "../src/supervisor/activity-store.mjs"; + +const NOW = "2026-07-13T00:00:00.000Z"; + +function makeStore(t) { + const root = mkdtempSync(join(os.tmpdir(), "crp-activity-")); + const path = join(root, "private", "activity.jsonl"); + t.after(() => rmSync(root, { recursive: true, force: true })); + return { path, store: new ActivityStore({ path, now: () => NOW }) }; +} + +function event(action, details = {}) { + return { + category: "provider", + action, + providerId: "provider-1", + result: "success", + errorCode: null, + details + }; +} + +function makeSecret(label) { + return `${label}-${crypto.randomUUID()}`; +} + +test("persists the exact activity-event allowlist and recursively redacts sensitive fields", (t) => { + const { path, store } = makeStore(t); + const values = { + authorization: makeSecret("authorization"), + cookie: makeSecret("cookie"), + token: makeSecret("token"), + secret: makeSecret("secret"), + apiKey: makeSecret("api-key") + }; + + store.append({ + category: "provider", + action: "test", + providerId: "provider-1", + result: "failed", + errorCode: "PROVIDER_TEST_HTTP_401", + details: { + Authorization: values.authorization, + "session.cookie": values.cookie, + access_token: values.token, + clientSecret: values.secret, + "API-Key": values.apiKey, + safe: { attempt: 1 } + }, + ignored: "must-not-persist" + }); + + const bytes = readFileSync(path, "utf8"); + for (const value of Object.values(values)) { + assert.equal(bytes.includes(value), false); + } + const event = JSON.parse(bytes.trim()); + assert.deepEqual(Object.keys(event), [ + "timestamp", + "category", + "action", + "providerId", + "result", + "errorCode", + "details" + ]); + assert.equal(event.timestamp, NOW); + assert.equal(event.details.Authorization, "[REDACTED]"); + assert.equal(event.details["session.cookie"], "[REDACTED]"); + assert.equal(event.details.access_token, "[REDACTED]"); + assert.equal(event.details.clientSecret, "[REDACTED]"); + assert.equal(event.details["API-Key"], "[REDACTED]"); + assert.deepEqual(event.details.safe, { attempt: 1 }); + assert.equal("ignored" in event, false); +}); + +test("serializes nested Error, cyclic, bigint, and unsupported detail values safely", (t) => { + const { path, store } = makeStore(t); + const errorSecret = makeSecret("error-message"); + const nested = { count: 2n, missing: undefined }; + nested.self = nested; + const error = new Error(errorSecret, { + cause: new Error(makeSecret("cause-message")) + }); + error.accessToken = makeSecret("error-token"); + nested.error = error; + nested.callback = () => errorSecret; + + assert.doesNotThrow(() => store.append({ + category: "provider", + action: "test", + providerId: "provider-1", + result: "failed", + errorCode: "PROVIDER_TEST_FAILED", + details: nested + })); + + const bytes = readFileSync(path, "utf8"); + assert.equal(bytes.includes(errorSecret), false); + assert.equal(bytes.includes(error.accessToken), false); + assert.equal(bytes.includes(error.cause.message), false); + const event = JSON.parse(bytes.trim()); + assert.equal(event.details.count, "2"); + assert.equal(event.details.self, "[CIRCULAR]"); + assert.deepEqual(event.details.error, { + name: "Error", + accessToken: "[REDACTED]" + }); + assert.equal(event.details.missing, "[UNSERIALIZABLE]"); + assert.equal(event.details.callback, "[UNSERIALIZABLE]"); +}); + +test("redacts orchestration metadata, bodies, headers, paths, causes, and stacks", (t) => { + const { path, store } = makeStore(t); + const values = Object.fromEntries([ + "credential-ref", + "request-body", + "response-body", + "cause", + "stack", + "headers", + "backup-path" + ].map((label) => [label, makeSecret(label)])); + + store.append(event("security-boundary", { + "Credential.Ref": values["credential-ref"], + REQUEST_body: values["request-body"], + "response-body": values["response-body"], + CaUsE: values.cause, + "error.stack": values.stack, + requestHeaders: { safe: values.headers }, + "backup-path": values["backup-path"] + })); + + const bytes = readFileSync(path, "utf8"); + for (const value of Object.values(values)) assert.equal(bytes.includes(value), false); + const details = JSON.parse(bytes).details; + for (const value of Object.values(details)) assert.equal(value, "[REDACTED]"); +}); + +test("retains only events within 30 days and the newest configured row limit", (t) => { + const root = mkdtempSync(join(os.tmpdir(), "crp-activity-retention-")); + const path = join(root, "private", "activity.jsonl"); + t.after(() => rmSync(root, { recursive: true, force: true })); + const store = new ActivityStore({ + path, + now: () => "2026-07-13T00:00:00.000Z", + maxEvents: 3, + retentionMs: 30 * 24 * 60 * 60 * 1_000 + }); + + for (const [timestamp, sequence] of [ + ["2026-06-01T00:00:00.000Z", 0], + ["2026-07-09T00:00:00.000Z", 1], + ["2026-07-10T00:00:00.000Z", 2], + ["2026-07-11T00:00:00.000Z", 3], + ["2026-07-12T00:00:00.000Z", 4] + ]) { + store.append({ ...event("retention", { sequence }), timestamp }); + } + + assert.deepEqual( + store.list().map((entry) => entry.details.sequence), + [4, 3, 2] + ); + assert.deepEqual( + readFileSync(path, "utf8").trim().split("\n") + .map((line) => JSON.parse(line).details.sequence), + [2, 3, 4] + ); +}); + +test("uses a private atomic replacement and preserves old bytes on rename failure", (t) => { + const { path, store } = makeStore(t); + store.append(event("created")); + const original = readFileSync(path); + const failingStore = new ActivityStore({ + path, + now: () => NOW, + fileOperations: { + ...realFileOperations, + renameSync(from, to) { + if (to === path && from.endsWith(".tmp")) { + const error = new Error("private rename detail"); + error.code = "EIO"; + throw error; + } + return realFileOperations.renameSync(from, to); + } + } + }); + + assert.throws( + () => failingStore.append(event("failed-write")), + (error) => error?.code === "ACTIVITY_STORE_WRITE_FAILED" + && !error.message.includes("private rename detail") + ); + assert.deepEqual(readFileSync(path), original); + assert.equal(existsSync(`${path}.crp.lock`), false); + assert.deepEqual( + readdirSync(join(path, "..")).filter((name) => name.endsWith(".tmp")), + [] + ); + + if (process.platform !== "win32") { + assert.equal(lstatSync(join(path, "..")).mode & 0o777, 0o700); + assert.equal(lstatSync(path).mode & 0o777, 0o600); + } +}); + +test("rejects a foreign activity lock without changing or deleting it", (t) => { + const { path, store } = makeStore(t); + store.append(event("created")); + const original = readFileSync(path); + const lockPath = `${path}.crp.lock`; + const foreign = Buffer.from("foreign-activity-owner\n", "utf8"); + writeFileSync(lockPath, foreign, { mode: 0o600 }); + + assert.throws( + () => store.append(event("blocked")), + (error) => error?.code === "ACTIVITY_STORE_BUSY" + ); + assert.deepEqual(readFileSync(path), original); + assert.deepEqual(readFileSync(lockPath), foreign); +}); + +test("restores a canonical blocker and stops later mutations after committed lock degradation", (t) => { + const root = mkdtempSync(join(os.tmpdir(), "crp-activity-degraded-")); + const path = join(root, "private", "activity.jsonl"); + const lockPath = `${path}.crp.lock`; + t.after(() => rmSync(root, { recursive: true, force: true })); + const store = new ActivityStore({ + path, + now: () => NOW, + fileOperations: { + ...realFileOperations, + rmSync(target, options) { + if (typeof target === "string" && target.endsWith(".release")) { + const error = new Error("private claim cleanup failure"); + error.code = "EACCES"; + throw error; + } + return realFileOperations.rmSync(target, options); + } + } + }); + + assert.throws( + () => store.append(event("committed")), + (error) => error?.code === "ACTIVITY_STORE_COMMITTED_LOCK_DEGRADED" + && error.details.committed === true + ); + assert.equal(readFileSync(path, "utf8").trim().split("\n").length, 1); + assert.equal(existsSync(lockPath), true); + assert.deepEqual( + readdirSync(join(path, "..")).filter((name) => name.endsWith(".release")), + [] + ); + assert.throws( + () => store.append(event("must-not-write")), + (error) => error?.code === "ACTIVITY_STORE_LOCK_DEGRADED" + ); + assert.equal(readFileSync(path, "utf8").includes("must-not-write"), false); +}); + +test("never deletes a foreign canonical replacement after lock initialization fails", (t) => { + const root = mkdtempSync(join(os.tmpdir(), "crp-activity-acquire-swap-")); + const path = join(root, "private", "activity.jsonl"); + const lockPath = `${path}.crp.lock`; + const displacedPath = `${lockPath}.displaced`; + const foreign = Buffer.from("foreign-replacement\n", "utf8"); + let swapped = false; + t.after(() => rmSync(root, { recursive: true, force: true })); + const store = new ActivityStore({ + path, + now: () => NOW, + fileOperations: { + ...realFileOperations, + writeFileSync(target, bytes, options) { + if (!swapped && typeof target === "number") { + swapped = true; + realFileOperations.renameSync(lockPath, displacedPath); + realFileOperations.writeFileSync(lockPath, foreign, { mode: 0o600 }); + const error = new Error("private lock initialization failure"); + error.code = "EIO"; + throw error; + } + return realFileOperations.writeFileSync(target, bytes, options); + } + } + }); + + assert.throws( + () => store.append(event("must-not-commit")), + (error) => error?.code === "ACTIVITY_STORE_LOCK_DEGRADED" + && error.details.committed === false + ); + assert.deepEqual(readFileSync(lockPath), foreign); + assert.equal(existsSync(path), false); + assert.throws( + () => store.append(event("must-stay-blocked")), + (error) => error?.code === "ACTIVITY_STORE_LOCK_DEGRADED" + ); + assert.deepEqual(readFileSync(lockPath), foreign); +}); diff --git a/node/test/migration.test.mjs b/node/test/migration.test.mjs new file mode 100644 index 0000000..e2c6fda --- /dev/null +++ b/node/test/migration.test.mjs @@ -0,0 +1,631 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import * as realFileOperations from "node:fs"; +import { + chmodSync, + existsSync, + lstatSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + renameSync, + rmSync, + symlinkSync, + writeFileSync +} from "node:fs"; +import os from "node:os"; +import { join } from "node:path"; + +import { migrateLegacyConfiguration } from "../src/supervisor/migration.mjs"; + +const NOW = "2026-07-13T01:00:00.000Z"; + +function makeSecret() { + return `migration-test-${crypto.randomUUID()}`; +} + +function makeHarness(t, legacyDocument) { + const root = mkdtempSync(join(os.tmpdir(), "crp-migration-")); + const globalHome = join(root, ".codex-remote-proxy"); + const legacyConfigPath = join(globalHome, "config.json"); + const runtimeConfigPath = join(globalHome, "node", "proxy-config.json"); + const registryPath = join(globalHome, "providers.json"); + const legacyBytes = Buffer.from(`${JSON.stringify(legacyDocument, null, 2)}\n`, "utf8"); + mkdirSync(globalHome, { recursive: true, mode: 0o700 }); + writeFileSync(legacyConfigPath, legacyBytes, { mode: 0o600 }); + chmodSync(legacyConfigPath, 0o600); + t.after(() => rmSync(root, { recursive: true, force: true })); + return { + root, + globalHome, + legacyConfigPath, + runtimeConfigPath, + registryPath, + legacyBytes, + paths: { globalHome, legacyConfigPath, runtimeConfigPath, registryPath } + }; +} + +function writeRuntime(harness, document) { + mkdirSync(join(harness.globalHome, "node"), { recursive: true, mode: 0o700 }); + const bytes = Buffer.from(`${JSON.stringify(document, null, 2)}\n`, "utf8"); + writeFileSync(harness.runtimeConfigPath, bytes, { mode: 0o600 }); + chmodSync(harness.runtimeConfigPath, 0o600); + return bytes; +} + +class MemoryCredentialStore { + constructor({ failSet = false, failDelete = false } = {}) { + this.values = new Map(); + this.failSet = failSet; + this.failDelete = failDelete; + this.deleted = []; + } + + async set(ref, secret) { + if (this.failSet) throw new Error(`private credential failure ${secret}`); + this.values.set(ref, secret); + } + + async delete(ref) { + this.deleted.push(ref); + if (this.failDelete) throw new Error("private delete failure"); + return this.values.delete(ref); + } +} + +test("transactionally migrates legacy config to one untested inactive Default provider", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret, + captureEnabled: false + }); + const credentials = new MemoryCredentialStore(); + + const result = await migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque" + }); + + assert.deepEqual(result, { migrated: true, providerId: "provider-default" }); + assert.equal(credentials.values.get("credential-opaque"), secret); + const registry = JSON.parse(readFileSync(harness.registryPath, "utf8")); + assert.equal(registry.schemaVersion, 2); + assert.equal(registry.activeProviderId, null); + assert.equal(registry.providers.length, 1); + assert.equal(registry.providers[0].name, "Default"); + assert.equal(registry.providers[0].credentialRef, "credential-opaque"); + assert.equal(registry.providers[0].lastTestStatus, "untested"); + assert.equal(registry.providers[0].lastTestAt, null); + assert.equal(registry.providers[0].lastTestCode, null); + assert.equal(readFileSync(harness.registryPath, "utf8").includes(secret), false); + + const scrubbed = JSON.parse(readFileSync(harness.legacyConfigPath, "utf8")); + assert.equal("apiKey" in scrubbed, false); + assert.equal(scrubbed.upstreamBaseUrl, "https://legacy.example/v1"); + const backups = readdirSync(harness.globalHome) + .filter((name) => name.startsWith("config.json.") && name.endsWith(".bak")); + assert.equal(backups.length, 1); + assert.deepEqual(readFileSync(join(harness.globalHome, backups[0])), harness.legacyBytes); + if (process.platform !== "win32") { + assert.equal(lstatSync(join(harness.globalHome, backups[0])).mode & 0o777, 0o600); + } + const serializedResult = JSON.stringify(result); + assert.equal(serializedResult.includes(secret), false); + assert.equal(serializedResult.includes(backups[0]), false); +}); + +test("restores original bytes and removes new state when credential persistence fails", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const credentials = new MemoryCredentialStore({ failSet: true }); + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque" + }), + (error) => error?.code === "MIGRATION_FAILED" + && !error.message.includes(secret) + && !JSON.stringify(error.details).includes(secret) + ); + + assert.deepEqual(readFileSync(harness.legacyConfigPath), harness.legacyBytes); + assert.equal(existsSync(harness.registryPath), false); + assert.deepEqual(credentials.deleted, ["credential-opaque"]); + const backups = readdirSync(harness.globalHome) + .filter((name) => name.startsWith("config.json.") && name.endsWith(".bak")); + assert.equal(backups.length, 1); + assert.deepEqual(readFileSync(join(harness.globalHome, backups[0])), harness.legacyBytes); +}); + +test("migrates the runtime flat config and scrubs every backed-up secret source", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { captureEnabled: true }); + const runtimeBytes = writeRuntime(harness, { + baseUrl: "https://runtime.example/v1", + apiKey: secret, + authHeader: "x-runtime-auth", + authScheme: "Token", + extraHeaders: { "x-region": "test" } + }); + const credentials = new MemoryCredentialStore(); + + await migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-runtime", + createCredentialRef: () => "credential-runtime" + }); + + const provider = JSON.parse(readFileSync(harness.registryPath, "utf8")).providers[0]; + assert.equal(provider.baseUrl, "https://runtime.example/v1"); + assert.equal(provider.authHeader, "x-runtime-auth"); + assert.equal(provider.authScheme, "Token"); + assert.deepEqual(provider.extraHeaders, { "x-region": "test" }); + assert.equal(provider.lastTestStatus, "untested"); + assert.equal(JSON.stringify(JSON.parse(readFileSync(harness.runtimeConfigPath))).includes(secret), false); + assert.equal(JSON.stringify(JSON.parse(readFileSync(harness.legacyConfigPath))).includes(secret), false); + const runtimeBackups = readdirSync(join(harness.globalHome, "node")) + .filter((name) => name.startsWith("proxy-config.json.") && name.endsWith(".bak")); + assert.equal(runtimeBackups.length, 1); + assert.deepEqual( + readFileSync(join(harness.globalHome, "node", runtimeBackups[0])), + runtimeBytes + ); +}); + +test("does not overwrite an exclusive backup collision and is idempotent after schema 2", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const collisionPath = `${harness.legacyConfigPath}.collision.bak`; + const collisionBytes = Buffer.from("foreign-backup\n", "utf8"); + writeFileSync(collisionPath, collisionBytes, { mode: 0o600 }); + const backupIds = ["collision", "unique"]; + const credentials = new MemoryCredentialStore(); + + const first = await migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque", + createBackupId: () => backupIds.shift() ?? "unexpected" + }); + assert.deepEqual(first, { migrated: true, providerId: "provider-default" }); + assert.deepEqual(readFileSync(collisionPath), collisionBytes); + const beforeRegistry = readFileSync(harness.registryPath); + const beforeBackups = readdirSync(harness.globalHome).filter((name) => name.endsWith(".bak")); + + const second = await migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: { + async set() { throw new Error("must not write a second credential"); }, + async delete() { throw new Error("must not delete the existing credential"); } + }, + createProviderId: () => { throw new Error("must not create a second provider"); }, + createCredentialRef: () => { throw new Error("must not create a second reference"); } + }); + assert.deepEqual(second, { migrated: false, reason: "already-current" }); + assert.deepEqual(readFileSync(harness.registryPath), beforeRegistry); + assert.deepEqual( + readdirSync(harness.globalHome).filter((name) => name.endsWith(".bak")), + beforeBackups + ); +}); + +test("rolls back scrubbed files, registry, and credential when a later scrub fails", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const runtimeBytes = writeRuntime(harness, { + upstream: { baseUrl: "https://legacy.example/v1", apiKey: secret } + }); + const credentials = new MemoryCredentialStore(); + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque", + fileOperations: { + ...realFileOperations, + renameSync(from, to) { + if (to === harness.runtimeConfigPath && from.endsWith(".tmp")) { + const error = new Error("private runtime scrub failure"); + error.code = "EIO"; + throw error; + } + return realFileOperations.renameSync(from, to); + } + } + }), + (error) => error?.code === "MIGRATION_FAILED" && error.details.committed === false + ); + + assert.deepEqual(readFileSync(harness.legacyConfigPath), harness.legacyBytes); + assert.deepEqual(readFileSync(harness.runtimeConfigPath), runtimeBytes); + assert.equal(existsSync(harness.registryPath), false); + assert.equal(credentials.values.size, 0); + assert.deepEqual(credentials.deleted, ["credential-opaque"]); +}); + +test("reports stable degraded rollback state and retains backups when compensation fails", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + writeRuntime(harness, { + upstream: { baseUrl: "https://legacy.example/v1", apiKey: secret } + }); + const credentials = new MemoryCredentialStore({ failDelete: true }); + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque", + fileOperations: { + ...realFileOperations, + renameSync(from, to) { + if (to === harness.runtimeConfigPath && from.endsWith(".tmp")) { + const error = new Error("private runtime scrub failure"); + error.code = "EIO"; + throw error; + } + return realFileOperations.renameSync(from, to); + } + } + }), + (error) => error?.code === "MIGRATION_ROLLBACK_DEGRADED" + && error.details.committed === false + && error.details.degraded === true + && !error.message.includes(secret) + ); + + assert.ok(readdirSync(harness.globalHome).some((name) => name.endsWith(".bak"))); + assert.ok(readdirSync(join(harness.globalHome, "node")).some((name) => name.endsWith(".bak"))); +}); + +test("serializes migration transactions with an exclusive preserved lock", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + let releaseSet; + const credentials = new MemoryCredentialStore(); + credentials.set = async (ref, value) => { + await new Promise((resolve) => { releaseSet = resolve; }); + credentials.values.set(ref, value); + }; + + const first = migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque" + }); + while (!releaseSet) await Promise.resolve(); + const lockPath = `${harness.registryPath}.migration.lock`; + const lockBytes = readFileSync(lockPath); + await assert.rejects( + () => migrateLegacyConfiguration({ paths: harness.paths, credentialStore: credentials }), + (error) => error?.code === "MIGRATION_BUSY" + ); + assert.deepEqual(readFileSync(lockPath), lockBytes); + releaseSet(); + await first; + assert.equal(existsSync(lockPath), false); +}); + +test("reports rollback degraded when a failed transaction lock cannot be released", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const credentials = new MemoryCredentialStore({ failSet: true }); + const lockPath = `${harness.registryPath}.migration.lock`; + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + fileOperations: { + ...realFileOperations, + renameSync(from, to) { + if (from === lockPath && to.endsWith(".release")) { + const error = new Error("private lock release failure"); + error.code = "EIO"; + throw error; + } + return realFileOperations.renameSync(from, to); + } + } + }), + (error) => error?.code === "MIGRATION_ROLLBACK_DEGRADED" + && error.details.committed === false + && error.details.degraded === true + && !error.message.includes(secret) + ); + assert.deepEqual(readFileSync(harness.legacyConfigPath), harness.legacyBytes); + assert.equal(existsSync(harness.registryPath), false); + assert.equal(existsSync(lockPath), true); +}); + +test("rejects symlink legacy and registry paths without reading or replacing their targets", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const legacyTarget = join(harness.root, "legacy-target.json"); + const legacyTargetBytes = readFileSync(harness.legacyConfigPath); + writeFileSync(legacyTarget, legacyTargetBytes, { mode: 0o600 }); + rmSync(harness.legacyConfigPath); + symlinkSync(legacyTarget, harness.legacyConfigPath); + const credentials = new MemoryCredentialStore(); + + await assert.rejects( + () => migrateLegacyConfiguration({ paths: harness.paths, credentialStore: credentials }), + (error) => error?.code === "MIGRATION_INPUT_INVALID" + ); + assert.equal(lstatSync(harness.legacyConfigPath).isSymbolicLink(), true); + assert.deepEqual(readFileSync(legacyTarget), legacyTargetBytes); + assert.deepEqual(credentials.values.size, 0); + + rmSync(harness.legacyConfigPath); + writeFileSync(harness.legacyConfigPath, legacyTargetBytes, { mode: 0o600 }); + const registryTarget = join(harness.root, "registry-target.json"); + const registryTargetBytes = Buffer.from(JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [], + settings: { + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: 15101, + captureEnabled: false + } + }), "utf8"); + writeFileSync(registryTarget, registryTargetBytes, { mode: 0o600 }); + symlinkSync(registryTarget, harness.registryPath); + await assert.rejects( + () => migrateLegacyConfiguration({ paths: harness.paths, credentialStore: credentials }), + (error) => error?.code === "MIGRATION_INPUT_INVALID" + ); + assert.equal(lstatSync(harness.registryPath).isSymbolicLink(), true); + assert.deepEqual(readFileSync(registryTarget), registryTargetBytes); +}); + +test("rollback preserves a foreign registry replacement it does not own", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const foreignRegistry = Buffer.from("foreign-registry-bytes\n", "utf8"); + const displacedRegistry = `${harness.registryPath}.owned-displaced`; + const credentials = new MemoryCredentialStore(); + const activityStore = { + append() { + renameSync(harness.registryPath, displacedRegistry); + writeFileSync(harness.registryPath, foreignRegistry, { mode: 0o600 }); + throw new Error("private activity failure"); + } + }; + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + activityStore, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque" + }), + (error) => error?.code === "MIGRATION_ROLLBACK_DEGRADED" + && error.details.degraded === true + ); + assert.deepEqual(readFileSync(harness.registryPath), foreignRegistry); + assert.deepEqual(readFileSync(harness.legacyConfigPath), harness.legacyBytes); + assert.equal(credentials.values.size, 0); +}); + +test("lock initialization failure preserves a foreign canonical replacement and blocks retry", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const lockPath = `${harness.registryPath}.migration.lock`; + const displacedLock = `${lockPath}.owned-displaced`; + const foreign = Buffer.from("foreign-migration-owner\n", "utf8"); + let swapped = false; + const credentials = new MemoryCredentialStore(); + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + fileOperations: { + ...realFileOperations, + writeFileSync(target, bytes, options) { + if (!swapped && typeof target === "number") { + swapped = true; + realFileOperations.renameSync(lockPath, displacedLock); + realFileOperations.writeFileSync(lockPath, foreign, { mode: 0o600 }); + const error = new Error("private lock init failure"); + error.code = "EIO"; + throw error; + } + return realFileOperations.writeFileSync(target, bytes, options); + } + } + }), + (error) => error?.code === "MIGRATION_ROLLBACK_DEGRADED" + && error.details.committed === false + ); + assert.deepEqual(readFileSync(lockPath), foreign); + await assert.rejects( + () => migrateLegacyConfiguration({ paths: harness.paths, credentialStore: credentials }), + (error) => error?.code === "MIGRATION_BUSY" + ); + assert.deepEqual(readFileSync(lockPath), foreign); +}); + +test("release token mismatch restores a canonical blocker before the next transaction", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const lockPath = `${harness.registryPath}.migration.lock`; + const credentials = new MemoryCredentialStore(); + const releaseDescriptors = new Set(); + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque", + fileOperations: { + ...realFileOperations, + openSync(path, flags, mode) { + const descriptor = realFileOperations.openSync(path, flags, mode); + if (typeof path === "string" && path.endsWith(".release")) { + releaseDescriptors.add(descriptor); + } + return descriptor; + }, + readFileSync(target, options) { + if (releaseDescriptors.has(target)) { + return "foreign-claim-token\n"; + } + return realFileOperations.readFileSync(target, options); + } + } + }), + (error) => error?.code === "MIGRATION_COMMITTED_LOCK_DEGRADED" + && error.details.committed === true + ); + assert.equal(existsSync(lockPath), true); + await assert.rejects( + () => migrateLegacyConfiguration({ paths: harness.paths, credentialStore: credentials }), + (error) => error?.code === "MIGRATION_BUSY" + ); +}); + +test("keeps committed schema 2 and credential when registry lock cleanup degrades", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const credentials = new MemoryCredentialStore(); + const registryLockPath = `${harness.registryPath}.crp.lock`; + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque", + fileOperations: { + ...realFileOperations, + rmSync(path, options) { + if (path === registryLockPath) { + const error = new Error("private registry lock cleanup failure"); + error.code = "EACCES"; + throw error; + } + return realFileOperations.rmSync(path, options); + } + } + }), + (error) => error?.code === "MIGRATION_COMMITTED_DEGRADED" + && error.details.committed === true + ); + + const registry = JSON.parse(readFileSync(harness.registryPath, "utf8")); + assert.equal(registry.providers[0].id, "provider-default"); + assert.equal(credentials.values.get("credential-opaque"), secret); + assert.equal("apiKey" in JSON.parse(readFileSync(harness.legacyConfigPath, "utf8")), false); + assert.equal(existsSync(registryLockPath), true); +}); + +test("exclusive registry creation preserves a foreign schema 2 created during credential set", async (t) => { + const secret = makeSecret(); + const harness = makeHarness(t, { + upstreamBaseUrl: "https://legacy.example/v1", + apiKey: secret + }); + const foreignBytes = Buffer.from(`${JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [], + settings: { + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: 15101, + captureEnabled: false + } + }, null, 2)}\n`, "utf8"); + const credentials = new MemoryCredentialStore(); + const originalSet = credentials.set.bind(credentials); + let foreignIdentity; + credentials.set = async (ref, value) => { + await originalSet(ref, value); + writeFileSync(harness.registryPath, foreignBytes, { flag: "wx", mode: 0o600 }); + const stats = lstatSync(harness.registryPath); + foreignIdentity = { dev: stats.dev, ino: stats.ino }; + }; + + await assert.rejects( + () => migrateLegacyConfiguration({ + paths: harness.paths, + credentialStore: credentials, + now: () => NOW, + createProviderId: () => "provider-default", + createCredentialRef: () => "credential-opaque" + }), + (error) => error?.code === "MIGRATION_REGISTRY_CONFLICT" + && error.status === 409 + ); + + const after = lstatSync(harness.registryPath); + assert.deepEqual({ dev: after.dev, ino: after.ino }, foreignIdentity); + assert.deepEqual(readFileSync(harness.registryPath), foreignBytes); + assert.deepEqual(readFileSync(harness.legacyConfigPath), harness.legacyBytes); + assert.equal(credentials.values.size, 0); + assert.deepEqual(credentials.deleted, ["credential-opaque"]); +}); diff --git a/node/test/provider-service.test.mjs b/node/test/provider-service.test.mjs new file mode 100644 index 0000000..dc7cd4a --- /dev/null +++ b/node/test/provider-service.test.mjs @@ -0,0 +1,857 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync } from "node:fs"; +import { createServer } from "node:http"; +import os from "node:os"; +import { join } from "node:path"; + +import { ProviderRegistry } from "../src/providers/provider-registry.mjs"; +import { CrpError } from "../src/shared/errors.mjs"; +import { ProviderService } from "../src/supervisor/provider-service.mjs"; + +const NOW = "2026-07-13T02:00:00.000Z"; + +function makeSecret(label = "provider") { + return `${label}-${crypto.randomUUID()}`; +} + +function providerInput(name = "Primary", baseUrl = "https://provider.example/v1") { + return { name, baseUrl }; +} + +class MemoryCredentials { + constructor() { + this.values = new Map(); + this.operations = []; + } + + async set(ref, secret) { + this.operations.push(["set", ref]); + this.values.set(ref, secret); + } + + async get(ref) { + this.operations.push(["get", ref]); + if (!this.values.has(ref)) { + const error = new Error("missing"); + error.code = "CREDENTIAL_NOT_FOUND"; + throw error; + } + return this.values.get(ref); + } + + async has(ref) { + this.operations.push(["has", ref]); + return this.values.has(ref); + } + + async delete(ref) { + this.operations.push(["delete", ref]); + return this.values.delete(ref); + } +} + +class MemoryActivity { + constructor() { + this.events = []; + } + + append(event) { + this.events.push(structuredClone(event)); + } +} + +class FakeWorkerManager { + constructor() { + this.phase = "stopped"; + this.generation = 0; + this.calls = []; + this.failure = null; + } + + getPublicState() { + return { + phase: this.phase, + pid: this.phase === "running" ? 9001 : null, + generation: this.generation, + state: this.phase === "running" ? { + phase: "running", + configured: true, + generation: this.generation, + listening: true, + listenHost: "127.0.0.1", + listenPort: 15100, + inFlight: 0 + } : null, + restartCount: 0, + startedAt: this.phase === "running" ? NOW : null, + error: null + }; + } + + async start(snapshot) { + this.calls.push(["start", structuredClone(snapshot)]); + if (this.failure) throw this.failure; + this.phase = "running"; + this.generation = snapshot.generation; + return this.getPublicState(); + } + + async applySnapshot(snapshot) { + this.calls.push(["applySnapshot", structuredClone(snapshot)]); + if (this.failure) throw this.failure; + this.generation = snapshot.generation; + return this.getPublicState(); + } + + async restart(snapshot) { + this.calls.push(["restart", structuredClone(snapshot)]); + if (this.failure) throw this.failure; + this.phase = "running"; + this.generation = snapshot.generation; + return this.getPublicState(); + } + + async stop() { + this.calls.push(["stop"]); + this.phase = "stopped"; + return this.getPublicState(); + } +} + +function makeHarness(t, overrides = {}) { + const root = mkdtempSync(join(os.tmpdir(), "crp-provider-service-")); + const registry = new ProviderRegistry({ + path: join(root, "providers.json"), + createId: (() => { + let index = 0; + return () => `provider-${++index}`; + })(), + now: () => NOW + }); + const credentials = new MemoryCredentials(); + const activity = new MemoryActivity(); + const workerManager = new FakeWorkerManager(); + let credentialIndex = 0; + const service = new ProviderService({ + registry, + credentialStore: credentials, + activityStore: activity, + workerManager, + now: () => NOW, + createCredentialRef: () => `credential-${++credentialIndex}`, + createTimeoutSignal: () => ({ aborted: false }), + paths: { + runtimeConfigPath: join(root, "node", "proxy-config.json"), + capturePath: join(root, "traffic.sqlite3") + }, + ...overrides + }); + t.after(() => rmSync(root, { recursive: true, force: true })); + return { root, registry, credentials, activity, workerManager, service }; +} + +function compatibleResponse() { + return { + ok: true, + status: 200, + async json() { + return { id: "resp-test", object: "response", output: [] }; + } + }; +} + +function committedError(code) { + return new CrpError( + code, + "Committed operation degraded.", + "Repair the residual state.", + { details: { committed: true } } + ); +} + +async function listen(server) { + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", resolve); + }); + return server.address().port; +} + +async function closeServer(server) { + await new Promise((resolve) => server.close(resolve)); +} + +test("CRUD returns only public provider fields and removes inactive credentials", async (t) => { + const secret = makeSecret(); + const { service, credentials, activity } = makeHarness(t); + + const created = await service.createProvider(providerInput(), secret, { + fallbackConsent: false + }); + assert.equal(created.id, "provider-1"); + assert.equal(created.credentialConfigured, true); + assert.equal("credentialRef" in created, false); + assert.equal(JSON.stringify(created).includes(secret), false); + assert.deepEqual(await service.listProviders(), [created]); + + const updated = await service.updateProvider(created.id, { name: "Primary Updated" }); + assert.equal(updated.name, "Primary Updated"); + assert.equal("credentialRef" in updated, false); + const deleted = await service.deleteProvider(created.id); + assert.equal(deleted.id, created.id); + assert.equal("credentialRef" in deleted, false); + assert.equal(credentials.values.size, 0); + assert.deepEqual(activity.events.map((event) => event.action), ["create", "update", "delete"]); + assert.equal(JSON.stringify(activity.events).includes(secret), false); +}); + +test("CRUD compensates credential changes and rejects active delete before credential access", async (t) => { + const oldSecret = makeSecret("old"); + const otherSecret = makeSecret("other"); + const replacementSecret = makeSecret("replacement"); + const { service, registry, credentials, activity } = makeHarness(t); + const primary = await service.createProvider(providerInput("Primary"), oldSecret); + const other = await service.createProvider( + providerInput("Other", "https://other.example/v1"), + otherSecret + ); + + await assert.rejects( + () => service.createProvider( + providerInput("primary", "https://duplicate.example/v1"), + makeSecret("duplicate") + ), + (error) => error?.code === "PROVIDER_NAME_CONFLICT" + ); + assert.equal(credentials.values.has("credential-3"), false); + + await assert.rejects( + () => service.updateProvider(primary.id, { name: "OTHER" }, replacementSecret), + (error) => error?.code === "PROVIDER_NAME_CONFLICT" + ); + assert.equal(credentials.values.get("credential-1"), oldSecret); + assert.equal(registry.get(primary.id).name, "Primary"); + + const originalDelete = registry.delete.bind(registry); + registry.delete = () => { + throw new CrpError( + "PROVIDER_REGISTRY_WRITE_FAILED", + "Registry write failed.", + "Retry the operation." + ); + }; + await assert.rejects( + () => service.deleteProvider(other.id), + (error) => error?.code === "PROVIDER_REGISTRY_WRITE_FAILED" + ); + assert.equal(credentials.values.get("credential-2"), otherSecret); + registry.delete = originalDelete; + + registry.setActive(primary.id); + credentials.operations.length = 0; + await assert.rejects( + () => service.deleteProvider(primary.id), + (error) => error?.code === "PROVIDER_ACTIVE" + ); + assert.deepEqual(credentials.operations, []); + assert.deepEqual( + activity.events.filter((entry) => entry.result === "failed") + .map((entry) => [entry.action, entry.errorCode]), + [ + ["create", "PROVIDER_NAME_CONFLICT"], + ["update", "PROVIDER_NAME_CONFLICT"], + ["delete", "PROVIDER_REGISTRY_WRITE_FAILED"], + ["delete", "PROVIDER_ACTIVE"] + ] + ); + assert.equal(JSON.stringify(activity.events).includes(oldSecret), false); + assert.equal(JSON.stringify(activity.events).includes(replacementSecret), false); +}); + +test("rejects active provider updates before credential or registry mutation", async (t) => { + const { service, registry, credentials, activity } = makeHarness(t); + const provider = await service.createProvider(providerInput(), makeSecret()); + registry.setActive(provider.id); + credentials.operations.length = 0; + let updateCalls = 0; + let markTestCalls = 0; + const originalUpdate = registry.update.bind(registry); + const originalMarkTest = registry.markTest.bind(registry); + registry.update = (...args) => { + updateCalls += 1; + return originalUpdate(...args); + }; + registry.markTest = (...args) => { + markTestCalls += 1; + return originalMarkTest(...args); + }; + + await assert.rejects( + () => service.updateProvider(provider.id, { name: "Must Not Change" }, makeSecret()), + (error) => error?.code === "PROVIDER_ACTIVE" && error.status === 409 + ); + assert.deepEqual(credentials.operations, []); + assert.equal(updateCalls, 0); + assert.equal(markTestCalls, 0); + assert.equal(registry.get(provider.id).name, "Primary"); + assert.equal(activity.events.at(-1).action, "update"); + assert.equal(activity.events.at(-1).errorCode, "PROVIDER_ACTIVE"); +}); + +test("keeps a created provider and credential when registry create reports committed", async (t) => { + const { service, registry, credentials, activity } = makeHarness(t); + const originalCreate = registry.create.bind(registry); + registry.create = (input) => { + originalCreate(input); + throw committedError("PROVIDER_REGISTRY_COMMITTED_LOCK_DEGRADED"); + }; + + await assert.rejects( + () => service.createProvider(providerInput(), makeSecret()), + (error) => error?.code === "PROVIDER_CREATE_COMMITTED_DEGRADED" + && error.details.committed === true + ); + assert.equal(registry.list().length, 1); + assert.equal(credentials.values.has("credential-1"), true); + assert.equal(activity.events.at(-1).result, "degraded"); + assert.equal(activity.events.at(-1).errorCode, "PROVIDER_CREATE_COMMITTED_DEGRADED"); +}); + +test("keeps replacement secret and metadata when registry update reports committed", async (t) => { + const { service, registry, credentials, activity } = makeHarness(t); + const provider = await service.createProvider(providerInput(), makeSecret("old")); + registry.markTest(provider.id, { status: "passed" }); + const replacement = makeSecret("replacement"); + const originalUpdate = registry.update.bind(registry); + registry.update = (id, patch) => { + originalUpdate(id, patch); + throw committedError("PROVIDER_REGISTRY_COMMITTED_LOCK_DEGRADED"); + }; + + await assert.rejects( + () => service.updateProvider(provider.id, { name: "Committed Name" }, replacement), + (error) => error?.code === "PROVIDER_UPDATE_COMMITTED_DEGRADED" + && error.details.committed === true + ); + assert.equal(registry.get(provider.id).name, "Committed Name"); + assert.equal(registry.get(provider.id).lastTestStatus, "untested"); + assert.equal(credentials.values.get("credential-1"), replacement); + assert.equal(activity.events.at(-1).result, "degraded"); +}); + +test("does not restore a credential when registry delete reports committed", async (t) => { + const { service, registry, credentials, activity } = makeHarness(t); + const provider = await service.createProvider(providerInput(), makeSecret()); + const originalDelete = registry.delete.bind(registry); + registry.delete = (id) => { + originalDelete(id); + throw committedError("PROVIDER_REGISTRY_COMMITTED_LOCK_DEGRADED"); + }; + + await assert.rejects( + () => service.deleteProvider(provider.id), + (error) => error?.code === "PROVIDER_DELETE_COMMITTED_DEGRADED" + && error.details.committed === true + ); + assert.deepEqual(registry.list(), []); + assert.equal(credentials.values.has("credential-1"), false); + assert.equal(activity.events.at(-1).result, "degraded"); +}); + +test("reconciles committed credential create, replacement, and delete mutations", async (t) => { + await t.test("credential create", async (t) => { + const { service, registry, credentials } = makeHarness(t); + const originalSet = credentials.set.bind(credentials); + let injected = false; + credentials.set = async (ref, secret, options) => { + await originalSet(ref, secret, options); + if (!injected) { + injected = true; + throw committedError("CREDENTIAL_STORE_COMMITTED_LOCK_DEGRADED"); + } + }; + await assert.rejects( + () => service.createProvider(providerInput(), makeSecret()), + (error) => error?.code === "PROVIDER_CREATE_COMMITTED_DEGRADED" + ); + assert.equal(registry.list().length, 1); + assert.equal(credentials.values.has("credential-1"), true); + }); + + await t.test("credential replacement", async (t) => { + const { service, registry, credentials } = makeHarness(t); + const provider = await service.createProvider(providerInput(), makeSecret("old")); + registry.markTest(provider.id, { status: "passed" }); + const replacement = makeSecret("new"); + const originalSet = credentials.set.bind(credentials); + credentials.set = async (ref, secret, options) => { + await originalSet(ref, secret, options); + if (secret === replacement) { + throw committedError("CREDENTIAL_STORE_COMMITTED_LOCK_DEGRADED"); + } + }; + await assert.rejects( + () => service.updateProvider(provider.id, { name: "New Name" }, replacement), + (error) => error?.code === "PROVIDER_UPDATE_COMMITTED_DEGRADED" + ); + assert.equal(registry.get(provider.id).name, "New Name"); + assert.equal(registry.get(provider.id).lastTestStatus, "untested"); + assert.equal(credentials.values.get("credential-1"), replacement); + }); + + await t.test("credential delete", async (t) => { + const { service, registry, credentials } = makeHarness(t); + const provider = await service.createProvider(providerInput(), makeSecret()); + const originalDelete = credentials.delete.bind(credentials); + credentials.delete = async (ref) => { + await originalDelete(ref); + throw committedError("CREDENTIAL_STORE_COMMITTED_LOCK_DEGRADED"); + }; + await assert.rejects( + () => service.deleteProvider(provider.id), + (error) => error?.code === "PROVIDER_DELETE_COMMITTED_DEGRADED" + ); + assert.deepEqual(registry.list(), []); + assert.equal(credentials.values.has("credential-1"), false); + }); +}); + +test("keeps provider not ready when replacement-secret rollback cannot restore the old secret", async (t) => { + const oldSecret = makeSecret("old"); + const replacement = makeSecret("replacement"); + const { service, registry, credentials, workerManager } = makeHarness(t); + const primary = await service.createProvider(providerInput("Primary"), oldSecret); + await service.createProvider(providerInput("Other", "https://other.example/v1"), makeSecret()); + registry.markTest(primary.id, { status: "passed" }); + const originalSet = credentials.set.bind(credentials); + credentials.set = async (ref, secret, options) => { + if (ref === "credential-1" && secret === oldSecret) { + throw new Error("private old-secret restore failure"); + } + return await originalSet(ref, secret, options); + }; + + await assert.rejects( + () => service.updateProvider(primary.id, { name: "OTHER" }, replacement), + (error) => error?.code === "PROVIDER_UPDATE_ROLLBACK_DEGRADED" + && error.details.degraded === true + ); + assert.equal(credentials.values.get("credential-1"), replacement); + assert.notEqual(registry.get(primary.id).lastTestStatus, "passed"); + credentials.operations.length = 0; + const workerCallCount = workerManager.calls.length; + await assert.rejects( + () => service.activate(primary.id), + (error) => error?.code === "PROVIDER_NOT_READY" + ); + assert.deepEqual( + credentials.operations.filter(([operation]) => operation === "get"), + [] + ); + assert.equal(workerManager.calls.length, workerCallCount); +}); + +test("testProvider sends the fixed Responses request and classifies stable failures", async (t) => { + const secret = makeSecret(); + const requests = []; + let nextFetch = async () => compatibleResponse(); + const { service, credentials } = makeHarness(t, { + fetchImpl: async (...args) => { + requests.push(args); + return nextFetch(...args); + } + }); + const provider = await service.createProvider({ + ...providerInput(), + authHeader: "x-provider-auth", + authScheme: "Token", + extraHeaders: { "x-region": "test" } + }, secret); + + assert.deepEqual(await service.testProvider(provider.id, "model-test"), { + ok: true, + code: null + }); + const [url, options] = requests.at(-1); + assert.equal(url, "https://provider.example/v1/responses"); + assert.equal(options.method, "POST"); + assert.equal(options.redirect, "manual"); + assert.deepEqual(JSON.parse(options.body), { + model: "model-test", + stream: false, + input: "Reply with OK." + }); + assert.equal(options.headers["x-provider-auth"], `Token ${secret}`); + assert.equal(options.headers["x-region"], "test"); + assert.ok(options.signal); + + const scenarios = [ + ["PROVIDER_TEST_DNS", async () => { const error = new Error("private dns"); error.code = "ENOTFOUND"; throw error; }], + ["PROVIDER_TEST_TLS", async () => { const error = new Error("private tls"); error.code = "CERT_HAS_EXPIRED"; throw error; }], + ["PROVIDER_TEST_TIMEOUT", async () => { const error = new Error("private timeout"); error.name = "TimeoutError"; throw error; }], + ["PROVIDER_TEST_AUTH", async () => ({ ok: false, status: 401, json: async () => ({ private: secret }) })], + ["PROVIDER_TEST_NOT_FOUND", async () => ({ ok: false, status: 404, json: async () => ({ private: secret }) })], + ["PROVIDER_TEST_HTTP", async () => ({ ok: false, status: 503, json: async () => ({ private: secret }) })], + ["PROVIDER_TEST_INVALID_JSON", async () => ({ ok: true, status: 200, json: async () => { throw new Error(secret); } })], + ["PROVIDER_TEST_INVALID_RESPONSES", async () => ({ ok: true, status: 200, json: async () => ({ id: "wrong" }) })], + ["PROVIDER_TEST_NETWORK", async () => { throw new Error(`private other ${secret}`); }] + ]; + for (const [code, fetchScenario] of scenarios) { + nextFetch = fetchScenario; + assert.deepEqual(await service.testProvider(provider.id, "model-test"), { + ok: false, + code + }); + } + + assert.deepEqual( + credentials.operations.filter(([operation]) => operation === "get") + .map(([, ref]) => ref), + Array(scenarios.length + 1).fill("credential-1") + ); + assert.equal(JSON.stringify(await service.listProviders()).includes(secret), false); +}); + +test("never follows a redirect or forwards custom authentication to a second origin", async (t) => { + const secret = makeSecret("redirect"); + const secondRequests = []; + const second = createServer((request, response) => { + secondRequests.push({ url: request.url, headers: { ...request.headers } }); + response.writeHead(200, { "content-type": "application/json" }); + response.end(JSON.stringify({ id: "resp", object: "response", output: [] })); + }); + const secondPort = await listen(second); + t.after(() => closeServer(second)); + const firstRequests = []; + const first = createServer((request, response) => { + firstRequests.push({ url: request.url, headers: { ...request.headers } }); + response.writeHead(302, { + location: `http://127.0.0.1:${secondPort}/stolen` + }); + response.end(); + }); + const firstPort = await listen(first); + t.after(() => closeServer(first)); + const { service, activity } = makeHarness(t, { + fetchImpl: globalThis.fetch, + createTimeoutSignal: () => AbortSignal.timeout(1_000) + }); + const provider = await service.createProvider({ + ...providerInput("Redirect", `http://127.0.0.1:${firstPort}/v1`), + authHeader: "x-private-auth", + authScheme: "Token" + }, secret); + + assert.deepEqual(await service.testProvider(provider.id, "model-test"), { + ok: false, + code: "PROVIDER_TEST_REDIRECT" + }); + assert.equal(firstRequests.length, 1); + assert.equal(firstRequests[0].headers["x-private-auth"], `Token ${secret}`); + assert.equal(secondRequests.length, 0); + assert.equal(JSON.stringify(activity.events).includes(secret), false); +}); + +test("activate persists then confirms increasing generations and rolls back failures", async (t) => { + const secretA = makeSecret("a"); + const secretB = makeSecret("b"); + const healthCalls = []; + const harness = makeHarness(t, { + verifyWorkerHealth: async (generation, state) => { + healthCalls.push([generation, state.generation]); + return true; + } + }); + const { service, registry, credentials, workerManager } = harness; + const providerA = await service.createProvider(providerInput("A"), secretA); + const providerB = await service.createProvider( + providerInput("B", "https://b.example/v1"), + secretB + ); + + await assert.rejects( + () => service.activate(providerA.id), + (error) => error?.code === "PROVIDER_NOT_READY" && error.status === 409 + ); + registry.markTest(providerA.id, { status: "passed" }); + const first = await service.activate(providerA.id); + assert.equal(first.activeProviderId, providerA.id); + assert.equal(first.generation, 1); + assert.equal(registry.getDocument().activeProviderId, providerA.id); + assert.equal(workerManager.calls[0][0], "start"); + assert.equal(workerManager.calls[0][1].generation, 1); + assert.equal(workerManager.calls[0][1].settings.upstream.apiKey, secretA); + assert.deepEqual(healthCalls, [[1, 1]]); + + registry.markTest(providerB.id, { status: "passed" }); + const originalApply = workerManager.applySnapshot.bind(workerManager); + let failCandidate = true; + workerManager.applySnapshot = async (snapshot) => { + if (failCandidate) { + failCandidate = false; + workerManager.failure = Object.assign(new Error(`private worker ${secretB}`), { + code: "WORKER_HEALTH_FAILED" + }); + try { + return await originalApply(snapshot); + } finally { + workerManager.failure = null; + } + } + return await originalApply(snapshot); + }; + await assert.rejects( + () => service.activate(providerB.id), + (error) => error?.code === "PROVIDER_ACTIVATION_FAILED" + && !error.message.includes(secretB) + ); + assert.equal(registry.getDocument().activeProviderId, providerA.id); + assert.equal((await service.getStatus()).generation, 3); + + const second = await service.activate(providerB.id); + assert.equal(second.generation, 4); + assert.equal(second.activeProviderId, providerB.id); + assert.equal(workerManager.calls.at(-1)[0], "applySnapshot"); + assert.equal(workerManager.calls.at(-1)[1].settings.upstream.apiKey, secretB); + const credentialGets = credentials.operations + .filter(([operation]) => operation === "get") + .map(([, ref]) => ref); + assert.deepEqual(credentialGets, ["credential-1", "credential-2", "credential-2"]); + const status = await service.getStatus(); + assert.equal(JSON.stringify(status).includes(secretA), false); + assert.equal(JSON.stringify(status).includes(secretB), false); + assert.equal(JSON.stringify(status).includes("credential-"), false); +}); + +test("serializes concurrent activations before credential reads and worker changes", async (t) => { + let releaseFirstStart; + let firstStartEntered; + const startEntered = new Promise((resolve) => { firstStartEntered = resolve; }); + const harness = makeHarness(t); + const { service, registry, credentials, workerManager } = harness; + const originalStart = workerManager.start.bind(workerManager); + workerManager.start = async (snapshot) => { + firstStartEntered(); + await new Promise((resolve) => { releaseFirstStart = resolve; }); + return await originalStart(snapshot); + }; + const providerA = await service.createProvider(providerInput("A"), makeSecret("a")); + const providerB = await service.createProvider( + providerInput("B", "https://b.example/v1"), + makeSecret("b") + ); + registry.markTest(providerA.id, { status: "passed" }); + registry.markTest(providerB.id, { status: "passed" }); + credentials.operations.length = 0; + + const first = service.activate(providerA.id); + const second = service.activate(providerB.id); + await startEntered; + await Promise.resolve(); + assert.deepEqual( + credentials.operations.filter(([operation]) => operation === "get"), + [["get", "credential-1"]] + ); + assert.equal(workerManager.calls.length, 0); + releaseFirstStart(); + await first; + const secondResult = await second; + + assert.equal(secondResult.activeProviderId, providerB.id); + assert.equal(secondResult.generation, 2); + assert.deepEqual(workerManager.calls.map(([operation, snapshot]) => ( + [operation, snapshot?.generation ?? null] + )), [["start", 1], ["applySnapshot", 2]]); + assert.deepEqual( + credentials.operations.filter(([operation]) => operation === "get"), + [["get", "credential-1"], ["get", "credential-2"]] + ); +}); + +test("restores the confirmed worker snapshot when post-ack health fails", async (t) => { + let failHealthGeneration = null; + const harness = makeHarness(t, { + verifyWorkerHealth: async (generation) => generation !== failHealthGeneration + }); + const { service, registry, credentials, workerManager } = harness; + const secretA = makeSecret("a"); + const secretB = makeSecret("b"); + const providerA = await service.createProvider(providerInput("A"), secretA); + const providerB = await service.createProvider( + providerInput("B", "https://b.example/v1"), + secretB + ); + registry.markTest(providerA.id, { status: "passed" }); + registry.markTest(providerB.id, { status: "passed" }); + await service.activate(providerA.id); + credentials.operations.length = 0; + failHealthGeneration = 2; + + await assert.rejects( + () => service.activate(providerB.id), + (error) => error?.code === "PROVIDER_ACTIVATION_FAILED" + ); + assert.equal(registry.getDocument().activeProviderId, providerA.id); + assert.equal((await service.getStatus()).generation, 3); + assert.equal(workerManager.generation, 3); + assert.deepEqual(workerManager.calls.slice(-2).map(([operation, snapshot]) => [ + operation, + snapshot.generation, + snapshot.settings.upstream.apiKey + ]), [ + ["applySnapshot", 2, secretB], + ["applySnapshot", 3, secretA] + ]); + assert.deepEqual( + credentials.operations.filter(([operation]) => operation === "get"), + [["get", "credential-2"]] + ); +}); + +test("reports degraded activation when a post-ack worker rollback cannot be confirmed", async (t) => { + let failHealthGeneration = null; + const harness = makeHarness(t, { + verifyWorkerHealth: async (generation) => generation !== failHealthGeneration + }); + const { service, registry, workerManager, activity } = harness; + const providerA = await service.createProvider(providerInput("A"), makeSecret("a")); + const providerB = await service.createProvider( + providerInput("B", "https://b.example/v1"), + makeSecret("b") + ); + registry.markTest(providerA.id, { status: "passed" }); + registry.markTest(providerB.id, { status: "passed" }); + await service.activate(providerA.id); + failHealthGeneration = 2; + const originalApply = workerManager.applySnapshot.bind(workerManager); + let applyCount = 0; + workerManager.applySnapshot = async (snapshot) => { + applyCount += 1; + if (applyCount === 2) throw new Error("private rollback failure"); + return await originalApply(snapshot); + }; + + await assert.rejects( + () => service.activate(providerB.id), + (error) => error?.code === "PROVIDER_ACTIVATION_ROLLBACK_DEGRADED" + && error.details.committed === false + && error.details.degraded === true + ); + assert.equal(registry.getDocument().activeProviderId, providerA.id); + assert.equal((await service.getStatus()).generation, 1); + assert.equal(workerManager.generation, 2); + assert.deepEqual(activity.events.at(-1), { + category: "provider", + action: "activate", + providerId: providerB.id, + result: "failed", + errorCode: "PROVIDER_ACTIVATION_ROLLBACK_DEGRADED", + details: { generation: 2 } + }); +}); + +test("reconciles committed active persistence and continues the selected worker snapshot", async (t) => { + const harness = makeHarness(t); + const { service, registry, workerManager, activity } = harness; + const providerA = await service.createProvider(providerInput("A"), makeSecret("a")); + const secretB = makeSecret("b"); + const providerB = await service.createProvider( + providerInput("B", "https://b.example/v1"), + secretB + ); + registry.markTest(providerA.id, { status: "passed" }); + registry.markTest(providerB.id, { status: "passed" }); + await service.activate(providerA.id); + const originalSetActive = registry.setActive.bind(registry); + let injected = false; + registry.setActive = (id) => { + const result = originalSetActive(id); + if (id === providerB.id && !injected) { + injected = true; + throw committedError("PROVIDER_REGISTRY_COMMITTED_LOCK_DEGRADED"); + } + return result; + }; + + await assert.rejects( + () => service.activate(providerB.id), + (error) => error?.code === "PROVIDER_ACTIVATION_COMMITTED_DEGRADED" + && error.details.committed === true + ); + assert.equal(registry.getDocument().activeProviderId, providerB.id); + assert.equal(workerManager.generation, 2); + assert.equal(workerManager.calls.at(-1)[0], "applySnapshot"); + assert.equal(workerManager.calls.at(-1)[1].settings.upstream.apiKey, secretB); + assert.equal((await service.getStatus()).generation, 2); + assert.equal(activity.events.at(-1).result, "degraded"); + assert.equal(activity.events.at(-1).errorCode, "PROVIDER_ACTIVATION_COMMITTED_DEGRADED"); +}); + +test("deterministically restores generation 3 after candidate apply commits but ACK is lost", async (t) => { + const harness = makeHarness(t); + const { service, registry, workerManager } = harness; + const secretA = makeSecret("a"); + const secretB = makeSecret("b"); + const providerA = await service.createProvider(providerInput("A"), secretA); + const providerB = await service.createProvider( + providerInput("B", "https://b.example/v1"), + secretB + ); + registry.markTest(providerA.id, { status: "passed" }); + registry.markTest(providerB.id, { status: "passed" }); + await service.activate(providerA.id); + const originalApply = workerManager.applySnapshot.bind(workerManager); + workerManager.applySnapshot = async (snapshot) => { + const state = await originalApply(snapshot); + if (snapshot.generation === 2) { + const error = new Error("private lost acknowledgement"); + error.code = "WORKER_ACK_TIMEOUT"; + throw error; + } + return state; + }; + + await assert.rejects( + () => service.activate(providerB.id), + (error) => error?.code === "PROVIDER_ACTIVATION_FAILED" + ); + assert.equal(registry.getDocument().activeProviderId, providerA.id); + assert.equal(workerManager.generation, 3); + assert.equal((await service.getStatus()).generation, 3); + assert.deepEqual(workerManager.calls.slice(-2).map(([operation, snapshot]) => [ + operation, + snapshot.generation, + snapshot.settings.upstream.apiKey + ]), [ + ["applySnapshot", 2, secretB], + ["applySnapshot", 3, secretA] + ]); +}); + +test("reports rollback degraded when first activation may commit and bounded stop fails", async (t) => { + const harness = makeHarness(t); + const { service, registry, workerManager, activity } = harness; + const provider = await service.createProvider(providerInput(), makeSecret()); + registry.markTest(provider.id, { status: "passed" }); + const originalStart = workerManager.start.bind(workerManager); + workerManager.start = async (snapshot) => { + await originalStart(snapshot); + const error = new Error("private lost first acknowledgement"); + error.code = "WORKER_ACK_TIMEOUT"; + throw error; + }; + workerManager.stop = async () => { + throw new Error("private bounded stop failure"); + }; + + await assert.rejects( + () => service.activate(provider.id), + (error) => error?.code === "PROVIDER_ACTIVATION_ROLLBACK_DEGRADED" + && error.details.degraded === true + ); + assert.equal(registry.getDocument().activeProviderId, null); + assert.equal((await service.getStatus()).generation, 0); + assert.equal(workerManager.generation, 1); + assert.equal(activity.events.at(-1).errorCode, "PROVIDER_ACTIVATION_ROLLBACK_DEGRADED"); +}); From 5d7a7840c76444b07087f76483939f0c60e5fe61 Mon Sep 17 00:00:00 2001 From: cluic Date: Mon, 13 Jul 2026 14:29:38 +0800 Subject: [PATCH 12/44] feat: add secured local Admin API --- docs/AGENT_COORDINATION.md | 2 +- docs/AI_HANDOFF.md | 8 +- docs/API.md | 28 +- docs/ARCHITECTURE.md | 16 +- docs/DATA_MODEL.md | 7 +- docs/PERMISSIONS.md | 6 + docs/STATUS.md | 7 +- docs/TESTING.md | 6 + .../plans/2026-07-10-crp-v1-implementation.md | 14 +- node/src/supervisor/admin-server.mjs | 699 ++++++++++++++++ node/src/supervisor/provider-service.mjs | 141 ++++ node/src/supervisor/session-auth.mjs | 283 +++++++ node/src/supervisor/supervisor-entry.mjs | 62 ++ node/src/supervisor/supervisor.mjs | 349 ++++++++ node/test/integration/admin-server.test.mjs | 749 ++++++++++++++++++ node/test/provider-service.test.mjs | 129 +++ node/test/session-auth.test.mjs | 213 +++++ 17 files changed, 2693 insertions(+), 26 deletions(-) create mode 100644 node/src/supervisor/admin-server.mjs create mode 100644 node/src/supervisor/session-auth.mjs create mode 100644 node/src/supervisor/supervisor-entry.mjs create mode 100644 node/src/supervisor/supervisor.mjs create mode 100644 node/test/integration/admin-server.test.mjs create mode 100644 node/test/session-auth.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 94a6507..13ca36e 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus Task 8 implementation agent | Resolve Task 8 security-review findings with one writer | Task 8 owns `node/src/supervisor/activity-store.mjs`, `node/src/supervisor/migration.mjs`, `node/src/supervisor/provider-service.mjs`, their three focused tests, and affected living docs | Review fixes implemented; deterministic gates passing; re-review pending; real HOME/keyring/network migration remains prohibited | Task 8 commit `fa5e485468065c3f2eff54d2c9ebcc69fbcf8e35` | Existing provider registry, credential adapters, worker manager, server/runtime settings, capture, Codex, CLI/bin, Admin API, UI, runners, package scripts, and every other module; no simultaneous writable agent | +| V1 sequential implementation | Primary Codex plus Task 9 implementation agent | Build the secured loopback Admin API and supervisor composition with one writer | Task 9 owns `node/src/supervisor/session-auth.mjs`, `node/src/supervisor/admin-server.mjs`, `node/src/supervisor/supervisor.mjs`, `node/src/supervisor/supervisor-entry.mjs`, `node/test/session-auth.test.mjs`, `node/test/integration/admin-server.test.mjs`, a minimal lifecycle facade in `node/src/supervisor/provider-service.mjs` with its focused tests in `node/test/provider-service.test.mjs`, and affected living docs | Task 9 implementation and deterministic gates are complete; independent review is next; real HOME, native keyrings, external network, UI, CLI, and runner changes remain prohibited | Task 8 commit `776265c8113f39b26f2bb5cd1418cfc275c4156e` | Provider registry, credential adapters, worker manager internals, server/runtime settings, capture, Codex adapter internals, migration/activity persistence, UI, CLI/bin, runners, package scripts, and every other module; no simultaneous writable agent | ## Shared Contracts diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index dd826cd..39bdf2e 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,11 +6,11 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 8 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, the versioned proxy-worker IPC entrypoint, reliable fixed-port worker lifecycle management, sanitized activity, transactional legacy migration, and provider orchestration. Task 9, the secured loopback Admin API, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 9 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, versioned worker IPC, reliable fixed-port lifecycle management, sanitized activity, transactional migration, provider orchestration, and the secured loopback Admin control plane. Task 10, routing CLI commands through the supervisor, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, idempotent Codex bootstrap, strict provider storage, secure credential adapters, immutable request snapshots, strict worker IPC, reliable worker management, bounded sanitized activity, transactional v0.2.2 migration, and serialized provider CRUD/test/activation with compensating rollback. Target: the long-lived secured supervisor control plane. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap, strict provider storage, secure credential adapters, immutable request snapshots, strict worker IPC, reliable worker management, bounded sanitized activity, transactional v0.2.2 migration, serialized provider orchestration, private local sessions, the exact Admin route/security boundary, and readiness-gated supervisor composition. Target: CLI parity and the actual Web UI. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 8 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 9 are complete. ## How To Run Current Code @@ -49,6 +49,7 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 Task 6 gate: `node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` passes 21/21, `npm run test:integration` passes 11/11, and `npm test` passes a nonduplicated 112/112 top-level group followed by 11/11 integration tests; `npm run lint` syntax-checks 18 source files. Coverage includes exact version-1 directional schemas, HTTPS-or-loopback upstreams, HTTP-token authentication fields, final authentication-value validation, sensitive and conflicting header rejection, parent-only configure secrets, fixed invalid-message fatal IDs, configure-before-listen, monotonic reconfiguration, configure rejection after drain begins, retained in-flight drain acknowledgement, idempotent duplicate drain, health/status, idle keep-alive closure, clean shutdown, bounded parent-disconnect cleanup with a hanging upstream before or during shutdown, safe startup failures, stale generations, port conflicts, port release, and child-process cleanup without fixed sleeps. Real-fork integration runs after the ordinary group so it cannot race watcher readiness in another test process. - Node 22.19 Task 7 gate: strict-unhandled `node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` passes 22/22; exact `npm test` passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests without duplication; `npm run test:unit` retains all-top-level coverage and passes 133/133. `npm run lint` syntax-checks 19 source files, and `npm audit --omit=dev` reports zero vulnerabilities. Coverage includes ready/configure/health startup, acknowledgement-atomic snapshot application, concurrent restart reuse before inspecting its snapshot, restart prevalidation before drain when no operation exists, immediate waiter observation and send-failure cancellation, graceful drain/shutdown, TERM/KILL escalation, retained child control after termination timeout, same-port real-worker restart with a changed PID, partial-start and port-release cleanup, correlated fatal and malformed-message sanitization, old-epoch isolation, cancellable injected-clock crash backoff, immediate failure on the fifth crash in 60 seconds, and idempotent retryable close without child, timer, listener, or port residue. The exact runner isolates the preexisting polling capture watcher between the core unit and real-fork integration groups so its registration baseline cannot race unrelated unit load. - Node 22.19 Task 8 gate: `node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs` passes 42/42; exact `npm test` passes 168/168 core assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run lint` syntax-checks 22 source files. Tests use only temporary paths, injected credential adapters/fetch responses, fake worker boundaries, and loopback redirect servers. Coverage includes recursive lifecycle-field redaction, ownership-checked activity locks, descriptor-safe migration paths, exclusive final-path registry creation, symlink and foreign-replacement preservation, committed-state reconciliation, redirect refusal, active-update rejection, replacement-secret compensation, selected credentials, explicit operation serialization, and conservative activation rollback `1 -> 2 -> 3` after health or lost-ACK uncertainty. Real HOME migration, native keyrings, cross-platform filesystem semantics, and live upstreams remain prohibited until L3 platform review. +- Node 22.19 Task 9 gate: `node --test test/session-auth.test.mjs test/provider-service.test.mjs test/integration/admin-server.test.mjs` passes 42/42; exact `npm test` passes 179/179 core assertions, 7/7 isolated capture assertions, and 23/23 integration tests; `npm run lint` syntax-checks 26 source files and `npm audit --omit=dev` reports zero vulnerabilities. Coverage includes descriptor-safe private control tokens, expiring browser sessions/CSRF, exact Host/Origin/CORS rules, bounded request schemas, every Admin route, response/error secret scans, static allowlisting without actual UI files, active-only lifecycle credentials with in-flight command reuse, migration-before-registry composition, readiness-gated private state, startup compensation, separate Codex/state adapters, and signal cleanup. Real HOME, native keyring, external provider traffic, actual UI assets, and cross-platform browser behavior remain prohibited until their later gates. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. @@ -94,6 +95,7 @@ Credential migration, localhost browser security, in-flight activation semantics - IPC acknowledgement promises must be observed at registration and cancelled synchronously when send fails. - A child reference and its listeners must remain owned after TERM/KILL timeout until a later lifecycle attempt confirms exit and fixed-port release. - Provider-service operations must use their own mutex; a worker-manager shared operation must never substitute for provider-level serialization. +- Provider lifecycle facades must reuse the current in-flight operation before queueing work or resolving credentials. - A post-ack activation rollback must advance to a new generation; never regress a worker generation to restore the confirmed provider. - Compensation tests must run and fail before compensation code is written. - Migration must create a missing registry exclusively at its final path and must never adopt, modify, or delete an `EEXIST` registry. diff --git a/docs/API.md b/docs/API.md index 00dbf14..74275d6 100644 --- a/docs/API.md +++ b/docs/API.md @@ -3,7 +3,7 @@ ## Conventions - Base URL: `http://127.0.0.1:15101/api/v1`. -- JSON request and response bodies. +- JSON request and response bodies use UTF-8 `application/json`; request bodies are capped at 64 KiB. - Mutations are idempotent where an idempotency key is practical. - Complete credentials are write-only. @@ -11,9 +11,11 @@ - Bind only to loopback and reject unexpected `Host` and `Origin` values. - Disable CORS. -- Establish a local SameSite session and require a CSRF token for mutations. +- `POST /session` exchanges the valid control-token bearer for an HttpOnly `SameSite=Strict` cookie and a CSRF token. +- Browser reads require the session cookie; browser mutations additionally require `X-CRP-CSRF`. CLI bearer mutations do not require CSRF. - CLI uses a `0600` local control token. -- Read endpoints return only `credentialConfigured` and a masked preview. +- Read endpoints return only the positive public provider/worker allowlists and `credentialConfigured`; credential references and complete or partial secrets are omitted. +- Unexpected `Host`, nonmatching `Origin`, every CORS preflight, and every `/api` path outside the versioned route table are rejected. ## Error Format @@ -29,12 +31,13 @@ } ``` -`details` must be sanitized and safe to display. +`details` uses a small positive allowlist for safe validation and committed-state fields. Sensitive field names remain visible only with `[REDACTED]`; unknown detail fields are omitted. Known paths with an unsupported method return `405 API_METHOD_NOT_ALLOWED`, while unmatched API paths return `404 API_NOT_FOUND`. ## Endpoints | Method | Path | Purpose | | --- | --- | --- | +| POST | `/session` | Exchange the local control-token bearer for a browser session and CSRF token | | GET | `/status` | Supervisor, worker, Codex bootstrap, and active-provider status | | GET | `/providers` | List masked provider summaries | | POST | `/providers` | Create a profile and write its credential | @@ -48,10 +51,25 @@ | POST | `/proxy/restart` | Drain, stop, release port, restart, and health-check worker | | GET | `/activity` | Page through sanitized lifecycle activity | | GET | `/settings` | Read non-secret local settings | -| PATCH | `/settings` | Update supported settings with validation | +| PATCH | `/settings` | Return `409 SETTINGS_READ_ONLY` while V1 fixed settings have no supported mutation | | POST | `/codex/bootstrap` | Back up and idempotently set the fixed Codex provider/proxy entry | | POST | `/diagnostics/export` | Produce a redacted local diagnostic bundle | +## Request Contracts + +- `POST /providers`: `{ "provider": { ...public input fields }, "credential": "write-only", "fallbackConsent": false }`. +- `PATCH /providers/:id`: `{ "patch": { ...editable fields }, "replacementCredential"?: "write-only" }`. +- `POST /providers/:id/test`: `{ "model": "non-empty-model" }`. +- `PATCH /settings`: currently accepts only a boolean `captureEnabled` shape, then returns `409 SETTINGS_READ_ONLY` without mutation. +- Session exchange, delete, activate, proxy lifecycle, Codex bootstrap, and diagnostics export require an empty body. +- Root objects reject unknown fields. Malformed JSON returns `400 API_BODY_INVALID`, unsupported media types return `415 API_CONTENT_TYPE_UNSUPPORTED`, and oversized input returns `413 API_BODY_TOO_LARGE`. + +`GET /activity` accepts one bounded `limit` from 1 through 100 and one `offset` from 0 through 9999. It returns `{ events, page: { limit, offset, nextOffset } }` with newest events first. + +## Static UI Boundary + +The Admin server supports only `index.html`, `styles.css`, and `app.js` with explicit MIME types, `Cache-Control: no-store`, and `X-Content-Type-Options: nosniff`. Extensionless client routes fall back to `index.html`; unknown asset extensions, unsafe decoded paths, and non-GET/HEAD UI methods are rejected. The actual UI assets land in Task 11. + ## State Conflict Rules - Activating an untested or failed provider returns `409 PROVIDER_NOT_READY`. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8e1a0f5..aa3c093 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 remains a Node CLI plus one proxy process. `crp start` writes a single runtime configuration, bootstraps Codex to use the `OpenAI` provider section, and spawns the proxy. Tasks 2 through 7 have landed shared path and public-error contracts, an idempotent Codex configuration adapter, strict provider-schema validation, atomic provider and credential persistence, immutable request-level runtime settings, an IPC-hosted worker entrypoint, and the worker lifecycle manager. The composed supervisor, provider service, and Admin API remain target-state architecture. +Version 0.2.2 remains the published Node CLI plus one proxy process. Tasks 2 through 9 have now landed the target control-plane foundations: safe shared contracts, atomic provider and credential persistence, immutable request settings, strict worker IPC, reliable worker lifecycle management, bounded activity, transactional migration, provider orchestration, and a composed secured loopback Admin API. CLI routing and the actual static Web UI remain target-state work. ## Target Overview @@ -40,7 +40,7 @@ Active OpenAI-compatible upstream ## Module Boundaries -Landed in Tasks 2 through 7: +Landed in Tasks 2 through 9: - `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. - `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. @@ -55,12 +55,14 @@ Landed in Tasks 2 through 7: - `worker-protocol`: validates exact version-1 parent messages (`configure`, `drain`, `shutdown`, `status`) and child messages (`ready`, `configured`, `drained`, `status`, `fatal`). Configure accepts HTTPS or explicit loopback HTTP only, requires HTTP-token authentication fields, validates the exact final authentication value with Node header rules, and rejects sensitive or authentication-conflicting extra headers. Resolved settings are accepted only inside parent configure messages; child output and sanitized projections contain only allowlisted lifecycle state and stable public errors, and invalid input receives the fixed `worker-fatal` correlation ID. - `proxy-worker`: creates one runtime settings source and forwarding app after the first valid configure, binds only then, applies strictly increasing generations while ready or running, rejects configure once drain begins, tracks in-flight requests independently of later phase changes, closes the listener and idle keep-alive connections during drain, acknowledges repeated drain requests without leaving the drained phase, closes capture and IPC resources during shutdown, and bounds forced cleanup after parent disconnect even when graceful shutdown is already waiting so a hanging upstream cannot orphan the worker. - `worker-manager`: serializes lifecycle operations, validates restart snapshots before disturbing the current worker, validates every child message before use, correlates immediately observed acknowledgements by request ID and child epoch, cancels waiters on send failure, confirms configuration health before running, drains and escalates termination within deadlines, retains child control after termination timeout, proves the fixed port free before replacement, and recovers unexpected exits with a cancellable rolling crash window and sanitized public state. +- `activity/migration/provider-service`: persists a bounded sanitized event allowlist, transactionally migrates descriptor-validated legacy files, serializes provider CRUD/test/activation compensation, and exposes start/stop/restart facades that resolve only the active credential and advance confirmed generations. +- `session-auth`: owns one private 32-byte control token plus in-memory expiring browser session/CSRF tokens; existing token files are descriptor-validated and unsafe, malformed, symbolic-link, or broadly accessible files fail closed. +- `admin-api`: enforces the exact loopback Host and Origin, disables CORS, authenticates CLI bearer and browser cookie/CSRF requests, bounds and validates request bodies, dispatches the versioned allowlisted routes, and positively projects provider, worker, status, settings, diagnostics, and error data. +- `supervisor`: composes activity and credentials before migration, constructs the registry only after migration, owns worker/provider/auth/Admin services, writes a positive `0600` state projection only after Admin readiness, and performs idempotent reverse-order cleanup on startup failure or signal shutdown. Remaining target-state boundaries: -- `supervisor`: now owns worker management, bounded activity, transactional migration, and provider orchestration; the Admin server remains the next composition layer. -- `provider-service`: coordinates compatibility tests, credentials, activation, and credential-aware deletion above the metadata registry. -- `admin-api`: loopback-only versioned HTTP contract used by both UI and CLI. +- `cli-supervisor-client`: routes lifecycle and provider commands through the same Admin contract. - `web-ui`: static local app with onboarding and daily management views. ## Lifecycle @@ -74,9 +76,9 @@ Remaining target-state boundaries: - Package remains distributed through npm. - The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; the provider registry owns `providers.json`, the explicit fallback adapter owns `secrets.json`, and the other new stores remain target state. - Provider metadata: `~/.codex-remote-proxy/providers.json`, atomically replaced with mode `0600` where supported after complete document validation. -- Supervisor runtime metadata target: `~/.codex-remote-proxy/state.json`, mode `0600` where supported. +- Supervisor runtime metadata: `~/.codex-remote-proxy/state.json`, atomically written with mode `0600` after Admin readiness and removed only by the owning supervisor instance. - Credentials: native OS store by default; `~/.codex-remote-proxy/secrets.json` only after strict explicit fallback consent. Native loader/factory failure before selection is a safe backend-unavailable error unless that consent is present. Once native is selected, operation failures remain native and are never replayed. Construction fallback exposes a `file` label that must be explicitly reused across restart; no credential migration is implicit. -- Activity events: bounded local JSONL or SQLite store without request/response bodies. +- Activity events: implemented bounded private JSONL without request/response bodies. - Codex configuration replacement holds an exclusive sidecar lock, compares bytes before writing, creates backups with exclusive-copy semantics, rechecks the source before rename, and preserves the source permission mode through same-directory temporary-file `fsync` and rename; unchanged content is neither rewritten nor backed up. - Provider-registry mutation holds an exclusive `0600` sidecar lock across disk reload, complete validation, same-directory temporary-file `fsync`, `chmod 0600`, rename, and in-memory replacement; validation or persistence failure leaves the prior in-memory document unchanged. Bounded cleanup preserves primary errors and distinguishes a durable committed/degraded result from a retryable failure; permanent residual locks require explicit repair and restart and are never auto-removed. - File-credential mutation follows the same conservative durable-commit rules while using a strict schema-version-1 secret-only document. Reads validate the parent and path before opening, verify descriptor identity, and never read secret bytes by path. An exclusive gate covers mutation. Release atomically renames the canonical gate to a unique claim, verifies and removes only that claim, and never deletes the canonical path. The canonical primary lock remains present throughout gate claim validation, so a competing instance that acquires the empty gate still reports busy; foreign or uncertain gate state must prove a canonical blocker before primary release, otherwise the primary lock is retained. Foreign or permanent claimed primary locks likewise restore a nonempty canonical blocker. Permanent secret-temp cleanup failure records uncommitted degradation and stops later mutations before another lock opens. Public errors contain no reference, secret, path, or file bytes. diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index a33ad5e..0b4f116 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -82,7 +82,11 @@ Configure settings use the exact runtime graph required by the forwarding server ## RuntimeState -Runtime state includes supervisor PID plus the worker manager's exact public projection `{ phase, pid, generation, state, restartCount, startedAt, error }`. Nested `state` is the child protocol's public lifecycle state, and `error` is either null or a stable `{ code, message }`; snapshots, credentials, raw causes, argv, environment, stdout, and stderr are excluded. The state is observational and can be reconstructed. +The implemented private state document is `{ schemaVersion: 1, supervisorPid, startedAt, admin, worker }`. `admin` is the ready loopback address and `worker` is the manager's exact public projection `{ phase, pid, generation, state, restartCount, startedAt, error }`. Nested `state` is the child protocol's public lifecycle state, and `error` is either null or a stable `{ code, message }`; snapshots, credentials, raw causes, argv, environment, stdout, and stderr are excluded. State is observational, atomically written with mode `0600` only after Admin readiness, and reconstructable. + +## LocalAuthState + +The control token and every session/CSRF token are independent 32-byte base64url values. Only the control token is persisted, in a descriptor-validated regular `0600` file below a private `0700` parent. Browser session IDs and CSRF tokens remain in memory with an expiry timestamp; the cookie is HttpOnly and `SameSite=Strict`. Public API data never includes the control token or session ID. ## ActivityEvent @@ -102,6 +106,7 @@ The provider service now implements the following lifecycle behavior: - Deleting the active profile is rejected until another profile is activated or the proxy is stopped. - Deleting a profile removes its native credential entry and writes an activity event. - Activity retention defaults to 30 days or 10,000 events, whichever limit is reached first. +- Proxy start and restart resolve only the active tested provider credential, advance the confirmed generation after worker acknowledgement and health, and return only worker public state. Stop is credential-free and idempotent. ## Migration diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md index 49f5fe9..940c855 100644 --- a/docs/PERMISSIONS.md +++ b/docs/PERMISSIONS.md @@ -50,6 +50,12 @@ Record provider creation/update/deletion, test result, activation, Codex bootstr - Provider tests must use a no-follow redirect policy; no 3xx target may receive the configured authentication header or credential. - Migration and activity cleanup may delete only atomically claimed paths whose descriptor identity or ownership token matches; foreign replacements must remain canonical blockers. - A `committed: true` persistence error must be reconciled from durable state before compensation; it must never trigger an inverse credential mutation by assumption. +- The Admin server must bind exactly `127.0.0.1`, require its exact loopback `Host`, reject every mismatched `Origin` and CORS preflight, and emit no access-control allow headers. +- CLI requests require the private control-token bearer. Browser reads require an expiring HttpOnly `SameSite=Strict` session cookie, and browser mutations additionally require the matching session CSRF token. +- Admin request bodies must use the exact route schema, UTF-8 JSON where a body is allowed, and a 64 KiB limit; routes defined with an empty body must reject nonempty input. +- API responses, status files, settings, diagnostics, and errors must use positive field allowlists. Error details may expose only stable validation/commit metadata or `[REDACTED]` sensitive fields. +- Static UI serving must map only the three packaged asset filenames and an extensionless index fallback; decoded traversal, unknown asset extensions, and non-GET/HEAD methods are prohibited. +- State and Codex filesystem adapters must remain separate, and supervisor cleanup must not delete state it cannot prove it owns. ## Credential Verification Boundary diff --git a/docs/STATUS.md b/docs/STATUS.md index 0b1e631..9685dd6 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 8 are complete and Task 9 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 9 are complete and Task 10 is next. ## Done @@ -26,14 +26,15 @@ V1 implementation: provider lifecycle end to end. - Completed Task 6 strict version-1 worker IPC and the proxy-worker child entrypoint. Configure enforces HTTPS-or-loopback upstreams, HTTP-token authentication fields, Node-compatible final authentication values, and sensitive/header-collision rejection; the worker remains unbound until valid configuration, rejects configure after drain begins without losing the pending drain acknowledgement, keeps repeated drain acknowledgements and status drained, bounds parent-disconnect cleanup when an upstream hangs before or during shutdown, uses a fixed correlation ID for invalid-message fatals, releases its port on shutdown, and emits only sanitized child messages. Coverage passes 21/21 focused and 11/11 integration tests; the full gate passes 112/112 top-level tests followed by 11/11 integration tests without duplication. Syntax checking covers 18 source files, including the sequential group runner that isolates real-fork load from watcher tests. - Completed Task 7 reliable worker lifecycle management and its review fixes. Startup requires ready, correlated configure acknowledgement, and matching health; snapshot generations advance only after matching acknowledgement; concurrent restart calls reuse the current operation before inspecting a new snapshot, while an unshared restart validates the complete snapshot before drain; send failures synchronously cancel observed acknowledgement waiters; stop and restart bound drain, TERM, and KILL, confirm exclusive fixed-port release, retain control after termination timeout, and isolate old child epochs. Unexpected exits recover with cancellable injected-clock backoff and fail immediately on the fifth crash in 60 seconds. Strict-unhandled coverage passes 22/22 focused tests; the exact full gate passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests; the all-top-level compatibility command passes 133/133. Syntax checking covers 19 source files, and the runtime audit reports zero vulnerabilities. - Completed Task 8 sanitized activity, transactional legacy migration, and provider orchestration. Activity uses an exact event allowlist, expanded recursive denylist, bounded retention, atomic replacement, and ownership-checked degraded locks. Migration uses descriptor-safe source/registry reads, symlink rejection, byte-exact exclusive backups, transaction locking, exclusive final-path registry creation, identity-checked rollback, committed-state reconciliation, and canonical blockers. Provider CRUD/test/activation rejects active edits, refuses redirects, reconciles committed mutations, serializes selected-credential operations, and deterministically rolls an uncertain candidate back with a newer generation. Focused coverage passes 42/42; exact full coverage passes 168/168 core, 7/7 capture, and 12/12 integration assertions; syntax covers 22 source files. All tests use temporary paths, injected adapters, and loopback mocks; real HOME/keyring/live-network migration remains an L3 gate. +- Completed Task 9 secured loopback Admin API and supervisor composition. Private 32-byte control/session/CSRF tokens, exact Host/Origin and disabled-CORS enforcement, bounded exact request contracts, positive public/error projections, every approved Admin route, activity pagination, read-only settings, static-asset allowlisting, active-only proxy lifecycle facades with in-flight command reuse, migration-before-registry composition, readiness-gated `0600` state, construction/listen rollback, separate Codex/state filesystem adapters, and idempotent signal cleanup are covered. Focused coverage passes 42/42; exact full coverage passes 179/179 core, 7/7 capture, and 23/23 integration assertions; syntax checks 26 source files and the runtime audit reports zero vulnerabilities. Actual UI assets, CLI routing, real HOME/keyring/network use, and cross-platform browser security remain later gates. ## Blocked -- No current blocker is recorded for Task 9. +- No current blocker is recorded for Task 10. ## Next -1. Execute Task 9: build the secured loopback Admin API. +1. Execute Task 10: route CLI commands through the supervisor. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/TESTING.md b/docs/TESTING.md index c0e854e..3641503 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -26,6 +26,7 @@ - Task 5 focused suite: `cd node && node --test test/runtime-settings.test.mjs test/server.test.mjs` - Task 6 focused suite: `cd node && node --test test/worker-protocol.test.mjs test/integration/worker-entry.test.mjs` - Task 7 focused suite: `cd node && node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` +- Task 9 focused suite: `cd node && node --test test/session-auth.test.mjs test/provider-service.test.mjs test/integration/admin-server.test.mjs` - Task 8 focused suite: `cd node && node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs` - Worker integration suite: `cd node && npm run test:integration` - Runtime audit: `cd node && npm audit --omit=dev` @@ -40,6 +41,8 @@ The Node 22.19 Task 6 focused suite passes 21/21, `npm run test:integration` pas The Node 22.19 Task 7 strict-unhandled focused suite passes 22/22. Exact `npm test` passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run test:unit` passes all 133/133 top-level assertions; `npm run lint` syntax-checks 19 source files. Coverage verifies ready/configure/health startup, strict request correlation, acknowledgement-atomic generation changes, concurrent restart Promise identity and zero snapshot inspection before shared-operation reuse, restart prevalidation without drain side effects when no operation exists, send-failure waiter cancellation without unhandled rejections, early acknowledgement retention, graceful drain/shutdown, TERM/KILL escalation, retained retryable control after termination timeout, partial-start error preservation, fixed-port cleanup, same-port real-worker PID replacement, sanitized fatal and malformed messages, old-epoch isolation, cancellable injected-clock crash recovery, immediate failure on the fifth crash in 60 seconds, and idempotent close with no child, timer, listener, process, or port residue. The fifth in-window crash does not schedule the 4000 ms capped delay; only the first four 250/500/1000/2000 ms delays execute before the threshold. +The Node 22.19 Task 9 focused suite passes 42/42. Exact `npm test` passes 179/179 core assertions, 7/7 isolated capture assertions, and 23/23 integration assertions; `npm run lint` syntax-checks 26 source files and `npm audit --omit=dev` reports zero vulnerabilities. Coverage verifies private descriptor-validated control tokens, browser expiry and CSRF, exact Host/Origin/CORS rejection, bounded exact JSON and empty-body contracts, every approved route, safe activity pagination, read-only settings, positive response/error/diagnostic projections, static asset allowlisting, active-only lifecycle credentials with in-flight command reuse, supervisor migration-before-registry composition, readiness-gated `0600` state, construction/listen rollback, independent Codex/state adapters, and idempotent signal cleanup. All new integration boundaries use temporary homes, injected stores/services, and loopback HTTP; real HOME, native keyring, external provider traffic, and actual UI assets remain outside this deterministic gate. + The Node 22.19 Task 8 focused suite passes 42/42. Exact `npm test` passes 168/168 core assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run lint` syntax-checks 22 source files. Coverage verifies the expanded activity denylist and full-file secret scans, Error/cycle/non-JSON handling, retention, atomic failure preservation, foreign lock replacement, and canonical degraded blocking; descriptor-safe temp-only migration, symlink rejection, byte-exact backups, exclusive final-path registry creation, identity/byte-owned rollback, foreign registry preservation, transaction locks, committed schema reconciliation, and canonical release blockers; active-update rejection, no-follow redirect behavior across two loopback origins, CRUD/credential committed outcomes, replacement rollback safety, selected credentials, explicit serialization, and deterministic `1 -> 2 -> 3` rollback after health or acknowledgement uncertainty. No test accesses a real HOME, native credential backend, or live upstream network. `npm run test:unit` retains its public behavior and runs all top-level `test/*.test.mjs` files. The exact `npm test` gate runs the same top-level set without duplication as two sequential groups: every non-capture unit file first, then `capture-store.test.mjs` alone, followed by recursively discovered `test/integration/**/*.test.mjs`. This isolates the polling watcher registration baseline from unrelated unit load and keeps real child-process integration after watcher cleanup. `test:e2e` and the combined `test:all` command are not current gates until the UI, Playwright configuration, and E2E specs land. @@ -78,6 +81,9 @@ The Node 22.19 Task 8 focused suite passes 42/42. Exact `npm test` passes 168/16 - Restart validation must complete before any drain message, phase change, or current-worker mutation. - IPC acknowledgement waiters must be observed immediately and cancelled on send failure. - Lifecycle code must retain child control after termination timeout until exit and port release are confirmed. +- Host-header security tests must use a transport that preserves the explicit `Host` value; browser-style fetch implementations may normalize restricted headers. +- Admin contract tests must scan every serialized success and failure for complete credentials, credential references, session/control tokens, raw causes, stacks, and unknown error details. +- Supervisor tests must hold Admin readiness behind an observable gate and prove state is absent before readiness, private afterward, and removed by idempotent reverse-order cleanup. - Activity tests must scan the complete persisted file for every generated secret and cover retention, atomic-write failure, foreign locks, and committed lock degradation. - Migration tests must use only injected temporary paths/adapters and cover byte-exact exclusive backups, schema-2 idempotency, reverse rollback, residual-lock degradation, and backup retention. - Provider-service compensation tests must fail before implementation and cover create, credential replacement, delete, active-delete ordering, stable test classifications, selected credentials, explicit concurrency serialization, and activation rollback. diff --git a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md index 0fc0918..39db0f9 100644 --- a/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md +++ b/docs/superpowers/plans/2026-07-10-crp-v1-implementation.md @@ -725,15 +725,15 @@ Actual Node 22.19 Task 8 verification after security review fixes: the focused a - Create: `node/test/session-auth.test.mjs` - Create: `node/test/integration/admin-server.test.mjs` -- [ ] **Step 1: Write failing browser-auth tests** +- [x] **Step 1: Write failing browser-auth tests** Assert a 32-byte control token is created with `0600` mode. Test `/api/v1/session` with a valid bearer token returns an HttpOnly `SameSite=Strict` cookie and CSRF token, then clears access when expired. Invalid Host, non-loopback Origin, missing CSRF, missing cookie, and CORS preflight must be rejected. -- [ ] **Step 2: Write failing API contract tests** +- [x] **Step 2: Write failing API contract tests** Exercise every endpoint in `docs/API.md` against injected provider and worker services. Assert mutation status codes, `409` conflicts, sanitized error format, activity pagination, static UI fallback, and that serialized responses never contain `credentialRef` or a complete key. -- [ ] **Step 3: Run and verify failures** +- [x] **Step 3: Run and verify failures** ```bash cd node @@ -742,15 +742,15 @@ node --test test/session-auth.test.mjs test/integration/admin-server.test.mjs Expected: FAIL because auth and Admin API modules do not exist. -- [ ] **Step 4: Implement auth and route table** +- [x] **Step 4: Implement auth and route table** Use `crypto.randomBytes(32).toString("base64url")` for control/session/CSRF tokens. Accept bearer auth for CLI and cookie+CSRF for browser mutations. Route only the methods and paths listed in `docs/API.md`; unmatched API paths return `404 API_NOT_FOUND`. Serve `ui/index.html`, `ui/styles.css`, and `ui/app.js` with explicit MIME types and `Cache-Control: no-store`. -- [ ] **Step 5: Compose the supervisor** +- [x] **Step 5: Compose the supervisor** `createSupervisor(options)` builds paths, migration, registry, credential store, activity store, worker manager, provider service, auth, and Admin API. `supervisor-entry.mjs` loads the selected home, listens on `127.0.0.1:15101`, writes state only after health is ready, and shuts down cleanly on `SIGTERM`/`SIGINT`. -- [ ] **Step 6: Verify and commit** +- [x] **Step 6: Verify and commit** ```bash cd node @@ -760,6 +760,8 @@ git add node/src/supervisor node/test/session-auth.test.mjs node/test/integratio git commit -m "feat: add secured local Admin API" ``` +Result: Node 22.19 focused coverage passes 42/42; the exact full gate passes 179/179 core, 7/7 isolated capture, and 23/23 integration assertions. Syntax checking covers 26 source files and the runtime dependency audit reports zero vulnerabilities. + ## Task 10: Route CLI Commands Through the Supervisor **Files:** diff --git a/node/src/supervisor/admin-server.mjs b/node/src/supervisor/admin-server.mjs new file mode 100644 index 0000000..d72b1fb --- /dev/null +++ b/node/src/supervisor/admin-server.mjs @@ -0,0 +1,699 @@ +import { randomBytes } from "node:crypto"; +import http from "node:http"; +import { readFile } from "node:fs/promises"; +import { extname, join } from "node:path"; + +import { sanitizeActivityValue } from "./activity-store.mjs"; +import { CrpError, toPublicError } from "../shared/errors.mjs"; + +const API_PREFIX = "/api/v1"; +const PUBLIC_PROVIDER_FIELDS = [ + "id", + "name", + "baseUrl", + "authHeader", + "authScheme", + "extraHeaders", + "modelMode", + "modelOverride", + "lastTestAt", + "lastTestStatus", + "lastTestCode", + "createdAt", + "updatedAt", + "credentialConfigured" +]; +const PUBLIC_WORKER_FIELDS = [ + "phase", + "pid", + "generation", + "state", + "restartCount", + "startedAt", + "error" +]; +const PUBLIC_CHILD_STATE_FIELDS = [ + "phase", + "configured", + "generation", + "listening", + "listenHost", + "listenPort", + "inFlight" +]; +const SAFE_ERROR_DETAIL_FIELDS = new Set([ + "field", + "reason", + "committed", + "degraded", + "generation", + "httpStatus" +]); + +function apiError(code, message, action, status) { + return new CrpError(code, message, action, { status }); +} + +function bodyError(code) { + const contracts = { + API_CONTENT_TYPE_UNSUPPORTED: [ + "The request content type is not supported.", + "Send a UTF-8 application/json request body.", + 415 + ], + API_BODY_TOO_LARGE: [ + "The request body is too large.", + "Reduce the request body and try again.", + 413 + ], + API_BODY_INVALID: [ + "The request body is invalid.", + "Send only the documented JSON fields and try again.", + 400 + ] + }; + const [message, action, status] = contracts[code] ?? contracts.API_BODY_INVALID; + return apiError(code, message, action, status); +} + +function isPlainObject(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) return false; + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function exactObject(value, { allowed, required = [] }) { + if (!isPlainObject(value) + || Object.keys(value).some((field) => !allowed.includes(field)) + || required.some((field) => !Object.hasOwn(value, field))) { + throw bodyError("API_BODY_INVALID"); + } + return value; +} + +function assertJsonContentType(contentType) { + if (typeof contentType !== "string") throw bodyError("API_CONTENT_TYPE_UNSUPPORTED"); + const [mediaType, ...parameters] = contentType.split(";").map((part) => part.trim().toLowerCase()); + if (mediaType !== "application/json" + || parameters.some((parameter) => parameter !== "charset=utf-8")) { + throw bodyError("API_CONTENT_TYPE_UNSUPPORTED"); + } +} + +async function readJsonBody(request, maxBodyBytes) { + assertJsonContentType(request.headers["content-type"]); + const declaredLength = Number(request.headers["content-length"]); + if (Number.isFinite(declaredLength) && declaredLength > maxBodyBytes) { + request.resume(); + throw bodyError("API_BODY_TOO_LARGE"); + } + const chunks = []; + let length = 0; + let tooLarge = false; + for await (const chunk of request) { + length += chunk.length; + if (length > maxBodyBytes) { + tooLarge = true; + chunks.length = 0; + continue; + } + if (!tooLarge) chunks.push(chunk); + } + if (tooLarge) throw bodyError("API_BODY_TOO_LARGE"); + if (length === 0) throw bodyError("API_BODY_INVALID"); + try { + return JSON.parse(Buffer.concat(chunks, length).toString("utf8")); + } catch { + throw bodyError("API_BODY_INVALID"); + } +} + +async function requireEmptyBody(request, maxBodyBytes) { + let length = 0; + for await (const chunk of request) { + length += chunk.length; + if (length > maxBodyBytes) throw bodyError("API_BODY_TOO_LARGE"); + } + if (length !== 0) throw bodyError("API_BODY_INVALID"); +} + +function setSafeHeaders(response) { + response.setHeader("cache-control", "no-store"); + response.setHeader("x-content-type-options", "nosniff"); + response.setHeader("referrer-policy", "no-referrer"); + response.setHeader("content-security-policy", "default-src 'self'; frame-ancestors 'none'; base-uri 'none'"); +} + +function sendJson(response, status, payload, extraHeaders = {}) { + const bytes = Buffer.from(`${JSON.stringify(payload)}\n`, "utf8"); + response.writeHead(status, { + "content-type": "application/json; charset=utf-8", + "content-length": bytes.length, + ...extraHeaders + }); + response.end(bytes); +} + +function sendBytes(response, status, bytes, contentType, { head = false } = {}) { + response.writeHead(status, { + "content-type": contentType, + "content-length": bytes.length + }); + response.end(head ? undefined : bytes); +} + +function pick(source, fields) { + const result = {}; + for (const field of fields) { + if (Object.hasOwn(source ?? {}, field)) result[field] = structuredClone(source[field]); + } + return result; +} + +function projectProvider(provider) { + return provider === null ? null : pick(provider, PUBLIC_PROVIDER_FIELDS); +} + +function projectWorker(worker) { + if (worker === null || typeof worker !== "object") return null; + const projected = pick(worker, PUBLIC_WORKER_FIELDS); + projected.state = worker.state === null ? null : pick(worker.state, PUBLIC_CHILD_STATE_FIELDS); + if (worker.error !== null && typeof worker.error === "object") { + projected.error = pick(worker.error, ["code", "message"]); + } + return projected; +} + +function projectProviderStatus(status) { + return { + activeProviderId: status?.activeProviderId ?? null, + activeProvider: projectProvider(status?.activeProvider ?? null), + generation: Number.isSafeInteger(status?.generation) ? status.generation : 0, + worker: projectWorker(status?.worker ?? null) + }; +} + +function projectTestResult(result) { + return { + ok: result?.ok === true, + code: typeof result?.code === "string" ? result.code : null + }; +} + +function projectActivation(result) { + return { + activeProviderId: typeof result?.activeProviderId === "string" + ? result.activeProviderId + : null, + activeProvider: projectProvider(result?.activeProvider ?? null), + generation: Number.isSafeInteger(result?.generation) ? result.generation : 0, + worker: projectWorker(result?.worker ?? null) + }; +} + +function projectActivityEvent(event) { + return { + timestamp: typeof event?.timestamp === "string" ? event.timestamp : null, + category: typeof event?.category === "string" ? event.category : null, + action: typeof event?.action === "string" ? event.action : null, + providerId: typeof event?.providerId === "string" ? event.providerId : null, + result: typeof event?.result === "string" ? event.result : null, + errorCode: typeof event?.errorCode === "string" ? event.errorCode : null, + details: sanitizeActivityValue(event?.details ?? {}) + }; +} + +function projectSettings(settings) { + return { + proxyHost: typeof settings?.proxyHost === "string" ? settings.proxyHost : null, + proxyPort: Number.isInteger(settings?.proxyPort) ? settings.proxyPort : null, + adminHost: typeof settings?.adminHost === "string" ? settings.adminHost : null, + adminPort: Number.isInteger(settings?.adminPort) ? settings.adminPort : null, + captureEnabled: settings?.captureEnabled === true, + credentialBackend: typeof settings?.credentialBackend === "string" + ? settings.credentialBackend + : null + }; +} + +function projectBootstrap(result) { + return { + changed: result?.changed === true, + backupCreated: typeof result?.backupPath === "string" && result.backupPath.length > 0 + }; +} + +function projectDiagnostics(result) { + return { + created: result?.created === true, + generatedAt: typeof result?.generatedAt === "string" ? result.generatedAt : null, + eventCount: Number.isSafeInteger(result?.eventCount) ? result.eventCount : null + }; +} + +function projectSupervisorState(state) { + return { + pid: Number.isSafeInteger(state?.pid) ? state.pid : null, + startedAt: typeof state?.startedAt === "string" ? state.startedAt : null + }; +} + +function projectCodexState(state) { + return { + configured: state?.configured === true, + modelProvider: typeof state?.modelProvider === "string" ? state.modelProvider : null, + proxyUrl: typeof state?.proxyUrl === "string" ? state.proxyUrl : null + }; +} + +function sanitizePublicError(error, requestId) { + const payload = toPublicError(error, requestId); + const sanitized = sanitizeActivityValue(payload.error.details); + const details = {}; + for (const [key, value] of Object.entries(sanitized)) { + if (SAFE_ERROR_DETAIL_FIELDS.has(key) || value === "[REDACTED]") details[key] = value; + } + payload.error.details = details; + return payload; +} + +function currentAddress(server, host, configuredPort) { + const address = server.address(); + const port = typeof address === "object" && address ? address.port : configuredPort; + return { + host, + port, + authority: `${host}:${port}`, + origin: `http://${host}:${port}` + }; +} + +function parseProviderRoute(pathname) { + const prefix = `${API_PREFIX}/providers/`; + if (!pathname.startsWith(prefix)) return null; + const rawParts = pathname.slice(prefix.length).split("/"); + if (rawParts.length < 1 || rawParts.length > 2 || rawParts.some((part) => part.length === 0)) { + return null; + } + let id; + try { + id = decodeURIComponent(rawParts[0]); + } catch { + return null; + } + if (id.length === 0 || id.length > 128 || /[\\/\u0000-\u001f\u007f]/.test(id)) return null; + const action = rawParts[1] ?? null; + if (action !== null && action !== "test" && action !== "activate") return null; + return { id, action }; +} + +function providerNotFound() { + return new CrpError( + "PROVIDER_NOT_FOUND", + "The provider does not exist.", + "Refresh the provider list and try again.", + { status: 404 } + ); +} + +function allowedMethods(pathname) { + const exact = new Map([ + [`${API_PREFIX}/session`, ["POST"]], + [`${API_PREFIX}/status`, ["GET"]], + [`${API_PREFIX}/providers`, ["GET", "POST"]], + [`${API_PREFIX}/proxy/start`, ["POST"]], + [`${API_PREFIX}/proxy/stop`, ["POST"]], + [`${API_PREFIX}/proxy/restart`, ["POST"]], + [`${API_PREFIX}/activity`, ["GET"]], + [`${API_PREFIX}/settings`, ["GET", "PATCH"]], + [`${API_PREFIX}/codex/bootstrap`, ["POST"]], + [`${API_PREFIX}/diagnostics/export`, ["POST"]] + ]); + if (exact.has(pathname)) return exact.get(pathname); + const providerRoute = parseProviderRoute(pathname); + if (!providerRoute) return null; + return providerRoute.action === null ? ["GET", "PATCH", "DELETE"] : ["POST"]; +} + +function positiveQueryInteger(url, name, fallback, { min = 0, max }) { + const values = url.searchParams.getAll(name); + if (values.length === 0) return fallback; + if (values.length !== 1 || !/^\d+$/.test(values[0])) { + throw bodyError("API_BODY_INVALID"); + } + const value = Number(values[0]); + if (!Number.isSafeInteger(value) || value < min || value > max) { + throw bodyError("API_BODY_INVALID"); + } + return value; +} + +function uiAsset(pathname) { + let decoded; + try { + decoded = decodeURIComponent(pathname); + } catch { + return null; + } + if (/[\\\u0000-\u001f\u007f]/.test(decoded) + || decoded.split("/").includes("..")) { + return null; + } + const explicit = new Map([ + ["/", ["index.html", "text/html; charset=utf-8"]], + ["/index.html", ["index.html", "text/html; charset=utf-8"]], + ["/styles.css", ["styles.css", "text/css; charset=utf-8"]], + ["/app.js", ["app.js", "text/javascript; charset=utf-8"]] + ]); + if (explicit.has(decoded)) return explicit.get(decoded); + if (extname(decoded) === "") return explicit.get("/"); + return null; +} + +export function createAdminServer({ + auth, + providerService, + activityStore, + settingsService, + codexService, + diagnosticsService, + getSupervisorState = () => ({ pid: process.pid, startedAt: null }), + uiDir, + host = "127.0.0.1", + port = 15101, + maxBodyBytes = 64 * 1_024, + createRequestId = () => randomBytes(12).toString("base64url") +} = {}) { + if (host !== "127.0.0.1" || !Number.isInteger(port) || port < 0 || port > 65_535 + || !Number.isSafeInteger(maxBodyBytes) || maxBodyBytes < 1 + || !auth || !providerService) { + throw new TypeError("Admin server options are invalid."); + } + + const server = http.createServer((request, response) => { + setSafeHeaders(response); + const requestId = createRequestId(); + response.setHeader("x-request-id", requestId); + void handleRequest(request, response, requestId).catch((error) => { + if (response.headersSent) { + response.destroy(); + return; + } + if (error?.clearCookie === true) response.setHeader("set-cookie", auth.clearCookie()); + const status = error instanceof CrpError ? error.status : 500; + sendJson(response, status, sanitizePublicError(error, requestId)); + }); + }); + + async function handleRequest(request, response, requestId) { + const address = currentAddress(server, host, port); + if (request.headers.host !== address.authority) { + throw apiError( + "API_HOST_INVALID", + "The local request host is invalid.", + "Open CRP through its configured loopback address.", + 403 + ); + } + const origin = request.headers.origin; + if (origin !== undefined && origin !== address.origin) { + throw apiError( + "API_ORIGIN_INVALID", + "The local request origin is invalid.", + "Open CRP through its configured loopback address.", + 403 + ); + } + if (request.method === "OPTIONS") { + throw apiError( + "API_CORS_FORBIDDEN", + "Cross-origin requests are not allowed.", + "Use the bundled local CRP UI or CLI.", + 403 + ); + } + + const url = new URL(request.url, address.origin); + if (url.pathname === `${API_PREFIX}/session`) { + if (request.method !== "POST") { + throw apiError( + "API_METHOD_NOT_ALLOWED", + "The API method is not allowed.", + "Use the documented method for this endpoint.", + 405 + ); + } + await requireEmptyBody(request, maxBodyBytes); + const session = auth.createBrowserSession(request.headers.authorization); + sendJson(response, 200, { + csrfToken: session.csrfToken, + expiresAt: session.expiresAt + }, { "set-cookie": session.setCookie }); + return; + } + + const apiNamespace = url.pathname === "/api" + || url.pathname.startsWith("/api/"); + if (!apiNamespace) { + const asset = uiAsset(url.pathname); + if (!asset) { + throw apiError( + "UI_NOT_FOUND", + "The local UI resource was not found.", + "Open the CRP UI root.", + 404 + ); + } + if (request.method !== "GET" && request.method !== "HEAD") { + response.setHeader("allow", "GET, HEAD"); + throw apiError( + "API_METHOD_NOT_ALLOWED", + "The API method is not allowed.", + "Use GET to load local UI resources.", + 405 + ); + } + if (typeof uiDir !== "string" || uiDir.length === 0) { + throw apiError( + "UI_NOT_FOUND", + "The local UI resource was not found.", + "Install the bundled CRP UI files and try again.", + 404 + ); + } + let bytes; + try { + bytes = await readFile(join(uiDir, asset[0])); + } catch { + throw apiError( + "UI_NOT_FOUND", + "The local UI resource was not found.", + "Install the bundled CRP UI files and try again.", + 404 + ); + } + sendBytes(response, 200, bytes, asset[1], { head: request.method === "HEAD" }); + return; + } + auth.authorize({ + authorization: request.headers.authorization, + cookie: request.headers.cookie, + csrfToken: request.headers["x-crp-csrf"], + mutation: request.method !== "GET" && request.method !== "HEAD" + }); + + if (url.pathname === `${API_PREFIX}/status` && request.method === "GET") { + const [providerStatus, codexStatus] = await Promise.all([ + providerService.getStatus(), + codexService?.getStatus?.() ?? { configured: false } + ]); + sendJson(response, 200, { + supervisor: projectSupervisorState(getSupervisorState()), + ...projectProviderStatus(providerStatus), + codex: projectCodexState(codexStatus) + }); + return; + } + if (url.pathname === `${API_PREFIX}/providers` && request.method === "GET") { + const providers = await providerService.listProviders(); + sendJson(response, 200, { providers: providers.map(projectProvider) }); + return; + } + if (url.pathname === `${API_PREFIX}/providers` && request.method === "POST") { + const body = exactObject(await readJsonBody(request, maxBodyBytes), { + allowed: ["provider", "credential", "fallbackConsent"], + required: ["provider", "credential"] + }); + if (!isPlainObject(body.provider) + || typeof body.credential !== "string" || body.credential.length === 0 + || (body.fallbackConsent !== undefined && typeof body.fallbackConsent !== "boolean")) { + throw bodyError("API_BODY_INVALID"); + } + const provider = await providerService.createProvider( + body.provider, + body.credential, + { fallbackConsent: body.fallbackConsent === true } + ); + sendJson(response, 201, { provider: projectProvider(provider) }); + return; + } + const providerRoute = parseProviderRoute(url.pathname); + if (providerRoute?.action === null && request.method === "GET") { + const providers = await providerService.listProviders(); + const provider = providers.find((candidate) => candidate.id === providerRoute.id); + if (!provider) throw providerNotFound(); + sendJson(response, 200, { provider: projectProvider(provider) }); + return; + } + if (providerRoute?.action === null && request.method === "PATCH") { + const body = exactObject(await readJsonBody(request, maxBodyBytes), { + allowed: ["patch", "replacementCredential"], + required: ["patch"] + }); + if (!isPlainObject(body.patch) + || (body.replacementCredential !== undefined + && (typeof body.replacementCredential !== "string" + || body.replacementCredential.length === 0))) { + throw bodyError("API_BODY_INVALID"); + } + const provider = await providerService.updateProvider( + providerRoute.id, + body.patch, + body.replacementCredential + ); + sendJson(response, 200, { provider: projectProvider(provider) }); + return; + } + if (providerRoute?.action === null && request.method === "DELETE") { + await requireEmptyBody(request, maxBodyBytes); + const provider = await providerService.deleteProvider(providerRoute.id); + sendJson(response, 200, { provider: projectProvider(provider) }); + return; + } + if (providerRoute?.action === "test" && request.method === "POST") { + const body = exactObject(await readJsonBody(request, maxBodyBytes), { + allowed: ["model"], + required: ["model"] + }); + if (typeof body.model !== "string" || body.model.trim().length === 0) { + throw bodyError("API_BODY_INVALID"); + } + const result = await providerService.testProvider(providerRoute.id, body.model); + sendJson(response, 200, { result: projectTestResult(result) }); + return; + } + if (providerRoute?.action === "activate" && request.method === "POST") { + await requireEmptyBody(request, maxBodyBytes); + const activation = await providerService.activate(providerRoute.id); + sendJson(response, 200, { activation: projectActivation(activation) }); + return; + } + if (url.pathname === `${API_PREFIX}/proxy/start` && request.method === "POST") { + await requireEmptyBody(request, maxBodyBytes); + const worker = await providerService.startProxy(); + sendJson(response, 200, { worker: projectWorker(worker) }); + return; + } + if (url.pathname === `${API_PREFIX}/proxy/stop` && request.method === "POST") { + await requireEmptyBody(request, maxBodyBytes); + const worker = await providerService.stopProxy(); + sendJson(response, 200, { worker: projectWorker(worker) }); + return; + } + if (url.pathname === `${API_PREFIX}/proxy/restart` && request.method === "POST") { + await requireEmptyBody(request, maxBodyBytes); + const worker = await providerService.restartProxy(); + sendJson(response, 200, { worker: projectWorker(worker) }); + return; + } + if (url.pathname === `${API_PREFIX}/activity` && request.method === "GET") { + for (const key of url.searchParams.keys()) { + if (key !== "limit" && key !== "offset") throw bodyError("API_BODY_INVALID"); + } + const limit = positiveQueryInteger(url, "limit", 50, { min: 1, max: 100 }); + const offset = positiveQueryInteger(url, "offset", 0, { min: 0, max: 9_999 }); + const events = activityStore.list({ limit: Math.min(10_000, offset + limit + 1) }); + const page = events.slice(offset, offset + limit).map(projectActivityEvent); + sendJson(response, 200, { + events: page, + page: { + limit, + offset, + nextOffset: events.length > offset + limit ? offset + limit : null + } + }); + return; + } + if (url.pathname === `${API_PREFIX}/settings` && request.method === "GET") { + sendJson(response, 200, { settings: projectSettings(await settingsService.getSettings()) }); + return; + } + if (url.pathname === `${API_PREFIX}/settings` && request.method === "PATCH") { + const patch = exactObject(await readJsonBody(request, maxBodyBytes), { + allowed: ["captureEnabled"] + }); + if (Object.keys(patch).length === 0 || typeof patch.captureEnabled !== "boolean") { + throw bodyError("API_BODY_INVALID"); + } + throw new CrpError( + "SETTINGS_READ_ONLY", + "Local settings are read-only in this version.", + "Keep the fixed proxy settings and use a supported provider operation.", + { status: 409 } + ); + } + if (url.pathname === `${API_PREFIX}/codex/bootstrap` && request.method === "POST") { + await requireEmptyBody(request, maxBodyBytes); + const result = await codexService.bootstrap(); + sendJson(response, 200, { result: projectBootstrap(result) }); + return; + } + if (url.pathname === `${API_PREFIX}/diagnostics/export` && request.method === "POST") { + await requireEmptyBody(request, maxBodyBytes); + const result = await diagnosticsService.exportDiagnostics(); + sendJson(response, 200, { diagnostics: projectDiagnostics(result) }); + return; + } + const methods = allowedMethods(url.pathname); + if (methods) { + response.setHeader("allow", methods.join(", ")); + throw apiError( + "API_METHOD_NOT_ALLOWED", + "The API method is not allowed.", + "Use the documented method for this endpoint.", + 405 + ); + } + throw apiError( + "API_NOT_FOUND", + "The API endpoint was not found.", + "Use a documented local API endpoint.", + 404 + ); + } + + return { + server, + async listen() { + if (!server.listening) { + await new Promise((resolvePromise, rejectPromise) => { + server.once("error", rejectPromise); + server.listen(port, host, () => { + server.off("error", rejectPromise); + resolvePromise(); + }); + }); + } + return currentAddress(server, host, port); + }, + async close() { + if (!server.listening) return; + await new Promise((resolvePromise, rejectPromise) => { + server.close((error) => { + if (error) rejectPromise(error); + else resolvePromise(); + }); + }); + } + }; +} diff --git a/node/src/supervisor/provider-service.mjs b/node/src/supervisor/provider-service.mjs index 5e73111..bbbb1c0 100644 --- a/node/src/supervisor/provider-service.mjs +++ b/node/src/supervisor/provider-service.mjs @@ -64,6 +64,22 @@ function serviceError(code, { status = 500, cause, details = {} } = {}) { PROVIDER_ACTIVATION_COMMITTED_DEGRADED: [ "The provider was activated, but its persistence cleanup degraded.", "Stop CRP and repair the residual active-provider state before restarting." + ], + PROXY_NOT_CONFIGURED: [ + "No active provider is configured for the proxy.", + "Test and activate a provider before starting the proxy." + ], + PROXY_START_FAILED: [ + "The proxy worker could not be started.", + "Review worker health and try again." + ], + PROXY_STOP_FAILED: [ + "The proxy worker could not be stopped.", + "Review worker health and try again." + ], + PROXY_RESTART_FAILED: [ + "The proxy worker could not be restarted.", + "Review worker health and try again." ] }; const [message, action] = contracts[code] ?? [ @@ -92,6 +108,7 @@ function committedServiceError(action, cause) { export class ProviderService { #operationTail = Promise.resolve(); + #lifecycleOperation = null; constructor({ registry, @@ -490,6 +507,37 @@ export class ProviderService { }); } + startProxy() { + return this.#runLifecycleOperation(async () => { + const current = this.workerManager.getPublicState(); + if (current?.phase === "running") { + const worker = publicWorkerState(current); + await this.#recordProxy("start", "success", null, { alreadyRunning: true }); + return worker; + } + return await this.#startOrRestartProxy("start"); + }); + } + + stopProxy() { + return this.#runLifecycleOperation(async () => { + try { + const state = await this.workerManager.stop(); + const worker = publicWorkerState(state); + await this.#recordProxy("stop", "success", null, {}); + return worker; + } catch (error) { + const failure = serviceError("PROXY_STOP_FAILED", { cause: error }); + await this.#safeRecordProxyFailure("stop", failure); + throw failure; + } + }); + } + + restartProxy() { + return this.#runLifecycleOperation(() => this.#startOrRestartProxy("restart")); + } + async getStatus() { const document = this.registry.getDocument(); const activeProfile = document.activeProviderId === null @@ -562,6 +610,64 @@ export class ProviderService { return toPublicProvider(profile, configured); } + async #startOrRestartProxy(action) { + let workerAttempted = false; + let generation = null; + try { + const document = this.registry.getDocument(); + if (document.activeProviderId === null) { + throw serviceError("PROXY_NOT_CONFIGURED", { status: 409 }); + } + const profile = document.providers.find(({ id }) => id === document.activeProviderId); + if (!profile || profile.lastTestStatus !== "passed") { + throw serviceError("PROXY_NOT_CONFIGURED", { status: 409 }); + } + const secret = await this.credentialStore.get(profile.credentialRef); + const observedGeneration = this.workerManager.getPublicState()?.generation; + generation = Math.max( + this.confirmedGeneration, + Number.isSafeInteger(observedGeneration) ? observedGeneration : 0 + ) + 1; + if (!Number.isSafeInteger(generation)) throw new Error("proxy generation is invalid"); + const snapshot = this.#buildSnapshot(profile, secret, generation); + workerAttempted = true; + const state = action === "restart" + ? await this.workerManager.restart(snapshot) + : await this.workerManager.start(snapshot); + if (!isConfirmedWorkerState(state, generation)) { + throw new Error("worker generation was not confirmed"); + } + if (await this.verifyWorkerHealth(generation, state) !== true) { + throw new Error("worker health was not confirmed"); + } + this.confirmedGeneration = generation; + this.confirmedSnapshot = structuredClone(snapshot); + const worker = publicWorkerState(state); + await this.#recordProxy(action, "success", null, { generation }); + return worker; + } catch (error) { + if (error instanceof CrpError && error.code === "PROXY_NOT_CONFIGURED") { + await this.#safeRecordProxyFailure(action, error); + throw error; + } + let cleanupFailure = null; + if (workerAttempted) { + try { + await this.workerManager.stop(); + } catch (caught) { + cleanupFailure = caught; + } + } + const code = action === "restart" ? "PROXY_RESTART_FAILED" : "PROXY_START_FAILED"; + const failure = serviceError(code, { + cause: cleanupFailure ?? error, + details: cleanupFailure ? { degraded: true } : {} + }); + await this.#safeRecordProxyFailure(action, failure, generation === null ? {} : { generation }); + throw failure; + } + } + #buildSnapshot(profile, secret, generation) { const document = this.registry.getDocument(); const runtimeConfigPath = this.paths.runtimeConfigPath; @@ -611,6 +717,30 @@ export class ProviderService { }); } + async #recordProxy(action, result, errorCode, details) { + await this.activityStore.append({ + category: "proxy", + action, + providerId: null, + result, + errorCode, + details + }); + } + + async #safeRecordProxyFailure(action, error, details = {}) { + try { + await this.#recordProxy( + action, + "failed", + stableErrorCode(error, `PROXY_${action.toUpperCase()}_FAILED`), + details + ); + } catch { + // Preserve the lifecycle error when the audit store is unavailable. + } + } + async #safeRecordFailure(action, providerId, error, details = {}) { const errorCode = stableErrorCode(error, `PROVIDER_${action.toUpperCase()}_FAILED`); try { @@ -624,6 +754,17 @@ export class ProviderService { await this.#record(action, providerId, "degraded", error.code, details); } + #runLifecycleOperation(operation) { + if (this.#lifecycleOperation) return this.#lifecycleOperation; + const run = this.#runExclusive(operation); + this.#lifecycleOperation = run; + const clear = () => { + if (this.#lifecycleOperation === run) this.#lifecycleOperation = null; + }; + void run.then(clear, clear); + return run; + } + #runExclusive(operation) { const run = this.#operationTail.then(operation, operation); this.#operationTail = run.catch(() => {}); diff --git a/node/src/supervisor/session-auth.mjs b/node/src/supervisor/session-auth.mjs new file mode 100644 index 0000000..bc2c0d2 --- /dev/null +++ b/node/src/supervisor/session-auth.mjs @@ -0,0 +1,283 @@ +import { randomBytes as cryptoRandomBytes, timingSafeEqual } from "node:crypto"; +import { + chmodSync, + closeSync, + constants, + fchmodSync, + fstatSync, + fsyncSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + writeFileSync +} from "node:fs"; +import { dirname } from "node:path"; + +import { CrpError } from "../shared/errors.mjs"; + +export const SESSION_COOKIE_NAME = "crp_session"; + +const TOKEN_BYTES = 32; +const TOKEN_PATTERN = /^[A-Za-z0-9_-]{43}$/; +const DEFAULT_SESSION_TTL_MS = 8 * 60 * 60 * 1_000; +const DEFAULT_FILE_OPERATIONS = { + chmodSync, + closeSync, + fchmodSync, + fstatSync, + fsyncSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + writeFileSync +}; + +function authError(code, { clearCookie = false, cause } = {}) { + const contracts = { + AUTH_CONTROL_TOKEN_INVALID: [ + "The local control token is invalid.", + "Stop CRP, repair the private control token file, and restart CRP.", + 500 + ], + AUTH_REQUIRED: [ + "Local authentication is required.", + "Open the local CRP UI again or provide the local control token.", + 401 + ], + AUTH_SESSION_EXPIRED: [ + "The local session has expired.", + "Open the local CRP UI again to create a new session.", + 401 + ], + AUTH_CSRF_INVALID: [ + "The request could not be verified.", + "Refresh the local CRP UI and try again.", + 403 + ] + }; + const [message, action, status] = contracts[code] ?? contracts.AUTH_REQUIRED; + const error = new CrpError(code, message, action, { status, cause }); + if (clearCookie) { + Object.defineProperty(error, "clearCookie", { value: true, enumerable: false }); + } + return error; +} + +function sameIdentity(left, right) { + return left.dev === right.dev && left.ino === right.ino; +} + +function isPrivateMode(stats, platform) { + return platform === "win32" || (stats.mode & 0o777) === 0o600; +} + +function canonicalToken(bytes) { + if (!Buffer.isBuffer(bytes) && !(bytes instanceof Uint8Array)) { + throw new TypeError("randomBytes must return bytes"); + } + const buffer = Buffer.from(bytes); + if (buffer.length !== TOKEN_BYTES) throw new TypeError("token byte count is invalid"); + return buffer.toString("base64url"); +} + +function parseStoredToken(text) { + if (typeof text !== "string") throw new TypeError("control token bytes are invalid"); + const token = text.endsWith("\n") ? text.slice(0, -1) : text; + if (!TOKEN_PATTERN.test(token) || text !== token && text !== `${token}\n`) { + throw new TypeError("control token format is invalid"); + } + const bytes = Buffer.from(token, "base64url"); + if (bytes.length !== TOKEN_BYTES || bytes.toString("base64url") !== token) { + throw new TypeError("control token encoding is invalid"); + } + return token; +} + +function secureEqual(left, right) { + if (typeof left !== "string" || typeof right !== "string") return false; + const leftBytes = Buffer.from(left, "utf8"); + const rightBytes = Buffer.from(right, "utf8"); + return leftBytes.length === rightBytes.length && timingSafeEqual(leftBytes, rightBytes); +} + +function ensurePrivateParent(path, fileOperations, platform) { + const parent = dirname(path); + fileOperations.mkdirSync(parent, { recursive: true, mode: 0o700 }); + let stats = fileOperations.lstatSync(parent); + if (!stats.isDirectory() || stats.isSymbolicLink()) { + throw new TypeError("control token parent is unsafe"); + } + if (platform !== "win32") { + fileOperations.chmodSync(parent, 0o700); + stats = fileOperations.lstatSync(parent); + if ((stats.mode & 0o777) !== 0o700) throw new TypeError("control token parent is not private"); + } +} + +function readExistingToken(path, fileOperations, platform) { + const before = fileOperations.lstatSync(path); + if (!before.isFile() || before.isSymbolicLink() || !isPrivateMode(before, platform)) { + throw new TypeError("control token path is unsafe"); + } + if (before.size < 43 || before.size > 44) throw new TypeError("control token size is invalid"); + + let descriptor; + try { + const noFollow = typeof constants.O_NOFOLLOW === "number" ? constants.O_NOFOLLOW : 0; + descriptor = fileOperations.openSync(path, constants.O_RDONLY | noFollow); + const opened = fileOperations.fstatSync(descriptor); + if (!opened.isFile() || !sameIdentity(before, opened) || !isPrivateMode(opened, platform)) { + throw new TypeError("control token identity changed"); + } + const text = fileOperations.readFileSync(descriptor, "utf8"); + const after = fileOperations.lstatSync(path); + if (!after.isFile() || after.isSymbolicLink() || !sameIdentity(opened, after)) { + throw new TypeError("control token identity changed"); + } + return parseStoredToken(text); + } finally { + if (descriptor !== undefined) fileOperations.closeSync(descriptor); + } +} + +function createTokenFile(path, token, fileOperations) { + let descriptor; + try { + descriptor = fileOperations.openSync(path, "wx", 0o600); + fileOperations.writeFileSync(descriptor, `${token}\n`, "utf8"); + fileOperations.fsyncSync(descriptor); + fileOperations.fchmodSync(descriptor, 0o600); + } finally { + if (descriptor !== undefined) fileOperations.closeSync(descriptor); + } +} + +function loadOrCreateControlToken({ + path, + randomBytes, + fileOperations, + platform +}) { + try { + ensurePrivateParent(path, fileOperations, platform); + try { + return readExistingToken(path, fileOperations, platform); + } catch (error) { + if (error?.code !== "ENOENT") throw error; + } + const token = canonicalToken(randomBytes(TOKEN_BYTES)); + try { + createTokenFile(path, token, fileOperations); + return token; + } catch (error) { + if (error?.code !== "EEXIST") throw error; + return readExistingToken(path, fileOperations, platform); + } + } catch (error) { + if (error instanceof CrpError) throw error; + throw authError("AUTH_CONTROL_TOKEN_INVALID", { cause: error }); + } +} + +function bearerToken(authorization) { + if (typeof authorization !== "string") return null; + const match = /^Bearer ([A-Za-z0-9_-]+)$/.exec(authorization); + return match?.[1] ?? null; +} + +function sessionCookie(cookie) { + if (typeof cookie !== "string") return null; + const matches = []; + for (const part of cookie.split(";")) { + const separator = part.indexOf("="); + if (separator === -1) continue; + const name = part.slice(0, separator).trim(); + if (name !== SESSION_COOKIE_NAME) continue; + matches.push(part.slice(separator + 1).trim()); + } + if (matches.length !== 1 || !TOKEN_PATTERN.test(matches[0])) return null; + return matches[0]; +} + +export class SessionAuth { + #controlToken; + #randomBytes; + #now; + #sessionTtlMs; + #sessions = new Map(); + + constructor({ + controlTokenPath, + randomBytes = cryptoRandomBytes, + now = () => Date.now(), + sessionTtlMs = DEFAULT_SESSION_TTL_MS, + fileOperations = DEFAULT_FILE_OPERATIONS, + platform = process.platform + } = {}) { + if (typeof controlTokenPath !== "string" || controlTokenPath.length === 0 + || !Number.isSafeInteger(sessionTtlMs) || sessionTtlMs < 1) { + throw authError("AUTH_CONTROL_TOKEN_INVALID"); + } + this.#randomBytes = randomBytes; + this.#now = now; + this.#sessionTtlMs = sessionTtlMs; + this.#controlToken = loadOrCreateControlToken({ + path: controlTokenPath, + randomBytes, + fileOperations, + platform + }); + } + + createBrowserSession(authorization) { + const token = bearerToken(authorization); + if (!secureEqual(token, this.#controlToken)) throw authError("AUTH_REQUIRED"); + this.#purgeExpired(); + const sessionId = canonicalToken(this.#randomBytes(TOKEN_BYTES)); + const csrfToken = canonicalToken(this.#randomBytes(TOKEN_BYTES)); + const expiresAtMs = this.#now() + this.#sessionTtlMs; + this.#sessions.set(sessionId, { csrfToken, expiresAtMs }); + return { + csrfToken, + expiresAt: new Date(expiresAtMs).toISOString(), + setCookie: `${SESSION_COOKIE_NAME}=${sessionId}; Path=/; HttpOnly; SameSite=Strict; Max-Age=${Math.ceil(this.#sessionTtlMs / 1_000)}` + }; + } + + authorize({ authorization, cookie, csrfToken, mutation = false } = {}) { + if (authorization !== undefined && authorization !== null) { + const token = bearerToken(authorization); + if (!secureEqual(token, this.#controlToken)) throw authError("AUTH_REQUIRED"); + return { kind: "cli" }; + } + + const sessionId = sessionCookie(cookie); + const session = sessionId === null ? null : this.#sessions.get(sessionId); + if (!session) throw authError("AUTH_REQUIRED"); + if (this.#now() >= session.expiresAtMs) { + this.#sessions.delete(sessionId); + throw authError("AUTH_SESSION_EXPIRED", { clearCookie: true }); + } + if (mutation && !secureEqual(csrfToken, session.csrfToken)) { + throw authError("AUTH_CSRF_INVALID"); + } + return { kind: "browser" }; + } + + clearCookie() { + return `${SESSION_COOKIE_NAME}=; Path=/; HttpOnly; SameSite=Strict; Max-Age=0`; + } + + close() { + this.#sessions.clear(); + } + + #purgeExpired() { + const now = this.#now(); + for (const [sessionId, session] of this.#sessions) { + if (now >= session.expiresAtMs) this.#sessions.delete(sessionId); + } + } +} diff --git a/node/src/supervisor/supervisor-entry.mjs b/node/src/supervisor/supervisor-entry.mjs new file mode 100644 index 0000000..943892b --- /dev/null +++ b/node/src/supervisor/supervisor-entry.mjs @@ -0,0 +1,62 @@ +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { createSupervisor } from "./supervisor.mjs"; + +export function isDirectExecution(metaUrl = import.meta.url, argv1 = process.argv[1]) { + if (typeof argv1 !== "string" || argv1.length === 0) return false; + return resolve(fileURLToPath(metaUrl)) === resolve(argv1); +} + +export async function runSupervisor({ + processRef = process, + createSupervisorImpl = createSupervisor, + supervisorOptions = {} +} = {}) { + const supervisor = await createSupervisorImpl(supervisorOptions); + let shutdownPromise = null; + + const removeSignalHandlers = () => { + processRef.off("SIGTERM", onSignal); + processRef.off("SIGINT", onSignal); + }; + const shutdown = () => { + if (shutdownPromise) return shutdownPromise; + shutdownPromise = Promise.resolve(supervisor.close()) + .then( + () => { + removeSignalHandlers(); + processRef.exitCode = 0; + }, + () => { + removeSignalHandlers(); + processRef.exitCode = 1; + } + ); + return shutdownPromise; + }; + const onSignal = () => { + void shutdown(); + }; + + processRef.once("SIGTERM", onSignal); + processRef.once("SIGINT", onSignal); + try { + await supervisor.listen(); + return supervisor; + } catch (error) { + removeSignalHandlers(); + await supervisor.close().catch(() => {}); + throw error; + } +} + +if (isDirectExecution()) { + const home = typeof process.env.CRP_HOME === "string" && process.env.CRP_HOME.length > 0 + ? process.env.CRP_HOME + : undefined; + void runSupervisor({ supervisorOptions: { home } }).catch(() => { + process.stderr.write("CRP supervisor failed to start.\n"); + process.exitCode = 1; + }); +} diff --git a/node/src/supervisor/supervisor.mjs b/node/src/supervisor/supervisor.mjs new file mode 100644 index 0000000..7ffeaff --- /dev/null +++ b/node/src/supervisor/supervisor.mjs @@ -0,0 +1,349 @@ +import { randomUUID } from "node:crypto"; +import { + chmodSync, + closeSync, + constants, + existsSync, + fchmodSync, + fsyncSync, + linkSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +} from "node:fs"; +import { fileURLToPath } from "node:url"; +import { basename, dirname, resolve } from "node:path"; + +import { bootstrapCodexConfig, patchCodexConfigText } from "../codex/codex-config.mjs"; +import { createCredentialStore } from "../credentials/credential-store.mjs"; +import { ProviderRegistry } from "../providers/provider-registry.mjs"; +import { getPaths } from "../shared/paths.mjs"; +import { ActivityStore } from "./activity-store.mjs"; +import { createAdminServer } from "./admin-server.mjs"; +import { migrateLegacyConfiguration } from "./migration.mjs"; +import { ProviderService } from "./provider-service.mjs"; +import { SessionAuth } from "./session-auth.mjs"; +import { WorkerManager } from "./worker-manager.mjs"; + +const DEFAULT_UI_DIR = fileURLToPath(new URL("../../ui", import.meta.url)); +const DEFAULT_FILE_OPERATIONS = { + chmodSync, + closeSync, + existsSync, + fchmodSync, + fsyncSync, + linkSync, + lstatSync, + mkdirSync, + openSync, + readFileSync, + renameSync, + rmSync, + writeFileSync +}; + +function completePaths(home, provided) { + const base = provided ?? getPaths(home); + return { + ...base, + runtimeConfigPath: base.runtimeConfigPath + ?? resolve(base.globalHome, "node", "proxy-config.json"), + capturePath: base.capturePath ?? resolve(base.globalHome, "traffic.sqlite3") + }; +} + +function publicChildState(state) { + if (state === null || typeof state !== "object") return null; + return { + phase: state.phase, + configured: state.configured, + generation: state.generation, + listening: state.listening, + listenHost: state.listenHost, + listenPort: state.listenPort, + inFlight: state.inFlight + }; +} + +function publicWorkerState(state) { + if (state === null || typeof state !== "object") return null; + return { + phase: state.phase, + pid: state.pid, + generation: state.generation, + state: publicChildState(state.state), + restartCount: state.restartCount, + startedAt: state.startedAt, + error: state.error === null || typeof state.error !== "object" + ? null + : { code: state.error.code, message: state.error.message } + }; +} + +function createCodexService({ + paths, + proxyUrl, + fileOperations, + bootstrapImpl +}) { + return { + bootstrap() { + const options = { + configPath: paths.codexConfigPath, + proxyUrl + }; + if (fileOperations !== undefined) options.fileOperations = fileOperations; + return bootstrapImpl(options); + }, + getStatus() { + let configured = false; + const operations = fileOperations ?? DEFAULT_FILE_OPERATIONS; + try { + if (operations.existsSync(paths.codexConfigPath)) { + const text = operations.readFileSync(paths.codexConfigPath, "utf8"); + configured = patchCodexConfigText(text, proxyUrl) === text; + } + } catch { + configured = false; + } + return { configured, modelProvider: "OpenAI", proxyUrl }; + } + }; +} + +function createSettingsService({ registry, credentialStore }) { + return { + getSettings() { + return { + ...registry.getDocument().settings, + credentialBackend: credentialStore.backend ?? null + }; + } + }; +} + +function createDiagnosticsService({ activityStore, now }) { + return { + exportDiagnostics() { + return { + created: true, + generatedAt: now(), + eventCount: activityStore.list({ limit: 10_000 }).length + }; + } + }; +} + +function writeState(path, document, fileOperations, createId) { + const parent = dirname(path); + fileOperations.mkdirSync(parent, { recursive: true, mode: 0o700 }); + try { + fileOperations.chmodSync(parent, 0o700); + } catch { + // Windows ACL validation remains an L3 platform gate. + } + const bytes = Buffer.from(`${JSON.stringify(document, null, 2)}\n`, "utf8"); + const tempPath = resolve(parent, `.${basename(path)}.${createId()}.tmp`); + let descriptor; + try { + descriptor = fileOperations.openSync(tempPath, "wx", 0o600); + fileOperations.writeFileSync(descriptor, bytes); + fileOperations.fsyncSync(descriptor); + fileOperations.fchmodSync(descriptor, 0o600); + fileOperations.closeSync(descriptor); + descriptor = undefined; + fileOperations.renameSync(tempPath, path); + const identity = fileOperations.lstatSync(path); + return { bytes, identity }; + } catch (error) { + if (descriptor !== undefined) { + try { fileOperations.closeSync(descriptor); } catch {} + } + try { fileOperations.rmSync(tempPath, { force: true }); } catch {} + throw error; + } +} + +function sameIdentity(left, right) { + return left.dev === right.dev && left.ino === right.ino; +} + +function removeOwnedState(path, owned, fileOperations, createId) { + if (!owned) return; + let before; + try { + before = fileOperations.lstatSync(path); + } catch (error) { + if (error?.code === "ENOENT") return; + throw error; + } + if (!sameIdentity(before, owned.identity)) return; + const claimPath = resolve(dirname(path), `.${basename(path)}.${createId()}.claim`); + fileOperations.renameSync(path, claimPath); + const claimed = fileOperations.lstatSync(claimPath); + if (!sameIdentity(claimed, owned.identity)) { + try { + fileOperations.linkSync(claimPath, path); + fileOperations.rmSync(claimPath); + } catch (error) { + if (error?.code !== "EEXIST") throw error; + } + return; + } + const bytes = fileOperations.readFileSync(claimPath); + if (!bytes.equals(owned.bytes)) { + try { + fileOperations.linkSync(claimPath, path); + fileOperations.rmSync(claimPath); + } catch (error) { + if (error?.code !== "EEXIST") throw error; + } + return; + } + fileOperations.rmSync(claimPath); +} + +export async function createSupervisor({ + home, + paths: providedPaths, + pid = process.pid, + now = () => new Date().toISOString(), + backend = "native", + fallbackConsent = false, + uiDir = DEFAULT_UI_DIR, + stateFileOperations = DEFAULT_FILE_OPERATIONS, + codexFileOperations, + bootstrapCodex = bootstrapCodexConfig, + createStateId = randomUUID, + activityStoreFactory = (options) => new ActivityStore(options), + credentialStoreFactory = (options) => createCredentialStore(options), + migrate = migrateLegacyConfiguration, + registryFactory = (options) => new ProviderRegistry(options), + workerManagerFactory = (options) => new WorkerManager(options), + providerServiceFactory = (options) => new ProviderService(options), + authFactory = (options) => new SessionAuth(options), + adminServerFactory = (options) => createAdminServer(options) +} = {}) { + const paths = completePaths(home, providedPaths); + const startedAt = now(); + const activityStore = activityStoreFactory({ path: paths.activityPath, now }); + const credentialStore = credentialStoreFactory({ + backend, + fallbackConsent, + paths + }); + await migrate({ paths, credentialStore, activityStore, now }); + const registry = registryFactory({ path: paths.registryPath, now }); + const settings = registry.getDocument().settings; + let workerManager; + let providerService; + let auth; + let codexService; + let settingsService; + let diagnosticsService; + let adminServer; + try { + workerManager = workerManagerFactory({ + host: settings.proxyHost, + port: settings.proxyPort + }); + providerService = providerServiceFactory({ + registry, + credentialStore, + activityStore, + workerManager, + now, + paths + }); + auth = authFactory({ controlTokenPath: paths.controlTokenPath }); + const proxyUrl = `http://${settings.proxyHost}:${settings.proxyPort}`; + codexService = createCodexService({ + paths, + proxyUrl, + fileOperations: codexFileOperations, + bootstrapImpl: bootstrapCodex + }); + settingsService = createSettingsService({ registry, credentialStore }); + diagnosticsService = createDiagnosticsService({ activityStore, now }); + adminServer = adminServerFactory({ + auth, + providerService, + activityStore, + settingsService, + codexService, + diagnosticsService, + getSupervisorState: () => ({ pid, startedAt }), + uiDir, + host: settings.adminHost, + port: settings.adminPort + }); + } catch (error) { + try { await auth?.close?.(); } catch {} + try { await workerManager?.close?.(); } catch {} + throw error; + } + let address = null; + let ownedState = null; + let listenPromise = null; + let closePromise = null; + + const getPublicState = () => ({ + supervisorPid: pid, + startedAt, + admin: address, + worker: publicWorkerState(workerManager.getPublicState()) + }); + const close = () => { + if (closePromise) return closePromise; + closePromise = (async () => { + let firstError = null; + for (const operation of [ + () => adminServer.close(), + () => auth.close(), + () => workerManager.close(), + () => removeOwnedState(paths.statePath, ownedState, stateFileOperations, createStateId) + ]) { + try { + await operation(); + } catch (error) { + firstError ??= error; + } + } + if (firstError) throw firstError; + })(); + return closePromise; + }; + + return { + paths, + getPublicState, + listen() { + if (listenPromise) return listenPromise; + listenPromise = (async () => { + try { + address = await adminServer.listen(); + const state = { + schemaVersion: 1, + ...getPublicState() + }; + ownedState = writeState( + paths.statePath, + state, + stateFileOperations, + createStateId + ); + return { ...address }; + } catch (error) { + await close().catch(() => {}); + throw error; + } + })(); + return listenPromise; + }, + close + }; +} diff --git a/node/test/integration/admin-server.test.mjs b/node/test/integration/admin-server.test.mjs new file mode 100644 index 0000000..d9338e9 --- /dev/null +++ b/node/test/integration/admin-server.test.mjs @@ -0,0 +1,749 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { existsSync, mkdtempSync, mkdirSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs"; +import { EventEmitter } from "node:events"; +import http from "node:http"; +import os from "node:os"; +import { join } from "node:path"; + +import { createAdminServer } from "../../src/supervisor/admin-server.mjs"; +import { SessionAuth } from "../../src/supervisor/session-auth.mjs"; +import { createSupervisor } from "../../src/supervisor/supervisor.mjs"; +import { runSupervisor } from "../../src/supervisor/supervisor-entry.mjs"; +import { getPaths } from "../../src/shared/paths.mjs"; +import { CrpError } from "../../src/shared/errors.mjs"; + +const SECRET = "admin-api-complete-secret-sentinel"; +const CREDENTIAL_REF = "credential-ref-must-not-pass"; + +function publicProvider(overrides = {}) { + return { + id: "provider-1", + name: "Primary", + baseUrl: "https://provider.example/v1", + authHeader: "authorization", + authScheme: "Bearer", + extraHeaders: {}, + modelMode: "passthrough", + modelOverride: null, + lastTestAt: null, + lastTestStatus: "untested", + lastTestCode: null, + createdAt: "2026-07-13T00:00:00.000Z", + updatedAt: "2026-07-13T00:00:00.000Z", + credentialConfigured: true, + credentialRef: CREDENTIAL_REF, + apiKey: SECRET, + ...overrides + }; +} + +function workerState(overrides = {}) { + return { + phase: "running", + pid: 12345, + generation: 1, + state: { + phase: "running", + configured: true, + generation: 1, + listening: true, + listenHost: "127.0.0.1", + listenPort: 15100, + inFlight: 0, + apiKey: SECRET + }, + restartCount: 0, + startedAt: "2026-07-13T00:00:00.000Z", + error: null, + settings: { apiKey: SECRET }, + ...overrides + }; +} + +function createServices() { + const calls = []; + const providers = [publicProvider()]; + const providerService = { + async listProviders() { + calls.push(["listProviders"]); + return providers.map((provider) => ({ ...provider })); + }, + async createProvider(input, secret, options) { + calls.push(["createProvider", input, secret, options]); + const created = publicProvider({ id: "provider-2", name: input.name }); + providers.push(created); + return created; + }, + async updateProvider(id, patch, replacementSecret) { + calls.push(["updateProvider", id, patch, replacementSecret]); + const current = providers.find((provider) => provider.id === id); + Object.assign(current, patch, { updatedAt: "2026-07-13T00:01:00.000Z" }); + return { ...current }; + }, + async deleteProvider(id) { + calls.push(["deleteProvider", id]); + if (id === "provider-1") { + throw new CrpError( + "PROVIDER_ACTIVE", + "The active provider cannot be deleted.", + "Activate another provider or stop the proxy first.", + { status: 409, details: { authorization: SECRET, note: SECRET } } + ); + } + const index = providers.findIndex((provider) => provider.id === id); + return providers.splice(index, 1)[0]; + }, + async testProvider(id, model) { + calls.push(["testProvider", id, model]); + return { ok: true, code: null, apiKey: SECRET }; + }, + async activate(id) { + calls.push(["activate", id]); + return { + activeProviderId: id, + activeProvider: providers.find((provider) => provider.id === id), + generation: 2, + worker: workerState({ generation: 2 }) + }; + }, + async getStatus() { + calls.push(["getStatus"]); + return { + activeProviderId: "provider-1", + activeProvider: providers[0], + generation: 1, + worker: workerState() + }; + }, + async startProxy() { + calls.push(["startProxy"]); + return workerState(); + }, + async stopProxy() { + calls.push(["stopProxy"]); + return workerState({ phase: "stopped", pid: null, generation: 1, state: null }); + }, + async restartProxy() { + calls.push(["restartProxy"]); + return workerState({ pid: 54321 }); + } + }; + const activityStore = { + list({ limit }) { + calls.push(["activity", limit]); + return Array.from({ length: 8 }, (_, index) => ({ + timestamp: `2026-07-13T00:00:0${index}.000Z`, + category: "provider", + action: "test", + providerId: "provider-1", + result: "success", + errorCode: null, + details: index === 0 ? { authorization: "[REDACTED]" } : { index } + })).reverse(); + } + }; + const settingsService = { + async getSettings() { + calls.push(["getSettings"]); + return { + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: 15101, + captureEnabled: false, + credentialBackend: "native", + apiKey: SECRET + }; + }, + async updateSettings(patch) { + calls.push(["updateSettings", patch]); + return { ...(await this.getSettings()), ...patch }; + } + }; + const codexService = { + async bootstrap() { + calls.push(["bootstrap"]); + return { changed: true, backupPath: `/private/${SECRET}` }; + }, + async getStatus() { + return { configured: true, modelProvider: "OpenAI", proxyUrl: "http://127.0.0.1:15100" }; + } + }; + const diagnosticsService = { + async exportDiagnostics() { + calls.push(["diagnostics"]); + return { created: true, apiKey: SECRET, credentialRef: CREDENTIAL_REF }; + } + }; + return { + calls, + providerService, + activityStore, + settingsService, + codexService, + diagnosticsService + }; +} + +async function createHarness(t, overrides = {}) { + const dir = mkdtempSync(join(os.tmpdir(), "crp-admin-server-")); + const uiDir = join(dir, "ui"); + mkdirSync(uiDir, { recursive: true }); + writeFileSync(join(uiDir, "index.html"), "CRP test UI\n"); + writeFileSync(join(uiDir, "styles.css"), "body { color: black; }\n"); + writeFileSync(join(uiDir, "app.js"), "globalThis.__crpTest = true;\n"); + const controlTokenPath = join(dir, "control-token"); + const auth = new SessionAuth({ controlTokenPath }); + const services = createServices(); + const admin = createAdminServer({ + auth, + ...services, + getSupervisorState: () => ({ pid: 9001, startedAt: "2026-07-13T00:00:00.000Z", apiKey: SECRET }), + uiDir, + host: "127.0.0.1", + port: 0, + ...overrides + }); + const address = await admin.listen(); + const controlToken = readFileSync(controlTokenPath, "utf8").trim(); + t.after(async () => { + await admin.close(); + auth.close(); + rmSync(dir, { recursive: true, force: true }); + }); + + async function request(path, options = {}) { + const target = new URL(`${address.origin}${path}`); + const received = await new Promise((resolvePromise, rejectPromise) => { + const outgoing = http.request({ + host: target.hostname, + port: target.port, + path: `${target.pathname}${target.search}`, + method: options.method ?? "GET", + headers: { ...(options.headers ?? {}) } + }, (response) => { + const chunks = []; + response.on("data", (chunk) => chunks.push(chunk)); + response.on("end", () => resolvePromise({ + status: response.statusCode, + headers: new Headers(response.headers), + text: Buffer.concat(chunks).toString("utf8") + })); + }); + outgoing.once("error", rejectPromise); + if (options.body !== undefined) outgoing.write(options.body); + outgoing.end(); + }); + const response = { status: received.status, headers: received.headers }; + const text = received.text; + let json = null; + if (text && response.headers.get("content-type")?.startsWith("application/json")) { + json = JSON.parse(text); + } + return { response, text, json }; + } + + return { + ...services, + admin, + address, + auth, + controlToken, + request + }; +} + +async function browserSession(harness, headers = {}) { + const created = await harness.request("/api/v1/session", { + method: "POST", + headers: { + authorization: `Bearer ${harness.controlToken}`, + origin: harness.address.origin, + ...headers + } + }); + assert.equal(created.response.status, 200); + return { + cookie: created.response.headers.get("set-cookie").split(";")[0], + csrfToken: created.json.csrfToken + }; +} + +function bearer(harness, extra = {}) { + return { authorization: `Bearer ${harness.controlToken}`, ...extra }; +} + +function assertNoSensitiveResponse(result) { + const serialized = `${result.text}\n${JSON.stringify(result.json)}`; + for (const forbidden of [SECRET, CREDENTIAL_REF, "credentialRef", "apiKey", "backupPath"]) { + assert.equal(serialized.includes(forbidden), false, `response leaked ${forbidden}`); + } +} + +test("enforces Host, Origin, disabled CORS, bearer, and browser read sessions", async (t) => { + const harness = await createHarness(t); + + const badHost = await harness.request("/api/v1/session", { + method: "POST", + headers: bearer(harness, { host: "attacker.example" }) + }); + assert.equal(badHost.response.status, 403); + assert.equal(badHost.json.error.code, "API_HOST_INVALID"); + + const badOrigin = await harness.request("/api/v1/session", { + method: "POST", + headers: bearer(harness, { origin: "https://attacker.example" }) + }); + assert.equal(badOrigin.response.status, 403); + assert.equal(badOrigin.json.error.code, "API_ORIGIN_INVALID"); + + const preflight = await harness.request("/api/v1/providers", { + method: "OPTIONS", + headers: { + origin: harness.address.origin, + "access-control-request-method": "POST" + } + }); + assert.equal(preflight.response.status, 403); + assert.equal(preflight.json.error.code, "API_CORS_FORBIDDEN"); + assert.equal(preflight.response.headers.has("access-control-allow-origin"), false); + + const unauthenticated = await harness.request("/api/v1/status"); + assert.equal(unauthenticated.response.status, 401); + assert.equal(unauthenticated.json.error.code, "AUTH_REQUIRED"); + + const session = await browserSession(harness); + const browserRead = await harness.request("/api/v1/status", { + headers: { cookie: session.cookie, origin: harness.address.origin } + }); + assert.equal(browserRead.response.status, 200); + const providers = await harness.request("/api/v1/providers", { + headers: { cookie: session.cookie, origin: harness.address.origin } + }); + assert.equal(providers.response.status, 200); + for (const result of [badHost, badOrigin, preflight, unauthenticated, browserRead, providers]) { + assert.equal(result.response.headers.has("access-control-allow-origin"), false); + assert.equal(result.response.headers.get("cache-control"), "no-store"); + assertNoSensitiveResponse(result); + } +}); + +test("requires CSRF for browser mutations while bearer mutations bypass CSRF", async (t) => { + const harness = await createHarness(t); + const session = await browserSession(harness); + const missingCsrf = await harness.request("/api/v1/proxy/stop", { + method: "POST", + headers: { cookie: session.cookie, origin: harness.address.origin } + }); + assert.equal(missingCsrf.response.status, 403); + assert.equal(missingCsrf.json.error.code, "AUTH_CSRF_INVALID"); + + const browserMutation = await harness.request("/api/v1/proxy/stop", { + method: "POST", + headers: { + cookie: session.cookie, + origin: harness.address.origin, + "x-crp-csrf": session.csrfToken + } + }); + assert.equal(browserMutation.response.status, 200); + + const cliMutation = await harness.request("/api/v1/proxy/start", { + method: "POST", + headers: bearer(harness) + }); + assert.equal(cliMutation.response.status, 200); + for (const result of [missingCsrf, browserMutation, cliMutation]) { + assert.equal(result.response.headers.has("access-control-allow-origin"), false); + assertNoSensitiveResponse(result); + } +}); + +test("routes every approved Admin API operation through injected services", async (t) => { + const harness = await createHarness(t); + const authHeaders = bearer(harness); + const requests = [ + ["GET", "/api/v1/status", undefined, 200], + ["GET", "/api/v1/providers", undefined, 200], + ["POST", "/api/v1/providers", { + provider: { name: "Backup", baseUrl: "https://backup.example/v1" }, + credential: SECRET, + fallbackConsent: false + }, 201], + ["GET", "/api/v1/providers/provider-2", undefined, 200], + ["PATCH", "/api/v1/providers/provider-2", { + patch: { name: "Backup Updated" }, + replacementCredential: SECRET + }, 200], + ["POST", "/api/v1/providers/provider-2/test", { model: "test-model" }, 200], + ["POST", "/api/v1/providers/provider-2/activate", undefined, 200], + ["POST", "/api/v1/proxy/start", undefined, 200], + ["POST", "/api/v1/proxy/stop", undefined, 200], + ["POST", "/api/v1/proxy/restart", undefined, 200], + ["GET", "/api/v1/activity?limit=3&offset=2", undefined, 200], + ["GET", "/api/v1/settings", undefined, 200], + ["PATCH", "/api/v1/settings", { captureEnabled: true }, 409, "SETTINGS_READ_ONLY"], + ["POST", "/api/v1/codex/bootstrap", undefined, 200], + ["POST", "/api/v1/diagnostics/export", undefined, 200], + ["DELETE", "/api/v1/providers/provider-2", undefined, 200] + ]; + + for (const [method, path, body, expectedStatus, expectedCode] of requests) { + const result = await harness.request(path, { + method, + headers: { ...authHeaders, ...(body === undefined ? {} : { "content-type": "application/json" }) }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }) + }); + assert.equal(result.response.status, expectedStatus, `${method} ${path}: ${result.text}`); + if (expectedCode) assert.equal(result.json.error.code, expectedCode); + assertNoSensitiveResponse(result); + } + + const activityCall = harness.calls.find((call) => call[0] === "activity"); + assert.deepEqual(activityCall, ["activity", 6]); + const createCall = harness.calls.find((call) => call[0] === "createProvider"); + assert.equal(createCall[2], SECRET); + assert.deepEqual(createCall[3], { fallbackConsent: false }); + assert.ok(harness.calls.some((call) => call[0] === "restartProxy")); + assert.ok(harness.calls.some((call) => call[0] === "bootstrap")); + assert.ok(harness.calls.some((call) => call[0] === "diagnostics")); + assert.equal(harness.calls.some((call) => call[0] === "updateSettings"), false); +}); + +test("enforces JSON content type, exact shape, and a bounded request body", async (t) => { + const harness = await createHarness(t, { maxBodyBytes: 256 }); + const headers = bearer(harness); + const valid = await harness.request("/api/v1/providers", { + method: "POST", + headers: { ...headers, "content-type": "application/json; charset=utf-8" }, + body: JSON.stringify({ + provider: { name: "Bounded", baseUrl: "https://bounded.example/v1" }, + credential: "write-only-test-value", + fallbackConsent: false + }) + }); + assert.equal(valid.response.status, 201, valid.text); + + const cases = [ + [{ + method: "POST", + headers: { ...headers, "content-type": "text/plain" }, + body: "not-json" + }, 415, "API_CONTENT_TYPE_UNSUPPORTED"], + [{ + method: "POST", + headers: { ...headers, "content-type": "application/json" }, + body: "{" + }, 400, "API_BODY_INVALID"], + [{ + method: "POST", + headers: { ...headers, "content-type": "application/json" }, + body: JSON.stringify({ credential: "x".repeat(512) }) + }, 413, "API_BODY_TOO_LARGE"], + [{ + method: "POST", + headers: { ...headers, "content-type": "application/json" }, + body: JSON.stringify({ provider: {}, credential: "x", unexpected: true }) + }, 400, "API_BODY_INVALID"] + ]; + for (const [options, status, code] of cases) { + const result = await harness.request("/api/v1/providers", options); + assert.equal(result.response.status, status, result.text); + assert.equal(result.json.error.code, code); + assertNoSensitiveResponse(result); + } + for (const [path, options] of [ + ["/api/v1/session", { + method: "POST", + headers: { ...bearer(harness), "content-type": "application/json" }, + body: "{}" + }], + ["/api/v1/proxy/start", { + method: "POST", + headers: { ...headers, "content-type": "application/json" }, + body: "{}" + }] + ]) { + const result = await harness.request(path, options); + assert.equal(result.response.status, 400, result.text); + assert.equal(result.json.error.code, "API_BODY_INVALID"); + } + assertNoSensitiveResponse(valid); +}); + +test("returns stable sanitized errors and strict route, method, and path failures", async (t) => { + const harness = await createHarness(t); + const headers = bearer(harness); + const conflict = await harness.request("/api/v1/providers/provider-1", { + method: "DELETE", + headers + }); + assert.equal(conflict.response.status, 409); + assert.equal(conflict.json.error.code, "PROVIDER_ACTIVE"); + assert.equal(typeof conflict.json.error.requestId, "string"); + assert.deepEqual(conflict.json.error.details, { authorization: "[REDACTED]" }); + + const cases = [ + ["/api/v1", { headers }, 404, "API_NOT_FOUND"], + ["/api/v1/missing", { headers }, 404, "API_NOT_FOUND"], + ["/api/v1/providers", { method: "PUT", headers }, 405, "API_METHOD_NOT_ALLOWED"], + ["/api/v1/providers/provider-2%2Factivate", { headers }, 404, "API_NOT_FOUND"] + ]; + for (const [path, options, status, code] of cases) { + const result = await harness.request(path, options); + assert.equal(result.response.status, status, `${path}: ${result.text}`); + assert.equal(result.json.error.code, code); + assertNoSensitiveResponse(result); + } + assertNoSensitiveResponse(conflict); +}); + +test("serves only explicit static assets with safe headers and an index fallback", async (t) => { + const harness = await createHarness(t); + for (const [path, type, content] of [ + ["/", "text/html; charset=utf-8", "CRP test UI"], + ["/index.html", "text/html; charset=utf-8", "CRP test UI"], + ["/styles.css", "text/css; charset=utf-8", "color: black"], + ["/app.js", "text/javascript; charset=utf-8", "__crpTest"], + ["/providers/provider-1", "text/html; charset=utf-8", "CRP test UI"] + ]) { + const result = await harness.request(path); + assert.equal(result.response.status, 200); + assert.equal(result.response.headers.get("content-type"), type); + assert.equal(result.response.headers.get("cache-control"), "no-store"); + assert.equal(result.response.headers.get("x-content-type-options"), "nosniff"); + assert.match(result.text, new RegExp(content)); + } + + const missingAsset = await harness.request("/missing.css"); + assert.equal(missingAsset.response.status, 404); + const postUi = await harness.request("/", { method: "POST" }); + assert.equal(postUi.response.status, 405); +}); + +function createGate() { + let resolve; + let reject; + const promise = new Promise((resolvePromise, rejectPromise) => { + resolve = resolvePromise; + reject = rejectPromise; + }); + return { promise, resolve, reject }; +} + +function supervisorDependencies(t, { listenGate = createGate() } = {}) { + const home = mkdtempSync(join(os.tmpdir(), "crp-supervisor-")); + const paths = { + ...getPaths(home), + runtimeConfigPath: join(home, ".codex-remote-proxy", "node", "proxy-config.json"), + capturePath: join(home, ".codex-remote-proxy", "traffic.sqlite3") + }; + const order = []; + const worker = { + getPublicState: () => workerState(), + close: () => { + order.push("worker.close"); + return Promise.resolve(); + } + }; + const activity = { append() {}, list: () => [] }; + const credentials = { backend: "native" }; + const registry = { getDocument: () => ({ settings: { + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: 15101, + captureEnabled: false + } }) }; + const provider = { getStatus: async () => ({ worker: workerState() }) }; + const auth = { + close() { order.push("auth.close"); } + }; + const admin = { + async listen() { + order.push("admin.listen"); + return await listenGate.promise; + }, + close() { + order.push("admin.close"); + return Promise.resolve(); + } + }; + const options = { + home, + paths, + pid: 4242, + now: () => "2026-07-13T03:00:00.000Z", + activityStoreFactory: ({ path }) => { + order.push("activity"); + assert.equal(path, paths.activityPath); + return activity; + }, + credentialStoreFactory: ({ paths: received }) => { + order.push("credential"); + assert.deepEqual(received, paths); + return credentials; + }, + migrate: async (input) => { + order.push("migration"); + assert.equal(input.credentialStore, credentials); + assert.equal(input.activityStore, activity); + return { migrated: false, reason: "no-legacy-config" }; + }, + registryFactory: ({ path }) => { + order.push("registry"); + assert.equal(path, paths.registryPath); + assert.ok(order.includes("migration")); + return registry; + }, + workerManagerFactory: () => { + order.push("worker"); + return worker; + }, + providerServiceFactory: (input) => { + order.push("provider"); + assert.equal(input.registry, registry); + assert.equal(input.credentialStore, credentials); + assert.equal(input.workerManager, worker); + return provider; + }, + authFactory: ({ controlTokenPath }) => { + order.push("auth"); + assert.equal(controlTokenPath, paths.controlTokenPath); + return auth; + }, + adminServerFactory: (input) => { + order.push("admin"); + assert.equal(input.auth, auth); + assert.equal(input.providerService, provider); + return admin; + } + }; + t.after(() => rmSync(home, { recursive: true, force: true })); + return { home, paths, order, worker, admin, listenGate, options }; +} + +test("supervisor migrates before registry construction and writes private state only after ready", async (t) => { + const harness = supervisorDependencies(t); + const supervisor = await createSupervisor(harness.options); + assert.deepEqual(harness.order, [ + "activity", + "credential", + "migration", + "registry", + "worker", + "provider", + "auth", + "admin" + ]); + + const listening = supervisor.listen(); + assert.equal(existsSync(harness.paths.statePath), false); + harness.listenGate.resolve({ + host: "127.0.0.1", + port: 15101, + authority: "127.0.0.1:15101", + origin: "http://127.0.0.1:15101" + }); + const address = await listening; + assert.equal(address.port, 15101); + assert.equal(existsSync(harness.paths.statePath), true); + if (process.platform !== "win32") { + assert.equal(statSync(harness.paths.statePath).mode & 0o777, 0o600); + } + const stateBytes = readFileSync(harness.paths.statePath, "utf8"); + const state = JSON.parse(stateBytes); + assert.equal(state.schemaVersion, 1); + assert.equal(state.supervisorPid, 4242); + assert.equal(state.startedAt, "2026-07-13T03:00:00.000Z"); + assert.equal(state.admin.origin, "http://127.0.0.1:15101"); + assert.equal(state.worker.generation, 1); + for (const forbidden of [SECRET, "apiKey", "settings", CREDENTIAL_REF]) { + assert.equal(stateBytes.includes(forbidden), false); + } + + const firstClose = supervisor.close(); + const secondClose = supervisor.close(); + assert.equal(firstClose, secondClose); + await firstClose; + assert.deepEqual(harness.order.slice(-3), ["admin.close", "auth.close", "worker.close"]); + assert.equal(existsSync(harness.paths.statePath), false); +}); + +test("supervisor cleans up in reverse order when Admin readiness fails", async (t) => { + const harness = supervisorDependencies(t); + const supervisor = await createSupervisor(harness.options); + const failure = new Error(`private listen failure ${SECRET}`); + harness.listenGate.reject(failure); + await assert.rejects(() => supervisor.listen(), (error) => error === failure); + assert.deepEqual(harness.order.slice(-3), ["admin.close", "auth.close", "worker.close"]); + assert.equal(existsSync(harness.paths.statePath), false); +}); + +test("supervisor cleans constructed resources when composition fails before listen", async (t) => { + const harness = supervisorDependencies(t); + const failure = new Error(`private composition failure ${SECRET}`); + harness.options.adminServerFactory = () => { + harness.order.push("admin"); + throw failure; + }; + await assert.rejects(() => createSupervisor(harness.options), (error) => error === failure); + assert.deepEqual(harness.order.slice(-2), ["auth.close", "worker.close"]); + assert.equal(existsSync(harness.paths.statePath), false); +}); + +test("supervisor keeps Codex and state filesystem adapters independent", async (t) => { + const harness = supervisorDependencies(t); + const codexFileOperations = { boundary: "codex-only" }; + let bootstrapInput = null; + let codexService = null; + harness.options.codexFileOperations = codexFileOperations; + harness.options.bootstrapCodex = (input) => { + bootstrapInput = input; + return { changed: false, backupPath: null }; + }; + harness.options.adminServerFactory = (input) => { + harness.order.push("admin"); + codexService = input.codexService; + return harness.admin; + }; + const supervisor = await createSupervisor(harness.options); + assert.deepEqual(codexService.bootstrap(), { changed: false, backupPath: null }); + assert.equal(bootstrapInput.fileOperations, codexFileOperations); + assert.equal(bootstrapInput.configPath, harness.paths.codexConfigPath); + await supervisor.close(); +}); + +test("supervisor entry shares idempotent signal shutdown without exiting early", async () => { + const processRef = new EventEmitter(); + processRef.exitCode = null; + const closed = createGate(); + let closeCalls = 0; + const supervisor = { + async listen() { + return { origin: "http://127.0.0.1:15101" }; + }, + close() { + closeCalls += 1; + return closed.promise; + } + }; + await runSupervisor({ + processRef, + createSupervisorImpl: async () => supervisor, + supervisorOptions: { home: "/unused/injected-home" } + }); + assert.equal(processRef.listenerCount("SIGTERM"), 1); + assert.equal(processRef.listenerCount("SIGINT"), 1); + processRef.emit("SIGTERM"); + processRef.emit("SIGINT"); + assert.equal(closeCalls, 1); + assert.equal(processRef.exitCode, null); + closed.resolve(); + await closed.promise; + await new Promise((resolvePromise) => setImmediate(resolvePromise)); + assert.equal(processRef.exitCode, 0); + assert.equal(processRef.listenerCount("SIGTERM"), 0); + assert.equal(processRef.listenerCount("SIGINT"), 0); +}); diff --git a/node/test/provider-service.test.mjs b/node/test/provider-service.test.mjs index dc7cd4a..32a974c 100644 --- a/node/test/provider-service.test.mjs +++ b/node/test/provider-service.test.mjs @@ -170,6 +170,14 @@ function committedError(code) { ); } +function createGate() { + let resolve; + const promise = new Promise((resolvePromise) => { + resolve = resolvePromise; + }); + return { promise, resolve }; +} + async function listen(server) { await new Promise((resolve, reject) => { server.once("error", reject); @@ -855,3 +863,124 @@ test("reports rollback degraded when first activation may commit and bounded sto assert.equal(workerManager.generation, 1); assert.equal(activity.events.at(-1).errorCode, "PROVIDER_ACTIVATION_ROLLBACK_DEGRADED"); }); + +test("proxy lifecycle facade resolves only the active credential and advances confirmed generations", async (t) => { + const harness = makeHarness(t); + const { service, registry, credentials, workerManager, activity } = harness; + const secretA = makeSecret("active"); + const secretB = makeSecret("inactive"); + const providerA = await service.createProvider(providerInput("A"), secretA); + await service.createProvider(providerInput("B", "https://b.example/v1"), secretB); + registry.markTest(providerA.id, { status: "passed" }); + await service.activate(providerA.id); + + credentials.operations.length = 0; + workerManager.calls.length = 0; + const stopped = await service.stopProxy(); + assert.equal(stopped.phase, "stopped"); + assert.deepEqual(credentials.operations, []); + + const started = await service.startProxy(); + assert.equal(started.phase, "running"); + assert.equal(started.generation, 2); + assert.equal(workerManager.calls.at(-1)[0], "start"); + assert.equal(workerManager.calls.at(-1)[1].settings.upstream.apiKey, secretA); + + const restarted = await service.restartProxy(); + assert.equal(restarted.phase, "running"); + assert.equal(restarted.generation, 3); + assert.equal(workerManager.calls.at(-1)[0], "restart"); + assert.equal(workerManager.calls.at(-1)[1].settings.upstream.apiKey, secretA); + assert.deepEqual( + credentials.operations.filter(([operation]) => operation === "get").map(([, ref]) => ref), + ["credential-1", "credential-1"] + ); + + const serialized = JSON.stringify({ stopped, started, restarted, events: activity.events }); + for (const forbidden of [secretA, secretB, "credential-1", "credential-2", "apiKey"]) { + assert.equal(serialized.includes(forbidden), false, `lifecycle output leaked ${forbidden}`); + } + assert.deepEqual( + activity.events.slice(-3).map(({ category, action, result }) => ({ category, action, result })), + [ + { category: "proxy", action: "stop", result: "success" }, + { category: "proxy", action: "start", result: "success" }, + { category: "proxy", action: "restart", result: "success" } + ] + ); +}); + +test("shares each pending proxy lifecycle operation across concurrent commands", async (t) => { + for (const leader of ["start", "stop", "restart"]) { + await t.test(leader, async (t) => { + const { service, registry, credentials, workerManager, activity } = makeHarness(t); + const provider = await service.createProvider(providerInput(), makeSecret()); + registry.markTest(provider.id, { status: "passed" }); + registry.setActive(provider.id); + if (leader === "stop") { + workerManager.phase = "running"; + } + credentials.operations.length = 0; + workerManager.calls.length = 0; + activity.events.length = 0; + + const gate = createGate(); + const method = `${leader}Proxy`; + const original = workerManager[leader].bind(workerManager); + let leaderCalls = 0; + workerManager[leader] = (...args) => { + leaderCalls += 1; + return gate.promise.then(() => original(...args)); + }; + + const current = service[method](); + const concurrent = [ + service.startProxy(), + service.stopProxy(), + service.restartProxy() + ]; + await new Promise((resolvePromise) => setImmediate(resolvePromise)); + const samePromises = concurrent.map((operation) => operation === current); + gate.resolve(); + const [result, ...concurrentResults] = await Promise.all([current, ...concurrent]); + + assert.deepEqual(samePromises, [true, true, true]); + assert.ok(concurrentResults.every((candidate) => candidate === result)); + assert.equal(leaderCalls, 1); + assert.deepEqual(workerManager.calls.map(([operation]) => operation), [leader]); + assert.equal( + credentials.operations.filter(([operation]) => operation === "get").length, + leader === "stop" ? 0 : 1 + ); + assert.deepEqual( + activity.events.map(({ category, action, result }) => ({ category, action, result })), + [{ category: "proxy", action: leader, result: "success" }] + ); + assert.equal(result.generation, leader === "stop" ? 0 : 1); + + const later = service.restartProxy(); + assert.notEqual(later, current); + const laterResult = await later; + assert.equal(laterResult.generation, leader === "stop" ? 1 : 2); + assert.equal(activity.events.length, 2); + }); + } +}); + +test("proxy start and restart reject without an active provider while stop remains idempotent", async (t) => { + const { service, credentials } = makeHarness(t); + await assert.rejects( + () => service.startProxy(), + (error) => error instanceof CrpError + && error.code === "PROXY_NOT_CONFIGURED" + && error.status === 409 + ); + await assert.rejects( + () => service.restartProxy(), + (error) => error instanceof CrpError + && error.code === "PROXY_NOT_CONFIGURED" + && error.status === 409 + ); + assert.equal((await service.stopProxy()).phase, "stopped"); + assert.deepEqual(credentials.operations, []); +}); diff --git a/node/test/session-auth.test.mjs b/node/test/session-auth.test.mjs new file mode 100644 index 0000000..960475a --- /dev/null +++ b/node/test/session-auth.test.mjs @@ -0,0 +1,213 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { + chmodSync, + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + statSync, + symlinkSync, + writeFileSync +} from "node:fs"; +import os from "node:os"; +import { join } from "node:path"; + +import { SessionAuth } from "../src/supervisor/session-auth.mjs"; +import { CrpError, toPublicError } from "../src/shared/errors.mjs"; + +const CONTROL_BYTES = Buffer.alloc(32, 0x11); +const SESSION_BYTES = Buffer.alloc(32, 0x22); +const CSRF_BYTES = Buffer.alloc(32, 0x33); +const CONTROL_TOKEN = CONTROL_BYTES.toString("base64url"); +const SESSION_TOKEN = SESSION_BYTES.toString("base64url"); +const CSRF_TOKEN = CSRF_BYTES.toString("base64url"); +const START_MS = Date.parse("2026-07-13T00:00:00.000Z"); + +function makeTempAuth(t, prefix = "crp-session-auth-") { + const dir = mkdtempSync(join(os.tmpdir(), prefix)); + const tokenPath = join(dir, "control-token"); + t.after(() => rmSync(dir, { recursive: true, force: true })); + return { dir, tokenPath }; +} + +function sequenceRandomBytes(...values) { + let index = 0; + return (size) => { + assert.equal(size, 32); + const value = values[index++]; + assert.ok(value, `unexpected randomBytes call ${index}`); + return Buffer.from(value); + }; +} + +function assertCrpError(code, status) { + return (error) => { + assert.ok(error instanceof CrpError); + assert.equal(error.code, code); + assert.equal(error.status, status); + assert.equal(typeof error.action, "string"); + assert.notEqual(error.action.length, 0); + return true; + }; +} + +test("creates one private 32-byte control token and reuses it after restart", (t) => { + const { tokenPath } = makeTempAuth(t); + const first = new SessionAuth({ + controlTokenPath: tokenPath, + randomBytes: sequenceRandomBytes(CONTROL_BYTES), + now: () => START_MS + }); + + assert.equal(readFileSync(tokenPath, "utf8"), `${CONTROL_TOKEN}\n`); + if (process.platform !== "win32") { + assert.equal(statSync(tokenPath).mode & 0o777, 0o600); + assert.equal(statSync(join(tokenPath, "..")).mode & 0o777, 0o700); + } + assert.deepEqual( + first.authorize({ authorization: `Bearer ${CONTROL_TOKEN}`, mutation: true }), + { kind: "cli" } + ); + + const restarted = new SessionAuth({ + controlTokenPath: tokenPath, + randomBytes: () => assert.fail("an existing control token must be reused"), + now: () => START_MS + }); + assert.deepEqual( + restarted.authorize({ authorization: `Bearer ${CONTROL_TOKEN}`, mutation: true }), + { kind: "cli" } + ); +}); + +test("exchanges bearer auth for an HttpOnly strict browser session and enforces CSRF", (t) => { + const { tokenPath } = makeTempAuth(t); + const auth = new SessionAuth({ + controlTokenPath: tokenPath, + randomBytes: sequenceRandomBytes(CONTROL_BYTES, SESSION_BYTES, CSRF_BYTES), + now: () => START_MS, + sessionTtlMs: 60_000 + }); + + const session = auth.createBrowserSession(`Bearer ${CONTROL_TOKEN}`); + assert.equal(session.csrfToken, CSRF_TOKEN); + assert.equal(session.expiresAt, "2026-07-13T00:01:00.000Z"); + assert.match(session.setCookie, new RegExp(`^crp_session=${SESSION_TOKEN};`)); + assert.match(session.setCookie, /Path=\//); + assert.match(session.setCookie, /HttpOnly/); + assert.match(session.setCookie, /SameSite=Strict/); + assert.match(session.setCookie, /Max-Age=60/); + assert.doesNotMatch(session.setCookie, /Secure/); + + const cookie = `unrelated=value; crp_session=${SESSION_TOKEN}`; + assert.deepEqual(auth.authorize({ cookie, mutation: false }), { kind: "browser" }); + assert.deepEqual( + auth.authorize({ cookie, csrfToken: CSRF_TOKEN, mutation: true }), + { kind: "browser" } + ); + assert.throws( + () => auth.authorize({ cookie, mutation: true }), + assertCrpError("AUTH_CSRF_INVALID", 403) + ); + assert.throws( + () => auth.authorize({ cookie, csrfToken: `${CSRF_TOKEN}x`, mutation: true }), + assertCrpError("AUTH_CSRF_INVALID", 403) + ); +}); + +test("expired browser sessions are rejected and request cookie clearing", (t) => { + const { tokenPath } = makeTempAuth(t); + let now = START_MS; + const auth = new SessionAuth({ + controlTokenPath: tokenPath, + randomBytes: sequenceRandomBytes(CONTROL_BYTES, SESSION_BYTES, CSRF_BYTES), + now: () => now, + sessionTtlMs: 1_000 + }); + const session = auth.createBrowserSession(`Bearer ${CONTROL_TOKEN}`); + + now += 1_001; + let caught; + try { + auth.authorize({ cookie: `crp_session=${SESSION_TOKEN}`, mutation: false }); + } catch (error) { + caught = error; + } + assertCrpError("AUTH_SESSION_EXPIRED", 401)(caught); + assert.equal(caught.clearCookie, true); + assert.equal(auth.clearCookie(), "crp_session=; Path=/; HttpOnly; SameSite=Strict; Max-Age=0"); + assert.equal(JSON.stringify(toPublicError(caught, "request-1")).includes(SESSION_TOKEN), false); + assert.equal(JSON.stringify(toPublicError(caught, "request-1")).includes(CSRF_TOKEN), false); + assert.equal(session.csrfToken, CSRF_TOKEN); +}); + +test("rejects invalid bearer, missing or duplicate cookies, and destroys sessions on close", (t) => { + const { tokenPath } = makeTempAuth(t); + const auth = new SessionAuth({ + controlTokenPath: tokenPath, + randomBytes: sequenceRandomBytes(CONTROL_BYTES, SESSION_BYTES, CSRF_BYTES), + now: () => START_MS + }); + auth.createBrowserSession(`Bearer ${CONTROL_TOKEN}`); + + for (const authorization of [ + CONTROL_TOKEN, + "Basic invalid", + "Bearer", + `Bearer ${CONTROL_TOKEN}x` + ]) { + assert.throws( + () => auth.authorize({ authorization, mutation: false }), + assertCrpError("AUTH_REQUIRED", 401) + ); + } + for (const cookie of [ + undefined, + "unrelated=value", + `crp_session=${SESSION_TOKEN}; crp_session=${SESSION_TOKEN}`, + `crp_session=${SESSION_TOKEN}x` + ]) { + assert.throws( + () => auth.authorize({ cookie, mutation: false }), + assertCrpError("AUTH_REQUIRED", 401) + ); + } + + auth.close(); + assert.throws( + () => auth.authorize({ cookie: `crp_session=${SESSION_TOKEN}`, mutation: false }), + assertCrpError("AUTH_REQUIRED", 401) + ); +}); + +test("rejects malformed, permissive, and symbolic-link control token files safely", (t) => { + const secrets = ["short-control-secret", CONTROL_TOKEN]; + for (const [label, prepare] of [ + ["malformed", ({ tokenPath }) => writeFileSync(tokenPath, `${secrets[0]}\n`, { mode: 0o600 })], + ["permissive", ({ tokenPath }) => { + writeFileSync(tokenPath, `${CONTROL_TOKEN}\n`, { mode: 0o600 }); + chmodSync(tokenPath, 0o644); + }], + ["symlink", ({ dir, tokenPath }) => { + const target = join(dir, "target-token"); + writeFileSync(target, `${CONTROL_TOKEN}\n`, { mode: 0o600 }); + symlinkSync(target, tokenPath); + }] + ]) { + if (label === "permissive" && process.platform === "win32") continue; + const paths = makeTempAuth(t, `crp-session-${label}-`); + mkdirSync(paths.dir, { recursive: true, mode: 0o700 }); + prepare(paths); + let caught; + try { + new SessionAuth({ controlTokenPath: paths.tokenPath }); + } catch (error) { + caught = error; + } + assertCrpError("AUTH_CONTROL_TOKEN_INVALID", 500)(caught); + const serialized = JSON.stringify(toPublicError(caught, "request-1")); + for (const secret of secrets) assert.equal(serialized.includes(secret), false); + assert.equal(serialized.includes(paths.tokenPath), false); + } +}); From 7a87466e631e079a15472468c5a59086c9cf1ed5 Mon Sep 17 00:00:00 2001 From: cluic Date: Mon, 13 Jul 2026 21:43:57 +0800 Subject: [PATCH 13/44] feat: manage CRP through supervisor CLI --- docs/AGENT_COORDINATION.md | 6 +- docs/AI_HANDOFF.md | 11 +- docs/ARCHITECTURE.md | 8 +- docs/ROADMAP.md | 16 +- docs/STATUS.md | 8 +- docs/TESTING.md | 5 + node/bin/crp.mjs | 505 ++++++++++-- node/src/supervisor/supervisor-client.mjs | 578 +++++++++++++ node/test/codex-config.test.mjs | 130 +-- node/test/crp.test.mjs | 823 ++++++++++++++++++- node/test/integration/crp-lifecycle.test.mjs | 309 +++++++ 11 files changed, 2198 insertions(+), 201 deletions(-) create mode 100644 node/src/supervisor/supervisor-client.mjs create mode 100644 node/test/integration/crp-lifecycle.test.mjs diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index 13ca36e..e14d078 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | Primary Codex plus Task 9 implementation agent | Build the secured loopback Admin API and supervisor composition with one writer | Task 9 owns `node/src/supervisor/session-auth.mjs`, `node/src/supervisor/admin-server.mjs`, `node/src/supervisor/supervisor.mjs`, `node/src/supervisor/supervisor-entry.mjs`, `node/test/session-auth.test.mjs`, `node/test/integration/admin-server.test.mjs`, a minimal lifecycle facade in `node/src/supervisor/provider-service.mjs` with its focused tests in `node/test/provider-service.test.mjs`, and affected living docs | Task 9 implementation and deterministic gates are complete; independent review is next; real HOME, native keyrings, external network, UI, CLI, and runner changes remain prohibited | Task 8 commit `776265c8113f39b26f2bb5cd1418cfc275c4156e` | Provider registry, credential adapters, worker manager internals, server/runtime settings, capture, Codex adapter internals, migration/activity persistence, UI, CLI/bin, runners, package scripts, and every other module; no simultaneous writable agent | +| V1 sequential implementation | None | Task 10 supervisor-backed CLI routing is complete; Task 11 actual Web UI assets are next but unclaimed | No files are currently claimed; the Task 11 writer must declare exact files and no-edit areas here before editing | Task 10 complete; Task 11 not started | Completed Tasks 1 through 10 | All files remain no-edit until the next single writer records a bounded Task 11 scope; no simultaneous writable agent | ## Shared Contracts @@ -23,10 +23,10 @@ ## Locks / Avoid Editing -- No parallel writable work is authorized; only one task implementer may write at a time, and review agents are read-only. +- No writable work is currently authorized; the next task implementer must claim a bounded scope here, and review agents remain read-only. - Future agents must not change shared contracts without first updating the owning doc and coordination row. - Credential, migration, and lifecycle work must be isolated from UI styling work until contracts pass review. ## Decisions Needed -Subagent-driven execution is selected. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. +Subagent-driven execution is selected. Task 11 actual Web UI assets are next but unclaimed. Its writer must derive the exact file boundary and verification gate from the approved plan and record them in the active-workstream row before editing. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index 39bdf2e..b62fa0c 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,11 +6,11 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 9 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, versioned worker IPC, reliable fixed-port lifecycle management, sanitized activity, transactional migration, provider orchestration, and the secured loopback Admin control plane. Task 10, routing CLI commands through the supervisor, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 10 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, versioned worker IPC, reliable fixed-port lifecycle management, sanitized activity, transactional migration, provider orchestration, the secured loopback Admin control plane, and supervisor-backed CLI routing. Task 11, the actual Web UI assets, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. ## Architecture -Landed: shared paths, safe public errors, idempotent Codex bootstrap, strict provider storage, secure credential adapters, immutable request snapshots, strict worker IPC, reliable worker management, bounded sanitized activity, transactional v0.2.2 migration, serialized provider orchestration, private local sessions, the exact Admin route/security boundary, and readiness-gated supervisor composition. Target: CLI parity and the actual Web UI. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. +Landed: shared paths, safe public errors, idempotent Codex bootstrap, strict provider storage, secure credential adapters, immutable request snapshots, strict worker IPC, reliable worker management, bounded sanitized activity, transactional v0.2.2 migration, serialized provider orchestration, private local sessions, the exact Admin route/security boundary, readiness-gated supervisor composition, and state-discovered CLI dispatch through the Admin API. Target: the actual Web UI. Codex remains on `model_provider = "OpenAI"` and fixed `http://127.0.0.1:15100`; supervisor Admin API defaults to `127.0.0.1:15101`. ## Data and API @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 9 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 10 are complete. ## How To Run Current Code @@ -50,6 +50,7 @@ Do not run `crp start` against a real home directory during tests because it mod - Node 22.19 Task 7 gate: strict-unhandled `node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` passes 22/22; exact `npm test` passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests without duplication; `npm run test:unit` retains all-top-level coverage and passes 133/133. `npm run lint` syntax-checks 19 source files, and `npm audit --omit=dev` reports zero vulnerabilities. Coverage includes ready/configure/health startup, acknowledgement-atomic snapshot application, concurrent restart reuse before inspecting its snapshot, restart prevalidation before drain when no operation exists, immediate waiter observation and send-failure cancellation, graceful drain/shutdown, TERM/KILL escalation, retained child control after termination timeout, same-port real-worker restart with a changed PID, partial-start and port-release cleanup, correlated fatal and malformed-message sanitization, old-epoch isolation, cancellable injected-clock crash backoff, immediate failure on the fifth crash in 60 seconds, and idempotent retryable close without child, timer, listener, or port residue. The exact runner isolates the preexisting polling capture watcher between the core unit and real-fork integration groups so its registration baseline cannot race unrelated unit load. - Node 22.19 Task 8 gate: `node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs` passes 42/42; exact `npm test` passes 168/168 core assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run lint` syntax-checks 22 source files. Tests use only temporary paths, injected credential adapters/fetch responses, fake worker boundaries, and loopback redirect servers. Coverage includes recursive lifecycle-field redaction, ownership-checked activity locks, descriptor-safe migration paths, exclusive final-path registry creation, symlink and foreign-replacement preservation, committed-state reconciliation, redirect refusal, active-update rejection, replacement-secret compensation, selected credentials, explicit operation serialization, and conservative activation rollback `1 -> 2 -> 3` after health or lost-ACK uncertainty. Real HOME migration, native keyrings, cross-platform filesystem semantics, and live upstreams remain prohibited until L3 platform review. - Node 22.19 Task 9 gate: `node --test test/session-auth.test.mjs test/provider-service.test.mjs test/integration/admin-server.test.mjs` passes 42/42; exact `npm test` passes 179/179 core assertions, 7/7 isolated capture assertions, and 23/23 integration tests; `npm run lint` syntax-checks 26 source files and `npm audit --omit=dev` reports zero vulnerabilities. Coverage includes descriptor-safe private control tokens, expiring browser sessions/CSRF, exact Host/Origin/CORS rules, bounded request schemas, every Admin route, response/error secret scans, static allowlisting without actual UI files, active-only lifecycle credentials with in-flight command reuse, migration-before-registry composition, readiness-gated private state, startup compensation, separate Codex/state adapters, and signal cleanup. Real HOME, native keyring, external provider traffic, actual UI assets, and cross-platform browser behavior remain prohibited until their later gates. +- Node 22.19 Task 10 gate: `node --test test/crp.test.mjs test/integration/crp-lifecycle.test.mjs` passes 27/27; exact `npm test` passes 202/202 core assertions, 7/7 isolated capture assertions, and 24/24 integration assertions; `npm run lint` syntax-checks 27 source files and `npm audit --omit=dev` reports zero vulnerabilities. Coverage includes state discovery, authenticated lifecycle and provider dispatch, `ui` browser-session discovery, legacy secret-bearing flag rejection, owner-identity-checked shutdown, startup waiting, and failed-spawn cleanup without process or state residue. `git diff --check` and the static secret-pattern scan pass. Tests use temporary homes and injected spawn/client boundaries; real HOME, native keyrings, external provider traffic, browser launch behavior, and cross-platform process identity and signal handling remain L3. - Node 24.2 stability: `node --test test/capture-store.test.mjs` passes 7/7 without hanging after replacing fixed watcher sleeps with bounded condition waits and pre-assertion cleanup. - Future V1 gate: the full matrix and acceptance flow in `docs/TESTING.md`. @@ -99,3 +100,7 @@ Credential migration, localhost browser security, in-flight activation semantics - A post-ack activation rollback must advance to a new generation; never regress a worker generation to restore the confirmed provider. - Compensation tests must run and fail before compensation code is written. - Migration must create a missing registry exclusively at its final path and must never adopt, modify, or delete an `EEXIST` registry. +- CLI discovery must use the private supervisor state and dispatch lifecycle and provider commands through the authenticated loopback Admin API; legacy secret-bearing flags are prohibited. +- Lifecycle fake waits must advance the injected clock and simulate owner cleanup. +- Before signaling a process, verify `pid` plus `startedAt` and never mutate state first. +- A failed supervisor spawn must receive bounded termination and state cleanup before the CLI returns its public error. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index aa3c093..be8b7cd 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -2,7 +2,7 @@ ## Current State -Version 0.2.2 remains the published Node CLI plus one proxy process. Tasks 2 through 9 have now landed the target control-plane foundations: safe shared contracts, atomic provider and credential persistence, immutable request settings, strict worker IPC, reliable worker lifecycle management, bounded activity, transactional migration, provider orchestration, and a composed secured loopback Admin API. CLI routing and the actual static Web UI remain target-state work. +Version 0.2.2 remains the published release, while Tasks 2 through 10 have landed the target control-plane foundations: safe shared contracts, atomic provider and credential persistence, immutable request settings, strict worker IPC, reliable worker lifecycle management, bounded activity, transactional migration, provider orchestration, a composed secured loopback Admin API, and supervisor-backed CLI routing. Only the actual static Web UI remains target-state work in the next task. ## Target Overview @@ -40,7 +40,7 @@ Active OpenAI-compatible upstream ## Module Boundaries -Landed in Tasks 2 through 9: +Landed in Tasks 2 through 10: - `shared/paths`: derives CRP registry, credential fallback, state, control token, activity, log, Codex configuration, and Codex auth paths from one home root. - `shared/errors`: defines stable `CrpError` fields and safe public serialization for known and unknown failures. @@ -59,10 +59,10 @@ Landed in Tasks 2 through 9: - `session-auth`: owns one private 32-byte control token plus in-memory expiring browser session/CSRF tokens; existing token files are descriptor-validated and unsafe, malformed, symbolic-link, or broadly accessible files fail closed. - `admin-api`: enforces the exact loopback Host and Origin, disables CORS, authenticates CLI bearer and browser cookie/CSRF requests, bounds and validates request bodies, dispatches the versioned allowlisted routes, and positively projects provider, worker, status, settings, diagnostics, and error data. - `supervisor`: composes activity and credentials before migration, constructs the registry only after migration, owns worker/provider/auth/Admin services, writes a positive `0600` state projection only after Admin readiness, and performs idempotent reverse-order cleanup on startup failure or signal shutdown. +- `cli-supervisor-client`: discovers and validates the private supervisor state, authenticates to the loopback Admin API, dispatches lifecycle and provider commands, creates browser sessions for `ui`, starts a missing supervisor through an injected spawn boundary, rejects legacy secret-bearing flags, verifies `pid` plus `startedAt` before shutdown signaling, and bounds failed-spawn cleanup. Remaining target-state boundaries: -- `cli-supervisor-client`: routes lifecycle and provider commands through the same Admin contract. - `web-ui`: static local app with onboarding and daily management views. ## Lifecycle @@ -74,7 +74,7 @@ Remaining target-state boundaries: ## Storage and Deployment - Package remains distributed through npm. -- The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; the provider registry owns `providers.json`, the explicit fallback adapter owns `secrets.json`, and the other new stores remain target state. +- The landed path contract reserves `~/.codex-remote-proxy/providers.json`, `secrets.json`, `state.json`, `control-token`, `activity.jsonl`, and `supervisor.log`; the provider registry owns `providers.json`, the explicit fallback adapter owns `secrets.json`, the supervisor owns `state.json` and `control-token`, the activity store owns `activity.jsonl`, and `supervisor.log` remains reserved target state. - Provider metadata: `~/.codex-remote-proxy/providers.json`, atomically replaced with mode `0600` where supported after complete document validation. - Supervisor runtime metadata: `~/.codex-remote-proxy/state.json`, atomically written with mode `0600` after Admin readiness and removed only by the owning supervisor instance. - Credentials: native OS store by default; `~/.codex-remote-proxy/secrets.json` only after strict explicit fallback consent. Native loader/factory failure before selection is a safe backend-unavailable error unless that consent is present. Once native is selected, operation failures remain native and are never replayed. Construction fallback exposes a `file` label that must be explicitly reused across restart; no credential migration is implicit. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index c6ef0ff..93f35da 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -2,24 +2,24 @@ ## Current Milestone -M1: design and harness for the multi-provider local management experience. +M2: V1 implementation, with Tasks 1 through 10 landed and the actual Web UI assets next. ## MVP Vertical Slices | Slice | User Value | Status | Verification | | --- | --- | --- | --- | | V0: Harness and approved design | Shared product, architecture, security, UI, and test contracts | Approved | Doc self-review and user approval | -| V1: Provider lifecycle end to end | Add, test, activate, switch, and restart through local UI without changing Codex provider | Planned | Full acceptance flow in `docs/TESTING.md` | -| V2: Existing-user migration | Upgrade v0.2.2 flat config without losing provider access | Planned | Migration/rollback fixtures on all platforms | +| V1: Provider lifecycle end to end | Add, test, activate, switch, and restart through local UI without changing Codex provider | In progress: Tasks 1-10 complete; Web UI and acceptance remain | Full acceptance flow in `docs/TESTING.md` | +| V2: Existing-user migration | Upgrade v0.2.2 flat config without losing provider access | Core implementation landed; L3 platform verification pending | Migration/rollback fixtures on all platforms | | V3: Cross-platform hardening | Reliable macOS/Windows UI and Linux CLI | Planned | Platform CI, E2E, screenshots, accessibility | ## V1 Internal Delivery Order -1. Supervisor/worker protocol and deterministic process tests. -2. Provider registry, credential adapters, and migration-safe storage. -3. Admin API and security boundary. -4. Onboarding and guided utility UI. -5. Codex bootstrap, end-to-end activation, and restart verification. +1. Landed: supervisor/worker protocol and deterministic process tests. +2. Landed: provider registry, credential adapters, and migration-safe storage. +3. Landed: Admin API, security boundary, and supervisor-backed CLI routing. +4. Next: onboarding and guided utility Web UI assets. +5. Core behavior landed; UI acceptance and L3 platform verification remain for Codex bootstrap, end-to-end activation, and restart. ## Later diff --git a/docs/STATUS.md b/docs/STATUS.md index 9685dd6..1ee5109 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 9 are complete and Task 10 is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 10 are complete and Task 11, the actual Web UI assets, is next. ## Done @@ -27,14 +27,15 @@ V1 implementation: provider lifecycle end to end. - Completed Task 7 reliable worker lifecycle management and its review fixes. Startup requires ready, correlated configure acknowledgement, and matching health; snapshot generations advance only after matching acknowledgement; concurrent restart calls reuse the current operation before inspecting a new snapshot, while an unshared restart validates the complete snapshot before drain; send failures synchronously cancel observed acknowledgement waiters; stop and restart bound drain, TERM, and KILL, confirm exclusive fixed-port release, retain control after termination timeout, and isolate old child epochs. Unexpected exits recover with cancellable injected-clock backoff and fail immediately on the fifth crash in 60 seconds. Strict-unhandled coverage passes 22/22 focused tests; the exact full gate passes 126/126 non-capture unit assertions, 7/7 isolated capture assertions, and 12/12 integration tests; the all-top-level compatibility command passes 133/133. Syntax checking covers 19 source files, and the runtime audit reports zero vulnerabilities. - Completed Task 8 sanitized activity, transactional legacy migration, and provider orchestration. Activity uses an exact event allowlist, expanded recursive denylist, bounded retention, atomic replacement, and ownership-checked degraded locks. Migration uses descriptor-safe source/registry reads, symlink rejection, byte-exact exclusive backups, transaction locking, exclusive final-path registry creation, identity-checked rollback, committed-state reconciliation, and canonical blockers. Provider CRUD/test/activation rejects active edits, refuses redirects, reconciles committed mutations, serializes selected-credential operations, and deterministically rolls an uncertain candidate back with a newer generation. Focused coverage passes 42/42; exact full coverage passes 168/168 core, 7/7 capture, and 12/12 integration assertions; syntax covers 22 source files. All tests use temporary paths, injected adapters, and loopback mocks; real HOME/keyring/live-network migration remains an L3 gate. - Completed Task 9 secured loopback Admin API and supervisor composition. Private 32-byte control/session/CSRF tokens, exact Host/Origin and disabled-CORS enforcement, bounded exact request contracts, positive public/error projections, every approved Admin route, activity pagination, read-only settings, static-asset allowlisting, active-only proxy lifecycle facades with in-flight command reuse, migration-before-registry composition, readiness-gated `0600` state, construction/listen rollback, separate Codex/state filesystem adapters, and idempotent signal cleanup are covered. Focused coverage passes 42/42; exact full coverage passes 179/179 core, 7/7 capture, and 23/23 integration assertions; syntax checks 26 source files and the runtime audit reports zero vulnerabilities. Actual UI assets, CLI routing, real HOME/keyring/network use, and cross-platform browser security remain later gates. +- Completed Task 10 supervisor-backed CLI routing. State discovery and authenticated Admin API dispatch now cover `ui`, lifecycle commands, and provider commands; legacy secret-bearing flags are rejected, owner identity is checked before shutdown signaling, and failed supervisor spawns receive bounded cleanup. Focused coverage passes 27/27; the exact full gate passes 202/202 core, 7/7 capture, and 24/24 integration assertions; syntax checks 27 source files, the runtime audit reports zero vulnerabilities, and diff plus static secret-pattern checks pass. Actual Web UI assets and real HOME/keyring/network, browser, signal, and cross-platform behavior remain later L3 gates. ## Blocked -- No current blocker is recorded for Task 10. +- No current blocker is recorded for Task 11. ## Next -1. Execute Task 10: route CLI commands through the supervisor. +1. Execute Task 11: land the actual Web UI assets against the existing Admin API. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks @@ -43,3 +44,4 @@ V1 implementation: provider lifecycle end to end. - Cross-platform credential APIs and restart semantics require real macOS and Windows verification. - Provider-registry atomic rename and permission semantics remain unverified on real Windows and Linux hosts. - Task 4 tests inject the native loader and never invoke the real addon loader or touch an OS credential store; native verification remains L3 on every supported system, including Windows and Linux. +- Task 10 tests use temporary homes and injected process/client boundaries; real HOME, native keyrings, external provider traffic, browser launch behavior, and cross-platform process identity and signal handling remain L3. diff --git a/docs/TESTING.md b/docs/TESTING.md index 3641503..411edfc 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -28,6 +28,7 @@ - Task 7 focused suite: `cd node && node --test test/worker-manager.test.mjs test/integration/worker-restart.test.mjs` - Task 9 focused suite: `cd node && node --test test/session-auth.test.mjs test/provider-service.test.mjs test/integration/admin-server.test.mjs` - Task 8 focused suite: `cd node && node --test test/activity-store.test.mjs test/migration.test.mjs test/provider-service.test.mjs` +- Task 10 focused suite: `cd node && node --test test/crp.test.mjs test/integration/crp-lifecycle.test.mjs` - Worker integration suite: `cd node && npm run test:integration` - Runtime audit: `cd node && npm audit --omit=dev` @@ -45,6 +46,8 @@ The Node 22.19 Task 9 focused suite passes 42/42. Exact `npm test` passes 179/17 The Node 22.19 Task 8 focused suite passes 42/42. Exact `npm test` passes 168/168 core assertions, 7/7 isolated capture assertions, and 12/12 integration tests; `npm run lint` syntax-checks 22 source files. Coverage verifies the expanded activity denylist and full-file secret scans, Error/cycle/non-JSON handling, retention, atomic failure preservation, foreign lock replacement, and canonical degraded blocking; descriptor-safe temp-only migration, symlink rejection, byte-exact backups, exclusive final-path registry creation, identity/byte-owned rollback, foreign registry preservation, transaction locks, committed schema reconciliation, and canonical release blockers; active-update rejection, no-follow redirect behavior across two loopback origins, CRUD/credential committed outcomes, replacement rollback safety, selected credentials, explicit serialization, and deterministic `1 -> 2 -> 3` rollback after health or acknowledgement uncertainty. No test accesses a real HOME, native credential backend, or live upstream network. +The Node 22.19 Task 10 focused suite passes 27/27. Exact `npm test` passes 202/202 core assertions, 7/7 isolated capture assertions, and 24/24 integration assertions; `npm run lint` syntax-checks 27 source files and `npm audit --omit=dev` reports zero vulnerabilities. Coverage verifies private state discovery, authenticated CLI dispatch for lifecycle and provider commands, browser-session discovery for `ui`, explicit legacy secret-bearing flag rejection, startup readiness and owner cleanup, `pid` plus `startedAt` shutdown identity, and bounded failed-spawn cleanup without child or state residue. `git diff --check` and the static secret-pattern scan pass. Tests use temporary homes and injected spawn/client boundaries; real HOME, native keyrings, external provider traffic, browser launch behavior, and cross-platform process identity and signal handling remain L3. + `npm run test:unit` retains its public behavior and runs all top-level `test/*.test.mjs` files. The exact `npm test` gate runs the same top-level set without duplication as two sequential groups: every non-capture unit file first, then `capture-store.test.mjs` alone, followed by recursively discovered `test/integration/**/*.test.mjs`. This isolates the polling watcher registration baseline from unrelated unit load and keeps real child-process integration after watcher cleanup. `test:e2e` and the combined `test:all` command are not current gates until the UI, Playwright configuration, and E2E specs land. ## Test Authoring Rules @@ -90,6 +93,8 @@ The Node 22.19 Task 8 focused suite passes 42/42. Exact `npm test` passes 168/16 - Post-ack activation rollback must use a newer generation and confirm both acknowledgement and health before replacing the confirmed snapshot. - A rejected worker start/apply Promise must be treated as an unknown commit once sending was attempted; prior state requires a higher-generation deterministic rollback, while no-prior state requires bounded stop. - Committed/degraded mutation tests must reconcile registry and credential facts without inverse compensation, and replacement rollback must never restore `passed` unless the old secret was restored successfully. +- Lifecycle fake waits must advance the injected clock and simulate owner cleanup. +- Before signaling a process, verify `pid` plus `startedAt` and never mutate state first. ## Test Matrix diff --git a/node/bin/crp.mjs b/node/bin/crp.mjs index e141b58..57e5996 100644 --- a/node/bin/crp.mjs +++ b/node/bin/crp.mjs @@ -5,10 +5,17 @@ import { resolve } from "node:path"; import net from "node:net"; import readline from "node:readline/promises"; import { setTimeout as delay } from "node:timers/promises"; +import { fileURLToPath } from "node:url"; import { DEFAULT_CAPTURE_DB_PATH } from "../src/capture-config.mjs"; import { bootstrapCodexConfig } from "../src/codex/codex-config.mjs"; import { getPaths } from "../src/shared/paths.mjs"; +import { + discoverSupervisor, + ensureSupervisor, + readControlToken, + readSupervisorState +} from "../src/supervisor/supervisor-client.mjs"; const PACKAGE_ROOT = resolve(import.meta.dirname, ".."); const { @@ -60,18 +67,22 @@ function parseCommandLine(argv) { return { command, options }; } -function printHelp() { - console.log("Usage:"); - console.log(" crp check [--json] [--codex-config PATH] [--auth PATH]"); - console.log(" crp init [--json] [--upstream-base-url URL] [--api-key KEY] [--listen-host 127.0.0.1] [--listen-port PORT] [--capture] [--no-capture] [--capture-db-path PATH]"); - console.log(" crp start [--json] [--upstream-base-url URL] [--api-key KEY] [--listen-host 127.0.0.1] [--listen-port PORT] [--capture] [--no-capture] [--capture-db-path PATH] [--debug]"); - console.log(" crp install [same as start]"); - console.log(" crp capture [--json]"); - console.log(" crp status [--json]"); - console.log(" crp stop [--json]"); - console.log(" crp setup [same as start]"); - console.log(" crp guide [--json]"); - console.log(" crp install-cli [--json]"); +function printHelp(writeLine = (line) => console.log(line)) { + writeLine("Usage:"); + writeLine(" crp check [--json] [--codex-config PATH] [--auth PATH]"); + writeLine(" crp init [--json] [--upstream-base-url URL] [--api-key KEY]"); + writeLine(" crp ui [--no-open] [--json]"); + writeLine(" crp start [--json]"); + writeLine(" crp install [same as start]"); + writeLine(" crp capture [--json]"); + writeLine(" crp status [--json]"); + writeLine(" crp stop [--json]"); + writeLine(" crp restart [--json]"); + writeLine(" crp shutdown [--json]"); + writeLine(" crp provider list|add|test|activate|delete [--json]"); + writeLine(" crp setup [same as start]"); + writeLine(" crp guide [--json]"); + writeLine(" crp install-cli [--json]"); } function maybePrintJson(options, payload) { @@ -305,10 +316,23 @@ function isProcessAlive(pid) { } function getManagedServiceInfo() { + const supervisorState = readSupervisorState({ path: STATE_FILE, adminPort: 15101 }); + if (supervisorState) { + return { + state: { + ...supervisorState, + alive: isProcessAlive(supervisorState.supervisorPid) + }, + staleStateRemoved: false + }; + } const state = loadManagedState(); if (!state) { return { state: null, staleStateRemoved: false }; } + if (!Number.isSafeInteger(state.pid) || state.pid < 1) { + return { state: null, staleStateRemoved: false }; + } const alive = Boolean(state.pid && isProcessAlive(state.pid)); if (!alive) { return { state: null, staleStateRemoved: removeManagedState() }; @@ -475,31 +499,31 @@ function buildGuideData() { preferredImplementation: "node", commands: { inspect: "crp check --json", - init: "crp init --upstream-base-url --api-key [--capture] [--capture-db-path PATH] --json", - start: "crp start --upstream-base-url --api-key [--capture] [--capture-db-path PATH] --json", - captureOn: "crp capture on --json", - captureOff: "crp capture off --json", - captureStatus: "crp capture status --json", + providerAdd: "crp provider add --name --base-url --api-key --json", + providerTest: "crp provider test --id --model --json", + providerActivate: "crp provider activate --id --json", + start: "crp start --json", status: "crp status --json", + ui: "crp ui --json", stop: "crp stop --json", + shutdown: "crp shutdown --json", installCli: "npm install -g @cluic/codex-remote-proxy", runWithoutInstall: "npx @cluic/codex-remote-proxy guide --json" }, expectedFlow: [ - "Run check --json first.", - "Read runtimeStatus and recommendedImplementation.", - "If node dependencies are ready, use the node path.", - "Optionally set [codex_remote_proxy] in ~/.codex/config.toml or run init once to save upstream settings under ~/.codex-remote-proxy/.", - "Run start. It will resolve settings from CLI flags, then environment variables, then ~/.codex/config.toml [codex_remote_proxy], then saved config, and only prompt as a last resort.", - "Use `crp capture on|off` for runtime capture toggling; manual edits to the runtime proxy config also hot-apply capture.enabled.", - "start launches the proxy in the background by default and patches ~/.codex/config.toml.", - "Use status --json to confirm the proxy is healthy." + "Add a provider with `crp provider add --name --base-url --api-key --json`.", + "Validate it with `crp provider test --id --model --json`.", + "Activate it with `crp provider activate --id --json`.", + "Start the proxy through the supervisor with `crp start --json`.", + "Confirm supervisor and worker health with `crp status --json`.", + "Open the local management UI with `crp ui --json`.", + "When finished, stop the supervisor with `crp shutdown --json`." ], notes: [ "The start command creates a backup only when it changes ~/.codex/config.toml.", - "The proxy configuration and state are stored under ~/.codex-remote-proxy/.", - "Use CRP_UPSTREAM_BASE_URL and CRP_UPSTREAM_API_KEY when you want non-interactive start without exposing secrets in later AI interactions.", - "The optional ~/.codex/config.toml [codex_remote_proxy] section supports upstream_base_url, upstream_api_key, capture_enabled, and capture_db_path as non-interactive sources." + "Provider credential is write-only and never echoed by CLI or Admin API responses.", + "The supervisor owns proxy lifecycle and applies the active provider when start is requested.", + "The proxy configuration and state are stored under ~/.codex-remote-proxy/." ] }; } @@ -529,15 +553,17 @@ function buildCheckData(options) { }, codexRemoteProxy: { upstreamBaseUrl: codexRemoteProxyUpstreamBaseUrl, - upstreamApiKeyPreview: typeof codexRemoteProxyUpstreamApiKey === "string" ? maskSecret(codexRemoteProxyUpstreamApiKey) : null, + credentialConfigured: typeof codexRemoteProxyUpstreamApiKey === "string" + && codexRemoteProxyUpstreamApiKey.length > 0, captureEnabled: codexRemoteProxyCaptureEnabled, captureDbPath: codexRemoteProxyCaptureDbPath }, auth: { authMode: authData.auth_mode ?? null, - openAiApiKeyPreview: typeof authData.OPENAI_API_KEY === "string" ? maskSecret(authData.OPENAI_API_KEY) : null, - accessTokenPrefix: typeof authData?.tokens?.access_token === "string" ? authData.tokens.access_token.slice(0, 2) : null, - accessTokenLength: typeof authData?.tokens?.access_token === "string" ? authData.tokens.access_token.length : 0 + openAiApiKeyConfigured: typeof authData.OPENAI_API_KEY === "string" + && authData.OPENAI_API_KEY.length > 0, + accessTokenConfigured: typeof authData?.tokens?.access_token === "string" + && authData.tokens.access_token.length > 0 }, runtimeStatus, configSources: { @@ -556,7 +582,19 @@ function buildCheckData(options) { implementation: { configPath: NODE_RUNTIME_CONFIG_PATH, configExists: existsSync(NODE_RUNTIME_CONFIG_PATH), - runtimeConfig: runtimeProxyConfig, + runtimeConfig: runtimeProxyConfig === null ? null : { + server: { + host: runtimeProxyConfig.server?.host ?? null, + port: runtimeProxyConfig.server?.port ?? null + }, + upstream: { + baseUrl: runtimeProxyConfig.upstream?.baseUrl ?? null + }, + capture: { + enabled: runtimeProxyConfig.capture?.enabled === true, + dbPath: runtimeProxyConfig.capture?.dbPath ?? null + } + }, startCommand: startCommand(NODE_RUNTIME_CONFIG_PATH) }, recommendedImplementation: "node", @@ -572,8 +610,8 @@ function printHumanCheck(data) { console.log(`Codex auth path: ${data.authPath}`); console.log(""); console.log(`auth_mode: ${data.auth.authMode || "(unknown)"}`); - console.log(`OPENAI_API_KEY: ${data.auth.openAiApiKeyPreview || "(missing)"}`); - console.log(`tokens.access_token: ${data.auth.accessTokenPrefix ? `${data.auth.accessTokenPrefix}..., len=${data.auth.accessTokenLength}` : "(missing)"}`); + console.log(`OPENAI_API_KEY configured: ${data.auth.openAiApiKeyConfigured ? "yes" : "no"}`); + console.log(`tokens.access_token configured: ${data.auth.accessTokenConfigured ? "yes" : "no"}`); console.log(""); console.log("Codex [model_providers.OpenAI]:"); console.log(` base_url: ${data.codexOpenAiProvider.baseUrl || "(missing)"}`); @@ -582,7 +620,7 @@ function printHumanCheck(data) { console.log(""); console.log("Codex [codex_remote_proxy]:"); console.log(` upstream_base_url: ${data.codexRemoteProxy.upstreamBaseUrl || "(missing)"}`); - console.log(` upstream_api_key: ${data.codexRemoteProxy.upstreamApiKeyPreview || "(missing)"}`); + console.log(` credential configured: ${data.codexRemoteProxy.credentialConfigured ? "yes" : "no"}`); console.log(` capture_enabled: ${data.codexRemoteProxy.captureEnabled ?? "(missing)"}`); console.log(` capture_db_path: ${data.codexRemoteProxy.captureDbPath || "(missing)"}`); console.log(""); @@ -889,11 +927,10 @@ function checkCommand(options) { function guideCommand(options) { const data = buildGuideData(); if (!maybePrintJson(options, data)) { - console.log("AI guide:"); - console.log(` Entrypoint: ${data.entrypoint}`); - console.log(` Inspect first: ${data.commands.inspect}`); - console.log(` Install: ${data.commands.install}`); - console.log(` Status: ${data.commands.status}`); + console.log("CRP V1 guide:"); + for (const step of data.expectedFlow) { + console.log(` ${step}`); + } } } @@ -947,11 +984,21 @@ async function captureCommand(options, action) { if (action === "status") { const runtime = loadRuntimeProxyConfig(); const state = loadManagedState(); + const supervisorState = readSupervisorState({ path: STATE_FILE, adminPort: 15101 }); const payload = { ok: true, - running: Boolean(state?.pid && isProcessAlive(state.pid)), - persistedConfig: loadUserConfig(), - runtimeConfig: runtime?.capture ?? null + running: supervisorState + ? isProcessAlive(supervisorState.supervisorPid) + : Boolean(state?.pid && isProcessAlive(state.pid)), + managedBySupervisor: supervisorState !== null, + persistedConfig: { + captureEnabled: loadUserConfig().captureEnabled === true, + captureDbPath: loadUserConfig().captureDbPath ?? null + }, + runtimeConfig: runtime?.capture ? { + enabled: runtime.capture.enabled === true, + dbPath: runtime.capture.dbPath ?? null + } : null }; if (state?.proxyUrl && payload.running) { try { @@ -972,6 +1019,9 @@ async function captureCommand(options, action) { return; } + if (readSupervisorState({ path: STATE_FILE, adminPort: 15101 })) { + throw new Error("Capture settings are read-only in this version."); + } const enabled = action === "on"; const persistedConfig = applyUserConfigPatch({ captureEnabled: enabled, @@ -981,7 +1031,10 @@ async function captureCommand(options, action) { const payload = { ok: true, action, - persistedConfig, + persistedConfig: { + captureEnabled: persistedConfig.captureEnabled === true, + captureDbPath: persistedConfig.captureDbPath ?? null + }, runtimeUpdated: false, message: "" }; @@ -1049,8 +1102,311 @@ async function installCliCommand(options) { } } -async function main() { - const argv = process.argv.slice(2); +function writePayload(options, payload, stdout, humanMessage) { + if (options.json) { + stdout(`${JSON.stringify(payload, null, 2)}\n`); + return; + } + stdout(`${humanMessage}\n`); +} + +export function openManagementUrl(url, { + platform = process.platform, + spawnImpl = spawn +} = {}) { + let parsed; + try { + parsed = new URL(url); + } catch { + throw new Error("The local management URL is invalid."); + } + if (parsed.protocol !== "http:" || parsed.hostname !== "127.0.0.1" + || !parsed.port || parsed.pathname !== "/" || parsed.search + || !/^#token=[A-Za-z0-9_-]{43}$/.test(parsed.hash)) { + throw new Error("The local management URL is invalid."); + } + let command; + let args; + if (platform === "darwin") { + command = "open"; + args = [url]; + } else if (platform === "win32") { + command = "cmd"; + args = ["/d", "/s", "/c", "start", "", url]; + } else { + command = "xdg-open"; + args = [url]; + } + const child = spawnImpl(command, args, { + detached: true, + stdio: "ignore", + shell: false, + windowsHide: true + }); + child.once?.("error", () => {}); + child.unref(); + return child; +} + +function parseProviderOptions(argv) { + const action = argv[1]; + if (!["list", "add", "test", "activate", "delete"].includes(action)) { + throw new Error("Unknown provider action."); + } + const { options } = parseCommandLine(["provider", ...argv.slice(2)]); + const allowed = { + list: new Set(["json"]), + add: new Set([ + "json", + "name", + "base-url", + "api-key", + "auth-header", + "auth-scheme", + "model-mode", + "model-override", + "fallback-consent" + ]), + test: new Set(["json", "id", "model"]), + activate: new Set(["json", "id"]), + delete: new Set(["json", "id"]) + }[action]; + if (Object.keys(options).some((field) => !allowed.has(field))) { + throw new Error("The provider command contains an unsupported option."); + } + return { action, options }; +} + +function requiredOption(options, name) { + const value = options[name]; + if (typeof value !== "string" || value.trim().length === 0) { + throw new Error(`The provider ${name} option is required.`); + } + return value.trim(); +} + +async function dispatchProviderCommand(argv, dependencies) { + const { action, options } = parseProviderOptions(argv); + const context = await dependencies.ensureSupervisorImpl({ + paths: dependencies.paths, + adminPort: dependencies.adminPort + }); + let method; + let path; + let body; + if (action === "list") { + method = "GET"; + path = "/providers"; + } else if (action === "add") { + method = "POST"; + path = "/providers"; + const provider = { + name: requiredOption(options, "name"), + baseUrl: requiredOption(options, "base-url") + }; + for (const [option, field] of [ + ["auth-header", "authHeader"], + ["auth-scheme", "authScheme"], + ["model-mode", "modelMode"], + ["model-override", "modelOverride"] + ]) { + if (typeof options[option] === "string") provider[field] = options[option]; + } + body = { + provider, + credential: requiredOption(options, "api-key"), + fallbackConsent: options["fallback-consent"] === true + }; + } else { + const id = encodeURIComponent(requiredOption(options, "id")); + if (action === "test") { + method = "POST"; + path = `/providers/${id}/test`; + body = { model: requiredOption(options, "model") }; + } else if (action === "activate") { + method = "POST"; + path = `/providers/${id}/activate`; + } else { + method = "DELETE"; + path = `/providers/${id}`; + } + } + const result = await context.client.request(method, path, body); + writePayload(options, { + ok: true, + action, + ...result + }, dependencies.stdout, `Provider ${action} completed.`); +} + +function parseSupervisorOptions(argv) { + const { command, options } = parseCommandLine(argv); + const allowed = { + ui: new Set(["json", "no-open"]), + start: new Set(["json"]), + install: new Set(["json"]), + setup: new Set(["json"]), + status: new Set(["json"]), + stop: new Set(["json"]), + restart: new Set(["json"]), + shutdown: new Set(["json"]) + }[command]; + if (Object.keys(options).some((field) => !allowed.has(field))) { + throw new Error(`The ${command} command contains an unsupported option.`); + } + return { command, options }; +} + +async function dispatchSupervisorCommand(argv, dependencies) { + const supervisorCommands = new Set([ + "ui", + "start", + "install", + "setup", + "status", + "stop", + "restart", + "shutdown", + "provider" + ]); + if (!supervisorCommands.has(argv[0])) return false; + if (argv[0] === "provider") { + await dispatchProviderCommand(argv, dependencies); + return true; + } + const { command, options } = parseSupervisorOptions(argv); + const { + paths, + adminPort, + ensureSupervisorImpl, + discoverSupervisorImpl, + stdout, + killProcess, + isProcessAliveImpl, + wait, + now, + shutdownTimeoutMs, + readControlTokenImpl, + openManagementUrlImpl + } = dependencies; + const discoveryOptions = { paths, adminPort }; + + if (command === "ui") { + const context = await ensureSupervisorImpl(discoveryOptions); + const token = readControlTokenImpl({ path: paths.controlTokenPath }); + const url = `${context.origin}/#token=${token}`; + const opened = options["no-open"] !== true; + if (opened) openManagementUrlImpl(url); + writePayload(options["no-open"] === true ? { ...options, json: true } : options, { + ok: true, + opened, + origin: context.origin, + supervisorPid: context.state.supervisorPid, + url + }, stdout, opened ? "CRP management UI opened." : url); + return true; + } + + if (command === "status") { + const context = await discoverSupervisorImpl(discoveryOptions); + const payload = context === null + ? { ok: true, running: false, reason: "supervisor_not_running" } + : { ok: true, running: true, ...context.status }; + writePayload(options, payload, stdout, payload.running + ? "CRP supervisor is running." + : "CRP supervisor is not running."); + return true; + } + + if (command === "stop") { + const context = await discoverSupervisorImpl(discoveryOptions); + if (context === null) { + writePayload(options, { + ok: true, + stopped: false, + reason: "supervisor_not_running" + }, stdout, "No running proxy worker to stop."); + return true; + } + const result = await context.client.request("POST", "/proxy/stop"); + const payload = { + ok: true, + stopped: result?.worker?.phase === "stopped", + worker: result?.worker ?? null + }; + writePayload(options, payload, stdout, "Proxy worker stopped."); + return true; + } + + if (command === "shutdown") { + const context = await discoverSupervisorImpl(discoveryOptions); + if (context === null) { + writePayload(options, { + ok: true, + shutdown: false, + reason: "supervisor_not_running" + }, stdout, "CRP supervisor is not running."); + return true; + } + const latest = await context.client.request("GET", "/status"); + const supervisorPid = context.state.supervisorPid; + if (latest?.supervisor?.pid !== supervisorPid + || latest?.supervisor?.startedAt !== context.state.startedAt) { + throw new Error("Supervisor identity changed; shutdown was cancelled."); + } + killProcess(supervisorPid, "SIGTERM"); + const deadline = now() + shutdownTimeoutMs; + while ((isProcessAliveImpl(supervisorPid) || existsSync(paths.statePath)) && now() < deadline) { + await wait(Math.min(100, deadline - now())); + } + if (isProcessAliveImpl(supervisorPid)) { + throw new Error("The supervisor did not stop in time."); + } + if (existsSync(paths.statePath)) { + throw new Error("The supervisor state was not cleaned up in time."); + } + writePayload(options, { + ok: true, + shutdown: true, + supervisorPid, + workerStopped: true + }, stdout, "CRP supervisor stopped."); + return true; + } + + const context = await ensureSupervisorImpl(discoveryOptions); + if (command === "restart") { + const result = await context.client.request("POST", "/proxy/restart"); + writePayload(options, { + ok: true, + command: "restart", + supervisorPid: context.state.supervisorPid, + worker: result?.worker ?? null + }, stdout, "Proxy worker restarted."); + return true; + } + + let codexBootstrap = null; + if (context.status?.codex?.configured !== true) { + const bootstrap = await context.client.request("POST", "/codex/bootstrap"); + codexBootstrap = bootstrap?.result ?? null; + } + const result = await context.client.request("POST", "/proxy/start"); + const payload = { + ok: true, + command: "start", + implementation: "node", + supervisorPid: context.state.supervisorPid, + proxyUrl: context.status?.codex?.proxyUrl ?? "http://127.0.0.1:15100", + worker: result?.worker ?? null, + codexBootstrap + }; + if (command === "install" || command === "setup") payload.deprecated = true; + writePayload(options, payload, stdout, "Codex Remote Proxy is ready."); + return true; +} + +async function main(argv = process.argv.slice(2)) { if (argv[0] === "capture") { const action = argv[1]; const options = {}; @@ -1082,9 +1438,54 @@ async function main() { throw new Error(`Unknown command: ${command}`); } -try { - await main(); -} catch (error) { - console.error(`Error: ${error.message}`); - process.exit(1); +export async function runCli(argv, { + stdout = (text) => process.stdout.write(text), + stderr = (text) => process.stderr.write(text), + paths = getPaths(), + adminPort = 15101, + ensureSupervisorImpl = ensureSupervisor, + discoverSupervisorImpl = discoverSupervisor, + readControlTokenImpl = readControlToken, + openManagementUrlImpl = (url) => openManagementUrl(url), + killProcess = (pid, signal) => process.kill(pid, signal), + isProcessAlive: isProcessAliveImpl = isProcessAlive, + wait = (milliseconds) => delay(milliseconds), + now = () => Date.now(), + shutdownTimeoutMs = 8_000 +} = {}) { + try { + if (argv.length === 0 || argv[0] === "--help" || argv[0] === "-h") { + printHelp((line) => stdout(`${line}\n`)); + return 0; + } + const handled = await dispatchSupervisorCommand(argv, { + paths, + adminPort, + ensureSupervisorImpl, + discoverSupervisorImpl, + stdout, + killProcess, + isProcessAliveImpl, + wait, + now, + shutdownTimeoutMs, + readControlTokenImpl, + openManagementUrlImpl + }); + if (!handled) await main(argv); + return 0; + } catch (error) { + stderr(`Error: ${error?.message || "CRP could not complete the command."}\n`); + return 1; + } +} + +function isDirectExecution(metaUrl = import.meta.url, argv1 = process.argv[1]) { + return typeof argv1 === "string" + && argv1.length > 0 + && resolve(fileURLToPath(metaUrl)) === resolve(argv1); +} + +if (isDirectExecution()) { + process.exitCode = await runCli(process.argv.slice(2)); } diff --git a/node/src/supervisor/supervisor-client.mjs b/node/src/supervisor/supervisor-client.mjs new file mode 100644 index 0000000..8116bdf --- /dev/null +++ b/node/src/supervisor/supervisor-client.mjs @@ -0,0 +1,578 @@ +import { spawn } from "node:child_process"; +import { + chmodSync, + closeSync, + constants, + fchmodSync, + fstatSync, + lstatSync, + mkdirSync, + openSync, + readFileSync +} from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { setTimeout as delay } from "node:timers/promises"; + +import { CrpError } from "../shared/errors.mjs"; + +const PACKAGE_ROOT = fileURLToPath(new URL("../..", import.meta.url)); +const SUPERVISOR_ENTRY = fileURLToPath(new URL("./supervisor-entry.mjs", import.meta.url)); +const TOKEN_PATTERN = /^[A-Za-z0-9_-]{43}$/; +const ERROR_CODE_PATTERN = /^[A-Z][A-Z0-9_]*$/; +const MAX_STATE_BYTES = 64 * 1_024; +const MAX_RESPONSE_BYTES = 1024 * 1_024; +const SAFE_ERROR_DETAIL_FIELDS = new Set([ + "field", + "reason", + "committed", + "degraded", + "generation", + "httpStatus" +]); +const STATE_FIELDS = new Set([ + "schemaVersion", + "supervisorPid", + "startedAt", + "admin", + "worker" +]); +const ADMIN_FIELDS = new Set(["host", "port", "authority", "origin"]); +const WORKER_FIELDS = new Set([ + "phase", + "pid", + "generation", + "state", + "restartCount", + "startedAt", + "error" +]); +const CHILD_STATE_FIELDS = new Set([ + "phase", + "configured", + "generation", + "listening", + "listenHost", + "listenPort", + "inFlight" +]); +const WORKER_ERROR_FIELDS = new Set(["code", "message"]); +const DEFAULT_FILE_OPERATIONS = { + chmodSync, + closeSync, + fchmodSync, + fstatSync, + lstatSync, + mkdirSync, + openSync, + readFileSync +}; +const pendingEnsures = new Map(); + +function clientError(code, { status = 500 } = {}) { + const contracts = { + SUPERVISOR_CLIENT_INPUT_INVALID: [ + "The supervisor client input is invalid.", + "Use the fixed local supervisor address and a documented Admin path." + ], + SUPERVISOR_TOKEN_INVALID: [ + "The local control token is invalid.", + "Stop CRP, repair the private control token file, and restart CRP." + ], + SUPERVISOR_UNAVAILABLE: [ + "The local supervisor is unavailable.", + "Start the local supervisor and try again." + ], + SUPERVISOR_RESPONSE_INVALID: [ + "The local supervisor returned an invalid response.", + "Restart CRP and try again." + ], + SUPERVISOR_START_FAILED: [ + "The local supervisor could not be started.", + "Review the supervisor log and try again." + ], + SUPERVISOR_START_TIMEOUT: [ + "The local supervisor did not become ready in time.", + "Review the supervisor log and try again." + ] + }; + const [message, action] = contracts[code] ?? contracts.SUPERVISOR_UNAVAILABLE; + return new CrpError(code, message, action, { status }); +} + +function isPlainObject(value) { + if (value === null || typeof value !== "object" || Array.isArray(value)) return false; + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function hasExactFields(value, fields) { + return isPlainObject(value) + && Object.keys(value).length === fields.size + && Object.keys(value).every((field) => fields.has(field)); +} + +function isIsoTimestamp(value) { + if (typeof value !== "string") return false; + try { + return new Date(value).toISOString() === value; + } catch { + return false; + } +} + +function isSafePid(value) { + return Number.isSafeInteger(value) && value > 0; +} + +function isNullableSafeInteger(value, { min = 0 } = {}) { + return value === null || (Number.isSafeInteger(value) && value >= min); +} + +function sameIdentity(left, right) { + return left.dev === right.dev && left.ino === right.ino; +} + +function isPrivateMode(stats, platform) { + return platform === "win32" || (stats.mode & 0o777) === 0o600; +} + +function readPrivateFile({ path, fileOperations, platform, maxBytes }) { + const parent = fileOperations.lstatSync(dirname(path)); + if (!parent.isDirectory() || parent.isSymbolicLink() + || platform !== "win32" && (parent.mode & 0o777) !== 0o700) { + throw new TypeError("private file parent is unsafe"); + } + const before = fileOperations.lstatSync(path); + if (!before.isFile() || before.isSymbolicLink() || !isPrivateMode(before, platform) + || before.size < 1 || before.size > maxBytes) { + throw new TypeError("private file path is unsafe"); + } + + let descriptor; + try { + const noFollow = typeof constants.O_NOFOLLOW === "number" ? constants.O_NOFOLLOW : 0; + descriptor = fileOperations.openSync(path, constants.O_RDONLY | noFollow); + const opened = fileOperations.fstatSync(descriptor); + if (!opened.isFile() || !sameIdentity(before, opened) || !isPrivateMode(opened, platform) + || opened.size < 1 || opened.size > maxBytes) { + throw new TypeError("private file identity changed"); + } + const bytes = fileOperations.readFileSync(descriptor); + const after = fileOperations.lstatSync(path); + if (!after.isFile() || after.isSymbolicLink() || !sameIdentity(opened, after)) { + throw new TypeError("private file identity changed"); + } + return Buffer.from(bytes); + } finally { + if (descriptor !== undefined) fileOperations.closeSync(descriptor); + } +} + +function validChildState(state) { + return hasExactFields(state, CHILD_STATE_FIELDS) + && typeof state.phase === "string" && state.phase.length > 0 + && typeof state.configured === "boolean" + && Number.isSafeInteger(state.generation) && state.generation >= 0 + && typeof state.listening === "boolean" + && (state.listenHost === null || typeof state.listenHost === "string") + && isNullableSafeInteger(state.listenPort) + && Number.isSafeInteger(state.inFlight) && state.inFlight >= 0; +} + +function validWorkerError(error) { + return hasExactFields(error, WORKER_ERROR_FIELDS) + && typeof error.code === "string" && ERROR_CODE_PATTERN.test(error.code) + && typeof error.message === "string" && error.message.length > 0; +} + +function validWorkerState(worker) { + return hasExactFields(worker, WORKER_FIELDS) + && typeof worker.phase === "string" && worker.phase.length > 0 + && isNullableSafeInteger(worker.pid, { min: 1 }) + && Number.isSafeInteger(worker.generation) && worker.generation >= 0 + && (worker.state === null || validChildState(worker.state)) + && Number.isSafeInteger(worker.restartCount) && worker.restartCount >= 0 + && (worker.startedAt === null || isIsoTimestamp(worker.startedAt)) + && (worker.error === null || validWorkerError(worker.error)); +} + +function expectedAdmin(adminPort) { + return { + host: "127.0.0.1", + port: adminPort, + authority: `127.0.0.1:${adminPort}`, + origin: `http://127.0.0.1:${adminPort}` + }; +} + +function validateSupervisorState(state, adminPort) { + const admin = expectedAdmin(adminPort); + return hasExactFields(state, STATE_FIELDS) + && state.schemaVersion === 1 + && isSafePid(state.supervisorPid) + && isIsoTimestamp(state.startedAt) + && hasExactFields(state.admin, ADMIN_FIELDS) + && Object.keys(admin).every((field) => state.admin[field] === admin[field]) + && validWorkerState(state.worker); +} + +export function readSupervisorState({ + path, + adminPort = 15101, + fileOperations = DEFAULT_FILE_OPERATIONS, + platform = process.platform +} = {}) { + if (typeof path !== "string" || path.length === 0 + || !Number.isInteger(adminPort) || adminPort < 1 || adminPort > 65_535) { + return null; + } + try { + const bytes = readPrivateFile({ + path, + fileOperations, + platform, + maxBytes: MAX_STATE_BYTES + }); + const state = JSON.parse(bytes.toString("utf8")); + return validateSupervisorState(state, adminPort) ? structuredClone(state) : null; + } catch { + return null; + } +} + +export function readControlToken({ + path, + fileOperations = DEFAULT_FILE_OPERATIONS, + platform = process.platform +} = {}) { + try { + if (typeof path !== "string" || path.length === 0) throw new TypeError("token path is invalid"); + const text = readPrivateFile({ path, fileOperations, platform, maxBytes: 44 }).toString("utf8"); + const token = text.endsWith("\n") ? text.slice(0, -1) : text; + if (!TOKEN_PATTERN.test(token) || text !== token && text !== `${token}\n`) { + throw new TypeError("token format is invalid"); + } + const bytes = Buffer.from(token, "base64url"); + if (bytes.length !== 32 || bytes.toString("base64url") !== token) { + throw new TypeError("token encoding is invalid"); + } + return token; + } catch (error) { + if (error instanceof CrpError) throw error; + throw clientError("SUPERVISOR_TOKEN_INVALID"); + } +} + +function validateOrigin(origin) { + try { + const url = new URL(origin); + if (url.protocol !== "http:" || url.hostname !== "127.0.0.1" + || !url.port || url.username || url.password + || url.pathname !== "/" || url.search || url.hash) { + throw new TypeError("origin is unsafe"); + } + return url.origin; + } catch { + throw clientError("SUPERVISOR_CLIENT_INPUT_INVALID", { status: 400 }); + } +} + +function validateRequest(method, path) { + if (!["GET", "POST", "PATCH", "DELETE"].includes(method) + || typeof path !== "string" || !path.startsWith("/") || path.startsWith("//") + || /[\\\u0000-\u001f\u007f]/.test(path)) { + throw clientError("SUPERVISOR_CLIENT_INPUT_INVALID", { status: 400 }); + } +} + +function sanitizeErrorDetails(details) { + if (!isPlainObject(details)) return {}; + const projected = {}; + for (const [key, value] of Object.entries(details)) { + if (SAFE_ERROR_DETAIL_FIELDS.has(key) || value === "[REDACTED]") { + projected[key] = structuredClone(value); + } + } + return projected; +} + +function publicResponseError(payload, status) { + const error = payload?.error; + if (!isPlainObject(error) + || typeof error.code !== "string" || !ERROR_CODE_PATTERN.test(error.code) + || typeof error.message !== "string" || error.message.length === 0 + || typeof error.action !== "string" || error.action.length === 0) { + return clientError("SUPERVISOR_RESPONSE_INVALID"); + } + const projected = new CrpError(error.code, error.message, error.action, { + status, + details: sanitizeErrorDetails(error.details) + }); + if (typeof error.requestId === "string" && error.requestId.length > 0) { + projected.requestId = error.requestId; + } + return projected; +} + +export class SupervisorClient { + #origin; + #controlToken; + #fetch; + #requestTimeoutMs; + + constructor({ + origin, + controlTokenPath, + fetchImpl = globalThis.fetch, + requestTimeoutMs = 8_000, + fileOperations = DEFAULT_FILE_OPERATIONS, + platform = process.platform + } = {}) { + if (typeof fetchImpl !== "function" + || !Number.isSafeInteger(requestTimeoutMs) || requestTimeoutMs < 1) { + throw clientError("SUPERVISOR_CLIENT_INPUT_INVALID", { status: 400 }); + } + this.#origin = validateOrigin(origin); + this.#controlToken = readControlToken({ + path: controlTokenPath, + fileOperations, + platform + }); + this.#fetch = fetchImpl; + this.#requestTimeoutMs = requestTimeoutMs; + } + + request(method, path, body) { + validateRequest(method, path); + return this.#performRequest(method, path, body); + } + + async #performRequest(method, path, body) { + const headers = { authorization: `Bearer ${this.#controlToken}` }; + const options = { + method, + headers, + signal: AbortSignal.timeout(this.#requestTimeoutMs) + }; + if (body !== undefined) { + headers["content-type"] = "application/json; charset=utf-8"; + options.body = JSON.stringify(body); + if (Buffer.byteLength(options.body, "utf8") > 64 * 1_024) { + throw clientError("SUPERVISOR_CLIENT_INPUT_INVALID", { status: 400 }); + } + } + + let response; + try { + response = await this.#fetch(`${this.#origin}/api/v1${path}`, options); + } catch { + throw clientError("SUPERVISOR_UNAVAILABLE"); + } + let text; + try { + text = await response.text(); + if (Buffer.byteLength(text, "utf8") > MAX_RESPONSE_BYTES) throw new Error("response too large"); + } catch { + throw clientError("SUPERVISOR_RESPONSE_INVALID"); + } + let payload; + try { + payload = JSON.parse(text); + } catch { + throw clientError("SUPERVISOR_RESPONSE_INVALID"); + } + if (!response.ok) throw publicResponseError(payload, response.status); + if (!isPlainObject(payload)) throw clientError("SUPERVISOR_RESPONSE_INVALID"); + return payload; + } +} + +function defaultIsProcessAlive(pid) { + try { + process.kill(pid, 0); + return true; + } catch { + return false; + } +} + +export function spawnDetachedSupervisor({ + paths, + home = dirname(paths?.globalHome ?? ""), + spawnImpl = spawn, + fileOperations = DEFAULT_FILE_OPERATIONS +} = {}) { + if (!paths || typeof paths.globalHome !== "string" || typeof paths.logPath !== "string" + || typeof home !== "string" || home.length === 0 || typeof spawnImpl !== "function") { + throw clientError("SUPERVISOR_START_FAILED"); + } + fileOperations.mkdirSync(paths.globalHome, { recursive: true, mode: 0o700 }); + try { fileOperations.chmodSync(paths.globalHome, 0o700); } catch {} + let logDescriptor; + try { + logDescriptor = fileOperations.openSync(paths.logPath, "a", 0o600); + try { fileOperations.fchmodSync(logDescriptor, 0o600); } catch {} + const child = spawnImpl(process.execPath, [SUPERVISOR_ENTRY], { + cwd: resolve(PACKAGE_ROOT), + env: { ...process.env, CRP_HOME: home }, + detached: true, + stdio: ["ignore", logDescriptor, logDescriptor], + shell: false + }); + child.once?.("error", () => {}); + child.unref(); + return child; + } catch { + throw clientError("SUPERVISOR_START_FAILED"); + } finally { + if (logDescriptor !== undefined) fileOperations.closeSync(logDescriptor); + } +} + +export async function discoverSupervisor({ + paths, + adminPort = 15101, + fetchImpl = globalThis.fetch, + fileOperations = DEFAULT_FILE_OPERATIONS, + platform = process.platform, + isProcessAlive = defaultIsProcessAlive, + requestTimeoutMs = 2_000 +} = {}) { + const state = readSupervisorState({ + path: paths?.statePath, + adminPort, + fileOperations, + platform + }); + if (!state || !isProcessAlive(state.supervisorPid)) return null; + const client = new SupervisorClient({ + origin: state.admin.origin, + controlTokenPath: paths.controlTokenPath, + fetchImpl, + requestTimeoutMs, + fileOperations, + platform + }); + let status; + try { + status = await client.request("GET", "/status"); + } catch (error) { + if (error?.code === "SUPERVISOR_UNAVAILABLE") return null; + throw error; + } + if (!isPlainObject(status.supervisor) + || status.supervisor.pid !== state.supervisorPid + || status.supervisor.startedAt !== state.startedAt) { + return null; + } + return { + origin: state.admin.origin, + state, + status, + client, + spawned: false + }; +} + +async function ensureSupervisorInternal({ + paths, + adminPort, + spawnSupervisor, + fetchImpl, + fileOperations, + platform, + isProcessAlive, + requestTimeoutMs, + timeoutMs, + pollIntervalMs, + now, + wait, + home +}) { + const discoveryOptions = { + paths, + adminPort, + fetchImpl, + fileOperations, + platform, + isProcessAlive, + requestTimeoutMs + }; + const existing = await discoverSupervisor(discoveryOptions); + if (existing) return existing; + + let spawnedChild; + try { + spawnedChild = spawnSupervisor({ paths, home }); + } catch { + throw clientError("SUPERVISOR_START_FAILED"); + } + + try { + const deadline = now() + timeoutMs; + while (now() <= deadline) { + const discovered = await discoverSupervisor(discoveryOptions); + if (discovered) return { ...discovered, spawned: true }; + if (now() >= deadline) break; + await wait(Math.min(pollIntervalMs, deadline - now())); + } + throw clientError("SUPERVISOR_START_TIMEOUT"); + } catch (error) { + try { + if (typeof spawnedChild?.kill === "function") spawnedChild.kill("SIGTERM"); + } catch { + // Preserve the readiness/discovery error when cleanup is unavailable. + } + throw error; + } +} + +export function ensureSupervisor({ + paths, + adminPort = 15101, + spawnSupervisor = (options) => spawnDetachedSupervisor(options), + fetchImpl = globalThis.fetch, + fileOperations = DEFAULT_FILE_OPERATIONS, + platform = process.platform, + isProcessAlive = defaultIsProcessAlive, + requestTimeoutMs = 2_000, + timeoutMs = 8_000, + pollIntervalMs = 100, + now = () => Date.now(), + wait = (milliseconds) => delay(milliseconds), + home = dirname(paths?.globalHome ?? "") +} = {}) { + if (!paths || typeof paths.statePath !== "string" || typeof paths.controlTokenPath !== "string" + || typeof paths.globalHome !== "string" || typeof paths.logPath !== "string" + || !Number.isInteger(adminPort) || adminPort < 1 || adminPort > 65_535 + || typeof spawnSupervisor !== "function" || typeof fetchImpl !== "function" + || typeof isProcessAlive !== "function" || typeof now !== "function" || typeof wait !== "function" + || !Number.isSafeInteger(timeoutMs) || timeoutMs < 1 + || !Number.isSafeInteger(pollIntervalMs) || pollIntervalMs < 1) { + return Promise.reject(clientError("SUPERVISOR_CLIENT_INPUT_INVALID", { status: 400 })); + } + const key = `${resolve(paths.statePath)}\u0000${adminPort}`; + if (pendingEnsures.has(key)) return pendingEnsures.get(key); + const operation = ensureSupervisorInternal({ + paths, + adminPort, + spawnSupervisor, + fetchImpl, + fileOperations, + platform, + isProcessAlive, + requestTimeoutMs, + timeoutMs, + pollIntervalMs, + now, + wait, + home + }); + pendingEnsures.set(key, operation); + const clear = () => { + if (pendingEnsures.get(key) === operation) pendingEnsures.delete(key); + }; + void operation.then(clear, clear); + return operation; +} diff --git a/node/test/codex-config.test.mjs b/node/test/codex-config.test.mjs index 8d2fd3b..0150d66 100644 --- a/node/test/codex-config.test.mjs +++ b/node/test/codex-config.test.mjs @@ -27,32 +27,6 @@ import { getPaths } from "../src/shared/paths.mjs"; const PROXY_URL = "http://127.0.0.1:15100"; const PACKAGE_ROOT = resolve(import.meta.dirname, ".."); const CLI_PATH = join(PACKAGE_ROOT, "bin", "crp.mjs"); -const START_PAYLOAD_KEYS = [ - "captureActive", - "captureConfigured", - "captureDbPath", - "captureRestartRequired", - "captureState", - "codexConfigBackup", - "codexConfigPath", - "configSource", - "failedWriteCount", - "health", - "implementation", - "lastWriteErrorAt", - "lastWriteErrorMessage", - "listenHost", - "listenPort", - "logFile", - "managedStatePath", - "message", - "ok", - "pid", - "proxyConfigPath", - "proxyUrl", - "upstreamBaseUrl" -].sort(); - function makeHomeEnv(homeDir) { return { ...process.env, @@ -71,68 +45,6 @@ function runCrp(args, env) { }); } -function isProcessAlive(pid) { - if (!Number.isInteger(pid) || pid <= 0) { - return false; - } - try { - process.kill(pid, 0); - return true; - } catch { - return false; - } -} - -async function waitForProcessExit(pid, timeoutMs = 2_000) { - const deadline = Date.now() + timeoutMs; - while (isProcessAlive(pid) && Date.now() < deadline) { - await new Promise((resolveWait) => setTimeout(resolveWait, 25)); - } - return !isProcessAlive(pid); -} - -async function cleanupCliHome({ homeDir, env, pid }) { - const statePath = join(homeDir, ".codex-remote-proxy", "state.json"); - let managedPid = pid; - let cleanupError; - - try { - if (!managedPid && existsSync(statePath)) { - try { - managedPid = JSON.parse(readFileSync(statePath, "utf8")).pid; - } catch { - managedPid = null; - } - } - if (existsSync(statePath)) { - runCrp(["stop", "--json"], env); - } - if (isProcessAlive(managedPid)) { - try { - process.kill(managedPid, "SIGTERM"); - } catch { - // The process may exit between the liveness check and the signal. - } - } - if (isProcessAlive(managedPid) && !(await waitForProcessExit(managedPid))) { - try { - process.kill(managedPid, "SIGKILL"); - } catch { - // The process may exit between the liveness check and the signal. - } - if (!(await waitForProcessExit(managedPid))) { - cleanupError = new Error(`Could not stop test proxy process ${managedPid}`); - } - } - } finally { - rmSync(homeDir, { recursive: true, force: true }); - } - - if (cleanupError) { - throw cleanupError; - } -} - test("patchCodexConfigText creates the fixed OpenAI provider and preserves custom providers", () => { const original = [ 'model_provider = "custom"', @@ -399,7 +311,7 @@ test("bootstrapCodexConfig keeps both backups when changes share a timestamp", ( }); for (const command of ["start", "install", "setup"]) { - test(`${command} preserves the JSON contract and propagates the Codex backup`, async (t) => { + test(`${command} safely rejects legacy direct-start options`, () => { const homeDir = mkdtempSync(join(os.tmpdir(), `crp-${command}-home-`)); const env = makeHomeEnv(homeDir); const codexDir = join(homeDir, ".codex"); @@ -408,13 +320,6 @@ for (const command of ["start", "install", "setup"]) { const originalConfig = 'model_provider = "custom"\n'; const placeholderCredential = ["placeholder", "credential", randomUUID()].join("-"); const upstreamBaseUrl = "http://127.0.0.1:1/v1"; - let managedPid = null; - let cleanupPromise; - const cleanup = () => { - cleanupPromise ??= cleanupCliHome({ homeDir, env, pid: managedPid }); - return cleanupPromise; - }; - t.after(cleanup); try { mkdirSync(codexDir, { recursive: true }); @@ -429,34 +334,17 @@ for (const command of ["start", "install", "setup"]) { placeholderCredential ], env); - assert.equal(result.status, 0, result.stderr); - const payload = JSON.parse(result.stdout); - managedPid = payload.pid; - - assert.deepEqual(Object.keys(payload).sort(), START_PAYLOAD_KEYS); - assert.equal(payload.ok, true); - assert.equal(payload.implementation, "node"); - assert.equal(payload.listenHost, "127.0.0.1"); - assert.equal(payload.upstreamBaseUrl, upstreamBaseUrl); - assert.equal(payload.codexConfigPath, codexConfigPath); - assert.equal(payload.configSource.upstreamBaseUrl, "cli"); - assert.equal(payload.configSource.apiKey, "cli"); - assert.equal(payload.configSource.listenPort, "auto"); - assert.equal(payload.message, "Proxy configured and started"); - assert.match(payload.proxyUrl, /^http:\/\/127\.0\.0\.1:\d+$/); - - const state = JSON.parse(readFileSync(statePath, "utf8")); - assert.equal(state.pid, payload.pid); - assert.equal(state.proxyUrl, payload.proxyUrl); - assert.equal(state.codexConfigBackup, payload.codexConfigBackup); - assert.equal(existsSync(payload.codexConfigBackup), true); - assert.equal(readFileSync(payload.codexConfigBackup, "utf8"), originalConfig); + assert.equal(result.status, 1); + assert.equal(result.stdout, ""); assert.equal( - readFileSync(codexConfigPath, "utf8"), - patchCodexConfigText(originalConfig, payload.proxyUrl) + result.stderr, + `Error: The ${command} command contains an unsupported option.\n` ); + assert.equal(result.stderr.includes(placeholderCredential), false); + assert.equal(readFileSync(codexConfigPath, "utf8"), originalConfig); + assert.equal(existsSync(statePath), false); } finally { - await cleanup(); + rmSync(homeDir, { recursive: true, force: true }); } }); } diff --git a/node/test/crp.test.mjs b/node/test/crp.test.mjs index 4fa57b9..e3ddfa3 100644 --- a/node/test/crp.test.mjs +++ b/node/test/crp.test.mjs @@ -1,12 +1,69 @@ import test from "node:test"; import assert from "node:assert/strict"; import { spawnSync } from "node:child_process"; -import { mkdtempSync, rmSync } from "node:fs"; +import * as realFileOperations from "node:fs"; +import { + chmodSync, + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + rmSync, + statSync, + writeFileSync +} from "node:fs"; import os from "node:os"; import { dirname, join } from "node:path"; -import { fileURLToPath } from "node:url"; +import { fileURLToPath, pathToFileURL } from "node:url"; + +import { getPaths } from "../src/shared/paths.mjs"; +import { + discoverSupervisor, + SupervisorClient, + ensureSupervisor, + readControlToken, + readSupervisorState, + spawnDetachedSupervisor +} from "../src/supervisor/supervisor-client.mjs"; const PACKAGE_ROOT = join(dirname(fileURLToPath(import.meta.url)), ".."); +const CLI_URL = pathToFileURL(join(PACKAGE_ROOT, "bin", "crp.mjs")).href; +const CONTROL_TOKEN = Buffer.alloc(32, 0x41).toString("base64url"); +const OTHER_CONTROL_TOKEN = Buffer.alloc(32, 0x42).toString("base64url"); + +function supervisorState(pid = 4242, adminPort = 15101) { + return { + schemaVersion: 1, + supervisorPid: pid, + startedAt: "2026-07-13T08:00:00.000Z", + admin: { + host: "127.0.0.1", + port: adminPort, + authority: `127.0.0.1:${adminPort}`, + origin: `http://127.0.0.1:${adminPort}` + }, + worker: { + phase: "stopped", + pid: null, + generation: 0, + state: null, + restartCount: 0, + startedAt: null, + error: null + } + }; +} + +function prepareSupervisorFiles(homeDir, { state = supervisorState(), token = CONTROL_TOKEN } = {}) { + const paths = getPaths(homeDir); + mkdirSync(paths.globalHome, { recursive: true, mode: 0o700 }); + chmodSync(paths.globalHome, 0o700); + writeFileSync(paths.statePath, `${JSON.stringify(state)}\n`, { mode: 0o600 }); + writeFileSync(paths.controlTokenPath, `${token}\n`, { mode: 0o600 }); + chmodSync(paths.statePath, 0o600); + chmodSync(paths.controlTokenPath, 0o600); + return paths; +} function makeTempHome() { return mkdtempSync(join(os.tmpdir(), "crp-home-")); @@ -28,6 +85,43 @@ function runCrp(args, env) { }); } +async function invokeCli(args, overrides = {}) { + const stdout = []; + const stderr = []; + const { runCli } = await import(CLI_URL); + const status = await runCli(args, { + stdout: (text) => stdout.push(text), + stderr: (text) => stderr.push(text), + ...overrides + }); + return { status, stdout: stdout.join(""), stderr: stderr.join("") }; +} + +function adminStatus(pid = 4242, worker = { phase: "stopped", pid: null, generation: 0 }) { + return { + supervisor: { pid, startedAt: "2026-07-13T08:00:00.000Z" }, + activeProviderId: "provider-1", + activeProvider: { id: "provider-1", name: "Primary", credentialConfigured: true }, + generation: worker.generation, + worker, + codex: { + configured: false, + modelProvider: "OpenAI", + proxyUrl: "http://127.0.0.1:15100" + } + }; +} + +function discoveredContext(client, status = adminStatus()) { + return { + origin: "http://127.0.0.1:15101", + state: supervisorState(status.supervisor.pid), + status, + client, + spawned: false + }; +} + test("check does not emit sqlite experimental warnings", () => { const homeDir = makeTempHome(); try { @@ -55,16 +149,731 @@ test("init without config fails cleanly without sqlite warnings", () => { } }); -test("start without config fails cleanly without sqlite warnings", () => { +test("start reports a safe supervisor failure and exit code without real spawn", async () => { + const result = await invokeCli(["start", "--json"], { + ensureSupervisorImpl: async () => { + throw new Error("The local supervisor could not be started."); + } + }); + + assert.equal(result.status, 1); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, "Error: The local supervisor could not be started.\n"); +}); + +test("imports the CLI module without executing a command", () => { + const cliUrl = pathToFileURL(join(PACKAGE_ROOT, "bin", "crp.mjs")).href; + const result = spawnSync(process.execPath, [ + "--input-type=module", + "--eval", + `await import(${JSON.stringify(cliUrl)}); process.stdout.write("imported\\n");` + ], { cwd: PACKAGE_ROOT, encoding: "utf8" }); + + assert.equal(result.status, 0, result.stderr); + assert.equal(result.stdout, "imported\n"); +}); + +test("prints the approved supervisor CLI help without discovery", async () => { + let discovered = false; + const result = await invokeCli(["--help"], { + discoverSupervisorImpl: async () => { discovered = true; }, + ensureSupervisorImpl: async () => { discovered = true; } + }); + + assert.equal(result.status, 0); + assert.equal(result.stderr, ""); + for (const line of [ + "crp ui [--no-open] [--json]", + "crp restart [--json]", + "crp shutdown [--json]", + "crp provider list|add|test|activate|delete [--json]" + ]) { + assert.match(result.stdout, new RegExp(line.replace(/[|[\]]/g, "\\$&"))); + } + assert.equal(discovered, false); +}); + +test("guide JSON describes the V1 supervisor flow without legacy mutations", () => { + const homeDir = makeTempHome(); + try { + const result = runCrp(["guide", "--json"], makeHomeEnv(homeDir)); + + assert.equal(result.status, 0, result.stderr); + const guide = JSON.parse(result.stdout); + const flowCommands = [ + ["providerAdd", "crp provider add --name --base-url --api-key --json"], + ["providerTest", "crp provider test --id --model --json"], + ["providerActivate", "crp provider activate --id --json"], + ["start", "crp start --json"], + ["status", "crp status --json"], + ["ui", "crp ui --json"], + ["shutdown", "crp shutdown --json"] + ]; + const expectedFlow = guide.expectedFlow.join("\n"); + let previousIndex = -1; + for (const [name, command] of flowCommands) { + assert.equal(guide.commands[name], command); + const commandIndex = expectedFlow.indexOf(command); + assert.ok(commandIndex > previousIndex, `${command} must appear in V1 flow order`); + previousIndex = commandIndex; + } + + const serializedGuide = JSON.stringify(guide); + assert.doesNotMatch(serializedGuide, /crp start[^"\n]*--(?:upstream-base-url|api-key|capture)/); + assert.doesNotMatch(serializedGuide, /crp capture (?:on|off)\b/); + assert.match(serializedGuide, /credential[^"\n]*write-only/i); + assert.match(serializedGuide, /credential[^"\n]*(?:not echoed|never echoed)/i); + } finally { + rmSync(homeDir, { recursive: true, force: true }); + } +}); + +test("guide human output presents the V1 flow without stale fields", () => { const homeDir = makeTempHome(); try { - const result = runCrp(["start"], makeHomeEnv(homeDir)); - const output = `${result.stdout}\n${result.stderr}`; + const result = runCrp(["guide"], makeHomeEnv(homeDir)); + + assert.equal(result.status, 0, result.stderr); + assert.equal(result.stderr, ""); + assert.doesNotMatch(result.stdout, /undefined/); + for (const command of [ + "crp provider add --name --base-url --api-key --json", + "crp start --json", + "crp ui --json" + ]) { + assert.match(result.stdout, new RegExp(command.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"))); + } + assert.doesNotMatch(result.stdout, /crp start[^\n]*--(?:upstream-base-url|api-key|capture)/); + assert.doesNotMatch(result.stdout, /crp capture (?:on|off)\b/); + } finally { + rmSync(homeDir, { recursive: true, force: true }); + } +}); + +test("status never starts a missing supervisor", async () => { + let ensureCalls = 0; + const result = await invokeCli(["status", "--json"], { + discoverSupervisorImpl: async () => null, + ensureSupervisorImpl: async () => { + ensureCalls += 1; + throw new Error("must not start"); + } + }); + + assert.equal(result.status, 0, result.stderr); + assert.deepEqual(JSON.parse(result.stdout), { + ok: true, + running: false, + reason: "supervisor_not_running" + }); + assert.equal(ensureCalls, 0); +}); + +test("rejects legacy start options before supervisor discovery or mutation", async () => { + const clientCalls = []; + let ensureCalls = 0; + let discoverCalls = 0; + const result = await invokeCli([ + "start", + "--upstream-base-url", "https://provider.example/v1", + "--api-key", "legacy-complete-secret", + "--json" + ], { + ensureSupervisorImpl: async () => { + ensureCalls += 1; + return discoveredContext({ + async request(...args) { + clientCalls.push(args); + return { worker: { phase: "running", pid: 8001, generation: 1 } }; + } + }); + }, + discoverSupervisorImpl: async () => { + discoverCalls += 1; + return null; + } + }); + + assert.equal(result.status, 1); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, "Error: The start command contains an unsupported option.\n"); + assert.doesNotMatch(result.stderr, /legacy-complete-secret/); + assert.equal(ensureCalls, 0); + assert.equal(discoverCalls, 0); + assert.deepEqual(clientCalls, []); +}); + +test("rejects misspelled restart options before supervisor discovery or mutation", async () => { + const clientCalls = []; + let ensureCalls = 0; + let discoverCalls = 0; + const result = await invokeCli(["restart", "--jsno"], { + ensureSupervisorImpl: async () => { + ensureCalls += 1; + return discoveredContext({ + async request(...args) { + clientCalls.push(args); + return { worker: { phase: "running", pid: 8002, generation: 1 } }; + } + }); + }, + discoverSupervisorImpl: async () => { + discoverCalls += 1; + return null; + } + }); + + assert.equal(result.status, 1); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, "Error: The restart command contains an unsupported option.\n"); + assert.equal(ensureCalls, 0); + assert.equal(discoverCalls, 0); + assert.deepEqual(clientCalls, []); +}); + +test("routes start, stop, and restart through exact Admin methods with empty bodies", async () => { + const calls = []; + const client = { + async request(method, path, body) { + calls.push([method, path, body]); + if (path === "/codex/bootstrap") return { result: { changed: true, backupCreated: true } }; + if (path === "/proxy/stop") { + return { worker: { phase: "stopped", pid: null, generation: 1 } }; + } + return { worker: { phase: "running", pid: path.endsWith("restart") ? 8002 : 8001, generation: 1 } }; + } + }; + const context = discoveredContext(client); + const dependencies = { + ensureSupervisorImpl: async () => context, + discoverSupervisorImpl: async () => context + }; + + const started = await invokeCli(["start", "--json"], dependencies); + assert.equal(started.status, 0, started.stderr); + assert.deepEqual(calls.splice(0), [ + ["POST", "/codex/bootstrap", undefined], + ["POST", "/proxy/start", undefined] + ]); + assert.equal(JSON.parse(started.stdout).worker.pid, 8001); + + const stopped = await invokeCli(["stop", "--json"], dependencies); + assert.equal(stopped.status, 0, stopped.stderr); + assert.deepEqual(calls.splice(0), [["POST", "/proxy/stop", undefined]]); + assert.equal(JSON.parse(stopped.stdout).stopped, true); + + const restarted = await invokeCli(["restart", "--json"], dependencies); + assert.equal(restarted.status, 0, restarted.stderr); + assert.deepEqual(calls.splice(0), [["POST", "/proxy/restart", undefined]]); + assert.equal(JSON.parse(restarted.stdout).worker.pid, 8002); +}); + +test("install and setup add only one deprecation field to start JSON", async () => { + const client = { + async request(method, path) { + assert.equal(method, "POST"); + assert.equal(path, "/proxy/start"); + return { worker: { phase: "running", pid: 8001, generation: 1 } }; + } + }; + const status = adminStatus(); + status.codex.configured = true; + const dependencies = { + ensureSupervisorImpl: async () => discoveredContext(client, status) + }; + const start = JSON.parse((await invokeCli(["start", "--json"], dependencies)).stdout); + + for (const alias of ["install", "setup"]) { + const result = await invokeCli([alias, "--json"], dependencies); + assert.equal(result.status, 0, result.stderr); + const payload = JSON.parse(result.stdout); + assert.equal(payload.deprecated, true); + delete payload.deprecated; + assert.deepEqual(payload, start); + } +}); + +test("shutdown cross-checks authenticated status before SIGTERM and never removes state", async (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = prepareSupervisorFiles(homeDir); + const calls = []; + const signals = []; + let alive = true; + let clock = 0; + let stateExistedBeforeSupervisorCleanup = false; + const client = { + async request(method, path, body) { + calls.push([method, path, body]); + if (path === "/status") return adminStatus(4242); + throw new Error("unexpected request"); + } + }; + const result = await invokeCli(["shutdown", "--json"], { + paths, + discoverSupervisorImpl: async () => discoveredContext(client), + killProcess(pid, signal) { + signals.push([pid, signal]); + alive = false; + }, + isProcessAlive: () => alive, + wait: async (milliseconds) => { + stateExistedBeforeSupervisorCleanup = existsSync(paths.statePath); + rmSync(paths.statePath, { force: true }); + clock += milliseconds; + }, + now: () => clock + }); + + assert.equal(result.status, 0, result.stderr); + assert.deepEqual(calls, [["GET", "/status", undefined]]); + assert.deepEqual(signals, [[4242, "SIGTERM"]]); + assert.deepEqual(JSON.parse(result.stdout), { + ok: true, + shutdown: true, + supervisorPid: 4242, + workerStopped: true + }); + assert.equal(stateExistedBeforeSupervisorCleanup, true); + assert.equal(existsSync(paths.statePath), false); +}); + +test("shutdown refuses a same-PID replacement without mutating or signalling it", async (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const calls = []; + const signals = []; + const replacementStatus = adminStatus(4242); + replacementStatus.supervisor.startedAt = "2026-07-13T08:01:00.000Z"; + const client = { + async request(method, path, body) { + calls.push([method, path, body]); + if (path === "/proxy/stop") { + return { worker: { phase: "stopped", pid: null, generation: 1 } }; + } + if (path === "/status") return replacementStatus; + throw new Error("unexpected request"); + } + }; + + const result = await invokeCli(["shutdown", "--json"], { + paths: getPaths(homeDir), + discoverSupervisorImpl: async () => discoveredContext(client), + killProcess(pid, signal) { + signals.push([pid, signal]); + }, + isProcessAlive: () => false + }); + + assert.equal(result.status, 1); + assert.match(result.stderr, /identity changed/); + assert.deepEqual(calls, [["GET", "/status", undefined]]); + assert.deepEqual(signals, []); +}); + +test("ui keeps the control token in the fragment and honors no-open", async () => { + const client = { request: async () => assert.fail("ui must not call an Admin mutation") }; + const context = discoveredContext(client); + let openCalls = 0; + const dependencies = { + ensureSupervisorImpl: async () => context, + readControlTokenImpl: () => CONTROL_TOKEN, + openManagementUrlImpl: () => { openCalls += 1; } + }; + + const noOpen = await invokeCli(["ui", "--no-open", "--json"], dependencies); + assert.equal(noOpen.status, 0, noOpen.stderr); + assert.deepEqual(JSON.parse(noOpen.stdout), { + ok: true, + opened: false, + origin: "http://127.0.0.1:15101", + supervisorPid: 4242, + url: `http://127.0.0.1:15101/#token=${CONTROL_TOKEN}` + }); + assert.equal(openCalls, 0); + + const opened = await invokeCli(["ui", "--json"], dependencies); + assert.equal(opened.status, 0, opened.stderr); + assert.equal(JSON.parse(opened.stdout).opened, true); + assert.equal(openCalls, 1); +}); + +test("opens management URLs with platform argv and never a shell", async () => { + const { openManagementUrl } = await import(CLI_URL); + const url = `http://127.0.0.1:15101/#token=${CONTROL_TOKEN}`; + const cases = [ + ["darwin", "open", [url]], + ["linux", "xdg-open", [url]], + ["win32", "cmd", ["/d", "/s", "/c", "start", "", url]] + ]; + for (const [platform, expectedCommand, expectedArgs] of cases) { + let received; + let unrefCalls = 0; + openManagementUrl(url, { + platform, + spawnImpl(command, args, options) { + received = { command, args, options }; + return { once() {}, unref() { unrefCalls += 1; } }; + } + }); + assert.equal(received.command, expectedCommand); + assert.deepEqual(received.args, expectedArgs); + assert.equal(received.options.shell, false); + assert.equal(received.options.detached, true); + assert.equal(unrefCalls, 1); + } +}); + +test("routes automation-safe provider commands without returning the write-only credential", async () => { + const secret = "provider-write-only-complete-secret"; + const calls = []; + const client = { + async request(method, path, body) { + calls.push([method, path, body]); + if (path === "/providers") { + return method === "GET" + ? { providers: [{ id: "provider-1", name: "Primary", credentialConfigured: true }] } + : { provider: { id: "provider-1", name: "Primary", credentialConfigured: true } }; + } + if (path.endsWith("/test")) return { result: { ok: true, code: null } }; + if (path.endsWith("/activate")) { + return { activation: { activeProviderId: "provider/1", generation: 1 } }; + } + return { provider: { id: "provider/1", name: "Primary", credentialConfigured: true } }; + } + }; + const dependencies = { ensureSupervisorImpl: async () => discoveredContext(client) }; + const commands = [ + ["list", [], ["GET", "/providers", undefined]], + ["add", [ + "--name", "Primary", + "--base-url", "https://provider.example/v1", + "--api-key", secret, + "--fallback-consent" + ], ["POST", "/providers", { + provider: { name: "Primary", baseUrl: "https://provider.example/v1" }, + credential: secret, + fallbackConsent: true + }]], + ["test", ["--id", "provider/1", "--model", "test-model"], [ + "POST", "/providers/provider%2F1/test", { model: "test-model" } + ]], + ["activate", ["--id", "provider/1"], [ + "POST", "/providers/provider%2F1/activate", undefined + ]], + ["delete", ["--id", "provider/1"], [ + "DELETE", "/providers/provider%2F1", undefined + ]] + ]; + + for (const [action, args, expectedCall] of commands) { + const result = await invokeCli(["provider", action, ...args, "--json"], dependencies); + assert.equal(result.status, 0, `${action}: ${result.stderr}`); + assert.deepEqual(calls.shift(), expectedCall); + assert.equal(result.stdout.includes(secret), false); + } + assert.deepEqual(calls, []); +}); +test("check and capture JSON positively project legacy config without complete keys", () => { + const homeDir = makeTempHome(); + const secret = "legacy-complete-secret-sentinel"; + try { + const paths = getPaths(homeDir); + const nodeDir = join(paths.globalHome, "node"); + const codexDir = join(homeDir, ".codex"); + mkdirSync(nodeDir, { recursive: true, mode: 0o700 }); + mkdirSync(codexDir, { recursive: true }); + writeFileSync(join(paths.globalHome, "config.json"), JSON.stringify({ + upstreamBaseUrl: "https://provider.example/v1", + apiKey: secret, + captureEnabled: true, + captureDbPath: join(paths.globalHome, "traffic.sqlite3") + }), { mode: 0o600 }); + writeFileSync(join(nodeDir, "proxy-config.json"), JSON.stringify({ + server: { host: "127.0.0.1", port: 15100 }, + upstream: { baseUrl: "https://provider.example/v1", apiKey: secret }, + capture: { enabled: true, dbPath: join(paths.globalHome, "traffic.sqlite3") } + }), { mode: 0o600 }); + writeFileSync(join(codexDir, "config.toml"), [ + "[codex_remote_proxy]", + 'upstream_base_url = "https://provider.example/v1"', + `upstream_api_key = "${secret}"`, + "" + ].join("\n")); + writeFileSync(join(codexDir, "auth.json"), JSON.stringify({ + auth_mode: "chatgpt", + OPENAI_API_KEY: secret, + tokens: { access_token: secret } + })); + const env = makeHomeEnv(homeDir); + + for (const args of [["check", "--json"], ["capture", "status", "--json"]]) { + const result = runCrp(args, env); + assert.equal(result.status, 0, result.stderr); + assert.doesNotMatch(`${result.stdout}\n${result.stderr}`, new RegExp(secret)); + JSON.parse(result.stdout); + } + } finally { + rmSync(homeDir, { recursive: true, force: true }); + } +}); + +test("capture mutation refuses a V1 supervisor state without modifying legacy config", () => { + const homeDir = makeTempHome(); + try { + const paths = prepareSupervisorFiles(homeDir, { state: supervisorState(process.pid) }); + const configPath = join(paths.globalHome, "config.json"); + const original = '{"captureEnabled":false}\n'; + writeFileSync(configPath, original, { mode: 0o600 }); + + const result = runCrp(["capture", "on", "--json"], makeHomeEnv(homeDir)); assert.equal(result.status, 1); - assert.match(output, /Error: Upstream base URL is required/); - assert.doesNotMatch(output, /ExperimentalWarning: SQLite/); + assert.match(result.stderr, /read-only in this version/); + assert.equal(readFileSync(configPath, "utf8"), original); } finally { rmSync(homeDir, { recursive: true, force: true }); } }); + +test("reads only the exact fixed-loopback supervisor state and private canonical token", (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = prepareSupervisorFiles(homeDir); + + assert.deepEqual(readSupervisorState({ path: paths.statePath, adminPort: 15101 }), supervisorState()); + assert.equal(readControlToken({ path: paths.controlTokenPath }), CONTROL_TOKEN); + + writeFileSync(paths.statePath, `${JSON.stringify({ + ...supervisorState(), + admin: { ...supervisorState().admin, origin: "http://attacker.example:15101" } + })}\n`, { mode: 0o600 }); + assert.equal(readSupervisorState({ path: paths.statePath, adminPort: 15101 }), null); + + writeFileSync(paths.statePath, `${JSON.stringify({ + ...supervisorState(), + supervisorPid: Number.MAX_SAFE_INTEGER + 1 + })}\n`, { mode: 0o600 }); + assert.equal(readSupervisorState({ path: paths.statePath, adminPort: 15101 }), null); + + chmodSync(paths.controlTokenPath, 0o644); + assert.throws( + () => readControlToken({ path: paths.controlTokenPath }), + (error) => error?.code === "SUPERVISOR_TOKEN_INVALID" + ); +}); + +test("rejects a control-token path swapped after its descriptor is opened", (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = prepareSupervisorFiles(homeDir); + const originalPath = `${paths.controlTokenPath}.original`; + let swapped = false; + const fileOperations = { + ...realFileOperations, + openSync(path, flags, mode) { + const descriptor = realFileOperations.openSync(path, flags, mode); + if (path === paths.controlTokenPath && !swapped) { + swapped = true; + realFileOperations.renameSync(path, originalPath); + realFileOperations.writeFileSync(path, `${OTHER_CONTROL_TOKEN}\n`, { mode: 0o600 }); + } + return descriptor; + } + }; + + assert.throws( + () => readControlToken({ path: paths.controlTokenPath, fileOperations }), + (error) => error?.code === "SUPERVISOR_TOKEN_INVALID" + ); + assert.equal(readFileSync(paths.controlTokenPath, "utf8"), `${OTHER_CONTROL_TOKEN}\n`); +}); + +test("SupervisorClient pins the Admin origin, bearer, JSON bodies, and safe public errors", async (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = prepareSupervisorFiles(homeDir); + const calls = []; + let response = new Response(JSON.stringify({ supervisor: { pid: 4242 } }), { + status: 200, + headers: { "content-type": "application/json" } + }); + const client = new SupervisorClient({ + origin: "http://127.0.0.1:15101", + controlTokenPath: paths.controlTokenPath, + fetchImpl: async (...args) => { + calls.push(args); + return response; + } + }); + + assert.deepEqual(await client.request("GET", "/status"), { supervisor: { pid: 4242 } }); + assert.equal(calls[0][0], "http://127.0.0.1:15101/api/v1/status"); + assert.equal(calls[0][1].headers.authorization, `Bearer ${CONTROL_TOKEN}`); + assert.equal("content-type" in calls[0][1].headers, false); + assert.equal("body" in calls[0][1], false); + + response = new Response(JSON.stringify({ worker: { phase: "running" } }), { + status: 200, + headers: { "content-type": "application/json" } + }); + await client.request("POST", "/proxy/start"); + assert.equal("content-type" in calls[1][1].headers, false); + assert.equal("body" in calls[1][1], false); + + response = new Response(JSON.stringify({ provider: { id: "provider-1" } }), { + status: 201, + headers: { "content-type": "application/json" } + }); + await client.request("POST", "/providers", { provider: { name: "Primary" }, credential: "write-only" }); + assert.equal(calls[2][1].headers["content-type"], "application/json; charset=utf-8"); + assert.equal(calls[2][1].body, JSON.stringify({ + provider: { name: "Primary" }, + credential: "write-only" + })); + + const secret = "must-not-escape-public-error"; + response = new Response(JSON.stringify({ + error: { + code: "PROVIDER_INPUT_INVALID", + message: "Provider settings are invalid.", + action: "Review the provider settings and try again.", + requestId: "request-1", + details: { field: "name", unknown: secret, authorization: "[REDACTED]" } + } + }), { status: 400, headers: { "content-type": "application/json" } }); + await assert.rejects( + () => client.request("POST", "/providers", {}), + (error) => error?.code === "PROVIDER_INPUT_INVALID" + && error.status === 400 + && error.details.field === "name" + && error.details.authorization === "[REDACTED]" + && JSON.stringify(error).includes(secret) === false + ); +}); + +test("spawns one detached supervisor and shares concurrent bounded discovery", async (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = getPaths(homeDir); + let spawnCalls = 0; + let clock = 0; + const spawnSupervisor = () => { + spawnCalls += 1; + prepareSupervisorFiles(homeDir); + return { pid: 4242 }; + }; + const options = { + paths, + adminPort: 15101, + spawnSupervisor, + isProcessAlive: () => true, + fetchImpl: async () => new Response(JSON.stringify({ + supervisor: { pid: 4242, startedAt: "2026-07-13T08:00:00.000Z" }, + worker: { phase: "stopped", pid: null } + }), { status: 200, headers: { "content-type": "application/json" } }), + now: () => clock, + wait: async (milliseconds) => { clock += milliseconds; }, + timeoutMs: 8_000, + pollIntervalMs: 100 + }; + + const first = ensureSupervisor(options); + const second = ensureSupervisor(options); + assert.equal(first, second); + const [left, right] = await Promise.all([first, second]); + assert.equal(left, right); + assert.equal(left.state.supervisorPid, 4242); + assert.equal(left.spawned, true); + assert.equal(spawnCalls, 1); + + const reused = await ensureSupervisor(options); + assert.equal(reused.spawned, false); + assert.equal(spawnCalls, 1); +}); + +test("discovery rejects a status with the same PID but a different startedAt", async (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = prepareSupervisorFiles(homeDir); + const status = adminStatus(4242); + status.supervisor.startedAt = "2026-07-13T08:01:00.000Z"; + + const discovered = await discoverSupervisor({ + paths, + adminPort: 15101, + isProcessAlive: () => true, + fetchImpl: async () => new Response(JSON.stringify(status), { + status: 200, + headers: { "content-type": "application/json" } + }) + }); + + assert.equal(discovered, null); +}); + +test("detached spawn redirects logs, sets CRP_HOME, and unrefs without a shell", (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = getPaths(homeDir); + let received = null; + let unrefCalls = 0; + const child = spawnDetachedSupervisor({ + paths, + home: homeDir, + spawnImpl(command, args, options) { + received = { command, args, options }; + return { + pid: 4242, + once() {}, + unref() { unrefCalls += 1; } + }; + } + }); + + assert.equal(child.pid, 4242); + assert.equal(received.command, process.execPath); + assert.match(received.args[0], /supervisor-entry\.mjs$/); + assert.equal(received.options.detached, true); + assert.deepEqual(received.options.stdio.slice(0, 1), ["ignore"]); + assert.equal(received.options.stdio[1], received.options.stdio[2]); + assert.equal(received.options.env.CRP_HOME, homeDir); + assert.equal(received.options.shell, false); + assert.equal(unrefCalls, 1); + if (process.platform !== "win32") { + assert.equal(statSync(paths.logPath).mode & 0o777, 0o600); + } +}); + +test("discovery timeout preserves an invalid state file", async (t) => { + const homeDir = makeTempHome(); + t.after(() => rmSync(homeDir, { recursive: true, force: true })); + const paths = prepareSupervisorFiles(homeDir); + const invalidBytes = '{"legacy":true}\n'; + writeFileSync(paths.statePath, invalidBytes, { mode: 0o600 }); + let clock = 0; + const signals = []; + + await assert.rejects( + () => ensureSupervisor({ + paths, + adminPort: 15101, + spawnSupervisor: () => ({ + pid: 5000, + kill(signal) { + signals.push(signal); + throw new Error("cleanup failed"); + } + }), + isProcessAlive: () => true, + fetchImpl: async () => assert.fail("invalid state must not receive the bearer"), + now: () => clock, + wait: async (milliseconds) => { clock += milliseconds; }, + timeoutMs: 200, + pollIntervalMs: 50 + }), + (error) => error?.code === "SUPERVISOR_START_TIMEOUT" + ); + assert.deepEqual(signals, ["SIGTERM"]); + assert.equal(readFileSync(paths.statePath, "utf8"), invalidBytes); +}); diff --git a/node/test/integration/crp-lifecycle.test.mjs b/node/test/integration/crp-lifecycle.test.mjs new file mode 100644 index 0000000..077f759 --- /dev/null +++ b/node/test/integration/crp-lifecycle.test.mjs @@ -0,0 +1,309 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { + chmodSync, + existsSync, + mkdirSync, + mkdtempSync, + rmSync, + writeFileSync +} from "node:fs"; +import { createServer } from "node:net"; +import os from "node:os"; +import { join } from "node:path"; + +import { runCli } from "../../bin/crp.mjs"; +import { getPaths } from "../../src/shared/paths.mjs"; +import { createAdminServer } from "../../src/supervisor/admin-server.mjs"; +import { + discoverSupervisor, + ensureSupervisor, + readControlToken, + readSupervisorState +} from "../../src/supervisor/supervisor-client.mjs"; +import { SessionAuth } from "../../src/supervisor/session-auth.mjs"; + +const SUPERVISOR_PID = 61001; +const STARTED_AT = "2026-07-13T09:00:00.000Z"; + +async function choosePort() { + const server = createServer(); + await new Promise((resolve, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", resolve); + }); + const port = server.address().port; + await new Promise((resolve, reject) => { + server.close((error) => error ? reject(error) : resolve()); + }); + return port; +} + +function providerProfile(id, name, baseUrl, overrides = {}) { + return { + id, + name, + baseUrl, + authHeader: "authorization", + authScheme: "Bearer", + extraHeaders: {}, + modelMode: "passthrough", + modelOverride: null, + lastTestAt: null, + lastTestStatus: "untested", + lastTestCode: null, + createdAt: STARTED_AT, + updatedAt: STARTED_AT, + credentialConfigured: true, + ...overrides + }; +} + +function createServices() { + const providers = []; + let activeProviderId = null; + let workerPid = null; + let generation = 0; + let nextWorkerPid = 62000; + let codexConfigured = false; + + const worker = () => ({ + phase: workerPid === null ? "stopped" : "running", + pid: workerPid, + generation, + state: workerPid === null ? null : { + phase: "running", + configured: true, + generation, + listening: true, + listenHost: "127.0.0.1", + listenPort: 15100, + inFlight: 0 + }, + restartCount: 0, + startedAt: workerPid === null ? null : STARTED_AT, + error: null + }); + + return { + providerService: { + async listProviders() { + return providers.map((provider) => structuredClone(provider)); + }, + async createProvider(input) { + const provider = providerProfile(`provider-${providers.length + 1}`, input.name, input.baseUrl); + providers.push(provider); + return structuredClone(provider); + }, + async deleteProvider(id) { + const index = providers.findIndex((provider) => provider.id === id); + return structuredClone(providers.splice(index, 1)[0]); + }, + async testProvider(id) { + const provider = providers.find((candidate) => candidate.id === id); + Object.assign(provider, { + lastTestAt: STARTED_AT, + lastTestStatus: "passed", + updatedAt: STARTED_AT + }); + return { ok: true, code: null }; + }, + async activate(id) { + activeProviderId = id; + return { + activeProviderId: id, + activeProvider: providers.find((provider) => provider.id === id), + generation, + worker: worker() + }; + }, + async startProxy() { + generation += 1; + workerPid = ++nextWorkerPid; + return worker(); + }, + async stopProxy() { + workerPid = null; + return worker(); + }, + async restartProxy() { + generation += 1; + workerPid = ++nextWorkerPid; + return worker(); + }, + async getStatus() { + return { + activeProviderId, + activeProvider: providers.find((provider) => provider.id === activeProviderId) ?? null, + generation, + worker: worker() + }; + } + }, + activityStore: { list: () => [] }, + settingsService: { + getSettings: async () => ({ + proxyHost: "127.0.0.1", + proxyPort: 15100, + adminHost: "127.0.0.1", + adminPort: null, + captureEnabled: false, + credentialBackend: "injected" + }) + }, + codexService: { + async bootstrap() { + codexConfigured = true; + return { changed: true, backupPath: "/private/injected-backup" }; + }, + async getStatus() { + return { + configured: codexConfigured, + modelProvider: "OpenAI", + proxyUrl: "http://127.0.0.1:15100" + }; + } + }, + diagnosticsService: { exportDiagnostics: async () => ({ created: true }) }, + getWorkerPid: () => workerPid + }; +} + +test("CLI manages one injected supervisor and replaceable worker through the real Admin client", async (t) => { + const home = mkdtempSync(join(os.tmpdir(), "crp-cli-lifecycle-")); + const paths = getPaths(home); + const adminPort = await choosePort(); + const auth = new SessionAuth({ controlTokenPath: paths.controlTokenPath }); + const services = createServices(); + const admin = createAdminServer({ + auth, + ...services, + getSupervisorState: () => ({ pid: SUPERVISOR_PID, startedAt: STARTED_AT }), + host: "127.0.0.1", + port: adminPort + }); + let supervisorAlive = false; + let startPromise = null; + let closePromise = null; + t.after(async () => { + await startPromise?.catch(() => {}); + await closePromise?.catch(() => {}); + await admin.close().catch(() => {}); + auth.close(); + rmSync(home, { recursive: true, force: true }); + }); + + const writeState = async () => { + const address = await admin.listen(); + mkdirSync(paths.globalHome, { recursive: true, mode: 0o700 }); + chmodSync(paths.globalHome, 0o700); + writeFileSync(paths.statePath, `${JSON.stringify({ + schemaVersion: 1, + supervisorPid: SUPERVISOR_PID, + startedAt: STARTED_AT, + admin: address, + worker: { + phase: "stopped", + pid: null, + generation: 0, + state: null, + restartCount: 0, + startedAt: null, + error: null + } + })}\n`, { mode: 0o600 }); + chmodSync(paths.statePath, 0o600); + supervisorAlive = true; + }; + const spawnSupervisor = () => { + startPromise ??= writeState(); + return { pid: SUPERVISOR_PID }; + }; + const discoveryOptions = { + paths, + adminPort, + isProcessAlive: () => supervisorAlive + }; + const ensureSupervisorImpl = () => ensureSupervisor({ + ...discoveryOptions, + spawnSupervisor, + wait: async () => { await startPromise; } + }); + const discoverSupervisorImpl = () => discoverSupervisor(discoveryOptions); + const killProcess = (pid, signal) => { + assert.equal(pid, SUPERVISOR_PID); + assert.equal(signal, "SIGTERM"); + supervisorAlive = false; + closePromise ??= admin.close().then(() => { + rmSync(paths.statePath, { force: true }); + }); + }; + const dependencies = { + paths, + adminPort, + ensureSupervisorImpl, + discoverSupervisorImpl, + readControlTokenImpl: () => readControlToken({ path: paths.controlTokenPath }), + openManagementUrlImpl: () => assert.fail("--no-open must not launch a browser"), + killProcess, + isProcessAlive: () => supervisorAlive, + wait: async () => { await closePromise; }, + now: () => 0 + }; + const outputs = []; + const invoke = async (args) => { + const stdout = []; + const stderr = []; + const status = await runCli(args, { + ...dependencies, + stdout: (text) => stdout.push(text), + stderr: (text) => stderr.push(text) + }); + const result = { status, stdout: stdout.join(""), stderr: stderr.join("") }; + outputs.push(result); + assert.equal(status, 0, result.stderr); + return JSON.parse(result.stdout); + }; + + const ui = await invoke(["ui", "--no-open", "--json"]); + assert.equal(ui.supervisorPid, SUPERVISOR_PID); + assert.equal(supervisorAlive, true); + + const secret = "integration-provider-complete-secret"; + const added = await invoke([ + "provider", "add", + "--name", "Primary", + "--base-url", "https://provider.example/v1", + "--api-key", secret, + "--json" + ]); + assert.equal(added.provider.id, "provider-1"); + assert.equal((await invoke(["provider", "list", "--json"])).providers.length, 1); + assert.equal((await invoke([ + "provider", "test", "--id", "provider-1", "--model", "test-model", "--json" + ])).result.ok, true); + assert.equal((await invoke([ + "provider", "activate", "--id", "provider-1", "--json" + ])).activation.activeProviderId, "provider-1"); + + const started = await invoke(["start", "--json"]); + const firstWorkerPid = started.worker.pid; + assert.equal(Number.isSafeInteger(firstWorkerPid), true); + const restarted = await invoke(["restart", "--json"]); + assert.notEqual(restarted.worker.pid, firstWorkerPid); + const statusAfterRestart = await invoke(["status", "--json"]); + assert.equal(statusAfterRestart.supervisor.pid, SUPERVISOR_PID); + assert.equal(statusAfterRestart.worker.pid, restarted.worker.pid); + + const stopped = await invoke(["stop", "--json"]); + assert.equal(stopped.worker.phase, "stopped"); + assert.equal(supervisorAlive, true); + assert.equal((await invoke(["status", "--json"])).running, true); + + const shutdown = await invoke(["shutdown", "--json"]); + assert.equal(shutdown.supervisorPid, SUPERVISOR_PID); + assert.equal(supervisorAlive, false); + assert.equal(existsSync(paths.statePath), false); + assert.equal(outputs.some(({ stdout, stderr }) => `${stdout}\n${stderr}`.includes(secret)), false); + assert.equal(services.getWorkerPid(), null); +}); From d9bb27158d3f9a7736ce7604a3d74c7f79eeca25 Mon Sep 17 00:00:00 2001 From: cluic Date: Mon, 13 Jul 2026 22:46:11 +0800 Subject: [PATCH 14/44] docs: define bilingual Task11 UI plan --- docs/AGENT_COORDINATION.md | 8 +- docs/AI_HANDOFF.md | 4 +- docs/ROADMAP.md | 6 +- docs/STATUS.md | 6 +- docs/UIUX.md | 22 +- ...07-13-crp-task11-ui-i18n-implementation.md | 448 ++++++++++++++++++ .../specs/2026-07-13-crp-ui-i18n-design.md | 161 +++++++ 7 files changed, 642 insertions(+), 13 deletions(-) create mode 100644 docs/superpowers/plans/2026-07-13-crp-task11-ui-i18n-implementation.md create mode 100644 docs/superpowers/specs/2026-07-13-crp-ui-i18n-design.md diff --git a/docs/AGENT_COORDINATION.md b/docs/AGENT_COORDINATION.md index e14d078..2c49405 100644 --- a/docs/AGENT_COORDINATION.md +++ b/docs/AGENT_COORDINATION.md @@ -4,7 +4,7 @@ | Workstream | Owner | Scope | Files/Areas | Status | Depends On | No-Edit Areas | | --- | --- | --- | --- | --- | --- | --- | -| V1 sequential implementation | None | Task 10 supervisor-backed CLI routing is complete; Task 11 actual Web UI assets are next but unclaimed | No files are currently claimed; the Task 11 writer must declare exact files and no-edit areas here before editing | Task 10 complete; Task 11 not started | Completed Tasks 1 through 10 | All files remain no-edit until the next single writer records a bounded Task 11 scope; no simultaneous writable agent | +| Task 11 guided bilingual Web UI | `/root/task11_ui` | Implement the user-approved English/Simplified Chinese static UI and deterministic browser acceptance from base `7a87466` | `node/ui/index.html`; `node/ui/styles.css`; `node/ui/app.js`; `node/playwright.config.mjs`; `node/test/e2e/onboarding.spec.mjs`; `node/test/e2e/provider-switch.spec.mjs`; `node/test/e2e/restart-and-errors.spec.mjs`; only if shared setup is required, `node/test/e2e/crp-ui-fixture.mjs` | Claimed; implementation not started | Completed Tasks 1-10; approved 2026-07-13 UI/i18n addendum and Task 11 plan | All `node/src/**`, `node/bin/**`, existing tests outside `node/test/e2e/**`, package manifests, root release docs, API/data/security contracts, credential/migration/lifecycle code, and every file not listed in Files/Areas; no simultaneous writable agent | ## Shared Contracts @@ -23,10 +23,12 @@ ## Locks / Avoid Editing -- No writable work is currently authorized; the next task implementer must claim a bounded scope here, and review agents remain read-only. +- `/root/task11_ui` is the only authorized Task 11 implementation writer. Review agents remain read-only. +- `node/package.json` and `node/package-lock.json` are no-edit by default because Playwright and the E2E scripts landed in Task 1; a discovered manifest defect must return to the coordinator for a separate scope decision. +- The optional shared E2E helper, if needed, must use the exact path `node/test/e2e/crp-ui-fixture.mjs`; no other fixture file is authorized. - Future agents must not change shared contracts without first updating the owning doc and coordination row. - Credential, migration, and lifecycle work must be isolated from UI styling work until contracts pass review. ## Decisions Needed -Subagent-driven execution is selected. Task 11 actual Web UI assets are next but unclaimed. Its writer must derive the exact file boundary and verification gate from the approved plan and record them in the active-workstream row before editing. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. +Subagent-driven execution is selected. The Task 11 writer must follow `docs/superpowers/specs/2026-07-13-crp-ui-i18n-design.md` and `docs/superpowers/plans/2026-07-13-crp-task11-ui-i18n-implementation.md`, return verification evidence to the coordinator, and must not expand the file boundary. Work remains on the dedicated branch `codex/harness-product-design`; no linked worktree or parallel writable work is in use. diff --git a/docs/AI_HANDOFF.md b/docs/AI_HANDOFF.md index b62fa0c..4ab9cd0 100644 --- a/docs/AI_HANDOFF.md +++ b/docs/AI_HANDOFF.md @@ -6,7 +6,7 @@ CRP preserves ChatGPT login/remote features while routing Codex model traffic to ## Current Scope -V1 implementation is underway. Tasks 1 through 10 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, versioned worker IPC, reliable fixed-port lifecycle management, sanitized activity, transactional migration, provider orchestration, the secured loopback Admin control plane, and supervisor-backed CLI routing. Task 11, the actual Web UI assets, is next. Read `docs/PRD.md`, the formal design spec, and `docs/superpowers/plans/2026-07-10-crp-v1-implementation.md` before changing code. +V1 implementation is underway. Tasks 1 through 10 have landed, including atomic provider metadata, secure credential adapters, snapshot-based proxy settings, versioned worker IPC, reliable fixed-port lifecycle management, sanitized activity, transactional migration, provider orchestration, the secured loopback Admin control plane, and supervisor-backed CLI routing. Task 11 is now the claimed implementation slice, but no UI code has landed. Read `docs/PRD.md`, the formal design spec, `docs/superpowers/specs/2026-07-13-crp-ui-i18n-design.md`, and `docs/superpowers/plans/2026-07-13-crp-task11-ui-i18n-implementation.md` before changing UI code. ## Architecture @@ -24,7 +24,7 @@ One authenticated local OS user. Admin API is loopback-only, origin/host checked ## Current Progress -Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The written specification and detailed V1 plan are approved, subagent-driven sequential execution is selected, and Tasks 1 through 10 are complete. +Architecture, provider model, core flows, UI direction, errors, testing, and MVP boundary were visually reviewed and approved on 2026-07-10. The user approved the Task 11 Overview visual and required complete English/Simplified Chinese UI coverage on 2026-07-13. The runtime-dictionary design and bounded single-writer plan are recorded; Tasks 1 through 10 are complete and Task 11 implementation has not started. ## How To Run Current Code diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 93f35da..0674a51 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -2,14 +2,14 @@ ## Current Milestone -M2: V1 implementation, with Tasks 1 through 10 landed and the actual Web UI assets next. +M2: V1 implementation, with Tasks 1 through 10 landed and the approved bilingual Web UI implementation in progress. ## MVP Vertical Slices | Slice | User Value | Status | Verification | | --- | --- | --- | --- | | V0: Harness and approved design | Shared product, architecture, security, UI, and test contracts | Approved | Doc self-review and user approval | -| V1: Provider lifecycle end to end | Add, test, activate, switch, and restart through local UI without changing Codex provider | In progress: Tasks 1-10 complete; Web UI and acceptance remain | Full acceptance flow in `docs/TESTING.md` | +| V1: Provider lifecycle end to end | Add, test, activate, switch, and restart through local UI without changing Codex provider | In progress: Tasks 1-10 complete; Overview visual and English/Simplified Chinese UI design approved; implementation and acceptance remain | Full acceptance flow in `docs/TESTING.md` | | V2: Existing-user migration | Upgrade v0.2.2 flat config without losing provider access | Core implementation landed; L3 platform verification pending | Migration/rollback fixtures on all platforms | | V3: Cross-platform hardening | Reliable macOS/Windows UI and Linux CLI | Planned | Platform CI, E2E, screenshots, accessibility | @@ -18,7 +18,7 @@ M2: V1 implementation, with Tasks 1 through 10 landed and the actual Web UI asse 1. Landed: supervisor/worker protocol and deterministic process tests. 2. Landed: provider registry, credential adapters, and migration-safe storage. 3. Landed: Admin API, security boundary, and supervisor-backed CLI routing. -4. Next: onboarding and guided utility Web UI assets. +4. In progress: bounded onboarding and guided utility Web UI assets with `en`/`zh-CN` runtime dictionaries. 5. Core behavior landed; UI acceptance and L3 platform verification remain for Codex bootstrap, end-to-end activation, and restart. ## Later diff --git a/docs/STATUS.md b/docs/STATUS.md index 1ee5109..7c36217 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -6,7 +6,7 @@ V1 implementation: provider lifecycle end to end. ## In Progress -- Sequential execution of the approved V1 plan; Tasks 1 through 10 are complete and Task 11, the actual Web UI assets, is next. +- Sequential execution of the approved V1 plan; Tasks 1 through 10 are complete. The Task 11 Overview visual and English/Simplified Chinese runtime-dictionary design are approved, its single-writer scope is claimed, and implementation has not started. ## Done @@ -31,11 +31,11 @@ V1 implementation: provider lifecycle end to end. ## Blocked -- No current blocker is recorded for Task 11. +- No current blocker is recorded for the bounded Task 11 implementation. ## Next -1. Execute Task 11: land the actual Web UI assets against the existing Admin API. +1. Execute the approved Task 11 bilingual UI plan against the existing Admin API without changing backend contracts. 2. Keep product implementation within the approved V1 task order and fixed provider/proxy invariants. ## Risks diff --git a/docs/UIUX.md b/docs/UIUX.md index 13b4bfd..0574dab 100644 --- a/docs/UIUX.md +++ b/docs/UIUX.md @@ -18,7 +18,7 @@ The admin UI is not designed for mobile or remote access. - Overview: supervisor/worker health, active provider, fixed proxy address, recent error, switch and restart actions. - Providers: list, create, edit, test, activate, delete, and replace credential. - Activity: sanitized lifecycle events and diagnostic export. -- Settings: ports, credential backend status, capture toggle, Codex bootstrap state. +- Settings: read-only ports, credential backend status, capture state, and Codex bootstrap state. ## Core Interaction Rules @@ -28,6 +28,10 @@ The admin UI is not designed for mobile or remote access. - Destructive profile deletion requires confirmation and is unavailable for the active profile. - Secret inputs are blank on edit; masked previews are informational, never form values. - Every error includes a plain-language cause and a next action. +- First-run provider setup persists the provider before testing it, then permits activation only after a passing test. +- Permission-restricted file credential fallback requires a dedicated, unchecked consent checkbox. +- Settings is read-only in V1; fixed ports, capture state, credential backend, and Codex bootstrap state are informational. +- An expired browser session becomes a read-only instruction to close the tab and run `crp ui` again; Task 11 does not refresh sessions. ## Visual Direction @@ -45,6 +49,18 @@ Approved direction: **guided utility console**. - Minimum 44px interactive targets and visible keyboard focus. - Eight-point spacing scale and restrained shadows. +## Internationalization + +- Task 11 ships complete `en` and `zh-CN` runtime dictionaries inside `app.js`; separate locale assets are outside the static-file allowlist. +- Locale priority is `localStorage["crp.locale"]`, then the first supported `navigator.languages` entry after skipping unsupported entries, then English. Browser-derived/default selection does not write storage; explicit selector use writes the preference, and invalid stored values are removed. +- The language menu offers `English` and `简体中文`, has a minimum 44px target, and updates `document.documentElement.lang` immediately. +- Dates and numbers use `Intl` with the active locale. Stable IDs, ports, URLs, request IDs, and error codes remain literal. +- Every JavaScript-rendered visible string and accessible name, including validation, empty, loading, success, confirmation, live-region, session-expiry, and error cause/action copy, comes from the dictionaries. A short static bilingual `