diff --git a/AGENTS.md b/AGENTS.md index 208f472..9b3f10b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -60,6 +60,16 @@ - Session-bootstrap routes must compare the raw request target, and tests must preserve non-canonical targets. - Asynchronous catalog refresh must preserve nonempty manual selections and clear provider-scoped catalogs before switching. - Zsh scripts must not use `path` as a variable name because it overrides the executable search path. +- Package-manager CLI entry checks must compare canonical filesystem paths so POSIX bin symlinks execute. +- Legacy persisted values accepted by storage validation must remain valid through every downstream runtime validator or be explicitly migrated. +- E2E Metrics fixtures must declare unsaturated conservation assumptions and return the exact requested window shape. +- Overlapping command-output ranges must be de-duplicated before reporting duplicate source content. +- Package and lockfile root versions must remain synchronized. +- UI command-label changes must update every role/name locator in the same edit. +- Debug previews and Capture bodies, metadata, paths, and errors must redact every configured protected value without changing forwarded bytes. +- Protected-value scanning must cover reversible percent, JSON-escape, Base64, base64url, and mixed-case hex representations without emitting decoded metadata. +- Diagnostics must be issued as separate commands instead of chaining them with shell separators. +- Integration tests must await terminal Metrics/Capture settlement before closing the server that owns those resources. ## Required Checks diff --git a/README.md b/README.md index c4edd13..4884169 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ Codex Remote Proxy (CRP) keeps Codex signed in with ChatGPT while routing model [简体中文](./README.zh-CN.md) -> Release status: npm `0.2.2` is still the published pre-supervisor version and does not include `crp ui`. The instructions below describe the pending next minor release; its external platform and L3 gates must pass before publication. +> Release status: npm `0.3.0` is the current release and includes the Supervisor and `crp ui`. Changes after `0.3.0` remain unreleased until their deterministic, platform, and human-review gates pass. -## Install After Release +## Install Node.js 22.13 or newer is required. @@ -46,9 +46,13 @@ The local UI supports the complete daily workflow: - review sanitized control-plane Activity and read-only System facts; - generate an in-memory diagnostic summary containing only creation state, generation time, and sanitized event count. -`Forwarding Records` is visible as a disabled `Coming soon` navigation item. This MVP has no forwarding-record route, request/response viewer, Capture control, or mock traffic data. Overview Metrics is anonymous aggregate state and remains independent from optional Capture. +`Forwarding Records` is visible as a disabled `Coming soon` navigation item. This MVP has no forwarding-record route, request/response viewer, Capture control, or mock traffic data. Overview Metrics is anonymous aggregate state and remains independent from optional Capture. Its 24-hour and 7-day series use fixed UTC hourly buckets. A request counts as successful only after a successful Responses terminal event or completed JSON response; if metric updates were dropped, the UI marks success rates unavailable instead of presenting a precise percentage. Provider and model distributions always retain an explicit grouped remainder. -Provider activation affects new requests. Requests already in flight keep the provider snapshot with which they started. The explicit activation route is also the production switch operation: it applies a new snapshot to a running Worker and starts a stopped Worker. The first-provider Setup path is deliberately different: a successful compatibility test uses compare-and-set selection while the Worker remains stopped. +Provider activation affects new requests. Requests already in flight keep the provider snapshot, including its model policy, with which they started. In `passthrough` mode CRP preserves the client model; in `override` mode it replaces only the top-level JSON `model` value with the Provider's configured model. The explicit activation route is also the production switch operation: it applies a new snapshot to a running Worker and starts a stopped Worker. Initial selection is deliberately different: after a successful compatibility test, Setup, CLI, and the ordinary Providers page all use a first-wins compare-and-set when no Provider is active, while the Worker remains stopped. + +Proxy pass-through streams request and response bytes with backpressure and does not auto-decompress request bodies. Model override performs a bounded 8 MiB JSON transformation, preserves gzip, deflate, Brotli, and native zstd encoding when possible, and removes stale body-integrity/signature headers after a rewrite. On Node versions without native zstd compression, a verified single-frame zstd override is forwarded as identity after rewriting; zstd frames that cannot be safely inspected remain byte-exact pass-through for non-override traffic. Client cancellation stops the corresponding upstream work. + +Optional Capture stores at most 1 MiB for each request and response body while retaining the total observed byte count. When configured protected values exist, truncated bodies, declared or detected compressed bodies, and bodies containing literal or recoverably encoded protected values are stored as `empty-truncated`; fully screened text/binary records still use explicit UTF-8/base64 encoding. Configured API keys and extra-header values are removed from captured headers, bodies, URL/ID metadata, and debug logs. Buffered Metrics body inspection is independently bounded to 8 MiB, SSE inspection is incremental with bounded events, and neither path enables Capture. ## Stable Codex Configuration @@ -133,11 +137,11 @@ The former compatibility aliases `crp init`, `crp install`, and `crp setup` have `provider test`, `activate`, `delete`, and `models` require exactly one selector: `--id` or `--name`. Names resolve by exact case-insensitive match against the unique public provider list. `provider models` performs an authenticated, no-redirect refresh from `/models`; the Admin API also exposes a cached read separately. Discovery is bounded and rejects any model ID containing the complete credential before it can reach cache or output. It is independent from Responses compatibility testing, so a missing or incompatible model endpoint does not change provider test or activation state and a failed refresh does not erase the last good catalog. -CLI-triggered compatibility tests, including `provider add --model`, request initial selection only when no provider is active. The first successful candidate wins an atomic compare-and-set while the Worker is stopped. Selection writes `activeProviderId` but never starts or reconfigures the Worker; run `crp start` explicitly. Admin callers that omit `activateIfNone` retain non-selecting test behavior. The conditional Web Setup explicitly opts in and runs `save provider -> test and compare-and-set select -> prepare Codex/history repair -> start Worker`; it does not call explicit activation during first setup. Ordinary Provider-page tests remain non-selecting until the user chooses a switch action. +CLI-triggered compatibility tests, including `provider add --model`, request initial selection only when no Provider is active. The ordinary Web Providers page now makes the same request. The first successful candidate wins an atomic compare-and-set while the Worker is stopped. Selection writes `activeProviderId` but never starts or reconfigures the Worker, never calls the readiness-gated explicit activation route, and is confirmed from refreshed server state; run `crp start` explicitly. Admin callers that omit `activateIfNone` retain non-selecting test behavior. The conditional Web Setup also opts in and runs `save provider -> test and compare-and-set select -> prepare Codex/history repair -> start Worker`. ## Upgrading From 0.2.2 -The next minor release migrates the pre-supervisor flat configuration to provider-registry schema 2 on first supervisor startup. +The `0.3` series migrates the pre-supervisor flat configuration to provider-registry schema 2 on first supervisor startup. 1. Stop the old managed proxy. 2. Make a private backup of `~/.codex-remote-proxy/` and `~/.codex/config.toml`. Treat every backup as secret-bearing. diff --git a/README.zh-CN.md b/README.zh-CN.md index cefcc02..26e2e66 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -6,9 +6,9 @@ Codex Remote Proxy（CRP）让 Codex 保持 ChatGPT 登录态，同时把模型 [English](./README.md) -> 发布状态：npm 当前发布的仍是 pre-supervisor `0.2.2`，其中不包含 `crp ui`。下文说明待发布的下一个 minor 版本；必须先通过外部平台门禁与 L3 确认才能发布。 +> 发布状态：npm 当前版本是 `0.3.0`，已经包含 Supervisor 和 `crp ui`。`0.3.0` 之后的变更仍需通过确定性测试、平台门禁和人工审查才会发布。 -## 发布后安装 +## 安装 需要 Node.js 22.13 或更高版本。 @@ -46,9 +46,13 @@ npx @cluic/codex-remote-proxy ui - 查看已脱敏的控制面 Activity 和只读系统事实； - 生成只包含创建状态、生成时间和已脱敏事件数量的内存诊断摘要。 -侧边栏会显示不可操作的 `转发记录 / 即将上线` 占位项。本 MVP 不提供转发记录路由、请求/响应查看器、Capture 控件或模拟流量数据；总览 Metrics 是独立于可选 Capture 的匿名聚合状态。 +侧边栏会显示不可操作的 `转发记录 / 即将上线` 占位项。本 MVP 不提供转发记录路由、请求/响应查看器、Capture 控件或模拟流量数据；总览 Metrics 是独立于可选 Capture 的匿名聚合状态。24 小时和 7 天序列使用固定 UTC 小时桶。只有成功的 Responses 终态事件或已完成 JSON 响应才计为成功；如果存在丢弃的指标更新，界面会把成功率标记为不可用，而不是展示看似精确的百分比。Provider 和模型分布始终保留明确的合并余量。 -提供商切换只影响新请求。已经在处理中的请求继续使用其开始时捕获的提供商快照。显式 activation 路由同时也是生产切换操作：Worker 运行时应用新快照，Worker 停止时会启动它。首次 Setup 路径有意不同：兼容性测试成功后只通过 compare-and-set 选中首个 Provider，Worker 保持停止。 +提供商切换只影响新请求。已经在处理中的请求继续使用其开始时捕获的提供商快照，包括模型策略。`passthrough` 模式保留客户端模型；`override` 模式只替换 JSON 顶层 `model` 值。显式 activation 路由同时也是生产切换操作：Worker 运行时应用新快照，Worker 停止时会启动它。首次选中有意不同：兼容性测试成功后，Setup、CLI 和普通 Providers 页面都会在尚无当前 Provider 时通过 first-wins compare-and-set 选中候选，Worker 保持停止。 + +代理透传会按背压流式转发请求和响应字节，不会自动解压请求体。模型覆盖只在 8 MiB 有界范围内改写 JSON；发生改写时会尽可能保留 gzip、deflate、Brotli 和原生 zstd 编码，并移除已经失效的正文完整性/签名头。Node 没有原生 zstd 压缩能力时，经过验证的单帧 zstd 覆盖请求会在改写后以 identity 转发；非覆盖流量中无法安全检查的 zstd 帧仍保持字节完全一致。客户端取消连接会终止对应的上游工作。 + +可选 Capture 对请求体和响应体各自最多保存 1 MiB，同时保留实际观测总字节数。存在已配置保护值时，截断正文、已声明或检测到的压缩正文，以及包含明文或可恢复编码保护值的正文都会记录为 `empty-truncated`；能够完整筛查的文本/二进制记录仍使用明确的 UTF-8/base64 编码。配置的 API key 和额外请求头值不会进入 Capture header、正文、URL/ID 元数据或 debug 日志。Metrics 的缓冲正文检查独立限制为 8 MiB，SSE 使用有界事件做增量检查，两者都不会隐式开启 Capture。 ## 固定的 Codex 配置 @@ -133,11 +137,11 @@ Detached Supervisor 启动只使用一次性、严格白名单化的 IPC 错误 `provider test`、`activate`、`delete` 和 `models` 必须且只能提供一个选择器：`--id` 或 `--name`。名称通过公开 provider 列表做精确的大小写不敏感匹配。`provider models` 会向 `/models` 发起带鉴权、禁止重定向的刷新；Admin API 另提供独立的缓存读取。模型发现有界，并会在进入缓存或输出前拒绝任何包含完整 credential 的模型 ID。它独立于 Responses 兼容性测试，因此模型端点缺失或不兼容不会修改 provider 的测试或激活状态，刷新失败也不会清除最后一次成功目录。 -CLI 发起的兼容性测试（包括 `provider add --model`）只会在当前没有 provider 时请求首次选中。第一个成功候选在 Worker 已停止时通过原子 compare-and-set 胜出。选中只写入 `activeProviderId`，绝不会启动或重新配置 Worker；仍需显式运行 `crp start`。未提供 `activateIfNone` 的 Admin 调用继续保持不自动选中。条件式 Web Setup 会明确选择该行为，并按 `保存 Provider -> 测试并 CAS 选中 -> 配置 Codex/修复历史 -> 启动 Worker` 执行；首次设置不调用显式 activation。Provider 日常页面的普通测试仍不自动选中，只有用户选择切换操作时才切换。 +CLI 发起的兼容性测试（包括 `provider add --model`）只会在当前没有 Provider 时请求首次选中，普通 Web Providers 页面现在也会发出同样的请求。第一个成功候选在 Worker 已停止时通过原子 compare-and-set 胜出。选中只写入 `activeProviderId`，绝不会启动或重新配置 Worker，也不会调用受 readiness gate 保护的显式 activation 路由；界面会刷新服务端状态确认结果。仍需显式运行 `crp start`。未提供 `activateIfNone` 的 Admin 调用继续保持不自动选中。条件式 Web Setup 同样会选择该行为，并按 `保存 Provider -> 测试并 CAS 选中 -> 配置 Codex/修复历史 -> 启动 Worker` 执行。 ## 从 0.2.2 升级 -下一个 minor 版本会在 Supervisor 首次启动时，把 pre-supervisor 扁平配置迁移到 provider registry schema 2。 +`0.3` 系列会在 Supervisor 首次启动时，把 pre-supervisor 扁平配置迁移到 provider registry schema 2。 1. 停止旧的托管代理。 2. 私下备份 `~/.codex-remote-proxy/` 和 `~/.codex/config.toml`；所有备份都应视为包含敏感信息。 diff --git a/node/.changeset/fix-audit-findings.md b/node/.changeset/fix-audit-findings.md new file mode 100644 index 0000000..7e3b66c --- /dev/null +++ b/node/.changeset/fix-audit-findings.md @@ -0,0 +1,9 @@ +--- +"@cluic/codex-remote-proxy": minor +--- + +Restore POSIX npm-bin symlink execution and let an ordinary Web compatibility test safely select the first Provider without starting the Worker or calling readiness-gated activation. + +Propagate Provider model policy into Worker snapshots and perform bounded, lexical top-level model rewrites while preserving unrelated JSON bytes and supported content encodings. Stream pass-through traffic with backpressure, cancel upstream work on client disconnect, strip invalid hop-by-hop/integrity headers, and bound/redact Capture and debug observations. + +Classify Responses success from semantic completion, conserve bounded Provider/model distribution remainders, persist saturated Metrics safely, and disclose dropped metric updates instead of showing a precise success rate. diff --git a/node/README.md b/node/README.md index 770b1c7..5dc2f2d 100644 --- a/node/README.md +++ b/node/README.md @@ -2,9 +2,9 @@ `@cluic/codex-remote-proxy` keeps Codex signed in with ChatGPT while routing model requests through a selected OpenAI-compatible provider. -> Release status: npm `0.2.2` is still the published pre-supervisor version and does not include `crp ui`. This document describes the pending next minor release, which must not be published until its external platform and L3 gates pass. +> Release status: npm `0.3.0` is current and includes the Supervisor and `crp ui`. Changes after `0.3.0` remain unreleased until their deterministic, platform, and human-review gates pass. -## Requirements and Install After Release +## Requirements and Install Node.js 22.13 or newer is required. @@ -25,9 +25,9 @@ The current development source is a responsive React + TypeScript SPA under `ui- ## Product Behavior -The UI can create, test, switch, update, and delete named providers; start, stop, and restart the proxy worker; inspect anonymous 24-hour/7-day aggregate Metrics; review sanitized Activity; inspect read-only System facts; and generate in-memory diagnostic summary metadata. A provider must pass an OpenAI Responses compatibility test before explicit activation. Provider cards expose the legal switch action directly. Explicit activation applies a snapshot to a running Worker and starts a stopped Worker; the Setup-only first-provider compare-and-set selection never starts or reconfigures it. The active provider cannot be updated or deleted, even while the worker is stopped; switch to another provider first. +The UI can create, test, switch, update, and delete named providers; start, stop, and restart the proxy worker; inspect anonymous 24-hour/7-day aggregate Metrics; review sanitized Activity; inspect read-only System facts; and generate in-memory diagnostic summary metadata. A provider must pass an OpenAI Responses compatibility test before explicit activation. Provider cards expose the legal switch action directly. Explicit activation applies a snapshot to a running Worker and starts a stopped Worker; first-provider compare-and-set selection from Setup, CLI, or the ordinary Providers page never starts or reconfigures it. The active provider cannot be updated or deleted, even while the worker is stopped; switch to another provider first. -`Forwarding Records` is a disabled coming-soon navigation item only. It has no route, traffic request, Capture control, payload viewer, or mock records. Overview Metrics is anonymous aggregation independent from optional Capture. +`Forwarding Records` is a disabled coming-soon navigation item only. It has no route, traffic request, Capture control, payload viewer, or mock records. Overview Metrics is anonymous aggregation independent from optional Capture. It uses fixed UTC hourly buckets, classifies Responses success from semantic terminal state, marks rates unavailable after dropped metric updates, and retains explicit grouped remainders for Provider and model distributions. Codex remains configured as: @@ -35,7 +35,11 @@ Codex remains configured as: model_provider = "OpenAI" ``` -Its proxy address remains fixed at `http://127.0.0.1:15100`. CRP switches upstreams internally for new requests while in-flight requests retain their starting snapshot. +Its proxy address remains fixed at `http://127.0.0.1:15100`. CRP switches upstreams internally for new requests while in-flight requests retain their starting snapshot, including the Provider model policy. Passthrough preserves the client's model; override replaces only the top-level JSON `model` value. + +Pass-through traffic streams with backpressure and preserves request encodings byte-for-byte. Model override is an 8 MiB bounded JSON transformation that preserves gzip, deflate, Brotli, and native zstd encoding when available, strips stale integrity/signature headers after a rewrite, and falls back to identity for a verified single-frame zstd request when Node has no native zstd compressor. Downstream cancellation destroys the corresponding upstream work. + +Optional Capture stores at most 1 MiB per request and response body and retains total observed byte counts. With configured protected values, truncated bodies, declared or detected compressed bodies, and bodies containing literal or recoverably encoded protected values are stored as `empty-truncated`; fully screened text/binary records retain explicit UTF-8/base64 encoding. Configured API keys and extra-header values are redacted from Capture headers, bodies, URL/ID metadata, and debug logs. Independent buffered Metrics inspection is bounded to 8 MiB, SSE inspection is incremental with bounded events, and Responses success requires semantic completion rather than HTTP 2xx alone. On a clean home, explicit `crp start` privately and atomically creates a missing `.codex` directory and `config.toml`, with no backup for a source that did not exist. New POSIX directory/file modes are `0700`/`0600`; a repeated bootstrap is byte-identical. Existing-file locking, identity/race checks, adjacent backup, mode preservation, and idempotency remain in force. @@ -93,7 +97,7 @@ The former `init`, `install`, and `setup` compatibility aliases are removed. Eac Optional `provider add --model ` uses that value only for the follow-up Responses test; routing override remains `--model-mode override --model-override `. It saves the profile first, then tests, and creation remains committed when the compatibility result fails or the second-stage request cannot complete. `provider test`, `activate`, `delete`, and `models` require exactly one of `--id` or case-insensitive exact `--name`. `provider models` refreshes the authenticated, no-redirect `/models` catalog and rejects a complete credential reflected in any model ID before cache or output; discovery failure preserves the last good cache and does not change provider test or activation state. -CLI tests request `activateIfNone` so the first successfully tested provider is selected through a first-wins compare-and-set while the Worker is stopped. That initial selection never starts or reconfigures the Worker; `crp start` remains explicit. Admin calls default `activateIfNone` to false. Conditional Web Setup opts in and runs `save -> test and CAS select -> Codex bootstrap/history repair -> Worker start`; ordinary Provider-page tests omit the flag. +CLI tests and ordinary Web Providers-page tests request `activateIfNone` so the first successfully tested Provider is selected through a first-wins compare-and-set while the Worker is stopped. That initial selection never starts or reconfigures the Worker, never calls the readiness-gated explicit activation route, and is confirmed from refreshed server state; `crp start` remains explicit. Admin calls default `activateIfNone` to false. Conditional Web Setup also opts in and runs `save -> test and CAS select -> Codex bootstrap/history repair -> Worker start`. ## Migration From 0.2.2 @@ -145,4 +149,4 @@ Final M2E/V8 local verification passes exact `npm test` 463/463 (`412` unit-core Supervisor discovery applies a 2-second liveness probe and returns a client with a separate 30-second operation timeout. Proxy forwarding joins base and incoming URLs structurally, preserving base paths and query parameters while avoiding duplicate path separators. The retained `provider add --api-key ` behavior and broader child-environment minimization remain explicit future follow-up work and do not block local core completion. -This release requires a minor Changeset. Do not run `npm run version-packages` or `npm run release` during feature preparation. See [RELEASING.md](./RELEASING.md) for local evidence and remaining remote/human gates. +This unreleased behavior change uses the repository-required minor Changeset. Do not run `npm run version-packages` or `npm run release` during feature preparation. See [RELEASING.md](./RELEASING.md) for local evidence and remaining remote/human gates. diff --git a/node/RELEASING.md b/node/RELEASING.md index bbfe9b8..73ceee3 100644 --- a/node/RELEASING.md +++ b/node/RELEASING.md @@ -1,6 +1,6 @@ # Releasing -This package publishes from `node/` through Changesets and GitHub Actions. The currently published version is `0.2.2`; the supervisor, multi-provider, and bilingual local-UI work is an unreleased minor change. +This package publishes from `node/` through Changesets and GitHub Actions. The currently published version is `0.3.0`, including the Supervisor, multi-provider routing, and bilingual local UI. This audit remediation is an unreleased behavior change. ## One-Time Publishing Setup @@ -14,7 +14,7 @@ Publishing uses GitHub OIDC and requires no long-lived `NPM_TOKEN`. ## Feature Pull Request -1. Add a minor Changeset under `node/.changeset/`. +1. Add a minor Changeset under `node/.changeset/` for package behavior changes, as required by release preflight. 2. Run every local gate below on the final tree. 3. Push a branch and open a pull request only after L3 human review is scheduled. 4. Wait for the macOS, Windows, Linux, and release-preflight workflows and retain their run URLs. @@ -88,7 +88,7 @@ V8.1 release-preparation rerun on 2026-07-17: `~/.codex-remote-proxy`, while the earlier `crpdev` wrapper is confirmed to be intentionally isolated through its injected temporary paths. -Publication remains blocked on deliberate working-tree staging, +Publication of changes after `0.3.0` remains blocked on deliberate working-tree staging, copied-real-history rehearsal, remote platform evidence, and final L3 approval. Current release-preparation adjustments include a shipped MIT License and a @@ -100,6 +100,16 @@ typecheck/build/exact-output, lint, runtime audit, package/release tests 21/21, the exact 34-file package dry run, and exact `npm test` 467/467 (`415` unit-core + `8` Capture + `43` ordinary integration + `1` serial core chain). +The 2026-07-18 audit-remediation branch supersedes those aggregate counts for +the current source tree: exact `npm test` passes 519/519 (`458` unit-core + +`9` isolated Capture + `51` ordinary integration + `1` serial core chain), +Chromium passes 46/46, lint checks 33 source files, UI typecheck and exact-build +verification pass, the runtime audit reports zero vulnerabilities, and the +package/release suite passes 21/21 against the exact 34-file dry-run package. +The required Changeset is minor and independent final review reports no +remaining P0-P2. These are local temporary-root and loopback results; they do +not replace the external release gates below. + Historical commits remain Task 11 implementation `d114061`, Task 11 docs `dd4de3f`, Task 12 package/platform gates `af918d5`, and credential-boundary hardening `210cb71`; the then-current M2D/V7 evidence was for its uncommitted reviewed working tree. These local results used temporary roots and synthetic history and do not prove copied-corpus real-home performance/recovery, remote Keychain/Credential Manager/Secret Service behavior, cross-platform filesystem semantics, browser launch behavior, or platform screenshots. ## Migration Review diff --git a/node/bin/crp.mjs b/node/bin/crp.mjs index fa48965..70fbe6d 100644 --- a/node/bin/crp.mjs +++ b/node/bin/crp.mjs @@ -1,6 +1,16 @@ #!/usr/bin/env node import { spawn, spawnSync } from "node:child_process"; -import { chmodSync, closeSync, existsSync, mkdirSync, openSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { + chmodSync, + closeSync, + existsSync, + mkdirSync, + openSync, + readFileSync, + realpathSync, + rmSync, + writeFileSync +} from "node:fs"; import { resolve } from "node:path"; import net from "node:net"; import readline from "node:readline/promises"; @@ -2876,9 +2886,15 @@ export async function runCli(argv, { } function isDirectExecution(metaUrl = import.meta.url, argv1 = process.argv[1]) { - return typeof argv1 === "string" - && argv1.length > 0 - && resolve(fileURLToPath(metaUrl)) === resolve(argv1); + if (typeof argv1 !== "string" || argv1.length === 0) return false; + try { + const modulePath = resolve(fileURLToPath(metaUrl)); + const entryPath = resolve(argv1); + if (modulePath === entryPath) return true; + return realpathSync(modulePath) === realpathSync(entryPath); + } catch { + return false; + } } if (isDirectExecution()) { diff --git a/node/package-lock.json b/node/package-lock.json index 4983332..1cd8adb 100644 --- a/node/package-lock.json +++ b/node/package-lock.json @@ -1,12 +1,12 @@ { "name": "@cluic/codex-remote-proxy", - "version": "0.2.2", + "version": "0.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@cluic/codex-remote-proxy", - "version": "0.2.2", + "version": "0.3.0", "dependencies": { "@napi-rs/keyring": "1.3.0", "fzstd": "^0.1.1" diff --git a/node/proxy-config.example.json b/node/proxy-config.example.json index 1ccd301..46f5036 100644 --- a/node/proxy-config.example.json +++ b/node/proxy-config.example.json @@ -15,7 +15,9 @@ }, "proxy": { "overrideAuthorization": true, - "requestIdHeader": "x-client-request-id" + "requestIdHeader": "x-client-request-id", + "modelMode": "passthrough", + "modelOverride": null }, "capture": { "enabled": false, diff --git a/node/src/capture-store.mjs b/node/src/capture-store.mjs index 34c15dc..fc37db1 100644 --- a/node/src/capture-store.mjs +++ b/node/src/capture-store.mjs @@ -139,12 +139,22 @@ export function redactHeaders(headersInput) { return result; } -export function encodeBody(buffer) { +export function encodeBody(buffer, { + totalBytes = buffer?.length ?? 0, + truncated = false +} = {}) { + if (!Number.isSafeInteger(totalBytes) || totalBytes < 0 || totalBytes < (buffer?.length ?? 0)) { + throw new Error("Captured body byte count is invalid"); + } + if (truncated !== (totalBytes > (buffer?.length ?? 0))) { + throw new Error("Captured body truncation marker is inconsistent"); + } + const suffix = truncated ? "-truncated" : ""; if (!buffer || buffer.length === 0) { return { body: "", - encoding: "empty", - bytes: 0 + encoding: `empty${suffix}`, + bytes: totalBytes }; } @@ -152,15 +162,15 @@ export function encodeBody(buffer) { if (Buffer.compare(Buffer.from(text, "utf8"), buffer) === 0) { return { body: text, - encoding: "utf8", - bytes: buffer.length + encoding: `utf8${suffix}`, + bytes: totalBytes }; } return { body: buffer.toString("base64"), - encoding: "base64", - bytes: buffer.length + encoding: `base64${suffix}`, + bytes: totalBytes }; } @@ -504,9 +514,15 @@ export class CaptureManager { return; } - const requestBody = encodeBody(record.requestBody); - const responseBody = encodeBody(record.responseBody); try { + const requestBody = encodeBody(record.requestBody, { + totalBytes: record.requestBodyBytes, + truncated: record.requestBodyTruncated === true + }); + const responseBody = encodeBody(record.responseBody, { + totalBytes: record.responseBodyBytes, + truncated: record.responseBodyTruncated === true + }); this.insertStatement.run({ started_at: record.startedAt, completed_at: record.completedAt, diff --git a/node/src/providers/provider-schema.mjs b/node/src/providers/provider-schema.mjs index 79f8285..41a2ee9 100644 --- a/node/src/providers/provider-schema.mjs +++ b/node/src/providers/provider-schema.mjs @@ -177,7 +177,7 @@ function normalizeExtraHeaders(value) { return extraHeaders; } -function normalizeModelPolicy(modeValue, overrideValue) { +function normalizeModelPolicy(modeValue, overrideValue, { allowControlCharacters = false } = {}) { const modelMode = modeValue === undefined ? "passthrough" : modeValue; if (modelMode !== "passthrough" && modelMode !== "override") { inputError("modelMode", "must be passthrough or override"); @@ -185,7 +185,9 @@ function normalizeModelPolicy(modeValue, overrideValue) { let modelOverride = overrideValue === undefined ? null : overrideValue; if (modelOverride !== null) { - if (typeof modelOverride !== "string" || modelOverride.trim().length === 0) { + if (typeof modelOverride !== "string" + || modelOverride.trim().length === 0 + || (!allowControlCharacters && CONTROL_CHARACTER_PATTERN.test(modelOverride))) { inputError("modelOverride", "must be a non-empty string or null"); } modelOverride = modelOverride.trim(); @@ -196,12 +198,12 @@ function normalizeModelPolicy(modeValue, overrideValue) { return { modelMode, modelOverride }; } -function normalizeInput(input) { +function normalizeInput(input, options) { if (!isPlainObject(input)) { inputError("provider", "must be an object"); } assertExactFields(input, INPUT_FIELDS, "provider"); - const modelPolicy = normalizeModelPolicy(input.modelMode, input.modelOverride); + const modelPolicy = normalizeModelPolicy(input.modelMode, input.modelOverride, options); return { name: normalizeRequiredString(input.name, "name"), baseUrl: normalizeBaseUrl(input.baseUrl), @@ -272,7 +274,7 @@ export function validateStoredProvider(profile) { extraHeaders: profile.extraHeaders, modelMode: profile.modelMode, modelOverride: profile.modelOverride - }); + }, { allowControlCharacters: true }); assertStoredValue( typeof profile.id === "string" && profile.id.trim() === profile.id && profile.id.length > 0, "id", diff --git a/node/src/server.mjs b/node/src/server.mjs index 7fef56b..857f461 100644 --- a/node/src/server.mjs +++ b/node/src/server.mjs @@ -6,11 +6,10 @@ import { readFileSync } from "node:fs"; import { resolve, dirname } from "node:path"; import { URL } from "node:url"; import zlib from "node:zlib"; -import { decompress as zstdDecompress } from "fzstd"; +import { Decompress as ZstdDecompress } from "fzstd"; import { createCaptureManager, - createNoopCaptureHandle, DEFAULT_CAPTURE_DB_PATH, headersToObject, normalizeCaptureConfig @@ -19,10 +18,15 @@ import { const CONFIG_ENV_VAR = "CODEX_PROXY_CONFIG"; const DEFAULT_CONFIG_PATH = resolve(import.meta.dirname, "..", "proxy-config.json"); const HEALTH_PATH = "/_proxy/health"; -const METRIC_MODEL_MAX_BYTES = 64 * 1024; const METRIC_USAGE_MAX_BYTES = 1024 * 1024; +const METRIC_BODY_INSPECTION_MAX_BYTES = 8 * 1024 * 1024; +const CAPTURE_BODY_MAX_BYTES = 1024 * 1024; +const MODEL_OVERRIDE_MAX_BYTES = 8 * 1024 * 1024; +const SSE_EVENT_MAX_BYTES = METRIC_USAGE_MAX_BYTES; +const SSE_EVENT_MAX_DATA_LINES = 16 * 1024; const METRIC_MAX_MODEL_CODE_POINTS = 256; const METRIC_MAX_OBSERVATION_TOKENS = 100_000_000; +const CONFIG_TEXT_CONTROL_PATTERN = /[\u0000-\u001f\u007f]/; const METRIC_TEXT_CONTROL_PATTERN = /[\u0000-\u001f\u007f-\u009f]/; const METRIC_LATENCY_BOUNDS_MS = [ 50, @@ -44,6 +48,7 @@ const HOP_BY_HOP_HEADERS = new Set([ "keep-alive", "proxy-authenticate", "proxy-authorization", + "proxy-connection", "te", "trailer", "trailers", @@ -75,6 +80,20 @@ export function loadConfig(configPath = resolveConfigPath()) { const proxy = parsed.proxy ?? {}; const capture = parsed.capture ?? {}; + const modelMode = proxy.modelMode ?? "passthrough"; + if (modelMode !== "passthrough" && modelMode !== "override") { + throw new Error("proxy.modelMode must be passthrough or override"); + } + const modelOverride = proxy.modelOverride ?? null; + if (modelOverride !== null && (typeof modelOverride !== "string" + || modelOverride.trim().length === 0 + || CONFIG_TEXT_CONTROL_PATTERN.test(modelOverride))) { + throw new Error("proxy.modelOverride must be a non-empty string or null"); + } + if (modelMode === "override" && modelOverride === null) { + throw new Error("proxy.modelOverride is required when proxy.modelMode is override"); + } + if (!upstream.baseUrl || typeof upstream.baseUrl !== "string") { throw new Error("upstream.baseUrl is required"); } @@ -100,7 +119,9 @@ export function loadConfig(configPath = resolveConfigPath()) { }, proxy: { overrideAuthorization: typeof proxy.overrideAuthorization === "boolean" ? proxy.overrideAuthorization : true, - requestIdHeader: typeof proxy.requestIdHeader === "string" && proxy.requestIdHeader ? proxy.requestIdHeader : "x-client-request-id" + requestIdHeader: typeof proxy.requestIdHeader === "string" && proxy.requestIdHeader ? proxy.requestIdHeader : "x-client-request-id", + modelMode, + modelOverride: typeof modelOverride === "string" ? modelOverride.trim() : null }, capture: normalizeCaptureConfig(capture, { baseDir: dirname(configPath), @@ -135,10 +156,129 @@ function debugLog(label, data, enabled) { } } -function safeBodyPreview(buffer, maxLen = 4096) { +function normalizeProtectedValues(values) { + return [...new Set(values.filter((value) => typeof value === "string" && value.length > 0))] + .sort((left, right) => right.length - left.length); +} + +function redactProtectedText(value, protectedValues) { + let result = String(value); + for (const secret of normalizeProtectedValues(protectedValues)) { + result = result.split(secret).join("[REDACTED]"); + } + return result; +} + +function decodeRecoverableText(value) { + return value + .replace(/(?:%[0-9a-f]{2})+/giu, (encoded) => { + const bytes = encoded.split("%").filter(Boolean).map((item) => Number.parseInt(item, 16)); + return Buffer.from(bytes).toString("utf8"); + }) + .replace(/\\u([0-9a-f]{4})/giu, (_match, code) => String.fromCharCode(Number.parseInt(code, 16))) + .replace(/\\(["\\/bfnrt])/gu, (_match, escaped) => ({ + "\"": "\"", + "\\": "\\", + "/": "/", + b: "\b", + f: "\f", + n: "\n", + r: "\r", + t: "\t" + })[escaped]); +} + +function containsRecoverableProtectedValue(value, protectedValues, { includeLiteral = true } = {}) { + const normalizedValues = normalizeProtectedValues(protectedValues); + const encodedRepresentations = normalizedValues.flatMap((secret) => { + const bytes = Buffer.from(secret); + const base64 = bytes.toString("base64"); + const paddedBase64Url = base64.replace(/\+/gu, "-").replace(/\//gu, "_"); + return [ + base64, + base64.replace(/=+$/u, ""), + paddedBase64Url, + bytes.toString("base64url") + ]; + }); + const hexRepresentations = normalizedValues.map((secret) => Buffer.from(secret).toString("hex")); + const matches = (candidate, matchLiteral) => (matchLiteral && normalizedValues.some( + (secret) => secret.length > 0 && candidate.includes(secret) + )) || encodedRepresentations.some( + (secret) => secret.length > 0 && candidate.includes(secret) + ) || hexRepresentations.some( + (secret) => secret.length > 0 && candidate.toLowerCase().includes(secret) + ); + let candidate = String(value); + for (let attempt = 0; attempt <= 3; attempt += 1) { + const formDecoded = candidate.replace(/\+/g, " "); + const compactBase64 = candidate.replace(/[\t\n\v\f\r ]/gu, ""); + const compactFormBase64 = formDecoded.replace(/[\t\n\v\f\r ]/gu, ""); + if (matches(candidate, includeLiteral || attempt > 0)) return true; + if (formDecoded !== candidate && matches(formDecoded, true)) return true; + if (compactBase64 !== candidate && matches(compactBase64, false)) return true; + if (compactFormBase64 !== formDecoded && matches(compactFormBase64, false)) return true; + if (attempt === 3) break; + const decoded = decodeRecoverableText(candidate); + if (decoded === candidate) break; + candidate = decoded; + } + return false; +} + +function redactRecoverableProtectedText(value, protectedValues) { + const source = String(value); + const encoded = containsRecoverableProtectedValue(source, protectedValues, { + includeLiteral: false + }); + const literal = redactProtectedText(source, protectedValues); + return encoded ? "[REDACTED]" : literal; +} + +function redactProtectedUrl(value, protectedValues) { + return redactRecoverableProtectedText(value, protectedValues); +} + +function redactProtectedBuffer(buffer, protectedValues) { + if (!Buffer.isBuffer(buffer) || buffer.length === 0) return buffer ?? Buffer.alloc(0); + const redacted = Buffer.from(buffer); + for (const secret of normalizeProtectedValues(protectedValues)) { + const needle = Buffer.from(secret); + let offset = 0; + while (offset <= redacted.length - needle.length) { + const index = redacted.indexOf(needle, offset); + if (index === -1) break; + redacted.fill(0x2a, index, index + needle.length); + offset = index + needle.length; + } + } + return redacted; +} + +function redactProtectedFields(fields, protectedValues) { + return Object.fromEntries(Object.entries(fields).map(([key, value]) => [ + key, + typeof value === "string" ? redactRecoverableProtectedText(value, protectedValues) : value + ])); +} + +function safeBodyPreview(buffer, maxLen = 4096, protectedValues = [], { + contentEncoding, + truncated = false +} = {}) { if (!buffer || !buffer.length) return "(empty)"; try { + const normalizedEncoding = singleContentEncoding(contentEncoding); + const opaqueEncoding = normalizedEncoding === null + || (normalizedEncoding !== "" && normalizedEncoding !== "identity"); const text = buffer.toString("utf-8"); + if (protectedValues.length > 0 + && (truncated + || opaqueEncoding + || containsRecoverableProtectedValue(text, protectedValues) + || isRecoverablyCompressedBody(buffer))) { + return "[REDACTED]"; + } return text.length > maxLen ? `${text.slice(0, maxLen)}... (${buffer.length} bytes total)` : text; } catch { return `(${buffer.length} bytes, binary)`; @@ -166,6 +306,14 @@ function formatAuthorization(upstream) { } const CONTENT_HEADERS = new Set(["content-encoding", "content-length"]); +const BODY_INTEGRITY_HEADERS = new Set([ + "content-md5", + "digest", + "content-digest", + "repr-digest", + "signature", + "signature-input" +]); const DEBUG_SENSITIVE_HEADER_NAMES = new Set([ "authorization", "proxy-authorization", @@ -175,49 +323,565 @@ const DEBUG_SENSITIVE_HEADER_NAMES = new Set([ ]); const DEBUG_SENSITIVE_HEADER_PARTS = ["token", "secret", "api-key"]; -function decompressBody(buffer, encoding) { +function declaredZstdContentSize(buffer) { + if (!Buffer.isBuffer(buffer) || buffer.length < 6 || buffer.readUInt32LE(0) !== 0xfd2fb528) { + return null; + } + const descriptor = buffer[4]; + if ((descriptor & 0x08) !== 0) return null; + const singleSegment = (descriptor & 0x20) !== 0; + const dictionaryFlag = descriptor & 0x03; + const dictionaryBytes = dictionaryFlag === 3 ? 4 : dictionaryFlag; + const contentFlag = descriptor >> 6; + const contentBytes = contentFlag === 0 ? (singleSegment ? 1 : 0) : (1 << contentFlag); + if (contentBytes === 0) return null; + const offset = (singleSegment ? 5 : 6) + dictionaryBytes; + if (offset + contentBytes > buffer.length) return null; + let size = 0n; + for (let index = 0; index < contentBytes; index += 1) { + size |= BigInt(buffer[offset + index]) << BigInt(index * 8); + } + if (contentFlag === 1) size += 256n; + return size <= BigInt(Number.MAX_SAFE_INTEGER) ? Number(size) : null; +} + +function declaredZstdWindowSize(buffer) { + if (!Buffer.isBuffer(buffer) || buffer.length < 6 || buffer.readUInt32LE(0) !== 0xfd2fb528) { + return null; + } + const descriptor = buffer[4]; + const singleSegment = (descriptor & 0x20) !== 0; + if (singleSegment) return declaredZstdContentSize(buffer); + const windowDescriptor = buffer[5]; + const base = 2 ** (10 + (windowDescriptor >> 3)); + return base + ((base / 8) * (windowDescriptor & 0x07)); +} + +function zstdFrameEnd(buffer) { + if (!Buffer.isBuffer(buffer) || buffer.length < 6 || buffer.readUInt32LE(0) !== 0xfd2fb528) { + return null; + } + const descriptor = buffer[4]; + const singleSegment = (descriptor & 0x20) !== 0; + const dictionaryFlag = descriptor & 0x03; + const dictionaryBytes = dictionaryFlag === 3 ? 4 : dictionaryFlag; + const contentFlag = descriptor >> 6; + const contentBytes = contentFlag === 0 ? (singleSegment ? 1 : 0) : (1 << contentFlag); + let offset = 5 + (singleSegment ? 0 : 1) + dictionaryBytes + contentBytes; + while (offset + 3 <= buffer.length) { + const header = buffer[offset] | (buffer[offset + 1] << 8) | (buffer[offset + 2] << 16); + const lastBlock = (header & 1) !== 0; + const blockType = (header >> 1) & 0x03; + const blockSize = header >>> 3; + if (blockType === 3) return null; + offset += 3 + (blockType === 1 ? 1 : blockSize); + if (offset > buffer.length) return null; + if (lastBlock) { + if ((descriptor & 0x04) !== 0) offset += 4; + return offset <= buffer.length ? offset : null; + } + } + return null; +} + +function decompressBody(buffer, encoding, maximumBytes) { + const enc = encoding.toLowerCase().trim(); + const options = Number.isFinite(maximumBytes) ? { maxOutputLength: maximumBytes } : undefined; + if (enc === "" || enc === "identity") return buffer; + if (enc === "gzip") return zlib.gunzipSync(buffer, options); + if (enc === "deflate") return zlib.inflateSync(buffer, options); + if (enc === "br") return zlib.brotliDecompressSync(buffer, options); + if (enc === "zstd" && typeof zlib.zstdDecompressSync === "function") { + return zlib.zstdDecompressSync(buffer, options); + } + if (enc === "zstd") { + const declaredSize = declaredZstdContentSize(buffer); + const frameEnd = zstdFrameEnd(buffer); + if (frameEnd === null) throw new Error("Invalid zstd frame"); + if (frameEnd !== buffer.length) throw new Error("Multi-frame zstd inspection is not supported"); + if ((buffer[4] & 0x04) !== 0) throw new Error("Checksummed zstd fallback inspection is not supported"); + if (!Number.isSafeInteger(declaredSize) || declaredSize < 0 || declaredSize > maximumBytes) { + throw new Error("Decompressed body exceeds the inspection limit"); + } + const windowSize = declaredZstdWindowSize(buffer); + if (!Number.isSafeInteger(windowSize) || windowSize < 0 || windowSize > maximumBytes) { + throw new Error("Zstd window exceeds the inspection limit"); + } + const collector = createBoundedCollector(maximumBytes); + const decompressor = new ZstdDecompress((chunk) => { + collector.append(Buffer.from(chunk)); + if (collector.truncated) throw new Error("Decompressed body exceeds the inspection limit"); + }); + decompressor.push(buffer, true); + if (collector.truncated || collector.totalBytes !== declaredSize) throw new Error("Invalid zstd body size"); + return collector.buffer(); + } + throw new Error(`Unsupported content encoding: ${encoding}`); +} + +function compressBody(buffer, encoding) { const enc = encoding.toLowerCase().trim(); - if (enc === "gzip") return zlib.gunzipSync(buffer); - if (enc === "deflate") return zlib.inflateSync(buffer); - if (enc === "br") return zlib.brotliDecompressSync(buffer); - if (enc === "zstd") return Buffer.from(zstdDecompress(buffer)); - return buffer; + if (enc === "" || enc === "identity") return buffer; + if (enc === "gzip") return zlib.gzipSync(buffer); + if (enc === "deflate") return zlib.deflateSync(buffer); + if (enc === "br") return zlib.brotliCompressSync(buffer); + if (enc === "zstd" && typeof zlib.zstdCompressSync === "function") { + return zlib.zstdCompressSync(buffer); + } + throw new Error(`Unsupported content encoding: ${encoding}`); +} + +function singleContentEncoding(value) { + if (value === undefined) return ""; + if (typeof value !== "string") return null; + const values = value.split(",").map((item) => item.trim().toLowerCase()).filter(Boolean); + return values.length === 1 ? values[0] : null; +} + +function hasSupportedCompressionEnvelope(buffer) { + if (buffer.length >= 2 && buffer[0] === 0x1f && buffer[1] === 0x8b) return true; + if (buffer.length >= 4 + && buffer[0] === 0x28 + && buffer[1] === 0xb5 + && buffer[2] === 0x2f + && buffer[3] === 0xfd) return true; + if (buffer.length >= 4 + && buffer[0] >= 0x50 + && buffer[0] <= 0x5f + && buffer[1] === 0x2a + && buffer[2] === 0x4d + && buffer[3] === 0x18) return true; + if (buffer.length < 2) return false; + const compressionMethod = buffer[0] & 0x0f; + const windowSize = buffer[0] >> 4; + return compressionMethod === 8 + && windowSize <= 7 + && (((buffer[0] << 8) | buffer[1]) % 31) === 0; +} + +function isRecoverablyCompressedBody(buffer) { + if (buffer.length === 0) return false; + if (hasSupportedCompressionEnvelope(buffer)) return true; + try { + zlib.brotliDecompressSync(buffer, { maxOutputLength: METRIC_BODY_INSPECTION_MAX_BYTES }); + return true; + } catch (error) { + return error?.code === "ERR_BUFFER_TOO_LARGE"; + } +} + +function captureBodySnapshot(collector, contentEncoding, protectedValues) { + const body = collector?.buffer() ?? Buffer.alloc(0); + const totalBytes = collector?.totalBytes ?? 0; + const normalizedEncoding = singleContentEncoding(contentEncoding); + const opaqueEncoding = protectedValues.length > 0 + && (normalizedEncoding === null + || (normalizedEncoding !== "" && normalizedEncoding !== "identity")); + const unscreenedPrefix = protectedValues.length > 0 && collector?.truncated === true; + if ((opaqueEncoding || unscreenedPrefix) && totalBytes > 0) { + return { body: Buffer.alloc(0), totalBytes, truncated: true }; + } + const recoverableSecret = protectedValues.length > 0 + && containsRecoverableProtectedValue(body.toString("utf8"), protectedValues); + const undeclaredCompression = protectedValues.length > 0 + && (normalizedEncoding === "" || normalizedEncoding === "identity") + && isRecoverablyCompressedBody(body); + if ((recoverableSecret || undeclaredCompression) && totalBytes > 0) { + return { body: Buffer.alloc(0), totalBytes, truncated: true }; + } + return { + body: redactProtectedBuffer(body, protectedValues), + totalBytes, + truncated: collector?.truncated ?? false + }; +} + +function decodeBoundedBody(buffer, encoding, maximumBytes) { + const normalized = singleContentEncoding(encoding); + if (normalized === null) return null; + try { + return decompressBody(buffer, normalized, maximumBytes); + } catch { + return null; + } +} + +function encodeBodyWithOriginalEncoding(buffer, encoding) { + const normalized = singleContentEncoding(encoding); + if (normalized === null) { + throw new Error("Multiple content encodings are not supported for model override"); + } + return compressBody(buffer, normalized); +} + +function createBoundedCollector(maximumBytes) { + let storage = null; + let length = 0; + let totalBytes = 0; + let truncated = false; + + return { + append(chunk) { + if (!Buffer.isBuffer(chunk)) chunk = Buffer.from(chunk); + totalBytes += chunk.length; + const remaining = maximumBytes - length; + if (remaining <= 0) { + if (chunk.length > 0) truncated = true; + return; + } + const copied = Math.min(remaining, chunk.length); + if (copied < chunk.length) truncated = true; + if (copied === 0) return; + const required = length + copied; + if (!storage || storage.length < required) { + const capacity = Math.min(maximumBytes, Math.max(required, storage ? storage.length * 2 : 4096)); + const replacement = Buffer.allocUnsafe(capacity); + if (storage && length > 0) storage.copy(replacement, 0, 0, length); + storage = replacement; + } + chunk.copy(storage, length, 0, copied); + length = required; + }, + buffer() { + return length === 0 ? Buffer.alloc(0) : Buffer.from(storage.subarray(0, length)); + }, + get length() { + return length; + }, + get totalBytes() { + return totalBytes; + }, + get truncated() { + return truncated; + } + }; +} + +function decodeJsonString(raw) { + try { + return JSON.parse(`"${raw}"`); + } catch { + return null; + } +} + +function createTopLevelJsonInspector({ stringKeys = [], tokenKeys = [] } = {}) { + const wantedStrings = new Set(stringKeys); + const wantedTokens = new Set(tokenKeys); + const strings = Object.create(null); + const tokens = Object.create(null); + const decoder = new TextDecoder("utf-8", { fatal: true }); + let rootStarted = false; + let complete = false; + let invalid = false; + let depth = 0; + let inString = false; + let escaped = false; + let stringRole = null; + let stringRaw = ""; + let stringOverflow = false; + let expectingKey = false; + let pendingKey = null; + let valueKey = null; + + function startString(role) { + inString = true; + escaped = false; + stringRole = role; + stringRaw = ""; + stringOverflow = false; + } + + function appendString(character) { + if (stringOverflow) return; + stringRaw += character; + if (stringRaw.length > 4096) { + stringRaw = ""; + stringOverflow = true; + } + } + + function finishString() { + const decoded = stringOverflow ? null : decodeJsonString(stringRaw); + if (stringRole === "key") { + pendingKey = typeof decoded === "string" ? decoded : null; + expectingKey = false; + } else if (stringRole === "value" && valueKey && typeof decoded === "string") { + strings[valueKey] = decoded; + } + stringRole = null; + valueKey = null; + } + + function inspectText(text) { + for (const character of text) { + if (invalid) break; + if (complete) { + if (!/\s/u.test(character)) invalid = true; + continue; + } + if (inString) { + if (escaped) { + appendString(character); + escaped = false; + } else if (character === "\\") { + appendString(character); + escaped = true; + } else if (character === "\"") { + inString = false; + finishString(); + } else { + appendString(character); + } + continue; + } + + if (!rootStarted) { + if (/\s/u.test(character)) continue; + if (character !== "{") { + invalid = true; + continue; + } + rootStarted = true; + depth = 1; + expectingKey = true; + continue; + } + + if (depth === 1) { + if (expectingKey && character === "\"") { + startString("key"); + continue; + } + if (pendingKey !== null) { + if (/\s/u.test(character)) continue; + if (character === ":") { + valueKey = pendingKey; + pendingKey = null; + continue; + } + invalid = true; + continue; + } + if (valueKey !== null) { + if (/\s/u.test(character)) continue; + const key = valueKey; + if (character === "\"") { + if (wantedStrings.has(key)) delete strings[key]; + startString(wantedStrings.has(key) ? "value" : null); + } else { + if (wantedStrings.has(key)) delete strings[key]; + if (wantedTokens.has(key)) tokens[key] = character; + valueKey = null; + if (character === "{" || character === "[") depth += 1; + } + continue; + } + if (character === ",") { + expectingKey = true; + continue; + } + if (character === "}") { + depth = 0; + complete = true; + continue; + } + } + + if (character === "\"") { + startString(null); + } else if (character === "{" || character === "[") { + depth += 1; + } else if (character === "}" || character === "]") { + depth -= 1; + if (depth < 0) invalid = true; + } + } + } + + return { + write(chunk) { + if (invalid) return; + try { + inspectText(decoder.decode(chunk, { stream: true })); + } catch { + invalid = true; + } + }, + end() { + if (!invalid) { + try { + inspectText(decoder.decode()); + } catch { + invalid = true; + } + } + return { strings, tokens, complete, invalid }; + }, + snapshot() { + return { strings, tokens, complete, invalid }; + } + }; } -function autoDecompress(buffer) { - if (buffer.length < 2) return null; - if (buffer[0] === 0x1f && buffer[1] === 0x8b) { - try { return zlib.gunzipSync(buffer); } catch { return null; } +function createSseInspector() { + const decoder = new TextDecoder("utf-8", { fatal: true }); + let pending = ""; + let previousWasCarriageReturn = false; + let eventName = ""; + let dataLines = []; + let eventBytes = 0; + let eventOverflow = false; + let terminal = null; + let usage = null; + + function resetEvent() { + eventName = ""; + dataLines = []; + eventBytes = 0; + eventOverflow = false; } - if (buffer[0] === 0x78 && (buffer[1] === 0x01 || buffer[1] === 0x5e || buffer[1] === 0x9c || buffer[1] === 0xda)) { - try { return zlib.inflateSync(buffer); } catch { return null; } + + function dispatch() { + if (eventOverflow || dataLines.length === 0) { + resetEvent(); + return; + } + const data = dataLines.join("\n"); + if (data === "[DONE]") { + resetEvent(); + return; + } + try { + const payload = JSON.parse(data); + const type = typeof payload?.type === "string" ? payload.type : eventName; + if (type === "response.completed") { + if (terminal !== "failure") { + terminal = payload?.response?.error == null ? "success" : "failure"; + } + usage = normalizeMetricUsage(payload?.response?.usage) ?? usage; + } else if (type === "response.failed" || type === "response.incomplete" || type === "error") { + terminal = "failure"; + } + } catch { + // Malformed or non-JSON events are not terminal protocol evidence. + } + resetEvent(); } - if (buffer.length >= 4 && buffer[0] === 0x28 && buffer[1] === 0xb5 && buffer[2] === 0x2f && buffer[3] === 0xfd) { - try { return Buffer.from(zstdDecompress(buffer)); } catch { return null; } + + function line(value) { + if (value.endsWith("\r")) value = value.slice(0, -1); + eventBytes += Buffer.byteLength(value) + 1; + if (eventBytes > SSE_EVENT_MAX_BYTES) { + eventOverflow = true; + dataLines = []; + } + if (value === "") { + dispatch(); + return; + } + if (value.startsWith(":")) return; + const separator = value.indexOf(":"); + const field = separator === -1 ? value : value.slice(0, separator); + let fieldValue = separator === -1 ? "" : value.slice(separator + 1); + if (fieldValue.startsWith(" ")) fieldValue = fieldValue.slice(1); + if (field === "event") { + eventName = fieldValue; + } else if (field === "data" && !eventOverflow) { + if (dataLines.length >= SSE_EVENT_MAX_DATA_LINES) { + eventOverflow = true; + dataLines = []; + } else { + dataLines.push(fieldValue); + } + } } - try { return zlib.brotliDecompressSync(buffer); } catch { return null; } + + function inspectText(text) { + for (const character of text) { + if (character === "\r") { + line(pending); + pending = ""; + previousWasCarriageReturn = true; + } else if (character === "\n") { + if (!previousWasCarriageReturn) line(pending); + pending = ""; + previousWasCarriageReturn = false; + } else { + previousWasCarriageReturn = false; + pending += character; + } + } + if (Buffer.byteLength(pending) > SSE_EVENT_MAX_BYTES) { + pending = ""; + eventOverflow = true; + } + } + + return { + write(chunk) { + if (terminal === "invalid") return; + try { + inspectText(decoder.decode(chunk, { stream: true })); + } catch { + terminal = "invalid"; + resetEvent(); + } + }, + end() { + if (terminal === "invalid") return { terminal: null, usage: null }; + try { + inspectText(decoder.decode()); + } catch { + return { terminal: null, usage: null }; + } + if (pending.length > 0) line(pending); + dispatch(); + return { terminal, usage }; + } + }; +} + +function protectedHeaderValues(settings) { + return normalizeProtectedValues([ + settings.upstream.apiKey, + ...Object.values(settings.upstream.extraHeaders) + ]); +} + +function headerContainsProtectedValue(value, protectedValues) { + const values = Array.isArray(value) ? value : [value]; + return values.some((item) => containsRecoverableProtectedValue(String(item), protectedValues)); } -function sanitizeHeadersForDebug(headersObject, authHeader = "authorization") { +function sanitizeHeadersForDebug(headersObject, authHeader = "authorization", protectedValues = []) { const result = {}; const activeAuthHeader = authHeader.toLowerCase(); for (const [key, value] of Object.entries(headersObject)) { const loweredKey = key.toLowerCase(); + const protectedName = containsRecoverableProtectedValue(key, protectedValues); + const protectedValue = headerContainsProtectedValue(value, protectedValues); const sensitive = loweredKey === activeAuthHeader || DEBUG_SENSITIVE_HEADER_NAMES.has(loweredKey) || DEBUG_SENSITIVE_HEADER_PARTS.some((part) => loweredKey.includes(part)); - result[key] = sensitive ? maskSecret(String(value)) : value; + const safeKey = protectedName ? "[REDACTED]" : key; + result[safeKey] = (protectedName || protectedValue) + ? "[REDACTED]" + : (sensitive ? maskSecret(String(value)) : value); } return result; } -function sanitizeHeadersForCapture(headersInput, authHeader) { +function sanitizeHeadersForCapture(headersInput, authHeader, protectedValues = []) { const result = headersToObject(headersInput); const activeAuthHeader = authHeader.toLowerCase(); for (const key of Object.keys(result)) { - if (key.toLowerCase() === activeAuthHeader) { - result[key] = "[REDACTED]"; - } + const protectedName = containsRecoverableProtectedValue(key, protectedValues); + const protectedValue = headerContainsProtectedValue(result[key], protectedValues); + const sensitive = key.toLowerCase() === activeAuthHeader || protectedName || protectedValue; + if (!sensitive) continue; + delete result[key]; + result[protectedName ? "[REDACTED]" : key] = "[REDACTED]"; } return result; } @@ -225,6 +889,7 @@ function sanitizeHeadersForCapture(headersInput, authHeader) { export function buildUpstreamHeaders(req, settings, targetUrl, { stripContentHeaders }) { const headers = []; const authHeader = settings.upstream.authHeader.toLowerCase(); + const connectionHeaders = connectionHeaderTokens(req.rawHeaders); for (let index = 0; index < req.rawHeaders.length; index += 2) { const key = req.rawHeaders[index]; @@ -233,6 +898,7 @@ export function buildUpstreamHeaders(req, settings, targetUrl, { stripContentHea if ( loweredKey === "host" || HOP_BY_HOP_HEADERS.has(loweredKey) || + connectionHeaders.has(loweredKey) || (stripContentHeaders && CONTENT_HEADERS.has(loweredKey)) ) { continue; @@ -245,17 +911,36 @@ export function buildUpstreamHeaders(req, settings, targetUrl, { stripContentHea upsertHeader(headers, "Host", targetUrl.host); - if (settings.proxy.overrideAuthorization) { + const generatedHeaderAllowed = (name) => { + const lowered = name.toLowerCase(); + return lowered !== "host" + && !HOP_BY_HOP_HEADERS.has(lowered) + && !connectionHeaders.has(lowered); + }; + + if (settings.proxy.overrideAuthorization && generatedHeaderAllowed(settings.upstream.authHeader)) { upsertHeader(headers, settings.upstream.authHeader, formatAuthorization(settings.upstream)); } for (const [key, value] of Object.entries(settings.upstream.extraHeaders)) { - upsertHeader(headers, key, value); + if (generatedHeaderAllowed(key)) upsertHeader(headers, key, value); } return headers; } +function connectionHeaderTokens(rawHeaders) { + const tokens = new Set(); + for (let index = 0; index < rawHeaders.length; index += 2) { + if (rawHeaders[index].toLowerCase() !== "connection") continue; + for (const token of rawHeaders[index + 1].split(",")) { + const normalized = token.trim().toLowerCase(); + if (normalized) tokens.add(normalized); + } + } + return tokens; +} + function upsertHeader(headers, key, value) { const lowered = key.toLowerCase(); for (let index = headers.length - 1; index >= 0; index -= 1) { @@ -266,11 +951,23 @@ function upsertHeader(headers, key, value) { headers.push([key, value]); } +function removeHeader(headers, key) { + const lowered = key.toLowerCase(); + for (let index = headers.length - 1; index >= 0; index -= 1) { + if (headers[index][0].toLowerCase() === lowered) headers.splice(index, 1); + } +} + +function stripBodyIntegrityHeaders(headers) { + for (const header of BODY_INTEGRITY_HEADERS) removeHeader(headers, header); +} + function writeHeadersToResponse(res, rawHeaders) { + const connectionHeaders = connectionHeaderTokens(rawHeaders); for (let index = 0; index < rawHeaders.length; index += 2) { const key = rawHeaders[index]; const value = rawHeaders[index + 1]; - if (HOP_BY_HOP_HEADERS.has(key.toLowerCase())) { + if (HOP_BY_HOP_HEADERS.has(key.toLowerCase()) || connectionHeaders.has(key.toLowerCase())) { continue; } res.appendHeader(key, value); @@ -306,6 +1003,108 @@ function parseBoundedJson(buffer, maximumBytes) { } } +function skipJsonWhitespace(source, offset) { + while (offset < source.length && /\s/u.test(source[offset])) offset += 1; + return offset; +} + +function jsonStringEnd(source, start) { + let escaped = false; + for (let offset = start + 1; offset < source.length; offset += 1) { + const character = source[offset]; + if (escaped) { + escaped = false; + } else if (character === "\\") { + escaped = true; + } else if (character === "\"") { + return offset + 1; + } + } + return -1; +} + +function jsonValueEnd(source, start) { + if (source[start] === "\"") return jsonStringEnd(source, start); + if (source[start] !== "{" && source[start] !== "[") { + let offset = start; + while (offset < source.length && !/[\s,}]/u.test(source[offset])) offset += 1; + return offset; + } + let depth = 0; + for (let offset = start; offset < source.length; offset += 1) { + const character = source[offset]; + if (character === "\"") { + offset = jsonStringEnd(source, offset) - 1; + } else if (character === "{" || character === "[") { + depth += 1; + } else if (character === "}" || character === "]") { + depth -= 1; + if (depth === 0) return offset + 1; + } + } + return -1; +} + +function rewriteTopLevelModel(buffer, modelOverride) { + if (!Buffer.isBuffer(buffer) || buffer.length === 0 || buffer.length > MODEL_OVERRIDE_MAX_BYTES) return null; + let source; + try { + source = new TextDecoder("utf-8", { fatal: true }).decode(buffer); + const parsed = JSON.parse(source); + if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return null; + } catch { + return null; + } + + let offset = skipJsonWhitespace(source, 0); + offset += 1; + let hasProperties = false; + const replacements = []; + let closingBrace = -1; + while (offset < source.length) { + offset = skipJsonWhitespace(source, offset); + if (source[offset] === "}") { + closingBrace = offset; + break; + } + hasProperties = true; + const keyStart = offset; + const keyEnd = jsonStringEnd(source, keyStart); + const key = JSON.parse(source.slice(keyStart, keyEnd)); + offset = skipJsonWhitespace(source, keyEnd) + 1; + offset = skipJsonWhitespace(source, offset); + const valueStart = offset; + const valueEnd = jsonValueEnd(source, valueStart); + if (key === "model") replacements.push({ start: valueStart, end: valueEnd }); + offset = skipJsonWhitespace(source, valueEnd); + if (source[offset] === ",") offset += 1; + } + if (closingBrace === -1) return null; + + const replacement = JSON.stringify(modelOverride); + if (replacements.length === 0) { + const insertion = `${hasProperties ? "," : ""}"model":${replacement}`; + const rewritten = Buffer.from(`${source.slice(0, closingBrace)}${insertion}${source.slice(closingBrace)}`); + return rewritten.length <= MODEL_OVERRIDE_MAX_BYTES + ? { body: rewritten, changed: true } + : { tooLarge: true }; + } + if (replacements.every(({ start, end }) => source.slice(start, end) === replacement)) { + return { body: buffer, changed: false }; + } + const chunks = []; + let cursor = 0; + for (const range of replacements) { + chunks.push(source.slice(cursor, range.start), replacement); + cursor = range.end; + } + chunks.push(source.slice(cursor)); + const rewritten = Buffer.from(chunks.join("")); + return rewritten.length <= MODEL_OVERRIDE_MAX_BYTES + ? { body: rewritten, changed: true } + : { tooLarge: true }; +} + function safeMetricModel(value, settings) { if (typeof value !== "string" || value.length === 0 || value.length > METRIC_MAX_MODEL_CODE_POINTS * 2 @@ -314,16 +1113,7 @@ function safeMetricModel(value, settings) { || METRIC_TEXT_CONTROL_PATTERN.test(value)) { return null; } - const protectedValues = [ - settings.upstream.apiKey, - ...Object.values(settings.upstream.extraHeaders) - ].filter((candidate) => typeof candidate === "string" && candidate.length > 0); - return protectedValues.some((secret) => value.includes(secret)) ? null : value; -} - -function extractMetricModel(body, settings) { - const parsed = parseBoundedJson(body, METRIC_MODEL_MAX_BYTES); - return parsed ? safeMetricModel(parsed.model, settings) : null; + return containsRecoverableProtectedValue(value, protectedHeaderValues(settings)) ? null : value; } function normalizeMetricUsage(value) { @@ -339,63 +1129,134 @@ function normalizeMetricUsage(value) { return { inputTokens, outputTokens }; } -function extractMetricUsage(body, stream) { - if (!Buffer.isBuffer(body) || body.length === 0 || body.length > METRIC_USAGE_MAX_BYTES) return null; - if (!stream) { - const parsed = parseBoundedJson(body, METRIC_USAGE_MAX_BYTES); - return normalizeMetricUsage(parsed?.usage); - } - let text; +function metricResultForStatus(statusCode) { + if (statusCode >= 200 && statusCode <= 299) return "success"; + if (statusCode >= 400 && statusCode <= 499) return "upstreamRejected"; + return "upstreamError"; +} + +function isResponsesRequest(requestUrl) { try { - text = new TextDecoder("utf-8", { fatal: true }).decode(body); + const pathname = new URL(requestUrl, "http://127.0.0.1").pathname.replace(/\/+$/, ""); + return pathname.endsWith("/responses"); } catch { - return null; + return false; } - let observed = null; - for (const line of text.split(/\r?\n/)) { - if (!line.startsWith("data:")) continue; - const data = line.slice(5).trimStart(); - if (data.length === 0 || data === "[DONE]") continue; - try { - const event = JSON.parse(data); - const usage = normalizeMetricUsage(event?.response?.usage); - if (usage) observed = usage; - } catch { - // Ignore non-JSON or partial SSE data without retaining it. - } +} + +function semanticResultForJson(body, fallbackInspection) { + const parsed = parseBoundedJson(body, METRIC_BODY_INSPECTION_MAX_BYTES); + if (parsed) { + const failed = parsed.error != null + || parsed.status === "failed" + || parsed.status === "incomplete" + || parsed.status === "cancelled"; + const completed = parsed.object === "response" + && parsed.status === "completed" + && Array.isArray(parsed.output); + return { + terminal: failed ? "failure" : (completed ? "success" : null), + usage: normalizeMetricUsage(parsed.usage) + }; + } + if (Buffer.isBuffer(body) && body.length > 0) { + return { terminal: null, usage: null }; } - return observed; + + const inspection = fallbackInspection?.snapshot?.() ?? fallbackInspection ?? {}; + if (inspection.invalid || inspection.complete !== true) { + return { terminal: null, usage: null }; + } + const status = inspection.strings?.status; + const explicitFailure = status === "failed" + || status === "incomplete" + || status === "cancelled" + || (inspection.tokens?.error && inspection.tokens.error !== "n"); + return { + // The lightweight scanner cannot validate an arbitrarily large JSON document. + terminal: explicitFailure ? "failure" : null, + usage: null + }; } -function metricResultForStatus(statusCode) { - if (statusCode >= 200 && statusCode <= 299) return "success"; - if (statusCode >= 400 && statusCode <= 499) return "upstreamRejected"; - return "upstreamError"; +function inspectCompletedResponse({ + statusCode, + stream, + responsesRequest, + contentEncoding, + metricCollector, + jsonInspector, + sseInspector +}) { + const statusResult = metricResultForStatus(statusCode); + let semantic = null; + let usage = null; + const encodedBody = metricCollector?.truncated ? null : metricCollector?.buffer(); + const decodedBody = encodedBody === null + ? null + : decodeBoundedBody(encodedBody, contentEncoding, METRIC_BODY_INSPECTION_MAX_BYTES); + + if (stream) { + const inspection = sseInspector + ? sseInspector.end() + : (decodedBody ? (() => { + const boundedSse = createSseInspector(); + boundedSse.write(decodedBody); + return boundedSse.end(); + })() : { terminal: null, usage: null }); + semantic = inspection.terminal; + usage = inspection.usage; + } else if (decodedBody) { + const inspection = semanticResultForJson(decodedBody, jsonInspector); + semantic = inspection.terminal; + usage = inspection.usage; + } else if (jsonInspector) { + semantic = semanticResultForJson(Buffer.alloc(0), jsonInspector).terminal; + } + + return { + result: statusResult === "success" && responsesRequest && semantic !== "success" + ? "upstreamError" + : statusResult, + usage + }; } function buildHealthPayload(settings, captureManager, settingsSource) { + const protectedValues = protectedHeaderValues(settings); + const captureState = redactProtectedFields(captureManager.getPublicState(), protectedValues); if (settingsSource) { - return { + return redactProtectedFields({ ok: true, ...settingsSource.publicState(), - ...captureManager.getPublicState() - }; + ...captureState + }, protectedValues); } - return { + return redactProtectedFields({ ok: true, - configPath: settings.configPath, + configPath: redactProtectedUrl(settings.configPath, protectedValues), listenHost: settings.server.host, listenPort: settings.server.port, - upstreamBaseUrl: settings.upstream.baseUrl, + upstreamBaseUrl: redactProtectedUrl(settings.upstream.baseUrl, protectedValues), overrideAuthorization: settings.proxy.overrideAuthorization, authHeader: settings.upstream.authHeader, authScheme: settings.upstream.authScheme, extraHeaderCount: Object.keys(settings.upstream.extraHeaders).length, - ...captureManager.getPublicState() - }; + ...captureState + }, protectedValues); } -function buildRequestContext({ req, settings, targetUrl, requestId, requestHeaders, requestBody, startedAt, captureHandle }) { +function buildRequestContext({ + req, + settings, + targetUrl, + requestId, + requestHeaders, + requestBody, + startedAt, + captureHandle, + protectedValues +}) { const turnMetadataHeader = req.headers["x-codex-turn-metadata"]; let turnMetadata = null; if (typeof turnMetadataHeader === "string") { @@ -406,19 +1267,26 @@ function buildRequestContext({ req, settings, targetUrl, requestId, requestHeade } } - const captureHeaders = sanitizeHeadersForCapture(requestHeaders, settings.upstream.authHeader); + const captureHeaders = sanitizeHeadersForCapture( + requestHeaders, + settings.upstream.authHeader, + protectedHeaderValues(settings) + ); return { requestId, - sessionId: typeof req.headers["session-id"] === "string" + sessionId: redactRecoverableProtectedText(typeof req.headers["session-id"] === "string" ? req.headers["session-id"] - : (typeof req.headers["session_id"] === "string" ? req.headers["session_id"] : (turnMetadata?.session_id || null)), - threadId: typeof req.headers["thread-id"] === "string" + : (typeof req.headers["session_id"] === "string" ? req.headers["session_id"] : (turnMetadata?.session_id || "")), protectedValues) || null, + threadId: redactRecoverableProtectedText(typeof req.headers["thread-id"] === "string" ? req.headers["thread-id"] - : (typeof req.headers["thread_id"] === "string" ? req.headers["thread_id"] : (turnMetadata?.thread_id || null)), - method: req.method || "GET", - incomingUrl: new URL(req.url, `http://${settings.server.host}:${settings.server.port}`).href, - targetUrl: targetUrl.href, + : (typeof req.headers["thread_id"] === "string" ? req.headers["thread_id"] : (turnMetadata?.thread_id || "")), protectedValues) || null, + method: redactRecoverableProtectedText(req.method || "GET", protectedValues), + incomingUrl: redactProtectedUrl( + new URL(req.url, `http://${settings.server.host}:${settings.server.port}`).href, + protectedValues + ), + targetUrl: redactProtectedUrl(targetUrl.href, protectedValues), requestHeaders: captureHeaders, requestBody, startedAt: new Date(startedAt).toISOString(), @@ -442,9 +1310,13 @@ function saveCaptureRecord(captureContext, fields) { targetUrl: captureContext.targetUrl, requestHeaders: captureContext.requestHeaders, requestBody: captureContext.requestBody, + requestBodyBytes: captureContext.requestBodyBytes, + requestBodyTruncated: captureContext.requestBodyTruncated, responseStatus: fields.responseStatus, responseHeaders: fields.responseHeaders ?? {}, responseBody: fields.responseBody ?? Buffer.alloc(0), + responseBodyBytes: fields.responseBodyBytes, + responseBodyTruncated: fields.responseBodyTruncated, isStream: fields.isStream ?? false, upstreamRequestId: fields.upstreamRequestId ?? null, errorType: fields.errorType ?? null, @@ -453,7 +1325,15 @@ function saveCaptureRecord(captureContext, fields) { } export function createServer(settings, { - captureManager = createCaptureManager({ configPath: settings.configPath, capture: settings.capture, log }).start(), + captureManager = createCaptureManager({ + configPath: settings.configPath, + capture: settings.capture, + log: (level, message, fields = {}) => log( + level, + message, + redactProtectedFields(fields, protectedHeaderValues(settings)) + ) + }).start(), logFn = log, settingsSource, recordMetric = () => {}, @@ -466,7 +1346,13 @@ export function createServer(settings, { } if (req.url === HEALTH_PATH) { - writeJson(res, 200, buildHealthPayload(settings, captureManager, settingsSource)); + let healthSettings = settings; + try { + healthSettings = settingsSource?.current().settings ?? settings; + } catch { + // An unconfigured source still exposes its existing public health contract. + } + writeJson(res, 200, buildHealthPayload(healthSettings, captureManager, settingsSource)); return; } @@ -484,268 +1370,594 @@ export function createServer(settings, { return; } const requestSettings = active.settings; + const requestProtectedValues = protectedHeaderValues(requestSettings); const requestDebugEnabled = requestSettings.server.logLevel.toLowerCase() === "debug"; - const requestId = req.headers[requestSettings.proxy.requestIdHeader] || req.headers["x-request-id"] || "-"; + const rawRequestId = req.headers[requestSettings.proxy.requestIdHeader] || req.headers["x-request-id"] || "-"; + const requestId = redactRecoverableProtectedText(String(rawRequestId), requestProtectedValues); const targetUrl = buildTargetUrl(requestSettings.upstream.baseUrl, req.url); + const safeRequestPath = redactProtectedUrl(req.url, requestProtectedValues); + const safeTargetUrl = redactProtectedUrl(targetUrl.href, requestProtectedValues); + const safeMethod = redactRecoverableProtectedText(req.method || "GET", requestProtectedValues); const transport = targetUrl.protocol === "https:" ? https : http; const startedAt = Date.now(); const metricStartedAt = metricNow(); + const responsesRequest = isResponsesRequest(req.url); + const overrideModel = requestSettings.proxy.modelMode === "override" + && typeof requestSettings.proxy.modelOverride === "string" + ? requestSettings.proxy.modelOverride + : null; + const requestEncoding = req.headers["content-encoding"]; + const normalizedRequestEncoding = singleContentEncoding(requestEncoding); + const directRequestInspection = normalizedRequestEncoding === "" || normalizedRequestEncoding === "identity"; + const captureHandle = captureManager.beginRecord(); + let requestCapture = captureHandle ? createBoundedCollector(CAPTURE_BODY_MAX_BYTES) : null; + const requestPreview = requestDebugEnabled ? createBoundedCollector(4096) : null; + const requestModelInspector = !overrideModel && directRequestInspection + ? createTopLevelJsonInspector({ stringKeys: ["model"] }) + : null; + const encodedRequestMetric = !overrideModel && !directRequestInspection + ? createBoundedCollector(METRIC_BODY_INSPECTION_MAX_BYTES) + : null; + const overrideCollector = overrideModel ? createBoundedCollector(MODEL_OVERRIDE_MAX_BYTES) : null; + let forwardedHeaders = buildUpstreamHeaders(req, requestSettings, targetUrl, { + stripContentHeaders: false + }); + let captureContext = null; + let captureSaved = false; + let metricSaved = false; + let terminal = false; + let requestEnded = false; + let requestModel = overrideModel ? safeMetricModel(overrideModel, requestSettings) : null; + let requestInspectionFinished = Boolean(overrideModel); + let upstreamRequest = null; + let upstreamResponse = null; + let responseState = null; + let responseStartBin = null; + let timedOut = false; + + function requestCaptureSnapshot() { + const snapshot = captureBodySnapshot(requestCapture, requestEncoding, requestProtectedValues); + return { + requestBody: snapshot.body, + requestBodyBytes: snapshot.totalBytes, + requestBodyTruncated: snapshot.truncated + }; + } - const chunks = []; - req.on("data", (chunk) => chunks.push(chunk)); - req.on("end", () => { - let body = Buffer.concat(chunks); - const contentEncoding = req.headers["content-encoding"]; - let bodyTransformed = false; - if (contentEncoding && body.length) { - try { - body = decompressBody(body, contentEncoding); - bodyTransformed = true; - } catch (error) { - logFn("warn", "Failed to decompress request body", { - encoding: contentEncoding, - error: error.message - }); - } - } else if (body.length >= 2 && contentEncoding === undefined) { - const decompressed = autoDecompress(body); - if (decompressed) { - debugLog("AUTODECOMP", { - originalSize: body.length, - decompressedSize: decompressed.length, - magicBytes: `0x${body[0].toString(16).padStart(2, "0")} 0x${body[1].toString(16).padStart(2, "0")}` - }, requestDebugEnabled); - body = decompressed; - bodyTransformed = true; - } - } - - const headers = buildUpstreamHeaders(req, requestSettings, targetUrl, { - stripContentHeaders: bodyTransformed - }); - if (bodyTransformed && body.length) { - upsertHeader(headers, "content-length", String(Buffer.byteLength(body))); - } - - const captureHandle = captureManager.beginRecord() ?? createNoopCaptureHandle(); - const captureContext = buildRequestContext({ + function ensureCaptureContext() { + if (!captureHandle || captureContext) return captureContext; + captureContext = buildRequestContext({ req, settings: requestSettings, targetUrl, requestId, - requestHeaders: headers, - requestBody: body, + requestHeaders: forwardedHeaders, + requestBody: Buffer.alloc(0), startedAt, - captureHandle + captureHandle, + protectedValues: requestProtectedValues }); - let captureSaved = false; - let metricSaved = false; - let responseCompleted = false; - let responseStartBin = null; - const metricModel = extractMetricModel(body, requestSettings); - - function finalizeCapture(fields) { - if (captureSaved) { - return; - } - captureSaved = true; - saveCaptureRecord(captureContext, fields); - } + return captureContext; + } + + function finalizeCapture(fields) { + if (captureSaved || !captureHandle) return; + captureSaved = true; + const context = ensureCaptureContext(); + Object.assign(context, requestCaptureSnapshot()); + saveCaptureRecord(context, fields); + } - function finalizeMetric(result, { responseBody = null, stream = false } = {}) { - if (metricSaved) return; - metricSaved = true; - if (!Number.isSafeInteger(active.generation) || active.generation <= 0) return; - const usage = result === "success" && responseBody !== null - ? extractMetricUsage(responseBody, stream) + function finishRequestInspection() { + if (requestInspectionFinished) return requestModel; + requestInspectionFinished = true; + if (requestModelInspector) { + const inspection = requestModelInspector.end(); + requestModel = inspection.complete && !inspection.invalid + ? safeMetricModel(inspection.strings.model, requestSettings) : null; - const observation = { - generation: active.generation, - result, - model: metricModel, - inputTokens: usage?.inputTokens ?? null, - outputTokens: usage?.outputTokens ?? null, - durationBin: metricLatencyBin(metricNow() - metricStartedAt), - responseStartBin - }; - try { - const pending = recordMetric(observation); - if (pending && typeof pending.then === "function") void pending.catch(() => {}); - } catch { - // Operational metrics must never affect proxy forwarding. + } else if (encodedRequestMetric && !encodedRequestMetric.truncated) { + const decoded = decodeBoundedBody( + encodedRequestMetric.buffer(), + requestEncoding, + METRIC_BODY_INSPECTION_MAX_BYTES + ); + if (decoded) { + const inspector = createTopLevelJsonInspector({ stringKeys: ["model"] }); + inspector.write(decoded); + const inspection = inspector.end(); + requestModel = inspection.complete && !inspection.invalid + ? safeMetricModel(inspection.strings.model, requestSettings) + : null; } } + return requestModel; + } + + function finalizeMetric(result, usage = null) { + if (metricSaved) return; + metricSaved = true; + if (!Number.isSafeInteger(active.generation) || active.generation <= 0) return; + const observation = { + generation: active.generation, + result, + model: requestEnded ? finishRequestInspection() : requestModel, + inputTokens: result === "success" ? (usage?.inputTokens ?? null) : null, + outputTokens: result === "success" ? (usage?.outputTokens ?? null) : null, + durationBin: metricLatencyBin(metricNow() - metricStartedAt), + responseStartBin + }; + try { + const pending = recordMetric(observation); + if (pending && typeof pending.then === "function") void pending.catch(() => {}); + } catch { + // Operational metrics must never affect proxy forwarding. + } + } + function clearResponseHeaders() { + if (res.headersSent) return; + for (const name of res.getHeaderNames()) res.removeHeader(name); + } + + function logRequestBody(body = null) { + if (!requestDebugEnabled) return; + const preview = body ?? requestPreview?.buffer() ?? Buffer.alloc(0); debugLog("REQUEST", { - method: req.method, - path: req.url, - targetUrl: targetUrl.href, + method: safeMethod, + path: safeRequestPath, + targetUrl: safeTargetUrl, incomingHeaders: sanitizeHeadersForDebug( Object.fromEntries(Object.entries(req.headers)), - requestSettings.upstream.authHeader + requestSettings.upstream.authHeader, + requestProtectedValues ), upstreamHeaders: sanitizeHeadersForDebug( - Object.fromEntries(headers), - requestSettings.upstream.authHeader + Object.fromEntries(forwardedHeaders), + requestSettings.upstream.authHeader, + requestProtectedValues ), - body: safeBodyPreview(body) - }, requestDebugEnabled); - - const upstreamRequest = transport.request( - { - method: req.method, - protocol: targetUrl.protocol, - hostname: targetUrl.hostname, - port: targetUrl.port || undefined, - path: `${targetUrl.pathname}${targetUrl.search}`, - headers, - rejectUnauthorized: requestSettings.upstream.verifySsl - }, - (upstreamResponse) => { - const stream = isEventStream(upstreamResponse.headers["content-type"]); - debugLog("RESPONSE HEADERS", { - status: upstreamResponse.statusCode, - headers: sanitizeHeadersForDebug( - upstreamResponse.headers, - requestSettings.upstream.authHeader - ) - }, requestDebugEnabled); - - const responseHeaders = sanitizeHeadersForCapture( - upstreamResponse.headers, - requestSettings.upstream.authHeader - ); - const respChunks = []; - upstreamResponse.on("data", (chunk) => { - if (responseStartBin === null && chunk.length > 0) { - responseStartBin = metricLatencyBin(metricNow() - metricStartedAt); - } - respChunks.push(chunk); - }); + body: safeBodyPreview(preview, 4096, requestProtectedValues, { + contentEncoding: requestEncoding, + truncated: requestPreview?.truncated === true + }), + bodyTruncated: body === null && requestPreview?.truncated === true + }, true); + } - res.statusCode = upstreamResponse.statusCode || 502; - writeHeadersToResponse(res, upstreamResponse.rawHeaders); - upstreamResponse.pipe(res); - upstreamResponse.on("end", () => { - responseCompleted = true; - const responseBody = Buffer.concat(respChunks); - if (responseBody.length) { - debugLog("RESPONSE BODY", { - status: upstreamResponse.statusCode, - body: safeBodyPreview(responseBody) - }, requestDebugEnabled); - } - finalizeCapture({ - responseStatus: upstreamResponse.statusCode || 502, - responseHeaders, - responseBody, - isStream: stream, - upstreamRequestId: typeof upstreamResponse.headers["x-request-id"] === "string" ? upstreamResponse.headers["x-request-id"] : null - }); - finalizeMetric(metricResultForStatus(upstreamResponse.statusCode || 502), { - responseBody, - stream - }); - logFn("info", "Proxied request", { - request_id: requestId, - method: req.method || "GET", - path: req.url, - status: upstreamResponse.statusCode || 502, - stream, - duration_ms: Date.now() - startedAt - }); - }); - upstreamResponse.once("aborted", () => finalizeMetric("networkError")); - upstreamResponse.once("error", () => finalizeMetric("networkError")); + function proxyErrorPayload(statusCode, errorType) { + return { + error: { + message: statusCode === 504 + ? "Upstream request timed out" + : (statusCode === 413 + ? "Request body is too large for model override" + : (statusCode === 400 + ? "Request body must be a valid JSON object for model override" + : (statusCode === 415 + ? "Request content encoding is not supported for model override" + : "Failed to reach upstream service"))), + type: errorType, + request_id: requestId } + }; + } + + function responseCaptureSnapshot() { + const snapshot = captureBodySnapshot( + responseState?.capture, + responseState?.contentEncoding, + requestProtectedValues ); + return { + responseBody: snapshot.body, + responseBodyBytes: snapshot.totalBytes, + responseBodyTruncated: snapshot.truncated + }; + } - upstreamRequest.setTimeout(requestSettings.upstream.timeoutMs, () => { - upstreamRequest.destroy(new Error("upstream timeout")); + function finishProxyFailure({ statusCode, errorType, result, error, responseStarted = false }) { + if (terminal) return; + terminal = true; + const message = redactRecoverableProtectedText( + error instanceof Error ? error.message : String(error ?? errorType), + requestProtectedValues + ); + upstreamResponse?.destroy(); + upstreamRequest?.destroy(); + req.resume?.(); + const payload = proxyErrorPayload(statusCode, errorType); + const proxyBody = Buffer.from(JSON.stringify(payload)); + const proxyHeaders = { + "content-type": "application/json; charset=utf-8", + "content-length": String(proxyBody.length) + }; + + debugLog("UPSTREAM ERROR", { + error: message, + code: redactRecoverableProtectedText(error?.code || "(none)", requestProtectedValues), + stack: error?.stack + ? redactRecoverableProtectedText(error.stack, requestProtectedValues) + : undefined + }, requestDebugEnabled); + if (!res.headersSent && !res.destroyed) { + clearResponseHeaders(); + writeJson(res, statusCode, payload); + } else if (!res.destroyed) { + res.destroy(); + } + finalizeCapture({ + responseStatus: responseStarted ? (responseState?.statusCode ?? statusCode) : statusCode, + responseHeaders: responseStarted ? (responseState?.captureHeaders ?? {}) : proxyHeaders, + ...(responseStarted + ? responseCaptureSnapshot() + : { + responseBody: proxyBody, + responseBodyBytes: proxyBody.length, + responseBodyTruncated: false + }), + isStream: responseState?.stream ?? false, + upstreamRequestId: responseStarted ? (responseState?.upstreamRequestId ?? null) : null, + errorType, + errorMessage: message + }); + finalizeMetric(result); + logFn("warn", "Proxy request failed", { + request_id: requestId, + method: safeMethod, + path: safeRequestPath, + status: responseStarted ? (responseState?.statusCode ?? statusCode) : statusCode, + duration_ms: Date.now() - startedAt, + error: JSON.stringify(message) }); + } - upstreamRequest.on("error", (error) => { - const statusCode = error.message === "upstream timeout" ? 504 : 502; - const errorType = statusCode === 504 ? "proxy_timeout" : "proxy_upstream_error"; - const payload = { - error: { - message: statusCode === 504 ? "Upstream request timed out" : "Failed to reach upstream service", - type: errorType, - request_id: requestId - } - }; - const responseBody = Buffer.from(JSON.stringify(payload)); - const responseHeaders = { - "content-type": "application/json; charset=utf-8", - "content-length": String(responseBody.length) - }; - - debugLog("UPSTREAM ERROR", { - error: error.message, - code: error.code || "(none)", - stack: error.stack + function handleClientAbort() { + if (terminal || res.writableFinished) return; + terminal = true; + upstreamResponse?.destroy(); + upstreamRequest?.destroy(); + finalizeCapture({ + responseStatus: responseState?.statusCode ?? (res.headersSent ? res.statusCode : null), + responseHeaders: responseState?.captureHeaders ?? {}, + ...responseCaptureSnapshot(), + isStream: responseState?.stream ?? false, + upstreamRequestId: responseState?.upstreamRequestId ?? null, + errorType: "proxy_client_abort", + errorMessage: "Client closed connection" + }); + finalizeMetric("clientAbort"); + } + + function maybeCompleteResponse() { + if (terminal || !responseState?.upstreamEnded || !responseState.downstreamFinished) return; + terminal = true; + if (!requestEnded) { + upstreamRequest?.destroy(); + req.resume?.(); + } + const inspected = inspectCompletedResponse({ + statusCode: responseState.statusCode, + stream: responseState.stream, + responsesRequest, + contentEncoding: responseState.contentEncoding, + metricCollector: responseState.metricCollector, + jsonInspector: responseState.jsonInspector, + sseInspector: responseState.sseInspector + }); + const captureSnapshot = responseCaptureSnapshot(); + const responseBody = captureSnapshot.responseBody; + const responsePreview = responseState.preview?.buffer() ?? responseBody; + if (responsePreview.length) { + debugLog("RESPONSE BODY", { + status: responseState.statusCode, + body: safeBodyPreview(responsePreview, 4096, requestProtectedValues, { + contentEncoding: responseState.contentEncoding, + truncated: responseState.preview?.truncated === true + }), + bodyTruncated: responseState.preview?.truncated === true }, requestDebugEnabled); - if (!res.headersSent) { - writeJson(res, statusCode, payload); - } else { - res.destroy(error); + } + finalizeCapture({ + responseStatus: responseState.statusCode, + responseHeaders: responseState.captureHeaders, + ...captureSnapshot, + isStream: responseState.stream, + upstreamRequestId: responseState.upstreamRequestId + }); + finalizeMetric(inspected.result, inspected.usage); + logFn("info", "Proxied request", { + request_id: requestId, + method: safeMethod, + path: safeRequestPath, + status: responseState.statusCode, + stream: responseState.stream, + duration_ms: Date.now() - startedAt + }); + } + + function onUpstreamResponse(incoming) { + if (terminal) { + incoming.destroy(); + return; + } + upstreamResponse = incoming; + const stream = isEventStream(incoming.headers["content-type"]); + const contentEncoding = incoming.headers["content-encoding"]; + const normalizedEncoding = singleContentEncoding(contentEncoding); + const directInspection = normalizedEncoding === "" || normalizedEncoding === "identity"; + responseState = { + statusCode: incoming.statusCode || 502, + stream, + contentEncoding, + captureHeaders: sanitizeHeadersForCapture( + incoming.headers, + requestSettings.upstream.authHeader, + requestProtectedValues + ), + upstreamRequestId: typeof incoming.headers["x-request-id"] === "string" + ? redactRecoverableProtectedText(incoming.headers["x-request-id"], requestProtectedValues) + : null, + capture: captureHandle ? createBoundedCollector(CAPTURE_BODY_MAX_BYTES) : null, + preview: requestDebugEnabled ? createBoundedCollector(4096) : null, + metricCollector: (!directInspection || !stream) + ? createBoundedCollector(METRIC_BODY_INSPECTION_MAX_BYTES) + : null, + jsonInspector: directInspection && !stream + ? createTopLevelJsonInspector({ + stringKeys: ["status"], + tokenKeys: ["error"] + }) + : null, + sseInspector: directInspection && stream ? createSseInspector() : null, + upstreamEnded: false, + downstreamFinished: false + }; + + debugLog("RESPONSE HEADERS", { + status: incoming.statusCode, + headers: sanitizeHeadersForDebug( + incoming.headers, + requestSettings.upstream.authHeader, + requestProtectedValues + ) + }, requestDebugEnabled); + + incoming.on("data", (chunk) => { + if (terminal) return; + if (responseStartBin === null && chunk.length > 0) { + responseStartBin = metricLatencyBin(metricNow() - metricStartedAt); } - finalizeCapture({ - responseStatus: statusCode, - responseHeaders, - responseBody, - errorType, - errorMessage: error.message, - upstreamRequestId: null + responseState.capture?.append(chunk); + responseState.preview?.append(chunk); + responseState.metricCollector?.append(chunk); + responseState.jsonInspector?.write(chunk); + responseState.sseInspector?.write(chunk); + }); + incoming.once("end", () => { + if (terminal) return; + responseState.jsonInspector?.end(); + responseState.upstreamEnded = true; + maybeCompleteResponse(); + }); + const responseFailure = (error) => finishProxyFailure({ + statusCode: timedOut ? 504 : 502, + errorType: timedOut ? "proxy_timeout" : "proxy_upstream_response_error", + result: timedOut ? "timeout" : "networkError", + error: error ?? new Error("Upstream response ended unexpectedly"), + responseStarted: res.headersSent + }); + incoming.once("aborted", responseFailure); + incoming.once("error", responseFailure); + incoming.once("close", () => { + if (!incoming.complete && !responseState?.upstreamEnded) responseFailure(new Error("Upstream response closed early")); + }); + + res.statusCode = responseState.statusCode; + writeHeadersToResponse(res, incoming.rawHeaders); + incoming.pipe(res); + } + + function createUpstreamRequest() { + if (terminal) return null; + try { + upstreamRequest = transport.request( + { + method: req.method, + protocol: targetUrl.protocol, + hostname: targetUrl.hostname, + port: targetUrl.port || undefined, + path: `${targetUrl.pathname}${targetUrl.search}`, + headers: forwardedHeaders, + rejectUnauthorized: requestSettings.upstream.verifySsl + }, + onUpstreamResponse + ); + } catch (error) { + finishProxyFailure({ + statusCode: 502, + errorType: "proxy_upstream_error", + result: "networkError", + error }); - finalizeMetric(statusCode === 504 ? "timeout" : "networkError"); - logFn("warn", "Proxy request failed", { - request_id: requestId, - method: req.method || "GET", - path: req.url, - status: statusCode, - duration_ms: Date.now() - startedAt, - error: JSON.stringify(error.message) + return null; + } + ensureCaptureContext(); + upstreamRequest.setTimeout(requestSettings.upstream.timeoutMs, () => { + timedOut = true; + upstreamRequest.destroy(new Error("upstream timeout")); + }); + upstreamRequest.on("error", (error) => { + if (terminal) return; + finishProxyFailure({ + statusCode: timedOut ? 504 : 502, + errorType: timedOut ? "proxy_timeout" : "proxy_upstream_error", + result: timedOut ? "timeout" : "networkError", + error, + responseStarted: Boolean(upstreamResponse && res.headersSent) }); }); + upstreamRequest.on("drain", () => { + if (!terminal) req.resume?.(); + }); + return upstreamRequest; + } - res.on("close", () => { - if (responseCompleted || res.writableFinished) { + res.on("finish", () => { + if (!responseState) return; + responseState.downstreamFinished = true; + maybeCompleteResponse(); + }); + res.on("close", handleClientAbort); + req.on("aborted", handleClientAbort); + req.on("error", handleClientAbort); + + if (overrideModel) { + req.on("data", (chunk) => { + if (terminal) return; + requestCapture?.append(chunk); + requestPreview?.append(chunk); + overrideCollector.append(chunk); + if (overrideCollector.truncated) { + logRequestBody(); + finishProxyFailure({ + statusCode: 413, + errorType: "proxy_request_too_large", + result: "upstreamError", + error: new Error("model override request exceeds the bounded transformation limit") + }); + } + }); + req.on("end", () => { + requestEnded = true; + if (terminal) return; + const encodedBody = overrideCollector.buffer(); + const normalizedEncoding = singleContentEncoding(requestEncoding); + const supportedEncoding = normalizedEncoding === "" + || normalizedEncoding === "identity" + || normalizedEncoding === "gzip" + || normalizedEncoding === "deflate" + || normalizedEncoding === "br" + || normalizedEncoding === "zstd"; + if (!supportedEncoding) { + logRequestBody(); + finishProxyFailure({ + statusCode: 415, + errorType: "proxy_unsupported_content_encoding", + result: "upstreamError", + error: new Error("model override content encoding is unsupported") + }); return; } - finalizeCapture({ - responseStatus: res.statusCode || null, - responseHeaders: {}, - responseBody: Buffer.alloc(0), - isStream: false, - upstreamRequestId: null, - errorType: "proxy_client_abort", - errorMessage: "Client closed connection" - }); - finalizeMetric("clientAbort"); + let decodedBody; + try { + decodedBody = decompressBody(encodedBody, normalizedEncoding, MODEL_OVERRIDE_MAX_BYTES); + } catch (error) { + const tooLarge = error?.code === "ERR_BUFFER_TOO_LARGE" + || String(error?.message).includes("exceeds the inspection limit"); + finishProxyFailure({ + statusCode: tooLarge ? 413 : 400, + errorType: tooLarge ? "proxy_request_too_large" : "proxy_bad_request", + result: "upstreamError", + error + }); + return; + } + const rewrite = rewriteTopLevelModel(decodedBody, overrideModel); + if (!rewrite) { + logRequestBody(); + finishProxyFailure({ + statusCode: 400, + errorType: "proxy_bad_request", + result: "upstreamError", + error: new Error("model override request is not a valid JSON object") + }); + return; + } + if (rewrite.tooLarge) { + finishProxyFailure({ + statusCode: 413, + errorType: "proxy_request_too_large", + result: "upstreamError", + error: new Error("rewritten model override request exceeds the bounded transformation limit") + }); + return; + } + let forwardedBody = encodedBody; + if (rewrite.changed) { + try { + if (normalizedEncoding === "zstd" && typeof zlib.zstdCompressSync !== "function") { + forwardedBody = rewrite.body; + removeHeader(forwardedHeaders, "content-encoding"); + } else { + forwardedBody = encodeBodyWithOriginalEncoding(rewrite.body, requestEncoding); + } + } catch (error) { + finishProxyFailure({ + statusCode: 415, + errorType: "proxy_unsupported_content_encoding", + result: "upstreamError", + error + }); + return; + } + stripBodyIntegrityHeaders(forwardedHeaders); + upsertHeader(forwardedHeaders, "content-length", String(forwardedBody.length)); + } + if (requestCapture) { + requestCapture = createBoundedCollector(CAPTURE_BODY_MAX_BYTES); + requestCapture.append(forwardedBody); + } + logRequestBody(forwardedBody.subarray(0, 4096)); + const outgoing = createUpstreamRequest(); + outgoing?.end(forwardedBody); }); + return; + } - upstreamRequest.end(body); + let outgoing = null; + req.on("data", (chunk) => { + if (terminal) return; + outgoing ??= createUpstreamRequest(); + requestCapture?.append(chunk); + requestPreview?.append(chunk); + requestModelInspector?.write(chunk); + encodedRequestMetric?.append(chunk); + if (outgoing && !outgoing.write(chunk)) req.pause?.(); + }); + req.on("end", () => { + requestEnded = true; + finishRequestInspection(); + logRequestBody(); + if (!terminal) { + outgoing ??= createUpstreamRequest(); + outgoing?.end(); + } }); }); } export function createApp(settings = loadConfig(), { settingsSource, recordMetric } = {}) { + const protectedValues = protectedHeaderValues(settings); const captureManager = createCaptureManager({ configPath: settings.configPath, capture: settings.capture, - log + log: (level, message, fields = {}) => log( + level, + message, + redactProtectedFields(fields, protectedValues) + ) }).start(); - log("info", "Loaded proxy config", { - config_path: settings.configPath, - upstream: settings.upstream.baseUrl, + log("info", "Loaded proxy config", redactProtectedFields({ + config_path: redactProtectedUrl(settings.configPath, protectedValues), + upstream: redactProtectedUrl(settings.upstream.baseUrl, protectedValues), auth_override: settings.proxy.overrideAuthorization, auth_header: settings.upstream.authHeader, api_key: maskSecret(settings.upstream.apiKey), capture_enabled: settings.capture.enabled, - capture_db_path: settings.capture.dbPath - }); + capture_db_path: redactProtectedUrl(settings.capture.dbPath, protectedValues) + }, protectedValues)); const server = createServer(settings, { captureManager, @@ -762,20 +1974,21 @@ export function createApp(settings = loadConfig(), { settingsSource, recordMetri export function startServer(settings = loadConfig()) { const app = createApp(settings); + const protectedValues = protectedHeaderValues(settings); app.server.on("error", (error) => { - log("error", "Node proxy failed to listen", { + log("error", "Node proxy failed to listen", redactProtectedFields({ host: settings.server.host, port: settings.server.port, - error: JSON.stringify(error.message) - }); + error: JSON.stringify(redactRecoverableProtectedText(error.message, protectedValues)) + }, protectedValues)); process.exit(1); }); app.server.listen(settings.server.port, settings.server.host, () => { - log("info", "Node proxy listening", { + log("info", "Node proxy listening", redactProtectedFields({ host: settings.server.host, port: settings.server.port - }); + }, protectedValues)); }); return app; diff --git a/node/src/supervisor/metrics-store.mjs b/node/src/supervisor/metrics-store.mjs index 631aa8d..66cabf7 100644 --- a/node/src/supervisor/metrics-store.mjs +++ b/node/src/supervisor/metrics-store.mjs @@ -313,6 +313,31 @@ function mergeBins(target, source) { } } +function projectRequestRows(rows, trackedRequests, limit = 16) { + const projected = []; + let remaining = trackedRequests; + for (const row of rows) { + if (projected.length === limit || remaining === 0) break; + const requests = Math.min(row.requests, remaining); + if (requests > 0) projected.push({ row, requests }); + remaining -= requests; + } + return projected; +} + +function leastRequestedIndex(rows, field) { + let selected = 0; + for (let index = 1; index < rows.length; index += 1) { + const row = rows[index]; + const current = rows[selected]; + if (row.requests < current.requests + || row.requests === current.requests && compareText(row[field], current[field]) > 0) { + selected = index; + } + } + return selected; +} + function percentileUpperBound(bins, percentile) { const total = bins.reduce((sum, count) => sum + count, 0); if (total === 0) return null; @@ -388,6 +413,13 @@ export class MetricsStore { this.buckets.push(bucket); } + if (bucket.requests === MAX_COUNTER) { + bucket.droppedObservations = Math.min(MAX_COUNTER, bucket.droppedObservations + 1); + this.dirty = true; + this.#scheduleFlush(); + return true; + } + addBounded(bucket, "requests", 1, MAX_COUNTER, bucket); addBounded(bucket.results, observation.result, 1, MAX_COUNTER, bucket); addBin(bucket.durationBins, observation.durationBin, bucket); @@ -402,7 +434,11 @@ export class MetricsStore { } let provider = bucket.providers.find((row) => row.providerId === observation.providerId); - if (!provider && bucket.providers.length < MAX_PROVIDERS_PER_BUCKET) { + if (!provider && bucket.providers.length === MAX_PROVIDERS_PER_BUCKET) { + const [evicted] = bucket.providers.splice(leastRequestedIndex(bucket.providers, "providerId"), 1); + addBounded(bucket, "providerOverflowRequests", evicted.requests, MAX_COUNTER, bucket); + } + if (!provider) { provider = { providerId: observation.providerId, requests: 0, @@ -414,26 +450,26 @@ export class MetricsStore { }; bucket.providers.push(provider); } - if (provider) { - addBounded(provider, "requests", 1, MAX_COUNTER, bucket); - if (observation.result === "success") { - addBounded(provider, "successfulRequests", 1, MAX_COUNTER, bucket); - } - addBin(provider.durationBins, observation.durationBin, bucket); - if (hasUsage) { - addBounded(provider, "usageObservedRequests", 1, MAX_COUNTER, bucket); - addBounded(provider, "inputTokens", observation.inputTokens, MAX_TOKEN_TOTAL, bucket); - addBounded(provider, "outputTokens", observation.outputTokens, MAX_TOKEN_TOTAL, bucket); - } - } else { - addBounded(bucket, "providerOverflowRequests", 1, MAX_COUNTER, bucket); + addBounded(provider, "requests", 1, MAX_COUNTER, bucket); + if (observation.result === "success") { + addBounded(provider, "successfulRequests", 1, MAX_COUNTER, bucket); + } + addBin(provider.durationBins, observation.durationBin, bucket); + if (hasUsage) { + addBounded(provider, "usageObservedRequests", 1, MAX_COUNTER, bucket); + addBounded(provider, "inputTokens", observation.inputTokens, MAX_TOKEN_TOTAL, bucket); + addBounded(provider, "outputTokens", observation.outputTokens, MAX_TOKEN_TOTAL, bucket); } if (observation.model === null) { addBounded(bucket, "unknownModelRequests", 1, MAX_COUNTER, bucket); } else { let model = bucket.models.find((row) => row.model === observation.model); - if (!model && bucket.models.length < MAX_MODELS_PER_BUCKET) { + if (!model && bucket.models.length === MAX_MODELS_PER_BUCKET) { + const [evicted] = bucket.models.splice(leastRequestedIndex(bucket.models, "model"), 1); + addBounded(bucket, "modelOverflowRequests", evicted.requests, MAX_COUNTER, bucket); + } + if (!model) { model = { model: observation.model, requests: 0, @@ -443,15 +479,11 @@ export class MetricsStore { }; bucket.models.push(model); } - if (model) { - addBounded(model, "requests", 1, MAX_COUNTER, bucket); - if (hasUsage) { - addBounded(model, "usageObservedRequests", 1, MAX_COUNTER, bucket); - addBounded(model, "inputTokens", observation.inputTokens, MAX_TOKEN_TOTAL, bucket); - addBounded(model, "outputTokens", observation.outputTokens, MAX_TOKEN_TOTAL, bucket); - } - } else { - addBounded(bucket, "modelOverflowRequests", 1, MAX_COUNTER, bucket); + addBounded(model, "requests", 1, MAX_COUNTER, bucket); + if (hasUsage) { + addBounded(model, "usageObservedRequests", 1, MAX_COUNTER, bucket); + addBounded(model, "inputTokens", observation.inputTokens, MAX_TOKEN_TOTAL, bucket); + addBounded(model, "outputTokens", observation.outputTokens, MAX_TOKEN_TOTAL, bucket); } } @@ -563,6 +595,18 @@ export class MetricsStore { .sort((left, right) => right.requests - left.requests || compareText(left.providerId, right.providerId)); const modelRows = [...modelTotals.values()] .sort((left, right) => right.requests - left.requests || compareText(left.model, right.model)); + const providerTrackedRequests = summary.requests - Math.min( + summary.requests, + summary.providerOverflowRequests + ); + const modelGroupedRequests = Math.min( + summary.requests, + summary.unknownModelRequests + summary.modelOverflowRequests + ); + const providerProjection = projectRequestRows(providerRows, providerTrackedRequests); + const modelProjection = projectRequestRows(modelRows, summary.requests - modelGroupedRequests); + const projectedProviderRequests = providerProjection.reduce((sum, entry) => sum + entry.requests, 0); + const projectedModelRequests = modelProjection.reduce((sum, entry) => sum + entry.requests, 0); return { window, bucketMinutes: METRICS_BUCKET_MINUTES, @@ -575,26 +619,26 @@ export class MetricsStore { responseStart: latencyProjection(summary.responseStartBins) }, series, - providers: providerRows.slice(0, 16).map((row) => ({ + providers: providerProjection.map(({ row, requests }) => ({ providerId: row.providerId, - requests: row.requests, - successfulRequests: row.successfulRequests, - tokens: tokenProjection(row), + requests, + successfulRequests: Math.min(row.successfulRequests, requests), + tokens: { + ...tokenProjection(row), + observedRequests: Math.min(row.usageObservedRequests, requests) + }, latency: latencyProjection(row.durationBins) })), - providerOtherRequests: providerRows.slice(16).reduce( - (sum, row) => Math.min(MAX_COUNTER, sum + row.requests), - 0 - ), - models: modelRows.slice(0, 16).map((row) => ({ + providerOtherRequests: summary.requests - projectedProviderRequests, + models: modelProjection.map(({ row, requests }) => ({ model: row.model, - requests: row.requests, - tokens: tokenProjection(row) + requests, + tokens: { + ...tokenProjection(row), + observedRequests: Math.min(row.usageObservedRequests, requests) + } })), - modelOtherRequests: modelRows.slice(16).reduce( - (sum, row) => Math.min(MAX_COUNTER, sum + row.requests), - 0 - ), + modelOtherRequests: summary.requests - projectedModelRequests, dataQuality: { unknownModelRequests: summary.unknownModelRequests, modelOverflowRequests: summary.modelOverflowRequests, @@ -726,6 +770,7 @@ export class MetricsStore { retentionBuckets: METRICS_RETENTION_BUCKETS, buckets: this.buckets }; + validateDocument(document); const bytes = Buffer.from(`${JSON.stringify(document)}\n`, "utf8"); if (bytes.length > METRICS_MAX_FILE_BYTES) throw new Error("Metrics document is too large."); const tempPath = join(parent, `.${basename(this.path)}.${this.createId()}.tmp`); diff --git a/node/src/supervisor/provider-service.mjs b/node/src/supervisor/provider-service.mjs index e7ae5a4..af8ca7d 100644 --- a/node/src/supervisor/provider-service.mjs +++ b/node/src/supervisor/provider-service.mjs @@ -930,7 +930,9 @@ export class ProviderService { }, proxy: { overrideAuthorization: true, - requestIdHeader: "x-client-request-id" + requestIdHeader: "x-client-request-id", + modelMode: profile.modelMode, + modelOverride: profile.modelOverride }, capture: { enabled: document.settings.captureEnabled, diff --git a/node/src/worker/protocol.mjs b/node/src/worker/protocol.mjs index 83f3b8f..74108c4 100644 --- a/node/src/worker/protocol.mjs +++ b/node/src/worker/protocol.mjs @@ -40,7 +40,12 @@ const UPSTREAM_FIELDS = new Set([ "authScheme", "extraHeaders" ]); -const PROXY_FIELDS = new Set(["overrideAuthorization", "requestIdHeader"]); +const PROXY_FIELDS = new Set([ + "overrideAuthorization", + "requestIdHeader", + "modelMode", + "modelOverride" +]); const CAPTURE_FIELDS = new Set(["enabled", "dbPath"]); const WORKER_PHASES = new Set(["ready", "running", "draining", "drained", "stopping", "failed"]); const REQUEST_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/; @@ -145,6 +150,20 @@ function isValidAuthenticationHeader(upstream) { } } +function isValidModelPolicy(proxy) { + if (proxy.modelMode !== "passthrough" && proxy.modelMode !== "override") { + return false; + } + // Schema 2 historically allowed internal controls, so existing snapshots must remain startable. + if (proxy.modelOverride !== null + && (typeof proxy.modelOverride !== "string" + || proxy.modelOverride.length === 0 + || proxy.modelOverride.trim() !== proxy.modelOverride)) { + return false; + } + return proxy.modelMode !== "override" || proxy.modelOverride !== null; +} + function isValidBaseUrl(value) { if (!isNonEmptyString(value)) { return false; @@ -213,6 +232,7 @@ function validateRuntimeSettings(settings) { || !hasExactFields(settings.proxy, PROXY_FIELDS) || typeof settings.proxy.overrideAuthorization !== "boolean" || !isValidHeaderName(settings.proxy.requestIdHeader) + || !isValidModelPolicy(settings.proxy) || (settings.proxy.overrideAuthorization && !isValidAuthenticationHeader(settings.upstream)) || (settings.proxy.overrideAuthorization && settings.upstream.apiKey.length === 0) diff --git a/node/test/capture-store.test.mjs b/node/test/capture-store.test.mjs index e6de39d..922b00c 100644 --- a/node/test/capture-store.test.mjs +++ b/node/test/capture-store.test.mjs @@ -59,6 +59,73 @@ test("encodeBody preserves utf8 and base64 encodes binary", () => { const binary = encodeBody(Buffer.from([0xff, 0x00, 0x10])); assert.equal(binary.encoding, "base64"); assert.equal(binary.bytes, 3); + + const truncated = encodeBody(Buffer.from("prefix", "utf8"), { + totalBytes: 4096, + truncated: true + }); + assert.deepEqual(truncated, { + body: "prefix", + encoding: "utf8-truncated", + bytes: 4096 + }); + assert.throws( + () => encodeBody(Buffer.from("too-long"), { totalBytes: 2, truncated: true }), + /byte count is invalid/ + ); + assert.throws( + () => encodeBody(Buffer.from("prefix"), { totalBytes: 4096, truncated: false }), + /truncation marker is inconsistent/ + ); +}); + +test("capture manager persists truncated prefixes with total observed byte counts", (t) => { + const dir = makeTempDir("crp-capture-truncated"); + mkdirSync(dir, { recursive: true }); + const runtimeConfigPath = join(dir, "proxy-config.json"); + const dbPath = join(dir, "traffic.sqlite3"); + writeFileSync(runtimeConfigPath, JSON.stringify({ capture: { enabled: true, dbPath } })); + const manager = new CaptureManager({ + configPath: runtimeConfigPath, + capture: { enabled: true, dbPath }, + watchRuntimeConfig: false + }).start(); + t.after(() => { + manager.close(); + rmSync(dir, { recursive: true, force: true }); + }); + manager.beginRecord().save({ + startedAt: "2026-05-19T00:00:00.000Z", + completedAt: "2026-05-19T00:00:01.000Z", + durationMs: 1000, + requestId: "req-truncated", + sessionId: null, + threadId: null, + method: "POST", + incomingUrl: "http://127.0.0.1:15100/responses", + targetUrl: "https://example.test/responses", + requestHeaders: {}, + requestBody: Buffer.from("request-prefix"), + requestBodyBytes: 2_000_000, + requestBodyTruncated: true, + responseStatus: 200, + responseHeaders: {}, + responseBody: Buffer.from([0xff, 0x00]), + responseBodyBytes: 3_000_000, + responseBodyTruncated: true, + isStream: false, + upstreamRequestId: null + }); + + const db = new DatabaseSync(dbPath); + const row = db.prepare("SELECT * FROM http_transactions WHERE request_id = ?").get("req-truncated"); + db.close(); + assert.equal(row.request_body, "request-prefix"); + assert.equal(row.request_body_encoding, "utf8-truncated"); + assert.equal(row.request_body_bytes, 2_000_000); + assert.equal(row.response_body, Buffer.from([0xff, 0x00]).toString("base64")); + assert.equal(row.response_body_encoding, "base64-truncated"); + assert.equal(row.response_body_bytes, 3_000_000); }); test("capture manager writes a complete request/response record", async () => { diff --git a/node/test/crp.test.mjs b/node/test/crp.test.mjs index 890ad62..8b142f0 100644 --- a/node/test/crp.test.mjs +++ b/node/test/crp.test.mjs @@ -11,6 +11,7 @@ import { readFileSync, rmSync, statSync, + symlinkSync, writeFileSync } from "node:fs"; import os from "node:os"; @@ -335,6 +336,57 @@ test("imports the CLI module without executing a command", () => { assert.equal(result.stdout, "imported\n"); }); +test("executes help through an npm-style POSIX bin symlink without making imports executable", { + skip: process.platform === "win32" +}, (t) => { + const tempRoot = mkdtempSync(join(os.tmpdir(), "crp-cli-bin-link-")); + t.after(() => rmSync(tempRoot, { recursive: true, force: true })); + + const binDir = join(tempRoot, "bin"); + const packageScopeDir = join(tempRoot, "lib", "node_modules", "@cluic"); + const packageInstallDir = join(packageScopeDir, "codex-remote-proxy"); + mkdirSync(binDir, { recursive: true }); + mkdirSync(packageScopeDir, { recursive: true }); + symlinkSync(PACKAGE_ROOT, packageInstallDir, "dir"); + + const cliLink = join(binDir, "crp"); + const homeDir = join(tempRoot, "home"); + symlinkSync("../lib/node_modules/@cluic/codex-remote-proxy/bin/crp.mjs", cliLink); + const environment = { + ...process.env, + CRP_LOCALE: "en", + HOME: homeDir, + USERPROFILE: homeDir, + LANG: "C", + LC_ALL: "C" + }; + + const executed = spawnSync(process.execPath, [cliLink, "--locale", "en", "--help"], { + cwd: tempRoot, + env: environment, + encoding: "utf8" + }); + assert.equal(executed.status, 0, executed.stderr); + assert.equal(executed.stderr, ""); + assert.match(executed.stdout, /^Usage:$/m); + assert.match(executed.stdout, /^ crp \[options\]$/m); + + const cliLinkUrl = pathToFileURL(cliLink).href; + const missingEntryPath = join(tempRoot, "missing-entry.mjs"); + const imported = spawnSync(process.execPath, [ + "--input-type=module", + "--eval", + `process.argv[1] = ${JSON.stringify(missingEntryPath)}; await import(${JSON.stringify(cliLinkUrl)}); process.stdout.write("imported\\n");` + ], { + cwd: tempRoot, + env: environment, + encoding: "utf8" + }); + assert.equal(imported.status, 0, imported.stderr); + assert.equal(imported.stderr, ""); + assert.equal(imported.stdout, "imported\n"); +}); + test("prints mature English-default CLI help without discovery", async () => { let discovered = false; const result = await invokeCli(["--help"], { diff --git a/node/test/e2e/crp-ui-fixture.mjs b/node/test/e2e/crp-ui-fixture.mjs index 9217631..b094250 100644 --- a/node/test/e2e/crp-ui-fixture.mjs +++ b/node/test/e2e/crp-ui-fixture.mjs @@ -14,8 +14,24 @@ const REPO_UI_ROOT = resolve(import.meta.dirname, "../../ui"); const STARTED_AT = "2026-07-13T08:00:00.000Z"; const MODEL_CATALOG_FETCHED_AT = "2026-07-13T08:15:00.000Z"; const MODEL_CATALOG_EXPIRES_AT = "2026-07-14T08:15:00.000Z"; +const MAX_METRIC_COUNTER = 1_000_000_000_000; +const MAX_METRIC_TOKENS = 9_000_000_000_000_000; +const METRIC_TEXT_CONTROL_PATTERN = /[\u0000-\u001f\u007f-\u009f]/; +const METRIC_LATENCY_BOUNDS = new Set([ + 50, 100, 250, 500, 1_000, 2_500, 5_000, 10_000, 30_000, 60_000, 120_000, 300_000 +]); +const METRIC_RESULTS = [ + "success", + "upstreamRejected", + "upstreamError", + "timeout", + "networkError", + "clientAbort" +]; function emptyMetrics(window = "24h") { + const bucketCount = window === "24h" ? 24 : 168; + const currentStart = Date.parse(STARTED_AT); return { window, bucketMinutes: 60, @@ -34,7 +50,19 @@ function emptyMetrics(window = "24h") { latency: { p50UpperBoundMs: null, p95UpperBoundMs: null, overflowRequests: 0 }, responseStart: { p50UpperBoundMs: null, p95UpperBoundMs: null, overflowRequests: 0 } }, - series: [], + series: Array.from({ length: bucketCount }, (_, index) => ({ + start: new Date(currentStart - ((bucketCount - 1 - index) * 60 * 60 * 1_000)).toISOString(), + requests: 0, + results: { + success: 0, + upstreamRejected: 0, + upstreamError: 0, + timeout: 0, + networkError: 0, + clientAbort: 0 + }, + tokens: { input: 0, output: 0, observedRequests: 0 } + })), providers: [], providerOtherRequests: 0, models: [], @@ -48,6 +76,204 @@ function emptyMetrics(window = "24h") { }; } +function distributeTotal(total, count) { + const base = Math.floor(total / count); + const remainder = total % count; + return Array.from({ length: count }, (_, index) => base + (index < remainder ? 1 : 0)); +} + +function distributeCapped(total, capacities) { + const distributed = Array(capacities.length).fill(0); + let remaining = total; + for (let index = 0; index < capacities.length && remaining > 0; index += 1) { + const fairShare = Math.ceil(remaining / (capacities.length - index)); + const amount = Math.min(capacities[index], fairShare); + distributed[index] = amount; + remaining -= amount; + } + assert.equal(remaining, 0); + return distributed; +} + +function metricSeries(window, summary) { + const bucketCount = window === "24h" ? 24 : 168; + const requests = distributeTotal(summary.requests, bucketCount); + const remaining = [...requests]; + const results = Array.from({ length: bucketCount }, () => ({ + success: 0, + upstreamRejected: 0, + upstreamError: 0, + timeout: 0, + networkError: 0, + clientAbort: 0 + })); + for (const result of ["upstreamRejected", "upstreamError", "timeout", "networkError", "clientAbort"]) { + const distributed = distributeCapped(summary.results[result], remaining); + for (let index = 0; index < bucketCount; index += 1) { + results[index][result] = distributed[index]; + remaining[index] -= distributed[index]; + } + } + assert.equal(remaining.reduce((sum, value) => sum + value, 0), summary.results.success); + const observed = distributeCapped(summary.tokens.observedRequests, requests); + const input = distributeTotal(summary.tokens.input, bucketCount); + const output = distributeTotal(summary.tokens.output, bucketCount); + const currentStart = Date.parse(STARTED_AT); + return Array.from({ length: bucketCount }, (_, index) => ({ + start: new Date(currentStart - ((bucketCount - 1 - index) * 60 * 60 * 1_000)).toISOString(), + requests: requests[index], + results: { ...results[index], success: remaining[index] }, + tokens: { + input: input[index], + output: output[index], + observedRequests: observed[index] + } + })); +} + +function assertUnsaturatedMetricsFixture(metrics, expectedWindow = metrics.window) { + assert.ok(expectedWindow === "24h" || expectedWindow === "7d", "metrics window must be supported"); + assert.equal(metrics.window, expectedWindow, "metrics payload window must match the requested window"); + assert.equal(metrics.bucketMinutes, 60, "metrics buckets must remain hourly"); + assert.ok(["ready", "degraded", "unavailable"].includes(metrics.storageState), + "metrics storage state must use the public enum"); + + const bucketCount = expectedWindow === "24h" ? 24 : 168; + assert.equal(metrics.series.length, bucketCount, `${expectedWindow} metrics must contain ${bucketCount} buckets`); + + const assertCounter = (value, label, maximum = MAX_METRIC_COUNTER) => { + assert.ok(Number.isSafeInteger(value) && value >= 0 && value <= maximum, + `${label} must be a bounded non-negative safe integer`); + }; + const assertBoundedText = (value, maximum, label) => { + assert.ok(typeof value === "string" + && value.length > 0 + && value.length <= maximum * 2 + && [...value].length <= maximum + && value.trim() === value + && !METRIC_TEXT_CONTROL_PATTERN.test(value), `${label} must be bounded safe text`); + }; + const assertLatency = (value, label) => { + assert.deepEqual(Object.keys(value).sort(), ["overflowRequests", "p50UpperBoundMs", "p95UpperBoundMs"], + `${label} must use the exact public latency fields`); + assert.ok(value.p50UpperBoundMs === null || METRIC_LATENCY_BOUNDS.has(value.p50UpperBoundMs), + `${label}.p50UpperBoundMs must use a fixed histogram boundary`); + assert.ok(value.p95UpperBoundMs === null || METRIC_LATENCY_BOUNDS.has(value.p95UpperBoundMs), + `${label}.p95UpperBoundMs must use a fixed histogram boundary`); + assertCounter(value.overflowRequests, `${label}.overflowRequests`); + }; + const resultTotal = (results, label) => METRIC_RESULTS.reduce((total, result) => { + assertCounter(results[result], `${label}.${result}`); + return total + results[result]; + }, 0); + const seriesTotals = { + requests: 0, + results: Object.fromEntries(METRIC_RESULTS.map((result) => [result, 0])), + input: 0, + output: 0, + observedRequests: 0 + }; + const currentStart = Date.parse(STARTED_AT); + + metrics.series.forEach((bucket, index) => { + const expectedStart = new Date(currentStart - ((bucketCount - 1 - index) * 60 * 60 * 1_000)).toISOString(); + assert.equal(bucket.start, expectedStart, `metrics bucket ${index} must use the fixed UTC fixture timeline`); + assertCounter(bucket.requests, `series[${index}].requests`); + assert.equal(resultTotal(bucket.results, `series[${index}].results`), bucket.requests, + `series[${index}] result counts must conserve requests`); + assertCounter(bucket.tokens.input, `series[${index}].tokens.input`, MAX_METRIC_TOKENS); + assertCounter(bucket.tokens.output, `series[${index}].tokens.output`, MAX_METRIC_TOKENS); + assertCounter(bucket.tokens.observedRequests, `series[${index}].tokens.observedRequests`); + assert.ok(bucket.tokens.observedRequests <= bucket.requests, + `series[${index}] observed token requests cannot exceed requests`); + + seriesTotals.requests += bucket.requests; + seriesTotals.input += bucket.tokens.input; + seriesTotals.output += bucket.tokens.output; + seriesTotals.observedRequests += bucket.tokens.observedRequests; + for (const result of METRIC_RESULTS) seriesTotals.results[result] += bucket.results[result]; + }); + + assertCounter(metrics.summary.requests, "summary.requests"); + assert.ok(metrics.summary.requests < MAX_METRIC_COUNTER, + "conserving E2E fixtures must stay below the aggregate counter saturation boundary"); + assert.equal(resultTotal(metrics.summary.results, "summary.results"), metrics.summary.requests, + "summary result counts must conserve requests"); + assertCounter(metrics.summary.tokens.input, "summary.tokens.input", MAX_METRIC_TOKENS); + assertCounter(metrics.summary.tokens.output, "summary.tokens.output", MAX_METRIC_TOKENS); + assert.ok(metrics.summary.tokens.input < MAX_METRIC_TOKENS + && metrics.summary.tokens.output < MAX_METRIC_TOKENS, + "conserving E2E fixtures must stay below the aggregate token saturation boundary"); + assertCounter(metrics.summary.tokens.observedRequests, "summary.tokens.observedRequests"); + assert.ok(metrics.summary.tokens.observedRequests <= metrics.summary.requests, + "summary observed token requests cannot exceed requests"); + assert.equal(seriesTotals.requests, metrics.summary.requests, "series requests must match the summary"); + assert.equal(seriesTotals.input, metrics.summary.tokens.input, "series input tokens must match the summary"); + assert.equal(seriesTotals.output, metrics.summary.tokens.output, "series output tokens must match the summary"); + assert.equal(seriesTotals.observedRequests, metrics.summary.tokens.observedRequests, + "series observed token requests must match the summary"); + for (const result of METRIC_RESULTS) { + assert.equal(seriesTotals.results[result], metrics.summary.results[result], + `series ${result} results must match the summary`); + } + assertLatency(metrics.summary.latency, "summary.latency"); + assertLatency(metrics.summary.responseStart, "summary.responseStart"); + + const qualityFields = [ + "unknownModelRequests", + "modelOverflowRequests", + "providerOverflowRequests", + "droppedObservations" + ]; + assert.deepEqual(Object.keys(metrics.dataQuality).sort(), [...qualityFields].sort(), + "data quality must use the exact public counters"); + for (const field of qualityFields) assertCounter(metrics.dataQuality[field], `dataQuality.${field}`); + + assert.ok(metrics.providers.length <= 16, "public metrics may contain at most 16 Provider rows"); + assert.equal(new Set(metrics.providers.map((provider) => provider.providerId)).size, + metrics.providers.length, "Provider rows must be unique"); + for (const [index, provider] of metrics.providers.entries()) { + assertBoundedText(provider.providerId, 128, `providers[${index}].providerId`); + assertCounter(provider.requests, `providers[${index}].requests`); + assertCounter(provider.successfulRequests, `providers[${index}].successfulRequests`); + assert.ok(provider.successfulRequests <= provider.requests, + `providers[${index}] successful requests cannot exceed requests`); + assertCounter(provider.tokens.input, `providers[${index}].tokens.input`, MAX_METRIC_TOKENS); + assertCounter(provider.tokens.output, `providers[${index}].tokens.output`, MAX_METRIC_TOKENS); + assertCounter(provider.tokens.observedRequests, `providers[${index}].tokens.observedRequests`); + assert.ok(provider.tokens.observedRequests <= provider.requests, + `providers[${index}] observed token requests cannot exceed requests`); + assertLatency(provider.latency, `providers[${index}].latency`); + } + assertCounter(metrics.providerOtherRequests, "providerOtherRequests"); + assert.equal(metrics.providers.reduce((total, provider) => total + provider.requests, 0) + + metrics.providerOtherRequests, metrics.summary.requests, + "provider distribution must conserve requests"); + assert.ok(metrics.dataQuality.providerOverflowRequests <= metrics.providerOtherRequests, + "grouped Provider requests must include the Provider overflow remainder"); + + assert.ok(metrics.models.length <= 16, "public metrics may contain at most 16 model rows"); + assert.equal(new Set(metrics.models.map((model) => model.model)).size, + metrics.models.length, "model rows must be unique"); + for (const [index, model] of metrics.models.entries()) { + assertBoundedText(model.model, 256, `models[${index}].model`); + assertCounter(model.requests, `models[${index}].requests`); + assertCounter(model.tokens.input, `models[${index}].tokens.input`, MAX_METRIC_TOKENS); + assertCounter(model.tokens.output, `models[${index}].tokens.output`, MAX_METRIC_TOKENS); + assertCounter(model.tokens.observedRequests, `models[${index}].tokens.observedRequests`); + assert.ok(model.tokens.observedRequests <= model.requests, + `models[${index}] observed token requests cannot exceed requests`); + } + assertCounter(metrics.modelOtherRequests, "modelOtherRequests"); + assert.equal(metrics.models.reduce((total, model) => total + model.requests, 0) + + metrics.modelOtherRequests, metrics.summary.requests, + "model distribution must conserve requests"); + assert.ok(Math.min( + metrics.summary.requests, + metrics.dataQuality.unknownModelRequests + metrics.dataQuality.modelOverflowRequests + ) <= metrics.modelOtherRequests, "other models must include unknown and overflow requests"); +} + function publicProvider(input = {}, index = 0) { const now = `2026-07-13T08:0${index}:00.000Z`; return { @@ -470,8 +696,14 @@ function createServices({ upstream }) { metricsService: { getOverview({ window }) { calls.push({ operation: "getMetrics", window }); - const metrics = state.metricsByWindow.get(window) ?? state.metrics; - return structuredClone({ ...metrics, window }); + const configured = state.metricsByWindow.get(window); + const response = structuredClone(configured ?? state.metrics); + if (configured === undefined && response.window !== window) { + response.window = window; + response.series = metricSeries(window, response.summary); + } + assertUnsaturatedMetricsFixture(response, window); + return response; } }, requestSupervisorShutdown() { @@ -819,39 +1051,24 @@ export async function createFixtureHarness({ failAt = null, onResource = () => { latency: { p50UpperBoundMs: 1000, p95UpperBoundMs: 5000, overflowRequests: 0 } }] : []) ]; + const metricsSummary = { + requests: 128, + results: { + success: 119, + upstreamRejected: 3, + upstreamError: 2, + timeout: 1, + networkError: 1, + clientAbort: 2 + }, + tokens: { input: 842000, output: 214000, observedRequests: 96 }, + latency: { p50UpperBoundMs: 1000, p95UpperBoundMs: 5000, overflowRequests: 0 }, + responseStart: { p50UpperBoundMs: 250, p95UpperBoundMs: 1000, overflowRequests: 0 } + }; services.state.metrics = { ...emptyMetrics(), - summary: { - requests: 128, - results: { - success: 119, - upstreamRejected: 3, - upstreamError: 2, - timeout: 1, - networkError: 1, - clientAbort: 2 - }, - tokens: { input: 842000, output: 214000, observedRequests: 96 }, - latency: { p50UpperBoundMs: 1000, p95UpperBoundMs: 5000, overflowRequests: 0 }, - responseStart: { p50UpperBoundMs: 250, p95UpperBoundMs: 1000, overflowRequests: 0 } - }, - series: Array.from({ length: 24 }, (_, index) => ({ - start: new Date(Date.parse(STARTED_AT) + index * 60 * 60 * 1_000).toISOString(), - requests: 3 + (index % 7), - results: { - success: 3 + (index % 5), - upstreamRejected: index % 6 === 0 ? 1 : 0, - upstreamError: index % 8 === 0 ? 1 : 0, - timeout: 0, - networkError: 0, - clientAbort: index % 11 === 0 ? 1 : 0 - }, - tokens: { - input: 12000 + index * 900, - output: 4000 + index * 300, - observedRequests: 3 + (index % 4) - } - })), + summary: metricsSummary, + series: metricSeries("24h", metricsSummary), providers: providerMetrics, models: [ { @@ -866,13 +1083,18 @@ export async function createFixtureHarness({ failAt = null, onResource = () => { } ] }; + assertUnsaturatedMetricsFixture(services.state.metrics); services.state.metricsByWindow.clear(); }, emptyMetrics(window = "24h") { return structuredClone(emptyMetrics(window)); }, + metricSeries(window, summary) { + return structuredClone(metricSeries(window, summary)); + }, setMetrics(metrics, { window = null } = {}) { const next = structuredClone(metrics); + assertUnsaturatedMetricsFixture(next, window ?? next.window); if (window === null) { services.state.metrics = next; services.state.metricsByWindow.clear(); diff --git a/node/test/e2e/provider-switch.spec.mjs b/node/test/e2e/provider-switch.spec.mjs index 9c7f597..c917166 100644 --- a/node/test/e2e/provider-switch.spec.mjs +++ b/node/test/e2e/provider-switch.spec.mjs @@ -142,11 +142,34 @@ test("renders populated Metrics and changes the aggregate window without stale d p95UpperBoundMs: null, overflowRequests: 2 }; - const sevenDay = structuredClone(crp.state.metrics); - sevenDay.window = "7d"; - sevenDay.summary.requests = 777; - sevenDay.summary.results.success = 700; - sevenDay.summary.tokens.observedRequests = 650; + const sevenDay = crp.emptyMetrics("7d"); + sevenDay.summary = { + requests: 777, + results: { + success: 700, + upstreamRejected: 30, + upstreamError: 20, + timeout: 10, + networkError: 7, + clientAbort: 10 + }, + tokens: { input: 5_000_000, output: 1_000_000, observedRequests: 650 }, + latency: { p50UpperBoundMs: 1000, p95UpperBoundMs: 5000, overflowRequests: 0 }, + responseStart: { p50UpperBoundMs: 250, p95UpperBoundMs: 1000, overflowRequests: 0 } + }; + sevenDay.series = crp.metricSeries("7d", sevenDay.summary); + sevenDay.providers = [{ + providerId: "provider-a", + requests: 777, + successfulRequests: 700, + tokens: { input: 5_000_000, output: 1_000_000, observedRequests: 650 }, + latency: { p50UpperBoundMs: 1000, p95UpperBoundMs: 5000, overflowRequests: 0 } + }]; + sevenDay.models = [{ + model: "gpt-5.1-codex-mini", + requests: 777, + tokens: { input: 5_000_000, output: 1_000_000, observedRequests: 650 } + }]; crp.setMetrics(sevenDay, { window: "7d" }); await openCrp(page, crp); @@ -170,12 +193,61 @@ test("renders populated Metrics and changes the aggregate window without stale d await assertNoSecrets(page, crp); }); +test("discloses incomplete Metrics rates and conserves the visible model remainder", async ({ page, crp }) => { + crp.state.metrics.models = [ + ...Array.from({ length: 8 }, (_, index) => ({ + model: `visible-model-${index + 1}`, + requests: 10 - index, + tokens: { input: 0, output: 0, observedRequests: 0 } + })) + ]; + crp.state.metrics.modelOtherRequests = 76; + crp.state.metrics.providers[0].requests -= 3; + crp.state.metrics.providerOtherRequests = 3; + crp.state.metrics.dataQuality = { + unknownModelRequests: 5, + modelOverflowRequests: 4, + providerOverflowRequests: 3, + droppedObservations: 2 + }; + + await openCrp(page, crp); + const successRate = page.locator(".metric-card").filter({ hasText: "Success rate" }); + await expect(successRate.locator("strong")).toHaveText("-"); + await expect(successRate).toContainText("Unavailable because 2 metric updates were dropped"); + await expect(page.getByRole("table").filter({ hasText: "Provider Alpha" })) + .toContainText("Not available"); + + const quality = page.getByRole("complementary", { name: "Data quality" }); + await expect(quality).toContainText("These counters are independent signals and may overlap."); + await expect(quality).toContainText("Unknown-model requests5"); + await expect(quality).toContainText("Grouped model requests4"); + await expect(quality).toContainText("Grouped provider requests3"); + await expect(quality).toContainText("Dropped metric updates2"); + await expect(page.getByText("Data quality: 14")).toHaveCount(0); + + const distribution = page.locator(".distribution-chart"); + await expect(distribution.locator("text").filter({ hasText: "Other models" })).toBeVisible(); + await expect(distribution.locator("text").filter({ hasText: /^79$/ })).toBeVisible(); + await expect(page.getByText("This view contains 24 UTC hourly buckets, including the current partial hour.")) + .toBeVisible(); + await page.setViewportSize({ width: 390, height: 844 }); + await assertLayoutIntegrity(page); + await assertNoSecrets(page, crp); +}); + test("shows empty and degraded Metrics without affecting proxy readiness", async ({ page, crp }) => { crp.setMetrics(crp.emptyMetrics()); await openCrp(page, crp); await expect(page.getByTestId("metrics-empty")).toBeVisible(); await expect(page.getByRole("heading", { name: "No proxy traffic in this window" })).toBeVisible(); await expect(page.getByRole("heading", { name: "Codex is securely connected" })).toBeVisible(); + await page.getByRole("button", { name: "7 days" }).click(); + await expect(page.getByRole("button", { name: "7 days" })).toHaveAttribute("aria-pressed", "true"); + await expect(page.getByTestId("metrics-empty")).toBeVisible(); + await expect.poll(() => crp.calls.filter((call) => ( + call.operation === "getMetrics" && call.window === "7d" + )).length).toBe(1); const degraded = crp.emptyMetrics(); degraded.storageState = "degraded"; @@ -285,6 +357,35 @@ test("creates, discovers models, tests, switches, edits, and deletes a provider await assertNoSecrets(page, crp, [replacement]); }); +test("ordinary provider test selects the first provider through no-start compare-and-set", async ({ page, crp }) => { + crp.seedProviders({ + providers: [{ id: "provider-a", name: "Provider Alpha", baseUrl: crp.upstreamBaseUrl }], + activeProviderId: null + }); + crp.seedProviderModels("provider-a", { models: ["fixture-model"] }); + await openCrp(page, crp); + await navigate(page, "Providers"); + + const card = page.getByTestId("provider-card-provider-a"); + const details = await openProviderDetails(page, "Provider Alpha"); + await expect(details.getByRole("button", { name: "Test", exact: true })).toHaveCount(0); + await details.getByRole("button", { name: "Test and select", exact: true }).click(); + const dialog = page.getByRole("dialog", { name: "Test provider" }); + const modelSelect = dialog.locator("select#test-model"); + await expect(modelSelect).toBeVisible(); + await modelSelect.selectOption("fixture-model"); + await dialog.getByRole("button", { name: "Test and select" }).click(); + + await expect(card.getByText("Current", { exact: true })).toBeVisible(); + expect(crp.state.activeProviderId).toBe("provider-a"); + expect(crp.state.worker.phase).toBe("stopped"); + expect(crp.calls.filter((call) => call.operation === "testProvider")).toEqual([ + { operation: "testProvider", id: "provider-a", model: "fixture-model", activateIfNone: true } + ]); + expect(crp.calls.filter((call) => call.operation === "activate")).toEqual([]); + await assertNoSecrets(page, crp); +}); + test("duplicates a provider configuration without copying credentials or state", async ({ page, crp }) => { const duplicateSecret = crp.registerSecret(`duplicate-${randomBytes(18).toString("base64url")}`); crp.seedProviders({ diff --git a/node/test/integration/core-real-chain.test.mjs b/node/test/integration/core-real-chain.test.mjs index 28890a1..9dbe224 100644 --- a/node/test/integration/core-real-chain.test.mjs +++ b/node/test/integration/core-real-chain.test.mjs @@ -171,6 +171,7 @@ function createResponsesFixture({ label, secret, heldInput = null, holdGate = nu response.end(JSON.stringify({ id: `response-${label}-${++sequence}`, object: "response", + status: "completed", output: [], provider: label, input: body.input @@ -313,6 +314,8 @@ test("real CLI core chain switches in-flight traffic and restarts on the fixed p "--name", "Provider B", "--base-url", `http://127.0.0.1:${portB}/v1`, "--api-key", SECRET_B, + "--model-mode", "override", + "--model-override", "provider-b-model", "--json" ]); const providerAId = addedA.provider.id; @@ -394,6 +397,7 @@ test("real CLI core chain switches in-flight traffic and restarts on the fixed p body: { id: "response-B-2", object: "response", + status: "completed", output: [], provider: "B", input: "new-B" @@ -407,6 +411,7 @@ test("real CLI core chain switches in-flight traffic and restarts on the fixed p body: { id: "response-A-2", object: "response", + status: "completed", output: [], provider: "A", input: "held-A" @@ -434,7 +439,7 @@ test("real CLI core chain switches in-flight traffic and restarts on the fixed p }); assert.deepEqual(compatibilityB.body, compatibilityA.body); assert.deepEqual(proxiedA.body, { model: "client-model", input: "held-A" }); - assert.deepEqual(proxiedB.body, { model: "client-model", input: "new-B" }); + assert.deepEqual(proxiedB.body, { model: "provider-b-model", input: "new-B" }); const restarted = await invoke(["restart", "--json"]); assert.equal(restarted.supervisorPid, process.pid); @@ -450,6 +455,23 @@ test("real CLI core chain switches in-flight traffic and restarts on the fixed p assert.equal(health.generation, restarted.worker.generation); assertSecretsAbsent("proxy health", JSON.stringify(health)); + const responseAfterRestart = await postResponses("after-restart"); + assert.deepEqual(responseAfterRestart, { + status: 200, + body: { + id: "response-B-3", + object: "response", + status: "completed", + output: [], + provider: "B", + input: "after-restart" + } + }); + assert.deepEqual(observationForInput(fixtureB, "after-restart").body, { + model: "provider-b-model", + input: "after-restart" + }); + const afterRestart = await invoke(["status", "--json"]); assert.equal(afterRestart.running, true); assert.equal(afterRestart.supervisor.pid, process.pid); diff --git a/node/test/integration/worker-entry.test.mjs b/node/test/integration/worker-entry.test.mjs index ed6051e..690b554 100644 --- a/node/test/integration/worker-entry.test.mjs +++ b/node/test/integration/worker-entry.test.mjs @@ -197,7 +197,9 @@ function makeSettings({ baseUrl, configPath, port = 0, apiKey = "worker-integrat }, proxy: { overrideAuthorization: true, - requestIdHeader: "x-client-request-id" + requestIdHeader: "x-client-request-id", + modelMode: "passthrough", + modelOverride: null }, capture: { enabled: false, diff --git a/node/test/integration/worker-restart.test.mjs b/node/test/integration/worker-restart.test.mjs index e168e76..44c1e60 100644 --- a/node/test/integration/worker-restart.test.mjs +++ b/node/test/integration/worker-restart.test.mjs @@ -51,7 +51,12 @@ function makeSnapshot({ generation, port, upstreamPort, dir }) { authScheme: "Bearer", extraHeaders: {} }, - proxy: { overrideAuthorization: true, requestIdHeader: "x-client-request-id" }, + proxy: { + overrideAuthorization: true, + requestIdHeader: "x-client-request-id", + modelMode: "passthrough", + modelOverride: null + }, capture: { enabled: false, dbPath: join(dir, "traffic.sqlite3") } } }; diff --git a/node/test/metrics-store.test.mjs b/node/test/metrics-store.test.mjs index 3d6d12a..b250a33 100644 --- a/node/test/metrics-store.test.mjs +++ b/node/test/metrics-store.test.mjs @@ -18,6 +18,7 @@ import { } from "../src/supervisor/metrics-store.mjs"; const HOUR_MS = 60 * 60 * 1_000; +const MAX_COUNTER = 1_000_000_000_000; function observation(overrides = {}) { return { @@ -51,6 +52,47 @@ function harness(t, { timestamp = Date.parse("2026-07-16T12:34:56.000Z") } = {}) }; } +function saturatedBucket(start, suffix) { + const durationBins = Array.from({ length: 13 }, (_, index) => index === 0 ? MAX_COUNTER : 0); + return { + start, + requests: MAX_COUNTER, + results: { + success: MAX_COUNTER, + upstreamRejected: 0, + upstreamError: 0, + timeout: 0, + networkError: 0, + clientAbort: 0 + }, + usageObservedRequests: 0, + inputTokens: 0, + outputTokens: 0, + durationBins, + responseStartBins: Array(13).fill(0), + unknownModelRequests: 0, + modelOverflowRequests: 0, + providerOverflowRequests: 0, + droppedObservations: 0, + providers: [{ + providerId: `provider-${suffix}`, + requests: MAX_COUNTER, + successfulRequests: MAX_COUNTER, + usageObservedRequests: 0, + inputTokens: 0, + outputTokens: 0, + durationBins + }], + models: [{ + model: `model-${suffix}`, + requests: MAX_COUNTER, + usageObservedRequests: 0, + inputTokens: 0, + outputTokens: 0 + }] + }; +} + test("empty metrics return stable zero-filled 24h and 7d chart series", (t) => { const state = harness(t); for (const [window, expectedBuckets] of [["24h", 24], ["7d", 168]]) { @@ -101,6 +143,7 @@ test("metrics store aggregates hourly observations and restores the strict priva overflowRequests: 0 }); assert.equal(overview.dataQuality.unknownModelRequests, 1); + assert.equal(overview.modelOtherRequests, 1); assert.deepEqual(overview.providers.map((provider) => ({ providerId: provider.providerId, requests: provider.requests, @@ -109,6 +152,14 @@ test("metrics store aggregates hourly observations and restores the strict priva assert.deepEqual(overview.models.map((model) => ({ model: model.model, requests: model.requests })), [ { model: "gpt-5-codex", requests: 1 } ]); + assert.equal( + overview.models.reduce((sum, model) => sum + model.requests, overview.modelOtherRequests), + overview.summary.requests + ); + assert.equal( + overview.providers.reduce((sum, provider) => sum + provider.requests, overview.providerOtherRequests), + overview.summary.requests + ); assert.equal(state.store.flush(), true); if (process.platform !== "win32") { @@ -158,6 +209,14 @@ test("metrics store bounds dimensions, prunes after seven days, and rejects unsa assert.equal(overview.dataQuality.modelOverflowRequests, 1); assert.equal(overview.dataQuality.providerOverflowRequests, 2); assert.equal(overview.dataQuality.unknownModelRequests, 33); + assert.equal( + overview.models.reduce((sum, model) => sum + model.requests, overview.modelOtherRequests), + overview.summary.requests + ); + assert.equal( + overview.providers.reduce((sum, provider) => sum + provider.requests, overview.providerOtherRequests), + overview.summary.requests + ); assert.equal(state.store.flush(), true); const persisted = readFileSync(state.path, "utf8"); @@ -175,6 +234,69 @@ test("metrics store bounds dimensions, prunes after seven days, and rejects unsa assert.equal(overview.models[0].model, "model-new"); }); +test("late high-volume dimensions replace bounded low-volume groups without losing requests", (t) => { + const state = harness(t); + for (let index = 0; index < 64; index += 1) { + assert.equal(state.store.record(observation({ model: `early-model-${index}` })), true); + } + for (let index = 0; index < 40; index += 1) { + assert.equal(state.store.record(observation({ model: "late-heavy-model" })), true); + } + for (let index = 0; index < 32; index += 1) { + assert.equal(state.store.record(observation({ + providerId: `early-provider-${index}`, + model: "late-heavy-model" + })), true); + } + for (let index = 0; index < 30; index += 1) { + assert.equal(state.store.record(observation({ + providerId: "late-heavy-provider", + model: "late-heavy-model" + })), true); + } + + const overview = state.store.getOverview(); + assert.equal(overview.models[0].model, "late-heavy-model"); + assert.equal(overview.models[0].requests, 102); + assert.equal(overview.providers.some((provider) => ( + provider.providerId === "late-heavy-provider" && provider.requests === 30 + )), true); + assert.ok(overview.dataQuality.modelOverflowRequests > 0); + assert.ok(overview.dataQuality.providerOverflowRequests > 0); + assert.equal( + overview.models.reduce((sum, model) => sum + model.requests, overview.modelOtherRequests), + overview.summary.requests + ); + assert.equal( + overview.providers.reduce((sum, provider) => sum + provider.requests, overview.providerOtherRequests), + overview.summary.requests + ); +}); + +test("hourly windows include the current UTC bucket and exclude the exact outer boundary", (t) => { + const day = harness(t, { timestamp: Date.parse("2026-07-01T12:00:00.000Z") }); + assert.equal(day.store.record(observation()), true); + day.advance((24 * HOUR_MS) - 1); + let overview = day.store.getOverview({ window: "24h" }); + assert.equal(overview.summary.requests, 1); + assert.equal(overview.series.length, 24); + assert.equal(overview.series[0].start, "2026-07-01T12:00:00.000Z"); + day.advance(1); + overview = day.store.getOverview({ window: "24h" }); + assert.equal(overview.summary.requests, 0); + assert.equal(overview.series[0].start, "2026-07-01T13:00:00.000Z"); + + const week = harness(t, { timestamp: Date.parse("2026-07-01T12:00:00.000Z") }); + assert.equal(week.store.record(observation()), true); + week.advance((168 * HOUR_MS) - 1); + overview = week.store.getOverview({ window: "7d" }); + assert.equal(overview.summary.requests, 1); + assert.equal(overview.series.length, 168); + week.advance(1); + overview = week.store.getOverview({ window: "7d" }); + assert.equal(overview.summary.requests, 0); +}); + test("maximum valid seven-day dimensions fit the metrics storage limit", (t) => { const state = harness(t, { timestamp: Date.parse("2026-07-09T00:05:00.000Z") }); const providerSuffix = "供".repeat(114); @@ -216,6 +338,86 @@ test("maximum valid seven-day dimensions fit the metrics storage limit", (t) => assert.equal(overview.modelOtherRequests, 168 * 48); }); +test("public distributions conserve a summary capped across valid maximum buckets", (t) => { + const dir = mkdtempSync(join(os.tmpdir(), "crp-metrics-capped-")); + const path = join(dir, "metrics.json"); + writeFileSync(path, `${JSON.stringify({ + schemaVersion: 1, + bucketMinutes: 60, + retentionBuckets: 168, + buckets: [ + saturatedBucket("2026-07-16T11:00:00.000Z", "a"), + saturatedBucket("2026-07-16T12:00:00.000Z", "b") + ] + })}\n`, { mode: 0o600 }); + const store = new MetricsStore({ + path, + now: () => Date.parse("2026-07-16T12:30:00.000Z"), + flushDelayMs: 60_000 + }); + t.after(() => { + store.close(); + rmSync(dir, { recursive: true, force: true }); + }); + + const overview = store.getOverview({ window: "24h" }); + assert.equal(overview.summary.requests, MAX_COUNTER); + assert.ok(overview.dataQuality.droppedObservations > 0); + assert.equal( + overview.providers.reduce((sum, provider) => sum + provider.requests, overview.providerOtherRequests), + overview.summary.requests + ); + assert.equal( + overview.models.reduce((sum, model) => sum + model.requests, overview.modelOtherRequests), + overview.summary.requests + ); + assert.equal(overview.providers.every((provider) => ( + provider.successfulRequests <= provider.requests + && provider.tokens.observedRequests <= provider.requests + )), true); + assert.equal(overview.models.every((model) => model.tokens.observedRequests <= model.requests), true); +}); + +test("recording after a saturated bucket preserves its persisted partition invariants", (t) => { + const dir = mkdtempSync(join(os.tmpdir(), "crp-metrics-saturated-record-")); + const path = join(dir, "metrics.json"); + writeFileSync(path, `${JSON.stringify({ + schemaVersion: 1, + bucketMinutes: 60, + retentionBuckets: 168, + buckets: [saturatedBucket("2026-07-16T12:00:00.000Z", "stable")] + })}\n`, { mode: 0o600 }); + const now = () => Date.parse("2026-07-16T12:30:00.000Z"); + const store = new MetricsStore({ path, now, flushDelayMs: 60_000 }); + t.after(() => { + store.close(); + rmSync(dir, { recursive: true, force: true }); + }); + + assert.equal(store.record(observation({ + providerId: "provider-new", + model: "model-new", + result: "upstreamError", + durationBin: 4 + })), true); + let overview = store.getOverview({ window: "24h" }); + assert.equal(overview.summary.requests, MAX_COUNTER); + assert.equal(overview.summary.results.success, MAX_COUNTER); + assert.equal(overview.summary.results.upstreamError, 0); + assert.equal(overview.dataQuality.droppedObservations, 1); + assert.equal(store.flush(), true); + store.close(); + + const restored = new MetricsStore({ path, now, flushDelayMs: 60_000 }); + t.after(() => restored.close()); + overview = restored.getOverview({ window: "24h" }); + assert.equal(overview.storageState, "ready"); + assert.equal(overview.summary.requests, MAX_COUNTER); + assert.equal(overview.summary.results.success, MAX_COUNTER); + assert.equal(overview.summary.results.upstreamError, 0); + assert.equal(overview.dataQuality.droppedObservations, 1); +}); + test("invalid canonical storage is unavailable and is never overwritten", (t) => { const dir = mkdtempSync(join(os.tmpdir(), "crp-metrics-invalid-")); const path = join(dir, "metrics.json"); diff --git a/node/test/provider-registry.test.mjs b/node/test/provider-registry.test.mjs index a8efddc..26f858a 100644 --- a/node/test/provider-registry.test.mjs +++ b/node/test/provider-registry.test.mjs @@ -395,6 +395,13 @@ test("validates passthrough and override model modes", () => { })), assertCrpError("PROVIDER_INPUT_INVALID", 400) ); + assert.throws( + () => validateProviderInput(validInput({ + modelMode: "override", + modelOverride: "model\ninvalid" + })), + assertCrpError("PROVIDER_INPUT_INVALID", 400) + ); assert.deepEqual(TEST_STATUSES, new Set(["untested", "passed", "failed"])); assert.deepEqual( @@ -424,6 +431,29 @@ test("validates passthrough and override model modes", () => { ); }); +test("loads a legacy controlled model override and allows replacing it safely", (t) => { + const { registryPath } = makeTempRegistry(t); + const legacy = normalizeProvider(validInput({ + modelMode: "override", + modelOverride: "legacy-model" + }), { id: "provider-legacy", now: FIXED_NOW }); + legacy.modelOverride = "legacy\tmodel"; + writeFileSync(registryPath, `${JSON.stringify({ + schemaVersion: 2, + activeProviderId: null, + providers: [legacy], + settings: DEFAULT_SETTINGS + })}\n`, { mode: 0o600 }); + + const registry = new ProviderRegistry({ + path: registryPath, + now: () => LATER_NOW + }); + assert.equal(registry.get("provider-legacy").modelOverride, "legacy\tmodel"); + const repaired = registry.update("provider-legacy", { modelOverride: "safe-model" }); + assert.equal(repaired.modelOverride, "safe-model"); +}); + test("marks passed and failed tests and manages active deletion", (t) => { const { registryPath } = makeTempRegistry(t); const registry = new ProviderRegistry({ diff --git a/node/test/provider-service.test.mjs b/node/test/provider-service.test.mjs index fd9cd79..a26b8d2 100644 --- a/node/test/provider-service.test.mjs +++ b/node/test/provider-service.test.mjs @@ -1,6 +1,6 @@ import test from "node:test"; import assert from "node:assert/strict"; -import { mkdtempSync, rmSync } from "node:fs"; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { createServer } from "node:http"; import os from "node:os"; import { join } from "node:path"; @@ -909,6 +909,34 @@ test("successful opt-in tests select the first provider without starting the Wor assert.equal(workerManager.getPublicState().generation, before.generation); }); +test("legacy controlled model overrides remain startable after initial selection", async (t) => { + const { root, service, registry, workerManager } = makeHarness(t, { + fetchImpl: async () => compatibleResponse() + }); + const provider = await service.createProvider({ + ...providerInput(), + modelMode: "override", + modelOverride: "legacy-model" + }, makeSecret("legacy-model")); + const registryPath = join(root, "providers.json"); + const document = JSON.parse(readFileSync(registryPath, "utf8")); + document.providers[0].modelOverride = "legacy\tmodel"; + writeFileSync(registryPath, `${JSON.stringify(document)}\n`, { mode: 0o600 }); + + const tested = await service.testProvider(provider.id, "model-test", { activateIfNone: true }); + assert.equal(tested.ok, true); + assert.equal(registry.getDocument().activeProviderId, provider.id); + assert.equal(workerManager.getPublicState().phase, "stopped"); + + await service.startProxy(); + assert.equal(workerManager.calls.at(-1)[0], "start"); + assert.equal(workerManager.calls.at(-1)[1].settings.proxy.modelOverride, "legacy\tmodel"); + + await service.restartProxy(); + assert.equal(workerManager.calls.at(-1)[0], "restart"); + assert.equal(workerManager.calls.at(-1)[1].settings.proxy.modelOverride, "legacy\tmodel"); +}); + test("provider test reports committed degradation when Activity fails after markTest", async (t) => { let fetchCalls = 0; const { service, registry, activity, workerManager } = makeHarness(t, { @@ -1105,7 +1133,11 @@ test("activate persists then confirms increasing generations and rolls back fail const { service, registry, credentials, workerManager } = harness; const providerA = await service.createProvider(providerInput("A"), secretA); const providerB = await service.createProvider( - providerInput("B", "https://b.example/v1"), + { + ...providerInput("B", "https://b.example/v1"), + modelMode: "override", + modelOverride: "model-b" + }, secretB ); @@ -1122,6 +1154,12 @@ test("activate persists then confirms increasing generations and rolls back fail assert.equal(workerManager.calls[0][1].providerId, providerA.id); assert.equal(workerManager.calls[0][1].generation, 1); assert.equal(workerManager.calls[0][1].settings.upstream.apiKey, secretA); + assert.deepEqual(workerManager.calls[0][1].settings.proxy, { + overrideAuthorization: true, + requestIdHeader: "x-client-request-id", + modelMode: "passthrough", + modelOverride: null + }); assert.deepEqual(healthCalls, [[1, 1]]); registry.markTest(providerB.id, { status: "passed" }); @@ -1155,6 +1193,8 @@ test("activate persists then confirms increasing generations and rolls back fail assert.equal(workerManager.calls.at(-1)[0], "applySnapshot"); assert.equal(workerManager.calls.at(-1)[1].providerId, providerB.id); assert.equal(workerManager.calls.at(-1)[1].settings.upstream.apiKey, secretB); + assert.equal(workerManager.calls.at(-1)[1].settings.proxy.modelMode, "override"); + assert.equal(workerManager.calls.at(-1)[1].settings.proxy.modelOverride, "model-b"); const credentialGets = credentials.operations .filter(([operation]) => operation === "get") .map(([, ref]) => ref); diff --git a/node/test/runtime-settings.test.mjs b/node/test/runtime-settings.test.mjs index ea6c9e1..0a41913 100644 --- a/node/test/runtime-settings.test.mjs +++ b/node/test/runtime-settings.test.mjs @@ -24,7 +24,9 @@ function makeSettings(label = "a") { }, proxy: { overrideAuthorization: true, - requestIdHeader: "x-client-request-id" + requestIdHeader: "x-client-request-id", + modelMode: "passthrough", + modelOverride: null }, capture: { enabled: false, diff --git a/node/test/server.test.mjs b/node/test/server.test.mjs index cefe38d..a121ff7 100644 --- a/node/test/server.test.mjs +++ b/node/test/server.test.mjs @@ -2,6 +2,7 @@ import test from "node:test"; import assert from "node:assert/strict"; import http from "node:http"; import https from "node:https"; +import zlib from "node:zlib"; import { Readable } from "node:stream"; import { mkdirSync, rmSync, writeFileSync } from "node:fs"; import os from "node:os"; @@ -9,7 +10,7 @@ import { join } from "node:path"; import { EventEmitter, once } from "node:events"; import { DatabaseSync } from "node:sqlite"; -import { buildTargetUrl, createApp, createServer, isDirectExecution } from "../src/server.mjs"; +import { buildTargetUrl, createApp, createServer, isDirectExecution, loadConfig } from "../src/server.mjs"; import { RuntimeSettingsSource } from "../src/worker/runtime-settings.mjs"; function makeTempDir(prefix) { @@ -51,6 +52,14 @@ function createSignal() { return { promise, resolve }; } +function withDeadline(promise, message, timeoutMs = 2000) { + let timer; + const timeout = new Promise((_, rejectPromise) => { + timer = setTimeout(() => rejectPromise(new Error(message)), timeoutMs); + }); + return Promise.race([promise, timeout]).finally(() => clearTimeout(timer)); +} + function makeSettings({ baseUrl, apiKey = "test-api-key", @@ -60,6 +69,8 @@ function makeSettings({ timeoutMs = 300000, verifySsl = true, requestIdHeader = "x-client-request-id", + modelMode = "passthrough", + modelOverride = null, logLevel = "info", captureEnabled = true }) { @@ -81,7 +92,9 @@ function makeSettings({ }, proxy: { overrideAuthorization: true, - requestIdHeader + requestIdHeader, + modelMode, + modelOverride }, capture: { enabled: captureEnabled, @@ -116,6 +129,22 @@ function createMemoryCaptureManager(publicState = {}) { }; } +function createInactiveCaptureManager(publicState = {}) { + return { + beginRecord() { + return null; + }, + getPublicState() { + return { + captureConfigured: false, + captureActive: false, + ...publicState + }; + }, + close() {} + }; +} + async function fetchJson(url, options) { const response = await fetch(url, options); return { @@ -137,6 +166,16 @@ function requestJson(url, body) { }); } +function completedResponse(extra = {}) { + return { + id: "resp_test", + object: "response", + status: "completed", + output: [], + ...extra + }; +} + test("buildTargetUrl joins base and request paths with one separator", () => { const cases = [ { @@ -283,6 +322,82 @@ test("server writes proxied request and response to sqlite", async () => { rmSync(dir, { recursive: true, force: true }); }); +test("large Capture bodies preserve totals while omitting prefixes that cannot be fully screened", async (t) => { + const dir = makeTempDir("crp-server-large-capture"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + const responseBody = Buffer.from(JSON.stringify(completedResponse({ + padding: "r".repeat(2 * 1024 * 1024), + usage: { input_tokens: 71, output_tokens: 29 } + }))); + let requestBytes = 0; + const upstream = http.createServer((req, res) => { + req.on("data", (chunk) => { + requestBytes += chunk.length; + }); + req.on("end", () => { + res.setHeader("content-type", "application/json"); + res.setHeader("content-length", String(responseBody.length)); + res.end(responseBody); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const dbPath = join(dir, "traffic.sqlite3"); + const settings = makeSettings({ baseUrl: `http://127.0.0.1:${upstreamPort}` }); + settings.configPath = join(dir, "proxy-config.json"); + settings.capture.dbPath = dbPath; + writeFileSync(settings.configPath, JSON.stringify({ capture: settings.capture })); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 19, settings }); + const metrics = []; + const metricRecorded = createSignal(); + const app = createApp(settings, { + settingsSource: source, + recordMetric(observation) { + metrics.push(structuredClone(observation)); + metricRecorded.resolve(); + } + }); + const proxyPort = await listen(app.server); + t.after(() => closeServer(app.server)); + const requestBody = Buffer.from(JSON.stringify({ + padding: "q".repeat(1536 * 1024), + model: "large-capture-model" + })); + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: requestBody + }); + assert.equal(response.status, 200); + assert.equal((await response.arrayBuffer()).byteLength, responseBody.length); + await withDeadline(metricRecorded.promise, "large Capture request did not finish settlement"); + await closeServer(app.server); + + const db = new DatabaseSync(dbPath); + const row = db.prepare("SELECT * FROM http_transactions").get(); + db.close(); + assert.equal(requestBytes, requestBody.length); + assert.equal(row.request_body_encoding, "empty-truncated"); + assert.equal(row.request_body_bytes, requestBody.length); + assert.equal(Buffer.byteLength(row.request_body), 0); + assert.equal(row.response_body_encoding, "empty-truncated"); + assert.equal(row.response_body_bytes, responseBody.length); + assert.equal(Buffer.byteLength(row.response_body), 0); + assert.deepEqual(metrics.map(({ result, model, inputTokens, outputTokens }) => ({ + result, + model, + inputTokens, + outputTokens + })), [{ + result: "success", + model: "large-capture-model", + inputTokens: 71, + outputTokens: 29 + }]); +}); + test("isDirectExecution handles both POSIX and Windows paths", () => { assert.equal( isDirectExecution("file:///Users/example/project/node/src/server.mjs", "/Users/example/project/node/src/server.mjs"), @@ -302,6 +417,427 @@ test("isDirectExecution handles both POSIX and Windows paths", () => { ); }); +test("standalone config defaults model passthrough and rejects unsafe overrides", (t) => { + const dir = makeTempDir("crp-server-model-config"); + mkdirSync(dir, { recursive: true }); + t.after(() => rmSync(dir, { recursive: true, force: true })); + const configPath = join(dir, "proxy-config.json"); + const base = { + upstream: { + baseUrl: "https://provider.example.test/v1", + apiKey: "config-secret" + }, + proxy: {} + }; + writeFileSync(configPath, JSON.stringify(base)); + assert.deepEqual( + (({ modelMode, modelOverride }) => ({ modelMode, modelOverride }))(loadConfig(configPath).proxy), + { modelMode: "passthrough", modelOverride: null } + ); + + writeFileSync(configPath, JSON.stringify({ + ...base, + proxy: { modelMode: "override", modelOverride: "unsafe\nmodel" } + })); + assert.throws(() => loadConfig(configPath), /proxy\.modelOverride/); +}); + +test("model override preserves compressed forwarding while Capture omits opaque protected bodies", async (t) => { + const observed = createSignal(); + const upstream = http.createServer((req, res) => { + const chunks = []; + req.on("data", (chunk) => chunks.push(chunk)); + req.on("end", () => { + const body = Buffer.concat(chunks); + observed.resolve({ + contentEncoding: req.headers["content-encoding"], + contentLength: req.headers["content-length"], + body, + json: JSON.parse(zlib.gunzipSync(body).toString("utf8")) + }); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(completedResponse())); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + modelMode: "override", + modelOverride: "provider-model" + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 7, settings }); + const captureManager = createMemoryCaptureManager(); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager, + recordMetric(observation) { + metrics.push(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const original = zlib.gzipSync(Buffer.from(JSON.stringify({ + model: "client-model", + input: "keep-me", + stream: false + }))); + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { + "content-type": "application/json", + "content-encoding": "gzip", + "content-length": String(original.length) + }, + body: original + }); + assert.equal(response.status, 200); + await response.arrayBuffer(); + + const forwarded = await observed.promise; + assert.equal(forwarded.contentEncoding, "gzip"); + assert.equal(Number(forwarded.contentLength), forwarded.body.length); + assert.deepEqual(forwarded.json, { + model: "provider-model", + input: "keep-me", + stream: false + }); + assert.equal(captureManager.records.length, 1); + assert.equal(captureManager.records[0].requestBody.length, 0); + assert.equal(captureManager.records[0].requestBodyTruncated, true); + assert.equal(captureManager.records[0].requestBodyBytes, forwarded.body.length); + assert.deepEqual(metrics.map(({ result, model }) => ({ result, model })), [ + { result: "success", model: "provider-model" } + ]); +}); + +test("model override changes only top-level model lexemes and strips stale integrity headers", async (t) => { + const observed = []; + const upstream = http.createServer((req, res) => { + const chunks = []; + req.on("data", (chunk) => chunks.push(chunk)); + req.on("end", () => { + observed.push({ headers: req.headers, body: Buffer.concat(chunks).toString("utf8") }); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(completedResponse())); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + modelMode: "override", + modelOverride: "provider-model", + captureEnabled: false + }); + const proxy = createServer(settings, { + captureManager: createInactiveCaptureManager(), + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + const source = [ + "{\n", + " \"large\": 9007199254740993123456789,\n", + " \"overflow\": 1e400,\n", + " \"negativeZero\": -0,\n", + " \"duplicate\": 1, \"duplicate\": 2,\n", + " \"model\": \"client-a\",\n", + " \"nested\": {\"model\":\"keep-nested\"},\n", + " \"model\": \"client-b\"\n", + "}" + ].join(""); + const expected = source + .replace('"model": "client-a"', '"model": "provider-model"') + .replace('"model": "client-b"', '"model": "provider-model"'); + const absent = '{"large":9007199254740993123456789,"value":-0\n}'; + const absentExpected = `${absent.slice(0, -1)},"model":"provider-model"}`; + + for (const body of [source, absent]) { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { + "content-type": "application/json", + "content-md5": "stale-md5", + digest: "sha-256=stale", + "content-digest": "sha-256=:stale:", + "repr-digest": "sha-256=:stale:", + signature: "sig1=:stale:", + "signature-input": "sig1=(\"content-digest\")" + }, + body + }); + assert.equal(response.status, 200); + await response.arrayBuffer(); + } + const invalid = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: '{"model":"client","broken":}' + }); + assert.equal(invalid.status, 400); + assert.equal(observed.length, 2); + assert.equal(observed[0].body, expected); + assert.equal(observed[1].body, absentExpected); + assert.match(observed[0].body, /"nested": \{"model":"keep-nested"\}/); + for (const { headers } of observed) { + for (const name of [ + "content-md5", + "digest", + "content-digest", + "repr-digest", + "signature", + "signature-input" + ]) { + assert.equal(headers[name], undefined, `${name} survived a body rewrite`); + } + } +}); + +test("zstd override falls back to identity forwarding when native Node compression is unavailable", async (t) => { + const compressed = Buffer.from( + "KLUv/SAnOQEAeyJtb2RlbCI6ImNsaWVudC1tb2RlbCIsImlucHV0Ijoia2VlcCJ9", + "base64" + ); + const observed = createSignal(); + const upstream = http.createServer((req, res) => { + const chunks = []; + req.on("data", (chunk) => chunks.push(chunk)); + req.on("end", () => { + observed.resolve({ + encoding: req.headers["content-encoding"] ?? null, + length: req.headers["content-length"], + body: Buffer.concat(chunks).toString("utf8") + }); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(completedResponse())); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + modelMode: "override", + modelOverride: "provider-model", + captureEnabled: false + }); + const proxy = createServer(settings, { + captureManager: createInactiveCaptureManager(), + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + const nativeCompress = zlib.zstdCompressSync; + const nativeDecompress = zlib.zstdDecompressSync; + zlib.zstdCompressSync = undefined; + zlib.zstdDecompressSync = undefined; + try { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { + "content-type": "application/json", + "content-encoding": "zstd" + }, + body: compressed + }); + await response.arrayBuffer(); + } finally { + zlib.zstdCompressSync = nativeCompress; + zlib.zstdDecompressSync = nativeDecompress; + } + assert.deepEqual(await observed.promise, { + encoding: null, + length: String(Buffer.byteLength('{"model":"provider-model","input":"keep"}')), + body: '{"model":"provider-model","input":"keep"}' + }); +}); + +test("oversized model override requests fail before any upstream request", async (t) => { + let upstreamHits = 0; + const upstream = http.createServer((req, res) => { + upstreamHits += 1; + req.resume(); + res.end(); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + modelMode: "override", + modelOverride: "provider-model", + captureEnabled: false + }); + const proxy = createServer(settings, { + captureManager: createInactiveCaptureManager(), + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const expanded = Buffer.from(JSON.stringify({ + model: "client-model", + padding: "x".repeat(8 * 1024 * 1024) + })); + for (const [body, contentEncoding] of [ + [Buffer.alloc((8 * 1024 * 1024) + 1, 0x20), null], + [zlib.gzipSync(expanded), "gzip"] + ]) { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { + "content-type": "application/json", + ...(contentEncoding ? { "content-encoding": contentEncoding } : {}) + }, + body + }); + assert.equal(response.status, 413); + assert.equal((await response.json()).error.type, "proxy_request_too_large"); + } + assert.equal(upstreamHits, 0); +}); + +test("metric model inspection handles large JSON and rejects invalid UTF-8", async (t) => { + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + if (req.url.includes("invalid")) { + res.statusCode = 400; + res.end(JSON.stringify({ error: { message: "invalid" } })); + return; + } + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(completedResponse())); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 11, settings }); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metrics.push(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const largeResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses?case=large`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ padding: "x".repeat(96 * 1024), model: "model-after-64k" }) + }); + await largeResponse.arrayBuffer(); + + const invalidUtf8 = Buffer.concat([ + Buffer.from('{"model":"must-not-be-recorded","padding":"'), + Buffer.from([0xff]), + Buffer.from('"}') + ]); + const invalidResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses?case=invalid`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: invalidUtf8 + }); + await invalidResponse.arrayBuffer(); + + assert.deepEqual(metrics.map(({ result, model }) => ({ result, model })), [ + { result: "success", model: "model-after-64k" }, + { result: "upstreamRejected", model: null } + ]); +}); + +test("compressed request model inspection covers 1-8 MiB and reports larger bodies as unknown", async (t) => { + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(completedResponse())); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 22, settings }); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metrics.push(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + const decoded = Buffer.from(JSON.stringify({ + padding: "x".repeat(1280 * 1024), + model: "compressed-large-model" + })); + const encodings = [ + ["gzip", (body) => zlib.gzipSync(body)], + ["deflate", (body) => zlib.deflateSync(body)], + ["br", (body) => zlib.brotliCompressSync(body)], + ["zstd", () => Buffer.from([ + "KLUv/aAvABQArAAAaHsicGFkZGluZyI6IngBAPD/OfgCAgAQeAIAEHgCABB4AgAQeAIAEHgCABB4", + "AgAQeAIAEHgCABB4XQEARAJ4IiwibW9kZWwiOiJjb21wcmVzc2VkLWxhcmdlLW1vZGVsIn0BAAYQ", + "Ag==" + ].join(""), "base64")] + ]; + for (const [encoding, compress] of encodings) { + const compressed = compress(decoded); + const nativeZstd = zlib.zstdDecompressSync; + if (encoding === "zstd") zlib.zstdDecompressSync = undefined; + try { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { + "content-type": "application/json", + "content-encoding": encoding + }, + body: compressed + }); + await response.arrayBuffer(); + } finally { + if (encoding === "zstd") zlib.zstdDecompressSync = nativeZstd; + } + } + const beyondCeiling = zlib.gzipSync(Buffer.from(JSON.stringify({ + padding: "y".repeat((8 * 1024 * 1024) + 1024), + model: "must-be-unknown" + }))); + const beyondResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { + "content-type": "application/json", + "content-encoding": "gzip" + }, + body: beyondCeiling + }); + await beyondResponse.arrayBuffer(); + + assert.deepEqual(metrics.map(({ result, model }) => ({ result, model })), [ + ...encodings.map(() => ({ result: "success", model: "compressed-large-model" })), + { result: "success", model: null } + ]); +}); + test("dynamic requests capture current settings exactly once before body listeners", () => { const settings = makeSettings({ baseUrl: "http://127.0.0.1:9" }); const runtime = new RuntimeSettingsSource(); @@ -384,6 +920,7 @@ test("TLS and timeout options stay pinned when settings change before the reques observed.timeoutMs = timeoutMs; }; request.destroy = (error) => request.emit("error", error); + request.write = () => true; request.end = () => { const response = Readable.from([Buffer.from(JSON.stringify({ ok: true }))]); response.statusCode = 200; @@ -448,11 +985,11 @@ test("in-flight request keeps A target, credential, headers, capture, and logs w receivedA.resolve(); releaseA.promise.then(() => { res.setHeader("content-type", "application/json"); - res.end(JSON.stringify({ upstream: "A" })); + res.end(JSON.stringify(completedResponse({ upstream: "A" }))); }); } else { res.setHeader("content-type", "application/json"); - res.end(JSON.stringify({ upstream: "A-unexpected-repeat" })); + res.end(JSON.stringify(completedResponse({ upstream: "A-unexpected-repeat" }))); } }); }); @@ -468,7 +1005,7 @@ test("in-flight request keeps A target, credential, headers, capture, and logs w req.on("end", () => { observedB.push({ headers: req.headers, body: Buffer.concat(chunks).toString("utf8") }); res.setHeader("content-type", "application/json"); - res.end(JSON.stringify({ upstream: "B" })); + res.end(JSON.stringify(completedResponse({ upstream: "B" }))); }); }); const portB = await listen(upstreamB); @@ -479,7 +1016,7 @@ test("in-flight request keeps A target, credential, headers, capture, and logs w apiKey: "a-api-key-sentinel", authHeader: "x-provider-a-auth", authScheme: "Token", - extraHeaders: { "x-snapshot-route": "A" }, + extraHeaders: { "x-snapshot-route": "snapshot-route-a" }, timeoutMs: 5000, verifySsl: false, requestIdHeader: "x-a-request-id" @@ -489,7 +1026,7 @@ test("in-flight request keeps A target, credential, headers, capture, and logs w apiKey: "b-api-key-sentinel", authHeader: "x-provider-b-auth", authScheme: "", - extraHeaders: { "x-snapshot-route": "B" }, + extraHeaders: { "x-snapshot-route": "snapshot-route-b" }, timeoutMs: 1000, verifySsl: true, requestIdHeader: "x-b-request-id" @@ -544,14 +1081,14 @@ test("in-flight request keeps A target, credential, headers, capture, and logs w releaseA.release(); const responseA = await responseAPromise; - assert.deepEqual(responseA, { status: 200, body: { upstream: "A" } }); - assert.deepEqual(responseB, { status: 200, body: { upstream: "B" } }); + assert.deepEqual(responseA, { status: 200, body: completedResponse({ upstream: "A" }) }); + assert.deepEqual(responseB, { status: 200, body: completedResponse({ upstream: "B" }) }); assert.equal(observedA.length, 1); assert.equal(observedA[0].headers["x-provider-a-auth"], "Token a-api-key-sentinel"); - assert.equal(observedA[0].headers["x-snapshot-route"], "A"); + assert.equal(observedA[0].headers["x-snapshot-route"], "snapshot-route-a"); assert.equal(observedB.length, 1); assert.equal(observedB[0].headers["x-provider-b-auth"], "b-api-key-sentinel"); - assert.equal(observedB[0].headers["x-snapshot-route"], "B"); + assert.equal(observedB[0].headers["x-snapshot-route"], "snapshot-route-b"); assert.deepEqual(captureManager.records.map((record) => new URL(record.targetUrl).host), [ `127.0.0.1:${portB}`, `127.0.0.1:${portA}` @@ -600,26 +1137,683 @@ test("metrics extract bounded JSON and SSE usage while screening credential-bear return; } res.setHeader("content-type", "application/json"); - res.end(JSON.stringify({ + res.end(JSON.stringify(completedResponse({ id: "response-private-id", usage: { input_tokens: 13, output_tokens: 5 } - })); + }))); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + apiKey: secret, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 9, settings }); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createMemoryCaptureManager(), + recordMetric(observation) { + metrics.push(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const jsonResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "model-json", stream: false }) + }); + assert.equal(jsonResponse.status, 200); + await jsonResponse.text(); + const streamResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: `prefix-${secret}-suffix`, stream: true }) + }); + assert.equal(streamResponse.status, 200); + await streamResponse.text(); + + const encodedModels = [ + Buffer.from(secret).toString("base64url"), + Buffer.from(secret).toString("hex").toUpperCase(), + [...secret] + .map((character) => `%${character.codePointAt(0).toString(16).padStart(2, "0")}`) + .join("") + ]; + for (const model of encodedModels) { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model, stream: false }) + }); + assert.equal(response.status, 200); + await response.text(); + } + + assert.equal(metrics.length, 5); + const serialized = JSON.stringify(metrics); + assert.equal(serialized.includes(secret), false); + for (const encodedModel of encodedModels) { + assert.equal(serialized.includes(encodedModel), false); + } + assert.equal(serialized.includes("response-private-id"), false); + assert.equal(serialized.includes("url"), false); + assert.equal(serialized.includes("headers"), false); + assert.equal(serialized.includes("body"), false); + assert.deepEqual(metrics.map(({ generation, result, model, inputTokens, outputTokens }) => ({ + generation, + result, + model, + inputTokens, + outputTokens + })), [ + { + generation: 9, + result: "success", + model: "model-json", + inputTokens: 13, + outputTokens: 5 + }, + { + generation: 9, + result: "success", + model: null, + inputTokens: 21, + outputTokens: 8 + }, + { + generation: 9, + result: "success", + model: null, + inputTokens: 13, + outputTokens: 5 + }, + { + generation: 9, + result: "success", + model: null, + inputTokens: 13, + outputTokens: 5 + }, + { + generation: 9, + result: "success", + model: null, + inputTokens: 13, + outputTokens: 5 + } + ]); +}); + +test("Responses metrics reject semantic JSON and SSE failures plus missing terminal completion", async (t) => { + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + const url = new URL(req.url, "http://upstream.test"); + const scenario = url.searchParams.get("case"); + if (scenario === "json-failed") { + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify({ + id: "resp_failed", + object: "response", + status: "failed", + output: [], + error: { message: "provider failed" } + })); + return; + } + if (scenario === "invalid-json") { + res.setHeader("content-type", "application/json"); + res.end(Buffer.concat([ + Buffer.from(JSON.stringify(completedResponse({ padding: "z".repeat(2 * 1024 * 1024) }))), + Buffer.from([0xff]) + ])); + return; + } + res.setHeader("content-type", "text/event-stream"); + if (scenario === "failed-then-completed") { + res.end([ + `data: ${JSON.stringify({ type: "response.failed", response: { error: { message: "failed" } } })}\n\n`, + `data: ${JSON.stringify({ + type: "response.completed", + response: { usage: { input_tokens: 99, output_tokens: 99 } } + })}\n\n` + ].join("")); + return; + } + res.end(`data: ${JSON.stringify({ type: "response.output_text.delta", delta: "partial" })}\n\n`); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 12, settings }); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metrics.push(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + for (const [scenario, stream] of [ + ["json-failed", false], + ["invalid-json", false], + ["failed-then-completed", true], + ["missing-terminal", true] + ]) { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses?case=${scenario}`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "semantic-model", stream }) + }); + assert.equal(response.status, 200); + await response.arrayBuffer(); + } + + assert.deepEqual(metrics.map(({ result, inputTokens, outputTokens }) => ({ + result, + inputTokens, + outputTokens + })), [ + { result: "upstreamError", inputTokens: null, outputTokens: null }, + { result: "upstreamError", inputTokens: null, outputTokens: null }, + { result: "upstreamError", inputTokens: null, outputTokens: null }, + { result: "upstreamError", inputTokens: null, outputTokens: null } + ]); +}); + +test("bounded SSE inspection accepts a completed event larger than the former 64 KiB limit", async (t) => { + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "text/event-stream"); + res.end(`data: ${JSON.stringify({ + type: "response.completed", + response: { + status: "completed", + output: [{ text: "x".repeat(96 * 1024) }], + usage: { input_tokens: 43, output_tokens: 19 } + } + })}\n\n`); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 18, settings }); + const metricSignal = createSignal(); + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metricSignal.resolve(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "large-event-model", stream: true }) + }); + assert.equal(response.status, 200); + await response.arrayBuffer(); + const metric = await withDeadline(metricSignal.promise, "large completed SSE event metric was not recorded"); + assert.deepEqual( + (({ result, inputTokens, outputTokens }) => ({ result, inputTokens, outputTokens }))(metric), + { result: "success", inputTokens: 43, outputTokens: 19 } + ); +}); + +test("SSE inspection bounds empty data lines including framing overhead", async (t) => { + const emptyLines = "data:\n".repeat(Math.ceil((1024 * 1024) / 6) + 128); + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "text/event-stream"); + res.end(`${emptyLines}\ndata: ${JSON.stringify({ + type: "response.completed", + response: { usage: { input_tokens: 47, output_tokens: 21 } } + })}\n\n`); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 21, settings }); + const metricSignal = createSignal(); + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metricSignal.resolve(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "empty-line-model", stream: true }) + }); + await response.arrayBuffer(); + const metric = await withDeadline(metricSignal.promise, "empty-line SSE metric was not recorded"); + assert.deepEqual( + (({ result, inputTokens, outputTokens }) => ({ result, inputTokens, outputTokens }))(metric), + { result: "success", inputTokens: 47, outputTokens: 21 } + ); +}); + +test("SSE inspection accepts CR-only and cross-chunk CRLF event delimiters", async (t) => { + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "text/event-stream"); + const crossChunk = req.url.includes("cross-chunk"); + const payload = JSON.stringify({ + type: "response.completed", + response: { + usage: crossChunk + ? { input_tokens: 53, output_tokens: 29 } + : { input_tokens: 51, output_tokens: 27 } + } + }); + if (!crossChunk) { + res.end(`data: ${payload}\r\r`); + return; + } + res.write(`data: ${payload}\r`); + setImmediate(() => res.end("\n\r\n")); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 24, settings }); + const metrics = []; + const metricsReady = createSignal(); + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metrics.push(structuredClone(observation)); + if (metrics.length === 2) metricsReady.resolve(); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + for (const scenario of ["cr-only", "cross-chunk"]) { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses?case=${scenario}`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "line-ending-model", stream: true }) + }); + await response.arrayBuffer(); + } + await withDeadline(metricsReady.promise, "line-ending SSE metrics were not recorded"); + assert.deepEqual(metrics.map(({ result, inputTokens, outputTokens }) => ({ + result, + inputTokens, + outputTokens + })), [ + { result: "success", inputTokens: 51, outputTokens: 27 }, + { result: "success", inputTokens: 53, outputTokens: 29 } + ]); +}); + +test("passthrough preserves declared and headerless compressed request bytes", async (t) => { + const observed = []; + const receivedBoth = createSignal(); + const upstream = http.createServer((req, res) => { + const chunks = []; + req.on("data", (chunk) => chunks.push(chunk)); + req.on("end", () => { + observed.push({ + encoding: req.headers["content-encoding"] ?? null, + body: Buffer.concat(chunks) + }); + if (observed.length === 2) receivedBoth.resolve(); + res.end("ok"); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const proxy = createServer(settings, { + captureManager: createInactiveCaptureManager(), + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + const compressed = zlib.gzipSync(Buffer.from(JSON.stringify({ model: "byte-exact-model" }))); + + for (const headers of [ + { "content-encoding": "gzip", "content-length": String(compressed.length) }, + { "content-length": String(compressed.length) } + ]) { + const response = await fetch(`http://127.0.0.1:${proxyPort}/passthrough`, { + method: "POST", + headers, + body: compressed + }); + assert.equal(await response.text(), "ok"); + } + await withDeadline(receivedBoth.promise, "upstream did not receive both compressed requests"); + assert.deepEqual(observed.map(({ encoding }) => encoding), ["gzip", null]); + assert.deepEqual(observed[0].body, compressed); + assert.deepEqual(observed[1].body, compressed); +}); + +test("Connection tokens remove dynamically nominated request and response headers", async (t) => { + const upstreamHeaders = createSignal(); + const upstream = http.createServer((req, res) => { + upstreamHeaders.resolve(req.headers); + req.resume(); + req.on("end", () => { + res.setHeader("connection", "keep-alive, x-response-remove"); + res.setHeader("x-response-remove", "must-not-reach-client"); + res.setHeader("x-response-keep", "visible"); + res.end("ok"); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + extraHeaders: { + connection: "configured-hop-secret", + "transfer-encoding": "configured-hop-secret", + upgrade: "configured-hop-secret", + "x-generated-keep": "visible" + }, + captureEnabled: false + }); + const proxy = createServer(settings, { + captureManager: createInactiveCaptureManager(), + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const received = await new Promise((resolvePromise, rejectPromise) => { + const request = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: "/passthrough", + method: "POST", + headers: { + connection: "keep-alive, x-remove-me", + "x-remove-me": "must-not-reach-upstream", + "x-request-keep": "visible" + } + }, (response) => { + response.resume(); + response.on("end", () => resolvePromise(response.headers)); + }); + request.on("error", rejectPromise); + request.end("body"); + }); + const forwarded = await upstreamHeaders.promise; + assert.equal(forwarded["x-remove-me"], undefined); + assert.equal(forwarded["x-request-keep"], "visible"); + assert.notEqual(forwarded.connection, "configured-hop-secret"); + assert.notEqual(forwarded["transfer-encoding"], "configured-hop-secret"); + assert.equal(forwarded.upgrade, undefined); + assert.equal(forwarded["x-generated-keep"], "visible"); + assert.equal(received["x-response-remove"], undefined); + assert.equal(received["x-response-keep"], "visible"); +}); + +test("compressed JSON and SSE usage is inspected without changing response bytes or headers", async (t) => { + const upstreamBodies = new Map(); + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + const scenario = new URL(req.url, "http://upstream.test").searchParams.get("case"); + const plain = scenario === "sse" + ? Buffer.from([ + `data: ${JSON.stringify({ + type: "response.output_text.delta", + delta: "s".repeat(2 * 1024 * 1024) + })}\n\n`, + `data: ${JSON.stringify({ + type: "response.completed", + response: { usage: { input_tokens: 41, output_tokens: 17 } } + })}\n\n` + ].join("")) + : Buffer.from(JSON.stringify(completedResponse({ + padding: "j".repeat(2 * 1024 * 1024), + usage: { input_tokens: 37, output_tokens: 14 } + }))); + const compressed = zlib.gzipSync(plain); + upstreamBodies.set(scenario, compressed); + res.setHeader("content-type", scenario === "sse" ? "text/event-stream" : "application/json"); + res.setHeader("content-encoding", "gzip"); + res.setHeader("content-length", String(compressed.length)); + res.end(compressed); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 13, settings }); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metrics.push(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + for (const scenario of ["json", "sse"]) { + const received = await new Promise((resolvePromise, rejectPromise) => { + const request = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: `/responses?case=${scenario}`, + method: "POST", + headers: { "content-type": "application/json" } + }, (response) => { + const chunks = []; + response.on("data", (chunk) => chunks.push(chunk)); + response.on("end", () => resolvePromise({ + encoding: response.headers["content-encoding"], + body: Buffer.concat(chunks) + })); + }); + request.on("error", rejectPromise); + request.end(JSON.stringify({ model: "compressed-model", stream: scenario === "sse" })); + }); + assert.equal(received.encoding, "gzip"); + assert.deepEqual(received.body, upstreamBodies.get(scenario)); + } + + assert.deepEqual(metrics.map(({ result, inputTokens, outputTokens }) => ({ + result, + inputTokens, + outputTokens + })), [ + { result: "success", inputTokens: 37, outputTokens: 14 }, + { result: "success", inputTokens: 41, outputTokens: 17 } + ]); +}); + +test("zstd usage inspection falls back to fzstd while passthrough remains byte exact", async (t) => { + const compressed = Buffer.from( + "KLUv/SB23QIAksUSGICpbZDc5t9K8V9vy04UJrJiELmChRk4AscBKCVg1nUHvPnmUjbMVzi6JgOB+Zvf+yEygchm3Z+riU8XHp2BgCZO14VJPYXNSdkIBAA2C6BS9GDqjjCbeA==", + "base64" + ); + const nativeDecompress = zlib.zstdDecompressSync; + zlib.zstdDecompressSync = undefined; + t.after(() => { + zlib.zstdDecompressSync = nativeDecompress; + }); + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "application/json"); + res.setHeader("content-encoding", "zstd"); + res.setHeader("content-length", String(compressed.length)); + res.end(compressed); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 14, settings }); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + metrics.push(structuredClone(observation)); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const received = await new Promise((resolvePromise, rejectPromise) => { + const request = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: "/responses", + method: "POST", + headers: { "content-type": "application/json" } + }, (response) => { + const chunks = []; + response.on("data", (chunk) => chunks.push(chunk)); + response.on("end", () => resolvePromise(Buffer.concat(chunks))); + }); + request.on("error", rejectPromise); + request.end(JSON.stringify({ model: "zstd-model", stream: false })); + }); + + assert.deepEqual(received, compressed); + assert.deepEqual(metrics.map(({ result, inputTokens, outputTokens }) => ({ + result, + inputTokens, + outputTokens + })), [ + { result: "success", inputTokens: 31, outputTokens: 12 } + ]); +}); + +test("fzstd inspection accepts the exact bound and rejects multi-frame input without changing bytes", async (t) => { + const exact = Buffer.from([ + "KLUv/YBYAACAAEwDALLGFRqAqS0GDCsSKCREfGO27MRgDKzoBTtmiiZhBMk3ktA5RW0NQQAwuDUh", + "IvfYpRKacSNSdG0Fwr1L7crtg4sJpUjzrWvL6jZqyjV4MOT2mpLgoRQX3wfNCAQA9P5xeW1qCJQi", + "xswmHgIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgC", + "ABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIA", + "EHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQ", + "eAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4", + "AgAQeAIAEHgCABB4AgAQeAIAEHgCABB4XQAAGHgifQEA+v85EAI=" + ].join(""), "base64"); + const single = Buffer.from( + "KLUv/SB23QIAksUSGICpbZDc5t9K8V9vy04UJrJiELmChRk4AscBKCVg1nUHvPnmUjbMVzi6JgOB+Zvf+yEygchm3Z+riU8XHp2BgCZO14VJPYXNSdkIBAA2C6BS9GDqjjCbeA==", + "base64" + ); + const multi = Buffer.concat([single, single]); + const validPrefix = Buffer.from(JSON.stringify(completedResponse({ + usage: { input_tokens: 71, output_tokens: 37 } + }))); + const prefixHeader = Buffer.alloc(3); + prefixHeader.writeUIntLE(validPrefix.length << 3, 0, 3); + const extraSize = 10; + const extraHeader = Buffer.alloc(3); + extraHeader.writeUIntLE((extraSize << 3) | 0x03, 0, 3); + const mismatchedSize = Buffer.concat([ + Buffer.from([0x28, 0xb5, 0x2f, 0xfd, 0x20, validPrefix.length]), + prefixHeader, + validPrefix, + extraHeader, + Buffer.from("A") + ]); + const checksummedBody = Buffer.from(JSON.stringify(completedResponse({ + usage: { input_tokens: 73, output_tokens: 39 } + }))); + const checksummedHeader = Buffer.alloc(3); + checksummedHeader.writeUIntLE((checksummedBody.length << 3) | 0x01, 0, 3); + const checksummed = Buffer.concat([ + Buffer.from([0x28, 0xb5, 0x2f, 0xfd, 0x24, checksummedBody.length]), + checksummedHeader, + checksummedBody, + Buffer.from([0xde, 0xad, 0xbe, 0xef]) + ]); + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + const body = req.url.includes("multi") + ? multi + : (req.url.includes("mismatch") + ? mismatchedSize + : (req.url.includes("checksum") ? checksummed : exact)); + res.setHeader("content-type", "application/json"); + res.setHeader("content-encoding", "zstd"); + res.setHeader("content-length", String(body.length)); + res.end(body); }); }); const upstreamPort = await listen(upstream); t.after(() => closeServer(upstream)); - const settings = makeSettings({ baseUrl: `http://127.0.0.1:${upstreamPort}`, - apiKey: secret, captureEnabled: false }); const source = new RuntimeSettingsSource(); - source.apply({ generation: 9, settings }); + source.apply({ generation: 23, settings }); const metrics = []; const proxy = createServer(settings, { settingsSource: source, - captureManager: createMemoryCaptureManager(), + captureManager: createInactiveCaptureManager(), recordMetric(observation) { metrics.push(structuredClone(observation)); }, @@ -627,51 +1821,45 @@ test("metrics extract bounded JSON and SSE usage while screening credential-bear }); const proxyPort = await listen(proxy); t.after(() => closeServer(proxy)); - - const jsonResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ model: "model-json", stream: false }) - }); - assert.equal(jsonResponse.status, 200); - await jsonResponse.text(); - const streamResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses`, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ model: `prefix-${secret}-suffix`, stream: true }) - }); - assert.equal(streamResponse.status, 200); - await streamResponse.text(); - - assert.equal(metrics.length, 2); - assert.deepEqual(metrics.map(({ generation, result, model, inputTokens, outputTokens }) => ({ - generation, + const nativeDecompress = zlib.zstdDecompressSync; + zlib.zstdDecompressSync = undefined; + try { + for (const [scenario, expected] of [ + ["exact", exact], + ["multi", multi], + ["mismatch", mismatchedSize], + ["checksum", checksummed] + ]) { + const received = await new Promise((resolvePromise, rejectPromise) => { + const request = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: `/responses?case=${scenario}`, + method: "POST", + headers: { "content-type": "application/json" } + }, (response) => { + const chunks = []; + response.on("data", (chunk) => chunks.push(chunk)); + response.on("end", () => resolvePromise(Buffer.concat(chunks))); + }); + request.on("error", rejectPromise); + request.end(JSON.stringify({ model: "zstd-bound-model" })); + }); + assert.deepEqual(received, expected); + } + } finally { + zlib.zstdDecompressSync = nativeDecompress; + } + assert.deepEqual(metrics.map(({ result, inputTokens, outputTokens }) => ({ result, - model, inputTokens, outputTokens })), [ - { - generation: 9, - result: "success", - model: "model-json", - inputTokens: 13, - outputTokens: 5 - }, - { - generation: 9, - result: "success", - model: null, - inputTokens: 21, - outputTokens: 8 - } + { result: "success", inputTokens: 61, outputTokens: 23 }, + { result: "upstreamError", inputTokens: null, outputTokens: null }, + { result: "upstreamError", inputTokens: null, outputTokens: null }, + { result: "upstreamError", inputTokens: null, outputTokens: null } ]); - const serialized = JSON.stringify(metrics); - assert.equal(serialized.includes(secret), false); - assert.equal(serialized.includes("response-private-id"), false); - assert.equal(serialized.includes("url"), false); - assert.equal(serialized.includes("headers"), false); - assert.equal(serialized.includes("body"), false); }); test("metrics response-start latency begins at the first response body byte", async (t) => { @@ -745,6 +1933,211 @@ test("metrics response-start latency begins at the first response body byte", as assert.equal(bodylessMetric.responseStartBin, null); }); +test("passthrough streams request and multi-megabyte SSE response before either side completes", async (t) => { + const firstRequestChunk = createSignal(); + const releaseResponse = createGate(); + let upstreamResponseEnded = false; + const upstream = http.createServer((req, res) => { + let observedRequestData = false; + req.on("data", () => { + if (!observedRequestData) { + observedRequestData = true; + firstRequestChunk.resolve(); + } + }); + req.on("end", () => { + res.setHeader("content-type", "text/event-stream"); + res.write(`data: ${JSON.stringify({ type: "response.created" })}\n\n`); + releaseResponse.promise.then(() => { + const delta = "x".repeat(60 * 1024); + for (let index = 0; index < 36; index += 1) { + res.write(`data: ${JSON.stringify({ type: "response.output_text.delta", delta })}\n\n`); + } + res.end(`data: ${JSON.stringify({ + type: "response.completed", + response: { usage: { input_tokens: 55, output_tokens: 34 } } + })}\n\n`); + upstreamResponseEnded = true; + }); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => { + releaseResponse.release(); + return closeServer(upstream); + }); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + captureEnabled: false + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 15, settings }); + const metricSignal = createSignal(); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager: createInactiveCaptureManager(), + recordMetric(observation) { + const clone = structuredClone(observation); + metrics.push(clone); + metricSignal.resolve(clone); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const firstResponseChunk = createSignal(); + const responseDone = createSignal(); + let responseBytes = 0; + const request = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: "/responses", + method: "POST", + headers: { "content-type": "application/json" } + }, (response) => { + response.on("data", (chunk) => { + responseBytes += chunk.length; + firstResponseChunk.resolve(); + }); + response.on("end", () => responseDone.resolve()); + }); + request.on("error", (error) => responseDone.resolve(error)); + request.write('{"model":"stream-model","stream":true,"input":"'); + await withDeadline(firstRequestChunk.promise, "upstream did not receive the request before client end"); + request.end('streamed"}'); + await withDeadline(firstResponseChunk.promise, "client did not receive the first response chunk"); + assert.equal(upstreamResponseEnded, false); + releaseResponse.release(); + const responseError = await withDeadline(responseDone.promise, "large streamed response did not finish", 5000); + if (responseError instanceof Error) throw responseError; + const metric = await withDeadline(metricSignal.promise, "large streamed response metric was not recorded"); + + assert.ok(responseBytes > 2 * 1024 * 1024, `received only ${responseBytes} bytes`); + assert.deepEqual( + (({ result, model, inputTokens, outputTokens }) => ({ result, model, inputTokens, outputTokens }))(metric), + { + result: "success", + model: "stream-model", + inputTokens: 55, + outputTokens: 34 + } + ); + assert.equal(metrics.length, 1); +}); + +test("downstream abort promptly cancels upstream work and records one clientAbort", async (t) => { + const upstreamClosed = createSignal(); + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "text/event-stream"); + res.on("close", () => upstreamClosed.resolve()); + res.write(`data: ${JSON.stringify({ type: "response.created" })}\n\n`); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ baseUrl: `http://127.0.0.1:${upstreamPort}` }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 16, settings }); + const captureManager = createMemoryCaptureManager(); + const metricSignal = createSignal(); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager, + recordMetric(observation) { + const clone = structuredClone(observation); + metrics.push(clone); + metricSignal.resolve(clone); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + await new Promise((resolvePromise, rejectPromise) => { + const request = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: "/responses", + method: "POST", + headers: { "content-type": "application/json" } + }, (response) => { + response.once("data", () => { + response.destroy(); + resolvePromise(); + }); + }); + request.on("error", rejectPromise); + request.end(JSON.stringify({ model: "abort-model", stream: true })); + }); + await withDeadline(upstreamClosed.promise, "upstream response was not cancelled after client abort"); + const metric = await withDeadline(metricSignal.promise, "client abort metric was not recorded"); + + assert.equal(metric.result, "clientAbort"); + assert.equal(metrics.length, 1); + assert.equal(captureManager.records.length, 1); + assert.equal(captureManager.records[0].errorType, "proxy_client_abort"); +}); + +test("partial upstream response failure closes downstream and finalizes capture plus metric once", async (t) => { + const upstream = http.createServer((req, res) => { + req.resume(); + req.on("end", () => { + res.writeHead(200, { "content-type": "text/event-stream" }); + res.write("data: partial-upstream-body\n\n"); + setImmediate(() => res.destroy(new Error("fixture partial response failure"))); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ baseUrl: `http://127.0.0.1:${upstreamPort}` }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 17, settings }); + const captureManager = createMemoryCaptureManager(); + const metricSignal = createSignal(); + const metrics = []; + const proxy = createServer(settings, { + settingsSource: source, + captureManager, + recordMetric(observation) { + const clone = structuredClone(observation); + metrics.push(clone); + metricSignal.resolve(clone); + }, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + await withDeadline(new Promise((resolvePromise, rejectPromise) => { + const request = http.request({ + host: "127.0.0.1", + port: proxyPort, + path: "/responses", + method: "POST", + headers: { "content-type": "application/json" } + }, (response) => { + response.resume(); + response.once("aborted", resolvePromise); + response.once("error", resolvePromise); + response.once("close", resolvePromise); + }); + request.on("error", rejectPromise); + request.end(JSON.stringify({ model: "partial-model", stream: true })); + }), "downstream did not close after partial upstream failure"); + const metric = await withDeadline(metricSignal.promise, "partial response metric was not recorded"); + + assert.equal(metric.result, "networkError"); + assert.equal(metrics.length, 1); + assert.equal(captureManager.records.length, 1); + assert.equal(captureManager.records[0].errorType, "proxy_upstream_response_error"); + assert.match(captureManager.records[0].responseBody.toString("utf8"), /partial-upstream-body/); +}); + test("an in-flight A request retains its longer timeout after B is applied", async (t) => { const releaseA = createGate(); t.after(() => releaseA.release()); @@ -754,7 +2147,7 @@ test("an in-flight A request retains its longer timeout after B is applied", asy req.resume(); req.on("end", () => { receivedA.resolve(); - releaseA.promise.then(() => res.end(JSON.stringify({ upstream: "A" }))); + releaseA.promise.then(() => res.end(JSON.stringify(completedResponse({ upstream: "A" })))); }); }); const portA = await listen(upstreamA); @@ -809,7 +2202,7 @@ test("an in-flight A request retains its longer timeout after B is applied", asy releaseA.release(); const responseA = await responseAPromise; - assert.deepEqual(responseA, { status: 200, body: { upstream: "A" } }); + assert.deepEqual(responseA, { status: 200, body: completedResponse({ upstream: "A" }) }); assert.deepEqual(metrics.map(({ generation, result }) => ({ generation, result })), [ { generation: 2, result: "timeout" }, { generation: 1, result: "success" } @@ -887,6 +2280,8 @@ test("debug and startup logs mask short keys and the active custom auth header", const staticSettings = makeSettings({ baseUrl: `http://127.0.0.1:${upstreamPort}`, apiKey: "static-api-key-sentinel", + authHeader: "static-api-key-sentinel", + authScheme: "", logLevel: "debug" }); const activeSettings = makeSettings({ @@ -927,14 +2322,285 @@ test("debug and startup logs mask short keys and the active custom auth header", } const output = lines.join("\n"); - assert.match(output, /DEBUG \[REQUEST\]/); + assert.equal(output.includes("static-api-key-sentinel"), false); assert.equal(output.includes("k3y"), false); assert.equal(output.includes("client-custom-auth-sentinel"), false); assert.equal(output.includes("response-custom-auth-secret"), false); assert.equal(output.includes("session-cookie-secret"), false); assert.equal(output.includes('"x-api-key": "tiny"'), false); + assert.equal(output.includes(JSON.stringify(source.current())), false); + assert.match(output, /DEBUG \[REQUEST\]/); assert.match(output, /"x-api-key": "\[REDACTED\]"/); assert.match(output, /"x-diagnostic": "trace-visible"/); - assert.equal(output.includes(JSON.stringify(source.current())), false); assert.equal(observedAuthHeader, "k3y"); }); + +test("debug and Capture redact configured protected values under otherwise safe header names", async (t) => { + const apiKey = "protected-api-key-complete-sentinel"; + const extraValue = "protected-extra-header-complete-sentinel"; + const shortValue = "k3y"; + let observedExtraHeader = null; + const upstream = http.createServer((req, res) => { + const compressed = req.headers["content-encoding"] === "gzip"; + observedExtraHeader = req.headers["x-route-marker"] ?? null; + req.resume(); + req.on("end", () => { + const responseBody = Buffer.from(JSON.stringify(completedResponse({ + output: [{ type: "message", text: `${apiKey}:${extraValue}:${shortValue}` }] + }))); + res.setHeader("content-type", "application/json"); + res.setHeader("x-request-id", apiKey); + res.setHeader("x-safe-api-echo", apiKey); + res.setHeader("x-safe-route-echo", `prefix-${extraValue}-suffix`); + res.setHeader("x-safe-short-echo", `${shortValue}-suffix`); + if (compressed) { + const encoded = zlib.gzipSync(responseBody); + res.setHeader("content-encoding", "gzip"); + res.setHeader("content-length", String(encoded.length)); + res.end(encoded); + } else { + res.end(responseBody); + } + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + apiKey, + extraHeaders: { + "x-route-marker": extraValue, + "x-short-marker": shortValue + }, + logLevel: "debug" + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 20, settings }); + const captureManager = createMemoryCaptureManager(); + const lines = []; + const originalConsoleLog = console.log; + console.log = (...args) => lines.push(args.join(" ")); + t.after(() => { + console.log = originalConsoleLog; + }); + const proxy = createServer(settings, { + settingsSource: source, + captureManager, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + const fullyEncodedApiKey = [...apiKey] + .map((character) => `%${character.codePointAt(0).toString(16).padStart(2, "0")}`) + .join(""); + let compressedBody; + try { + const escapedApiKey = [...apiKey] + .map((character) => `\\u${character.codePointAt(0).toString(16).padStart(4, "0")}`) + .join(""); + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses?trace=${fullyEncodedApiKey}&bad=%ZZ`, { + method: "POST", + headers: { + "content-type": "application/json", + "x-client-note": apiKey, + "x-client-request-id": apiKey, + "session-id": extraValue, + "thread-id": shortValue + }, + body: `{"model":"safe-model","input":"${escapedApiKey}"}` + }); + await response.arrayBuffer(); + compressedBody = zlib.gzipSync(Buffer.from(JSON.stringify({ + model: "safe-model", + input: `${apiKey}:${extraValue}:${shortValue}` + }))); + const compressedResponse = await fetch(`http://127.0.0.1:${proxyPort}/responses?case=compressed`, { + method: "POST", + headers: { + "content-type": "application/json", + "content-encoding": "gzip" + }, + body: compressedBody + }); + await compressedResponse.arrayBuffer(); + } finally { + console.log = originalConsoleLog; + } + + const serialized = `${lines.join("\n")}\n${JSON.stringify(captureManager.records)}`; + assert.equal(serialized.includes(apiKey), false); + assert.equal(serialized.includes(extraValue), false); + assert.equal(serialized.includes(shortValue), false); + assert.equal(serialized.includes(fullyEncodedApiKey), false); + assert.equal(observedExtraHeader, extraValue); + assert.equal(captureManager.records[0].requestHeaders["x-client-note"], "[REDACTED]"); + assert.equal(captureManager.records[0].requestHeaders["x-route-marker"], "[REDACTED]"); + assert.equal(captureManager.records[0].responseHeaders["x-safe-api-echo"], "[REDACTED]"); + assert.equal(captureManager.records[0].responseHeaders["x-safe-route-echo"], "[REDACTED]"); + assert.equal(captureManager.records[0].responseHeaders["x-safe-short-echo"], "[REDACTED]"); + assert.equal(captureManager.records[0].requestId, "[REDACTED]"); + assert.equal(captureManager.records[0].sessionId, "[REDACTED]"); + assert.equal(captureManager.records[0].threadId, "[REDACTED]"); + assert.equal(captureManager.records[0].upstreamRequestId, "[REDACTED]"); + assert.equal(captureManager.records[0].incomingUrl, "[REDACTED]"); + assert.equal(captureManager.records[0].targetUrl, "[REDACTED]"); + assert.equal(captureManager.records[0].requestBody.length, 0); + assert.equal(captureManager.records[0].requestBodyTruncated, true); + assert.equal(captureManager.records[0].responseBody.length, 0); + assert.equal(captureManager.records[0].responseBodyTruncated, true); + for (const body of [ + captureManager.records[0].requestBody.toString("utf8"), + captureManager.records[0].responseBody.toString("utf8") + ]) { + assert.equal(body.includes(apiKey), false); + assert.equal(body.includes(extraValue), false); + assert.equal(body.includes(shortValue), false); + } + assert.equal(captureManager.records[1].requestBody.length, 0); + assert.equal(captureManager.records[1].requestBodyTruncated, true); + assert.equal(captureManager.records[1].requestBodyBytes, compressedBody.length); + assert.equal(captureManager.records[1].responseBody.length, 0); + assert.equal(captureManager.records[1].responseBodyTruncated, true); + assert.ok(captureManager.records[1].responseBodyBytes > 0); +}); + +test("Capture omits independently recoverable encodings and undeclared compressed bodies", async (t) => { + const apiKey = "protected-api-key-complete-sentinel"; + const base64ProtectedValue = "!!>x"; + const base64NameProtectedValue = "j'!"; + const base64HeaderName = Buffer.from(base64NameProtectedValue).toString("base64url"); + const encodedApiKey = Buffer.from(apiKey).toString("base64"); + const percentEncodedApiKey = [...apiKey] + .map((character) => `%${character.codePointAt(0).toString(16).padStart(2, "0")}`) + .join(""); + const observedMethods = []; + const upstream = http.createServer((req, res) => { + observedMethods.push(req.method); + req.resume(); + req.on("end", () => { + res.setHeader("content-type", "application/json"); + res.setHeader("x-request-id", encodedApiKey); + res.end(JSON.stringify(completedResponse())); + }); + }); + const upstreamPort = await listen(upstream); + t.after(() => closeServer(upstream)); + const settings = makeSettings({ + baseUrl: `http://127.0.0.1:${upstreamPort}`, + apiKey, + extraHeaders: { + "x-encoded-marker": base64ProtectedValue, + "x-base64-name-marker": base64NameProtectedValue, + "x-method-marker": "POST" + }, + logLevel: "debug" + }); + const source = new RuntimeSettingsSource(); + source.apply({ generation: 21, settings }); + const captureManager = createMemoryCaptureManager({ + captureDbPath: `${apiKey}:${encodedApiKey}` + }); + const lines = []; + const originalConsoleLog = console.log; + console.log = (...args) => lines.push(args.join(" ")); + t.after(() => { + console.log = originalConsoleLog; + }); + const proxy = createServer(settings, { + settingsSource: source, + captureManager, + logFn() {} + }); + const proxyPort = await listen(proxy); + t.after(() => closeServer(proxy)); + + const hex = Buffer.from(apiKey).toString("hex"); + const mixedCaseHex = [...hex].map((character, index) => ( + /[a-f]/u.test(character) && index % 2 === 0 ? character.toUpperCase() : character + )).join(""); + const unpaddedStandardBase64 = Buffer.from(base64ProtectedValue) + .toString("base64") + .replace(/=+$/u, ""); + assert.match(unpaddedStandardBase64, /\+/u); + const paddedBase64Url = Buffer.from(base64ProtectedValue) + .toString("base64") + .replace(/\+/gu, "-") + .replace(/\//gu, "_"); + assert.match(paddedBase64Url, /-.*=+$/u); + const whitespaceBase64 = encodedApiKey.replace(/.{4}/gu, "$& \r\n"); + assert.equal(Buffer.from(whitespaceBase64, "base64").toString("utf8") === apiKey, true); + const zstdBody = typeof zlib.zstdCompressSync === "function" + ? zlib.zstdCompressSync(Buffer.from(apiKey)) + : Buffer.from( + "KLUv/SB23QIAksUSGICpbZDc5t9K8V9vy04UJrJiELmChRk4AscBKCVg1nUHvPnmUjbMVzi6JgOB+Zvf+yEygchm3Z+riU8XHp2BgCZO14VJPYXNSdkIBAA2C6BS9GDqjjCbeA==", + "base64" + ); + const skippableZstdBody = Buffer.concat([ + Buffer.from([0x50, 0x2a, 0x4d, 0x18, 0x00, 0x00, 0x00, 0x00]), + zstdBody + ]); + const bodies = [ + Buffer.from(mixedCaseHex), + Buffer.from(unpaddedStandardBase64), + Buffer.from(paddedBase64Url), + Buffer.from(whitespaceBase64), + zlib.gzipSync(Buffer.from(apiKey)), + zlib.deflateSync(Buffer.from(apiKey)), + zlib.brotliCompressSync(Buffer.from(apiKey)), + zstdBody, + skippableZstdBody + ]; + + for (const [index, body] of bodies.entries()) { + const response = await fetch(`http://127.0.0.1:${proxyPort}/responses?case=${index}`, { + method: "POST", + headers: { + "content-type": "application/octet-stream", + ...(index === 0 ? { + "x-client-request-id": encodedApiKey, + "session-id": percentEncodedApiKey, + "thread-id": mixedCaseHex, + "x-client-note": paddedBase64Url, + [apiKey]: "literal-protected-name", + [mixedCaseHex]: "hex-protected-name", + [base64HeaderName]: "base64url-protected-name" + } : {}) + }, + body + }); + assert.equal(response.status, 200); + await response.arrayBuffer(); + } + + const healthResponse = await fetch(`http://127.0.0.1:${proxyPort}/_proxy/health`); + assert.equal(healthResponse.status, 200); + const health = await healthResponse.json(); + const serialized = `${lines.join("\n")}\n${JSON.stringify(captureManager.records)}\n${JSON.stringify(health)}`; + for (const recoverable of [ + apiKey, + encodedApiKey, + percentEncodedApiKey, + mixedCaseHex, + unpaddedStandardBase64, + paddedBase64Url, + whitespaceBase64, + base64HeaderName + ]) { + assert.equal(serialized.includes(recoverable), false); + } + assert.equal(captureManager.records.length, bodies.length); + for (const [index, record] of captureManager.records.entries()) { + assert.equal(record.requestBody.length, 0); + assert.equal(record.requestBodyTruncated, true); + assert.equal(record.requestBodyBytes, bodies[index].length); + assert.equal(record.upstreamRequestId, "[REDACTED]"); + } + assert.equal(observedMethods[0], "POST"); + assert.equal(captureManager.records[0].method, "[REDACTED]"); + assert.equal(captureManager.records[0].requestId, "[REDACTED]"); + assert.equal(captureManager.records[0].sessionId, "[REDACTED]"); + assert.equal(captureManager.records[0].threadId, "[REDACTED]"); + assert.equal(captureManager.records[0].requestHeaders["x-client-note"], "[REDACTED]"); + assert.equal(captureManager.records[0].requestHeaders["[REDACTED]"], "[REDACTED]"); + assert.equal(health.captureDbPath, "[REDACTED]"); +}); diff --git a/node/test/worker-manager.test.mjs b/node/test/worker-manager.test.mjs index febce74..015cc04 100644 --- a/node/test/worker-manager.test.mjs +++ b/node/test/worker-manager.test.mjs @@ -22,7 +22,12 @@ function makeSnapshot(generation = 1, port = 15100, providerId = null) { authScheme: "Bearer", extraHeaders: {} }, - proxy: { overrideAuthorization: true, requestIdHeader: "x-client-request-id" }, + proxy: { + overrideAuthorization: true, + requestIdHeader: "x-client-request-id", + modelMode: "passthrough", + modelOverride: null + }, capture: { enabled: false, dbPath: "/tmp/crp-worker-manager/traffic.sqlite3" } } }; diff --git a/node/test/worker-protocol.test.mjs b/node/test/worker-protocol.test.mjs index 61b25da..5146c1a 100644 --- a/node/test/worker-protocol.test.mjs +++ b/node/test/worker-protocol.test.mjs @@ -30,7 +30,9 @@ function makeSettings() { }, proxy: { overrideAuthorization: true, - requestIdHeader: "x-client-request-id" + requestIdHeader: "x-client-request-id", + modelMode: "passthrough", + modelOverride: null }, capture: { enabled: false, @@ -110,6 +112,16 @@ test("configure rejects incomplete, extra, and invalid runtime settings before w emptyApiKey.upstream.apiKey = ""; const invalidExtraHeaders = makeSettings(); invalidExtraHeaders.upstream.extraHeaders = { "x-region": 123 }; + const invalidModelMode = makeSettings(); + invalidModelMode.proxy.modelMode = "automatic"; + const missingModelOverride = makeSettings(); + missingModelOverride.proxy.modelMode = "override"; + const blankModelOverride = makeSettings(); + blankModelOverride.proxy.modelMode = "override"; + blankModelOverride.proxy.modelOverride = " "; + const controlModelOverride = makeSettings(); + controlModelOverride.proxy.modelMode = "override"; + controlModelOverride.proxy.modelOverride = "model\ninvalid"; for (const settings of [ missingServer, @@ -118,7 +130,10 @@ test("configure rejects incomplete, extra, and invalid runtime settings before w invalidPort, invalidTimeout, emptyApiKey, - invalidExtraHeaders + invalidExtraHeaders, + invalidModelMode, + missingModelOverride, + blankModelOverride ]) { assert.throws( () => validateParentMessage({ @@ -132,6 +147,14 @@ test("configure rejects incomplete, extra, and invalid runtime settings before w && !String(error.message).includes("must-not-pass") ); } + + assert.doesNotThrow(() => validateParentMessage({ + version: 1, + type: "configure", + requestId: "configure-legacy-model", + generation: 1, + settings: controlModelOverride + })); }); test("configure enforces provider URL and header security contracts", () => { diff --git a/node/ui-src/src/app.tsx b/node/ui-src/src/app.tsx index 898a0fb..455c86a 100644 --- a/node/ui-src/src/app.tsx +++ b/node/ui-src/src/app.tsx @@ -316,10 +316,12 @@ export function App() { async () => { const test = await api.testProvider(id, model, activateIfNone); if (!test.ok) throw ApiError.fromTestResult(test); - if (switchAfter) await api.activateProvider(id); + if (switchAfter && !activateIfNone) await api.activateProvider(id); return true; }, - switchAfter ? "notice.providerSwitched" : "notice.providerTested" + activateIfNone + ? "notice.providerTested" + : switchAfter ? "notice.providerSwitched" : "notice.providerTested" ); return result === true; }, [api, executeMutation]); diff --git a/node/ui-src/src/i18n.ts b/node/ui-src/src/i18n.ts index 88a9664..558782a 100644 --- a/node/ui-src/src/i18n.ts +++ b/node/ui-src/src/i18n.ts @@ -122,8 +122,11 @@ const en = { "overview.metricsHelp": "Hourly aggregates only. Request content and identifiers are never stored.", "overview.window24": "24 hours", "overview.window7": "7 days", + "overview.window24Detail": "This view contains 24 UTC hourly buckets, including the current partial hour.", + "overview.window7Detail": "This view contains 168 UTC hourly buckets, including the current partial hour.", "overview.requestVolume": "Requests", "overview.successRate": "Success rate", + "overview.successRateUnavailable": "Unavailable because {count} metric updates were dropped", "overview.observedTokens": "Observed tokens", "overview.tokenCoverage": "{value}% usage coverage", "overview.responseStart": "P95 response start", @@ -151,6 +154,11 @@ const en = { "overview.other": "Other", "overview.otherModels": "Other models", "overview.dataQuality": "Data quality", + "overview.dataQualityHelp": "These counters are independent signals and may overlap.", + "overview.unknownModelRequests": "Unknown-model requests", + "overview.modelOverflowRequests": "Grouped model requests", + "overview.providerOverflowRequests": "Grouped provider requests", + "overview.droppedObservations": "Dropped metric updates", "metrics.success": "Success", "metrics.rejected": "Rejected", "metrics.upstreamError": "Upstream error", @@ -455,8 +463,11 @@ const zh: Record = { "overview.metricsHelp": "仅保留每小时聚合数据，绝不存储请求内容和标识符。", "overview.window24": "24 小时", "overview.window7": "7 天", + "overview.window24Detail": "此视图包含 24 个 UTC 小时桶，其中包括当前未结束的小时。", + "overview.window7Detail": "此视图包含 168 个 UTC 小时桶，其中包括当前未结束的小时。", "overview.requestVolume": "请求量", "overview.successRate": "成功率", + "overview.successRateUnavailable": "由于丢失了 {count} 次指标更新，当前不可用", "overview.observedTokens": "已观测 Token", "overview.tokenCoverage": "用量覆盖率 {value}%", "overview.responseStart": "P95 首字节响应", @@ -484,6 +495,11 @@ const zh: Record = { "overview.other": "其他", "overview.otherModels": "其他模型", "overview.dataQuality": "数据质量", + "overview.dataQualityHelp": "这些计数是相互独立且可能重叠的信号。", + "overview.unknownModelRequests": "未知模型请求", + "overview.modelOverflowRequests": "已归组模型请求", + "overview.providerOverflowRequests": "已归组提供商请求", + "overview.droppedObservations": "丢失的指标更新", "metrics.success": "成功", "metrics.rejected": "上游拒绝", "metrics.upstreamError": "上游错误", diff --git a/node/ui-src/src/pages/Overview.tsx b/node/ui-src/src/pages/Overview.tsx index aa3155f..07738ab 100644 --- a/node/ui-src/src/pages/Overview.tsx +++ b/node/ui-src/src/pages/Overview.tsx @@ -119,12 +119,17 @@ export function OverviewPage({ ? metrics.summary.tokens.input + metrics.summary.tokens.output : null; const providerNames = new Map(providers.map((provider) => [provider.id, provider.name])); - const dataQualityTotal = metrics - ? metrics.dataQuality.unknownModelRequests - + metrics.dataQuality.modelOverflowRequests - + metrics.dataQuality.providerOverflowRequests - + metrics.dataQuality.droppedObservations - : 0; + const droppedObservations = metrics?.dataQuality.droppedObservations ?? 0; + const successRateComplete = droppedObservations === 0; + const successRateUnavailable = t("overview.successRateUnavailable", { + count: formatNumber(locale, droppedObservations) + }); + const dataQualitySignals = metrics ? [ + { label: t("overview.unknownModelRequests"), value: metrics.dataQuality.unknownModelRequests }, + { label: t("overview.modelOverflowRequests"), value: metrics.dataQuality.modelOverflowRequests }, + { label: t("overview.providerOverflowRequests"), value: metrics.dataQuality.providerOverflowRequests }, + { label: t("overview.droppedObservations"), value: droppedObservations } + ].filter((signal) => signal.value > 0) : []; const metricsUnavailable = metricsError !== null || metrics === null || metrics.storageState === "unavailable"; @@ -182,7 +187,9 @@ export function OverviewPage({

{t("overview.metricsTitle")}

-

{t("overview.metricsHelp")}

+

{t("overview.metricsHelp")} {t(metricsWindow === "24h" + ? "overview.window24Detail" + : "overview.window7Detail")}

)} - {dataQualityTotal > 0 ? ( -

- {t("overview.dataQuality")}: {formatNumber(locale, dataQualityTotal)} · {t("overview.other")} -

+ {dataQualitySignals.length > 0 ? ( + ) : null} )} diff --git a/node/ui-src/src/pages/Providers.tsx b/node/ui-src/src/pages/Providers.tsx index f122b5d..78bb382 100644 --- a/node/ui-src/src/pages/Providers.tsx +++ b/node/ui-src/src/pages/Providers.tsx @@ -364,7 +364,12 @@ export function ProvidersPage({ const runTest = async () => { if (!selected || !model.trim()) return; - const complete = await onTest(selected.id, model.trim(), switchAfterTest); + const complete = await onTest( + selected.id, + model.trim(), + switchAfterTest, + activeProviderId === null + ); if (complete) close(); }; @@ -442,14 +447,16 @@ export function ProvidersPage({ variant="primary" busy={pending === `provider-switch-${provider.id}`} disabled={readOnly || pending !== null || !provider.credentialConfigured} - onClick={() => eligible + onClick={() => eligible && activeProviderId !== null ? void onActivate(provider.id) : open("test", provider, true)} >