diff --git a/devlog/_fin/260716_grok_build_hardening/000_plan.md b/devlog/_fin/260716_grok_build_hardening/000_plan.md new file mode 100644 index 00000000..efa8e158 --- /dev/null +++ b/devlog/_fin/260716_grok_build_hardening/000_plan.md @@ -0,0 +1,77 @@ +# 000 — Grok Build 대조 기반 xAI OAuth/캐시 경로 강화 로드맵 + +## Objective + +공개된 공식 Grok Build 클라이언트(로컬 미러: `/Users/jun/Developer/codex/180_grok-build`)의 +auth/wire/cache 계약을 기준으로, OpenCodex의 xAI 서빙 경로에서 확인된 4개 High + 2개 +Medium 격차를 닫는다. 근거는 2026-07-16 두 SOL 독립 감사(소스 인용 기반, verdict FAIL)와 +실제 프록시 스모크(GROK_SMOKE_OK, cached_tokens=128)이다. + +## Constraints / Boundaries + +- IN: xAI 경로 한정 — `src/responses/parser.ts`, `src/adapters/openai-chat.ts`, + `src/oauth/{local-token-detect,xai,store,index}.ts`, `src/server/responses.ts`, + `src/providers/xai-transport.ts` + 각 테스트. +- OUT: xAI Responses-native 업스트림 마이그레이션, usage `context_details`/cost-tick, + 타 프로바이더, dashboard UI, `git push`(로컬 커밋만; push는 사용자 승인 필요). +- 실행 중인 ocx 프로세스(127.0.0.1:10100)는 서빙 경로 변경 후 재시작해야 반영된다 + (레포 관례: green tests ≠ live proxy 반영). +- 각 work-phase = 1 PABCD 사이클. 두 decade doc을 한 B에서 구현하지 않는다. + +## Loop-spec (C3, HOTL) + +- Archetype: spec-satisfaction repair (verifier가 done을 정의). +- Trigger: 공식 소스 공개로 계약 격차가 확정됨. +- Goal: 위 5개 격차가 회귀 테스트 + 라이브 스모크로 닫힌 로컬 커밋. +- Non-goals: OUT 항목 전부. +- Verifier: `bun test ` + `bunx tsc --noEmit`(구성 시) + 서빙 phase는 + `curl 127.0.0.1:10100/v1/responses` xai/grok-4.5 completed 스모크. +- Stop: 5 phase 완료(DONE) / 외부 인증 불능(BLOCKED) / ~4h(BUDGET_EXHAUSTED). +- Memory artifact: 본 유닛 decade docs + goalplan ledger. +- Escalation: 저장소 보안 약화가 필요해지면 UNSAFE로 중단. + +## Dependency-ordered phase map + +| Doc | Work-phase | 내용 | 의존 | +|-----|-----------|------|------| +| 010 | wp1_reasoning_fold | [reasoning, assistant] → 단일 assistant + reasoning_content 폴딩, parser-to-wire 회귀 | — (wire 형태가 이후 스모크의 기준) | +| 020 | wp2_localcli_ownership | `~/.grok/auth.json` 임포트 자격증명 단일 소유권(refresh 전 재읽기/세대 채택) | — | +| 030 | wp3_refresh_lock | `~/.opencodex/auth.json` cross-process 락 + 세대 재확인 + 토큰 교환 bounded retry | 020 (소유권 규칙 위에 직렬화) | +| 040 | wp4_401_replay | OAuth xAI 요청 업스트림 401 → singleflight 강제 refresh + 1회 replay | 030 (replay가 락/세대 경로를 사용) | +| 050 | wp5_header_parity | 공식 per-request 헤더 패리티 + 클라이언트 버전 호환 프로파일 | 010 (동일 transport 파일, 순서 고정) | + +효과 기반 버킷 없음 — 각 phase는 이전 phase의 검증된 산출물을 소비한다(PHASE-SPLIT-01). + +## 공식 계약 앵커 (감사 확정) + +- folding: `xai-grok-sampling-types/src/conversation.rs:1814` (+회귀 8413) +- auth 단일 소유/락: `xai-grok-shell/src/auth/manager.rs:1529,1560,1604` +- 401 회복: `xai-grok-shell/src/auth/recovery.rs:440`, `manager_tests.rs:1293` +- retry: `xai-grok-sampler/src/retry.rs:14,210,225` +- 헤더: `xai-grok-sampler/src/client.rs:43-70,485` + +## SoT sync target (SOT-SYNC-01) + +확정: 레포에 CHANGELOG 파일과 xAI 전용 docs 문서는 없음. SoT는 +`structure/04_transports-and-sidecars.md`(전송/사이드카 구조 문서)로 하고, +xAI 전송 계층의 락/401 replay/헤더 프로파일 변경을 각 phase D에서 반영한다. +`README.md:202`의 xAI 표는 프로토콜/모드 변경이 없는 한 그대로 둔다. + +## Verification commands (확정) + +- `bun test --isolate ./tests/xai-transport.test.ts` (+ phase별 신규 테스트 파일) +- `bun run typecheck` (= `bun x tsc --noEmit`) +- 서빙 phase 라이브 스모크: ocx 재시작 후 + `curl -sS 127.0.0.1:10100/v1/responses -d '{"model":"xai/grok-4.5",...}'` → `status: completed` + +## Acceptance criteria (활성화 시나리오 포함) + +- c1: 새 parser-to-wire 테스트가 `[reasoning(summary), assistant]`와 + `[reasoning(encrypted-only), assistant]` 두 입력에서 assistant 메시지 1개를 단언. + 활성화: 테스트가 parser 경계를 실제 통과(기존 테스트처럼 수동 조립 금지). +- c2: Grok 저장소에 더 새로운 refresh 세대가 있을 때 OpenCodex가 IdP 호출 없이 채택. + 활성화: temp HOME에 모의 `~/.grok/auth.json` 세대 교체 후 refresh 경로 호출. +- c3: 동시 refresh 2건 → 토큰 엔드포인트 호출 1건. 활성화: mock fetch 카운터. +- c4: mock 업스트림 401 → refresh 1회 + replay 1회, 두 번째 401은 그대로 전파. + 활성화: 서빙 경로 통합 테스트 + ocx 재시작 후 라이브 스모크 completed. +- c5: 아웃바운드 헤더 스냅샷 테스트 + 라이브 스모크 completed. diff --git a/devlog/_fin/260716_grok_build_hardening/001_audit_synthesis.md b/devlog/_fin/260716_grok_build_hardening/001_audit_synthesis.md new file mode 100644 index 00000000..89581663 --- /dev/null +++ b/devlog/_fin/260716_grok_build_hardening/001_audit_synthesis.md @@ -0,0 +1,52 @@ +# 001 — A-round 1 감사 합성 (reviewer: sol/Aristotle, VERDICT: FAIL, blockers 9) + +전 블로커 수용. 반박 없음. 문서별 처분: + +| # | 대상 | 결정 | 수정 방향 | +|---|------|------|-----------| +| 1 | 010 | 수용 | pendingReasoning의 경계 규칙 완전 정의: user/tool-result/agent 경계에서 clear, function/tool call은 보존(공식 BackendToolCall 규칙), 복수 reasoning sibling은 공식대로 newline join, trailing reasoning은 폐기. 각 경계 활성화 테스트 추가 | +| 2 | 020 | 수용 | placeholder 주석 제거, 완전한 before/after 블록, `hasComparableGrokIdentity` 정의·export, Kiro/merge 분기 명시 보존 | +| 3 | 020 | 수용 | "newer" 판정을 단조 규칙으로 교체: authority = 더 늦은 `expiresAt` 세대(고정 수명 토큰에서 늦은 만료 = 늦은 발급), 동률이면 disk(Grok 소유) 우선. 단순 불일치를 newer로 간주 금지. stale-different-generation 테스트 추가 | +| 4 | 030 | 수용 | 락 획득/해제, stale takeover, retry 루프, index 통합, 테스트 본문 전부 컴파일 가능한 코드로 | +| 5 | 030 | 수용 | persist 직전 store-version compare-and-retry(CAS): 락 진입 시 스냅샷한 파일 fingerprint와 persist 직전 재읽기 비교, 불일치 시 재병합 후 기록. 무관 계정 mutate 경쟁 테스트 추가 | +| 6 | 030+040 | 수용 | 세대 표현 통일: opaque string fingerprint(SHA-256) + equality 비교. numeric/greater-than 제거 | +| 7 | 040 | 수용 | `...`/prose 블록을 완전한 코드로. 네트워크 seam은 030 확정 심볼 기준으로 서술하되 stale-check에 rebase 의무 유지 | +| 8 | 050 | 수용 | `x-grok-req-id` 생성을 transport resolve 시점이 아니라 outbound attempt(fetch wrapper 내부) 시점으로 이동. retry/sidecar 경로에서 attempt별 상이 ID + 고정 session/conv ID 테스트 | +| 9 | 050 | 수용 | 테스트를 산문 지시가 아닌 완전한 교체 코드로 | +| 10 | 020-050 | 수용(Medium) | 각 문서에 Risk/rollback 섹션 추가(재태깅·adopt·needsReauth 상태의 롤백 의미 포함) | + +리뷰어 확인 사항(유지): 공식 앵커 전부 실재, 401은 recovery loop에 실제 도달(사이드카 경로는 미커버로 명시), 020 Grok 재읽기는 030 락과 재귀 없음. + +## Round 2 (VERDICT: FAIL, blockers 5 — 1,2,6,7,9,10 해소 확인) + +전 블로커 수용. 반박 없음. + +| # | 대상 | 결정 | 수정 방향 | +|---|------|------|-----------| +| R2-1 | 030 | 수용 | authority 규칙 자체 재구현 금지: 020의 `shouldAdoptGrokGeneration(stored, disk, now, REFRESH_SKEW_MS)`를 import해 사용. 사용 불능(skew창 내) disk 세대는 refresh 입력 금지. later-but-unusable/equal-expiry/missing-expiry 조합 테스트 추가 | +| R2-2 | 030 | 수용 | 읽기-재병합-원자적 rename은 CAS가 아님. 결정: 모든 whole-store writer(`persist` 경유 전부)가 동일 글로벌 락을 짧게 획득(로그인·계정전환 포함, 쓰기 빈도 낮아 비용 무시 가능), refresh는 교환 전체 동안 보유. CAS-read 직후 결정적 테스트 seam 추가, 무관 writer 경쟁 테스트를 그 seam에서 수행 | +| R2-3 | 030 | 수용 | cleanup/release 전 exact-byte/stat 스냅샷 검증(스탈 takeover와 동일 기법). 부분 기록 실패·release-검사-unlink 사이 교체 테스트 추가 | +| R2-4 | 050 | 수용 | adapter-level fetchResponse hook이 서버의 `fetchWithHeaderTimeout`(responses.ts:1228) 경로를 우회함. req-id 주입을 timeout-capable fetch 경계 안쪽(transport fetch wrapper)으로 이동, `ctx.timeoutMs`/`ctx.stream` 의미 보존. timeout 활성화 테스트 추가 | +| R2-5 | 010 | 수용 | 서명 무결성: newline join은 unsigned plaintext sibling에만 적용. `ocxr1` 서명 sibling은 각각 독립 thinking part로 보존(서명↔텍스트 대응 유지). 서명 2개 sibling의 Anthropic replay 테스트 추가 | + +## Round 3 (VERDICT: FAIL, blockers 3 — R2-1/R2-3/R2-5 해소 확인) + 아키텍처 재계획 + +LOOP-REPAIR-01 3회 도달 → 메인 에이전트가 계획을 구조적으로 변경(단순 재수정 아님). + +| # | 대상 | 결정 | 아키텍처 변경 | +|---|------|------|---------------| +| R3-1,2 | 030 | 수용 | 글로벌 동기 락 폐기. **이원 락 설계**로 교체: (a) refresh-intent 락 — provider+account 단위 락 파일, IdP 교환 전체 동안 비동기 보유(이중 소비 방지); (b) store-write 락 — 짧게 잡는 글로벌 락, load-merge-persist 임계구역만 보호, 네트워크 I/O 절대 포함 금지. 모든 whole-store writer는 async 직렬 큐 + store-write 락 경유(동기 `Atomics.wait` 금지 — Bun 이벤트 루프 차단). 기존 동기 호출자는 async로 전파 전환(호출부 실측 후 명시). refresh 흐름: intent 락 → 세대 재확인 → 교환 → store-write 락 안에서 재읽기·병합·persist → 해제. 동일 프로세스 writer-during-refresh 활성화 테스트 필수 | +| R3-3 | 050 | 수용 | 서버 seam을 050이 명시 소유: `fetchWithHeaderTimeout`(responses.ts:1228)에 executor 파라미터 추가(기본 global fetch, xAI route는 `route.provider.fetch` 전달), ordinary/recovery 호출자 전부 before/after 제시. 테스트는 실제 서버 경로 통과로 UUID 회전 + TimeoutError 증명. 040과 responses.ts 중복은 stale-check rebase 의무로 관리 | + +## Round 4 (VERDICT: FAIL, blockers 1 — R3 3건 전부 해소 확인) + +| # | 대상 | 결정 | 수정 방향 | +|---|------|------|-----------| +| R4-1 | 030 | 수용(메인 직접 수정) | 교환 중 동일 계정 write 경쟁: `mergeAccountCredential`에 `expectedGeneration` 가드 추가 — 저장 세대가 시도 세대와 다르면 덮어쓰지 않고 superseded 반환, 사용 가능하면 채택/불가하면 fail-closed(`OAuthLoginRequiredError`, 제3자 세대 재귀 소비 금지 근거 기록). `needsReauth`는 `markAccountNeedsReauthIfGeneration`으로 세대 조건부. 성공/터미널 실패 양쪽의 replacement-during-exchange 테스트 추가 | + +## Round 5 (blockers 2 — R4-1 본체 해소 확인) + +| # | 대상 | 결정 | 수정 방향 | +|---|------|------|-----------| +| R5-1 | 030 | 수용(메인 직접 수정) | 020-adoption 분기(pre-IdP)에도 동일 `expectedGeneration` 가드 적용: superseded 시 usable→채택, 아니면 fail-closed. adoption 경쟁 테스트 방침 기록 | +| R5-2 | 030 | 수용(메인 직접 수정) | 테스트 코드 결함: `XaiTokenRequestError` 인자 순서 `(status, oauthError, message)`로 정정, 가상 픽스처 별칭 제거 — 파일 내 실제 헬퍼(`seed`/`def`/`saveCredential`/`getAccountSet`)로 재작성 | diff --git a/devlog/_fin/260716_grok_build_hardening/010_reasoning_fold.md b/devlog/_fin/260716_grok_build_hardening/010_reasoning_fold.md new file mode 100644 index 00000000..fd74fbf2 --- /dev/null +++ b/devlog/_fin/260716_grok_build_hardening/010_reasoning_fold.md @@ -0,0 +1,584 @@ +# 010 — Responses reasoning을 후속 assistant에 fold + +## Work-phase outcome + +`parseRequest`가 Responses 입력의 `[reasoning, assistant message]`를 두 개의 +`OcxAssistantMessage`로 분리하지 않고, reasoning part를 바로 뒤 assistant의 `content` 앞에 +붙인다. 따라서 Grok reasoning 모델을 위한 `preserveReasoningContentModels`가 활성화된 경우 +chat-completions wire에는 assistant가 정확히 하나만 나가며 그 메시지에 +`reasoning_content`와 answer `content`가 함께 존재한다. + +공식 기준은 +`/Users/jun/Developer/codex/180_grok-build/crates/codegen/xai-grok-sampling-types/src/conversation.rs:1814-1860`의 +pending-reasoning fold와 `:8413-8446` 회귀 테스트다. OpenCodex의 현재 분리 지점은 +`src/responses/parser.ts:324-350`, 독립 직렬화 지점은 +`src/adapters/openai-chat.ts:104-134`, opaque encrypted blob의 decode 한계는 +`src/responses/reasoning-envelope.ts:34-52`다. + +## IN / OUT + +IN: + +- `parseRequest`의 바로 앞 reasoning → 후속 assistant message fold. +- summary/content 또는 `ocxr1` envelope에서 복구 가능한 thinking text 보존. +- native/non-`ocxr1` encrypted-only reasoning을 assistant placeholder 없이 폐기. +- `parseRequest`부터 Grok chat wire까지 통과하는 여섯 개의 회귀 테스트와, 두 signed + sibling을 Anthropic wire까지 replay하는 activation test. + +OUT: + +- `openai-chat.ts`의 `reasoning_content` 직렬화 정책 변경. +- `reasoning-envelope.ts`에서 native OpenAI/xAI encrypted blob 복호화. +- Anthropic envelope 형식, provider registry, production runtime 변경. + +## File change map + +| Marker | Path | Change | +|---|---|---| +| MODIFY | `src/responses/parser.ts` | pending reasoning을 만들되 메시지를 즉시 생성하지 않고, 다음 assistant message에 prepend | +| MODIFY | `tests/xai-transport.test.ts` | 수동 `OcxAssistantMessage` 테스트를 parser-to-wire 회귀 여섯 건으로 교체 | +| MODIFY | `tests/anthropic-thinking-signature.test.ts` | 두 signed sibling의 text/signature 대응을 parser-to-Anthropic wire에서 고정 | +| READ ONLY | `src/adapters/openai-chat.ts` | 기존 thinking → `reasoning_content` 변환을 그대로 소비 | +| READ ONLY | `src/adapters/anthropic.ts` | 각 signed thinking part를 독립 block으로 replay하는 기존 변환을 그대로 소비 | +| READ ONLY | `src/responses/reasoning-envelope.ts` | `decodeReasoningEnvelope(...) === null`인 opaque blob 판정 계약을 그대로 소비 | + +## Diff specification + +### MODIFY — `src/responses/parser.ts` + +#### 1. Pending state 추가 + +`parseRequest`의 현재 local state(`src/responses/parser.ts:227-236`)에 한 turn짜리 pending +reasoning entry list를 추가한다. 각 entry는 thinking part와 `ocxr1` envelope signature의 +실재 여부를 함께 기록한다. 이 boolean은 `signature` 문자열 자체로 재추론하지 않는다. +현재 parser가 non-envelope reasoning에 넣는 `JSON.stringify(reasoning)` synthetic signature도 +유지하지만, join 분류에서는 **unsigned**다. 오직 `envelope?.sig`가 있는 part만 signed다. +새 helper/type/file은 만들지 않는다. + +Before: + +```ts + const messages: OcxMessage[] = []; + const systemPrompt: string[] = []; + // Tool specs surfaced by a prior tool_search (deferred tools, e.g. subagents). Codex does not + // re-list these in `tools`, but chat models can only call listed tools — so we re-inject them. + const loadedToolSpecs: unknown[] = []; +``` + +After: + +```ts + const messages: OcxMessage[] = []; + const systemPrompt: string[] = []; + // Responses reasoning siblings belong to the following assistant, including across call items. + // Keep them off the message list until that assistant arrives; turn boundaries clear the array. + const pendingReasoning: Array<{ part: OcxThinkingContent; envelopeSigned: boolean }> = []; + // Tool specs surfaced by a prior tool_search (deferred tools, e.g. subagents). Codex does not + // re-list these in `tools`, but chat models can only call listed tools — so we re-inject them. + const loadedToolSpecs: unknown[] = []; +``` + +#### 2. Assistant message가 pending reasoning을 소비 + +현재 assistant branch(`src/responses/parser.ts:324-327`)에서 pending part를 text보다 앞에 +prepend하고 즉시 clear한다. 순서는 wire cache prefix와 `thinking`/answer 결합을 고정한다. + +Before: + +```ts + case "assistant": { + const parts = outputTextOf(msg.content as unknown[] | string | undefined); + messages.push({ role: "assistant", content: parts, model: data.model, timestamp: now }); + break; + } +``` + +After: + +```ts + case "assistant": { + const parts = outputTextOf(msg.content as unknown[] | string | undefined); + messages.push({ + role: "assistant", + content: pendingReasoning.length > 0 + ? [...pendingReasoning.map(entry => entry.part), ...parts] + : parts, + model: data.model, + timestamp: now, + }); + pendingReasoning.length = 0; + break; + } +``` + +#### 3. Reasoning branch는 decodable text만 pending으로 보존 + +현재 reasoning branch(`src/responses/parser.ts:333-350`)는 빈 thinking까지 +`ensureAssistantPlaceholder`에 넣는다. 이를 다음 규칙으로 교체한다. + +- 보존: summary text 우선, 없으면 content text, 없으면 `ocxr1` envelope의 `txt`. +- 함께 보존: decodable `ocxr1`의 `sig`, `red`, reasoning `id`. +- 폐기: summary/content가 비고 `decodeReasoningEnvelope`가 `null`인 native/non-`ocxr1` + encrypted-only blob. 원문 encrypted bytes와 빈 thinking part는 wire에 남기지 않는다. + plaintext가 있는 non-envelope reasoning의 기존 synthetic JSON signature는 유지하되 unsigned로 분류한다. +- `envelope?.sig`가 있는 reasoning은 signed entry로 append하며 절대 다른 sibling과 join하지 + 않는다. signature, redacted, itemId, thinking text의 대응을 그대로 보존한다. +- `envelope?.sig`가 없는 reasoning은 unsigned다. 바로 앞 pending entry도 unsigned일 때만 + `"\n"`으로 합쳐 하나의 part로 만든다. signed entry가 사이에 있으면 새 unsigned part를 + 시작한다. 합쳐진 unsigned part의 metadata는 마지막 sibling 것을 사용한다. +- non-envelope reasoning의 `JSON.stringify(reasoning)` signature는 기존 adapter rejection + contract를 위해 part에 남지만 synthetic signature이므로 unsigned join 대상이다. + +Before: + +```ts + if (effectiveType === "reasoning") { + const reasoning = item as { id?: string; summary?: { text: string }[]; content?: { text: string }[]; encrypted_content?: string }; + const fromSummary = (reasoning.summary ?? []).map(c => c.text).join(""); + const text = fromSummary || (reasoning.content ?? []).map(c => c.text).join(""); + // ocxr1 envelope: the REAL Anthropic signature (+ redacted blocks, + hidden signed text) + // captured by the bridge. Native OpenAI-encrypted blobs decode to null and keep today's + // placeholder signature (which the anthropic adapter correctly rejects on replay). + const envelope = typeof reasoning.encrypted_content === "string" + ? decodeReasoningEnvelope(reasoning.encrypted_content) + : null; + const thinking: OcxThinkingContent = { + type: "thinking", + thinking: envelope?.txt || text, + signature: envelope?.sig ?? JSON.stringify(reasoning), + ...(envelope?.red ? { redacted: envelope.red } : {}), + ...(reasoning.id ? { itemId: reasoning.id } : {}), + }; + ensureAssistantPlaceholder(messages, data.model, now).content.push(thinking); + continue; + } +``` + +After: + +```ts + if (effectiveType === "reasoning") { + const reasoning = item as { id?: string; summary?: { text: string }[]; content?: { text: string }[]; encrypted_content?: string }; + const fromSummary = (reasoning.summary ?? []).map(c => c.text).join(""); + const text = fromSummary || (reasoning.content ?? []).map(c => c.text).join(""); + const envelope = typeof reasoning.encrypted_content === "string" + ? decodeReasoningEnvelope(reasoning.encrypted_content) + : null; + const thinkingText = envelope?.txt || text; + + // Native/non-ocxr1 encrypted-only reasoning is opaque here. Do not create a detached + // assistant turn or invent replayable plaintext/signatures from the encrypted payload. + if (thinkingText.length > 0) { + const part: OcxThinkingContent = { + type: "thinking", + thinking: thinkingText, + signature: envelope?.sig ?? JSON.stringify(reasoning), + ...(envelope?.red ? { redacted: envelope.red } : {}), + ...(reasoning.id ? { itemId: reasoning.id } : {}), + }; + const envelopeSigned = typeof envelope?.sig === "string"; + const previous = pendingReasoning[pendingReasoning.length - 1]; + + if (!envelopeSigned && previous && !previous.envelopeSigned) { + previous.part = { + ...part, + thinking: `${previous.part.thinking}\n${part.thinking}`, + }; + } else { + pendingReasoning.push({ part, envelopeSigned }); + } + } + continue; + } +``` + +#### 4. 모든 후속 item class의 pending 규칙 + +`src/responses/parser.ts:245-479`에서 reasoning 뒤 도달 가능한 class와 정확한 동작은 다음과 +같다. `compaction` 계열은 user-role history를 만들므로 user boundary와 동일하게 clear한다. +`additional_tools`/`compaction_trigger`는 conversation item이 아니므로 pending을 소비하거나 +clear하지 않는다. + +| 다음 item class | 현재 동작 | 변경 후 pending 동작 | +|---|---|---| +| assistant message | 새 assistant message push | signed는 개별 유지하고 연속 unsigned만 newline-join한 parts를 prepend한 뒤 clear | +| user/developer message | user/developer message push | push 전에 clear | +| system message | `systemPrompt`에 append | append 전에 clear | +| agent message | user-role external turn push | push 전에 clear | +| function/custom/local-shell/tool-search call | `ensureAssistantPlaceholder`가 assistant tool-call placeholder를 생성/재사용 | placeholder에는 call만 넣고 pending은 보존; 다음 explicit assistant가 reasoning을 소비 | +| web-search call | placeholder에 synthetic assistant text append | 공식 BackendToolCall과 같은 call-side evidence이므로 pending 보존 | +| function/custom/tool-search output | toolResult push | push 전에 clear | +| another reasoning | 현재 placeholder에 sibling append | signed면 별도 append; 바로 앞 entry와 모두 unsigned일 때만 `"\n"` merge | +| end-of-input | 현재 reasoning placeholder가 남음 | flush하지 않고 pending array를 자연 폐기; 대응 assistant가 없는 취소/고아 reasoning은 chat-completions 표현이 없으므로 drop | + +Boundary branches에는 push/append보다 먼저 아래 한 줄을 넣는다. + +```ts +pendingReasoning.length = 0; +``` + +정확한 삽입 지점은 다음과 같다. + +- `compaction`/`compaction_summary`/payload가 있는 `context_compaction`: `messages.push` 직전. +- `agent_message`: `messages.push` 직전. +- `message`의 `system`, `user`, `developer`: 각 branch의 content 변환 전. +- `tool_search_output`, `function_call_output`, `custom_tool_call_output`: output 해석 및 + `findToolById` 전. tool-result boundary가 pending을 뒤 assistant로 넘기지 않는 것이 우선이다. + +Call branch(`function_call`, `custom_tool_call`, `local_shell_call`, `web_search_call`, +`tool_search_call`)에는 clear 코드를 추가하지 않는다. 현재 parser의 placeholder consumer는 +tool call을 별도 assistant history item에 유지하는 역할만 한다. 따라서 +`[reasoning, function_call, assistant]`는 `[{assistant toolCall}, {assistant thinking+answer}]`로 +파싱되고 pending reasoning은 call placeholder에 붙지 않은 채 explicit assistant까지 보존된다. +이는 공식 `[Reasoning, BackendToolCall, Assistant]` 보존 규칙과 일치한다. + +#### 5. Multiple thinking part serializer 확인 + +- OpenAI Chat은 assistant content에서 thinking part를 원래 순서대로 filter한 뒤 text를 + `join("")`하여 하나의 `reasoning_content`로 만든다 + (`src/adapters/openai-chat.ts:104-115`). Parser가 unsigned sibling 내부에 정확히 하나의 + newline을 넣으므로 Grok wire는 `first\nsecond`가 되고, signed part가 여러 개여도 이 + adapter에서는 part 순서대로 자연스럽게 concatenate된다. 이 phase에서 adapter 변경은 없다. +- Anthropic은 assistant content를 순서대로 순회한다 (`src/adapters/anthropic.ts:396-420`). + 각 thinking part의 redacted payload를 먼저 replay하고 (`:404-410`), 그 **동일 part**의 + signature가 real-signature gate를 통과하면 `{type:"thinking", thinking, signature}`를 + 독립 block으로 push한다 (`:411-413`). 따라서 parser가 두 signed sibling을 두 part로 + 유지하면 각 signature가 자기 text에 붙어 원래 순서대로 replay된다. 빈 assistant만 + 제외하고 완성된 content 배열을 그대로 push하는 지점은 `:421-422`다. + +### MODIFY — `tests/xai-transport.test.ts` + +Replace the current manually assembled test at `tests/xai-transport.test.ts:239-263`. It bypasses +the parser boundary and therefore stays green while parser output is split. + +Before: + +```ts + test("assistant thinking parts round-trip as reasoning_content on grok-4.5 history", () => { + // xAI docs: dropped reasoning_content is the top cause of multi-turn cache misses + // (docs.x.ai prompt-caching/multi-turn, 2026-07-13). + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const assistant: OcxAssistantMessage = { + role: "assistant", + content: [ + { type: "thinking", thinking: "cached chain" }, + { type: "text", text: "answer" }, + ], + timestamp: 0, + }; + const req: OcxParsedRequest = { + modelId: "grok-4.5", + context: { messages: [{ role: "user", content: "q1", timestamp: 0 }, assistant, { role: "user", content: "q2", timestamp: 0 }] }, + stream: false, + options: {}, + }; + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const replayed = body.messages.find(m => m.role === "assistant"); + expect(replayed?.reasoning_content).toBe("cached chain"); + }); +``` + +After: + +```ts + test("parseRequest folds summary reasoning into one Grok assistant wire message", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const req = parseRequest({ + model: "xai/grok-4.5", + input: [ + { type: "message", role: "user", content: [{ type: "input_text", text: "q1" }] }, + { type: "reasoning", id: "r1", summary: [{ type: "summary_text", text: "cached chain" }] }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "q2" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toHaveLength(1); + expect(assistants[0]).toMatchObject({ content: "answer", reasoning_content: "cached chain" }); + }); + + test("parseRequest drops opaque encrypted-only reasoning without detaching an assistant wire message", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const req = parseRequest({ + model: "xai/grok-4.5", + input: [ + { type: "message", role: "user", content: [{ type: "input_text", text: "q1" }] }, + { type: "reasoning", id: "r-opaque", summary: [], encrypted_content: "opaque-native-blob" }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "q2" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toHaveLength(1); + expect(assistants[0]).toEqual({ role: "assistant", content: "answer" }); + expect(assistants[0]).not.toHaveProperty("reasoning_content"); + }); + + test("parseRequest clears pending reasoning at a user boundary", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const req = parseRequest({ + model: "xai/grok-4.5", + input: [ + { type: "reasoning", id: "r-orphan", summary: [{ type: "summary_text", text: "must drop" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "new turn" }] }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toEqual([{ role: "assistant", content: "answer" }]); + expect(assistants[0]).not.toHaveProperty("reasoning_content"); + }); + + test("parseRequest preserves pending reasoning across a function call", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const req = parseRequest({ + model: "xai/grok-4.5", + input: [ + { type: "reasoning", id: "r-call", summary: [{ type: "summary_text", text: "call chain" }] }, + { type: "function_call", call_id: "call_1", name: "lookup", arguments: "{\"q\":\"x\"}" }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toHaveLength(2); + expect(assistants[0]).toMatchObject({ + content: "", + tool_calls: [{ id: "call_1", type: "function", function: { name: "lookup", arguments: "{\"q\":\"x\"}" } }], + }); + expect(assistants[1]).toMatchObject({ content: "answer", reasoning_content: "call chain" }); + }); + + test("parseRequest newline-joins reasoning siblings before one assistant", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const req = parseRequest({ + model: "xai/grok-4.5", + input: [ + { type: "reasoning", id: "r1", summary: [{ type: "summary_text", text: "first" }] }, + { type: "reasoning", id: "r2", summary: [{ type: "summary_text", text: "second" }] }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + const parsedAssistant = req.context.messages.find(message => message.role === "assistant") as OcxAssistantMessage; + const thinkingParts = parsedAssistant.content.filter(part => part.type === "thinking"); + + expect(thinkingParts).toHaveLength(1); + expect(thinkingParts[0]).toMatchObject({ thinking: "first\nsecond", itemId: "r2" }); + expect(assistants).toHaveLength(1); + expect(assistants[0]).toMatchObject({ content: "answer", reasoning_content: "first\nsecond" }); + }); + + test("parseRequest drops trailing reasoning without creating an assistant", () => { + const req = parseRequest({ + model: "xai/grok-4.5", + input: [ + { type: "message", role: "user", content: [{ type: "input_text", text: "q1" }] }, + { type: "reasoning", id: "r-trailing", summary: [{ type: "summary_text", text: "unfinished" }] }, + ], + }); + + expect(req.context.messages.filter(message => message.role === "assistant")).toHaveLength(0); + expect(req.context.messages).toHaveLength(1); + }); +``` + +After replacement, remove now-unused type imports only if TypeScript reports them: +`OcxParsedRequest`. Keep `OcxAssistantMessage` for the one-part parser assertion and keep +`OcxProviderConfig`. + +### MODIFY — `tests/anthropic-thinking-signature.test.ts` + +Add this activation inside `describe("parser ocxr1 decode + anthropic replay", ...)`. It goes +through `parseRequest`, confirms two parser parts, then checks the actual Anthropic request blocks; +it fails if sibling plaintext is joined while only the final signature survives. + +같은 describe의 기존 parser decode/replay fixture도 stale-check한다. Fold 구현 후에는 +`reasoning` 바로 뒤 `user`가 boundary clear이므로, signed thinking을 검사하는 fixture는 +`reasoning`과 다음 `user` 사이에 explicit assistant message를 두어 실제 Responses ordering을 +활성화해야 한다. Assertions의 signature/redacted/text 의미는 유지한다. + +```ts + test("two signed reasoning siblings replay with each signature attached to its own text", async () => { + const adapter = createAnthropicAdapter(provider); + const firstEnvelope = encodeReasoningEnvelope({ + sig: "FirstRealSignature123456==", + txt: "first signed chain", + }); + const secondEnvelope = encodeReasoningEnvelope({ + sig: "SecondRealSignature123456==", + txt: "second signed chain", + }); + const parsed = parseRequest({ + model: "anthropic/claude-x", + input: [ + { type: "reasoning", id: "rs_first", summary: [], encrypted_content: firstEnvelope }, + { type: "reasoning", id: "rs_second", summary: [], encrypted_content: secondEnvelope }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "next" }] }, + ], + }); + const parsedAssistant = parsed.context.messages.find(message => message.role === "assistant") as { + content: OcxThinkingContent[]; + }; + const parsedThinking = parsedAssistant.content.filter(part => part.type === "thinking"); + + expect(parsedThinking).toHaveLength(2); + expect(parsedThinking.map(part => ({ + thinking: part.thinking, + signature: part.signature, + }))).toEqual([ + { thinking: "first signed chain", signature: "FirstRealSignature123456==" }, + { thinking: "second signed chain", signature: "SecondRealSignature123456==" }, + ]); + + const request = await adapter.buildRequest(parsed) as { body: string }; + const body = JSON.parse(request.body) as { + messages: Array<{ + role: string; + content: Array<{ type: string; thinking?: string; signature?: string; text?: string }>; + }>; + }; + const replayedAssistant = body.messages.find(message => message.role === "assistant"); + const replayedThinking = replayedAssistant?.content.filter(block => block.type === "thinking"); + + expect(replayedThinking).toEqual([ + { type: "thinking", thinking: "first signed chain", signature: "FirstRealSignature123456==" }, + { type: "thinking", thinking: "second signed chain", signature: "SecondRealSignature123456==" }, + ]); + }); +``` + +## Acceptance criteria and activation scenarios + +1. **Summary fold** — activate with input ordering + `[user, reasoning(summary="cached chain"), assistant("answer"), user]` through + `parseRequest`; Grok wire has exactly one assistant, with + `{content:"answer", reasoning_content:"cached chain"}`. +2. **Opaque encrypted-only drop** — activate with + `[user, reasoning(summary=[], encrypted_content="opaque-native-blob"), assistant("answer"), user]`; + Grok wire has exactly one assistant with answer content and no `reasoning_content`. +3. **User boundary clear** — `[reasoning, user, assistant]` drops the orphaned reasoning and the + later assistant has no `reasoning_content`. +4. **Call preservation** — `[reasoning, function_call, assistant]` keeps the existing assistant + tool-call placeholder and folds reasoning only into the following explicit assistant. +5. **Sibling join** — `[reasoning("first"), reasoning("second"), assistant]` emits + exactly one parser thinking part and `reasoning_content: "first\nsecond"`. +6. **Trailing drop** — `[user, reasoning]` creates no assistant placeholder and leaves only user history. +7. **Signed sibling integrity** — two `ocxr1` siblings with distinct `sig`/`txt` values before + one assistant remain two parser thinking parts, and Anthropic emits two thinking blocks in the + same order with each signature attached to its original text. +8. **No detached parser message** — fold/drop activations assert the expected assistant count; + `req.context.messages.filter(m => m.role === "assistant")` may additionally be asserted as + length 1; the required wire assertion remains authoritative. +9. **Preservation boundary** — decodable plaintext/`ocxr1` thinking metadata remains on the + following `OcxAssistantMessage`; only `envelope?.sig` marks a signed part. Synthetic JSON + signatures remain unsigned for joining; undecodable encrypted bytes and invented plaintext are dropped. +10. **No adapter regression** — registry preset and Anthropic signature suites remain green; no + change is made to `src/adapters/openai-chat.ts` or `src/adapters/anthropic.ts`. + +Exact new test names: + +- `parseRequest folds summary reasoning into one Grok assistant wire message` +- `parseRequest drops opaque encrypted-only reasoning without detaching an assistant wire message` +- `parseRequest clears pending reasoning at a user boundary` +- `parseRequest preserves pending reasoning across a function call` +- `parseRequest newline-joins reasoning siblings before one assistant` +- `parseRequest drops trailing reasoning without creating an assistant` + +Anthropic activation test name: + +- `two signed reasoning siblings replay with each signature attached to its own text` + +## Verification commands + +Run from `/Users/jun/Developer/new/700_projects/opencodex`: + +```bash +bun test tests/xai-transport.test.ts tests/anthropic-thinking-signature.test.ts +bun run typecheck +``` + +Optional affected-suite confidence gate after the focused commands: + +```bash +bun test tests/responses-parser.test.ts tests/xai-transport.test.ts tests/anthropic-adapter.test.ts tests/anthropic-thinking-signature.test.ts +``` + +If an optional path does not exist at implementation time, use `rg --files tests` to select the +actual parser/Anthropic replay suites; do not create a placeholder suite merely to satisfy this doc. + +## Risk and rollback + +Primary risk is assistant-history grouping: reasoning currently uses +`ensureAssistantPlaceholder`, so function/custom/local-shell/tool-search calls may rely on that +placeholder to share a turn. Parser tests involving reasoning adjacent to tool calls, signed +Anthropic thinking replay, trailing reasoning, or user/tool boundaries could fail. The most likely +affected suites are parser tests, Anthropic extended-thinking/signature tests, and +`tests/xai-transport.test.ts`; run the optional gate above after locating their current names. + +Secondary risk is semantic loss: native encrypted-only reasoning is intentionally unavailable to +routed chat models. This phase drops that opaque reasoning item while preserving the following +assistant answer. It must never serialize the blob as `reasoning_content`, expose it as text, or +manufacture a detached empty assistant. + +Rollback is a three-file revert of this work-phase: restore immediate +`ensureAssistantPlaceholder(...).content.push(thinking)`, restore the prior manual Grok wire test, +and restore the pre-fold Anthropic parser fixtures. +Rollback restores previous compatibility but also restores the Grok exact-prefix cache break, so +record that regression explicitly in the revert/commit message. + +## Stale-check checklist for implementing P + +- Re-read the full `parser.ts` input loop and confirm every assistant, user/developer/system, + agent-message, call, tool-output, reasoning, and end-of-input path still matches the boundary table. +- Confirm all call-like branches still use `ensureAssistantPlaceholder`; preserve pending across calls, + and clear it before every user/tool-result/agent/system boundary push or append. +- Confirm sibling reasoning still joins with exactly one newline and trailing pending state is never flushed. +- Confirm only `envelope?.sig` sets `envelopeSigned`; synthetic `JSON.stringify(reasoning)` signatures + remain unsigned, adjacent unsigned siblings become one part, and signed siblings remain separate parts. +- Confirm `openai-chat.ts:104-115` still concatenates multiple thinking parts in original order and + forwards the non-empty result only for `preserveReasoningContentModels`. +- Confirm `anthropic.ts:396-422` still iterates multiple thinking parts in order and emits every + real signature on the same block as that part's own thinking text. +- Confirm `decodeReasoningEnvelope` still returns `null` for non-`ocxr1` blobs. +- Validate all seven proposed request fixtures against the current Responses schema before editing. +- Re-check current test filenames/imports and `package.json` `test`/`typecheck` scripts. + +## Implementation record (B, 2026-07-16) + +구현은 본 문서대로 착지했고, 모호했던 "call 항목을 가로지르는 보존"은 다음처럼 확정했다: +pending reasoning은 tool/function/shell/web/tool_search call이 assistant placeholder를 만들 때 +그 **동일 assistant turn에 폴딩**된다(`assistantHolderWithReasoning` helper). 근거: Grok +chat-completions wire는 reasoning_content와 tool_calls를 같은 assistant 메시지로 내보내며, +Anthropic replay는 thinking이 같은 turn의 tool_use 앞에 와야 한다. 이에 따라 초안 테스트 +기대값(다음 text assistant로 skip)과 구계약 픽스처 +`tests/responses-parser-agent-message.test.ts`(경계 앞 reasoning의 detached assistant 유지)를 +신계약으로 갱신했다. 검증: 대상 3개 스위트 33 pass, 풀 스위트 2596 pass / 0 fail, +`bun run typecheck` 통과. diff --git a/devlog/_fin/260716_grok_build_hardening/020_localcli_ownership.md b/devlog/_fin/260716_grok_build_hardening/020_localcli_ownership.md new file mode 100644 index 00000000..b0c1d96e --- /dev/null +++ b/devlog/_fin/260716_grok_build_hardening/020_localcli_ownership.md @@ -0,0 +1,509 @@ +# 020 — `local-cli` xAI credential ownership handoff + +## Work-phase contract + +- Work-phase: `wp2_localcli_ownership` +- Class: C4 (OAuth rotating-refresh ownership); implementation is limited to the xAI credential path and its regression tests. +- Goal: while an xAI credential is tagged `source: "local-cli"`, treat `~/.grok/auth.json` as the authority and adopt a newer disk generation before any IdP refresh. +- Dependency: none. Phase 030 may add OpenCodex-store cross-process serialization after this ownership rule exists. + +## Threat / failure contract + +| Item | Contract | +|---|---| +| Asset | xAI access token, rotating refresh token, and continued Grok account session | +| Entrypoints | login-time local import and request/guardian-driven refresh | +| Trust boundary | Grok Build-owned `~/.grok/auth.json` → OpenCodex-owned `~/.opencodex/auth.json` → xAI token endpoint | +| Failure | Grok CLI and OpenCodex independently spend the same refresh generation; one receives `invalid_grant`, or the last persisted stale generation forces re-login | +| Official invariant | one `AuthManager` owns a store and holds `auth.json.lock` across the IdP exchange; a follower adopts the rotated disk token instead of calling the IdP (`/Users/jun/Developer/codex/180_grok-build/crates/codegen/xai-grok-shell/src/auth/manager.rs:64-68,1529-1561,1604-1642`) | +| Contract proof | two managers sharing one store produce exactly one IdP call (`/Users/jun/Developer/codex/180_grok-build/crates/codegen/xai-grok-shell/src/auth/refresh/auth_backend_contract_tests.rs:351-388`) | + +## Ownership decision + +Choose **adopt while attached; explicit detach on unavoidable stale refresh**. + +1. A credential remains attached only while its persisted source is `"local-cli"`. +2. Before either login-time refresh or normal refresh, re-read `~/.grok/auth.json`. +3. If the Grok credential is authoritative under the monotonic expiry rule below and its access token is usable, persist/adopt it in OpenCodex and make **zero** discovery/token-endpoint calls. +4. If Grok is also stale and OpenCodex must refresh to serve the request, call the IdP once with the currently selected Grok refresh token, persist the successor only to OpenCodex with `source: "oauth"`, and emit a one-time-per-refresh warning that this account has detached from Grok CLI ownership. Subsequent OpenCodex refreshes do not consult Grok storage. +5. Never write `~/.grok/auth.json` from OpenCodex. + +Rationale: writing the successor into another application's store would need exact format preservation, permission preservation, atomic replacement, and interoperability with Grok's `auth.json.lock`. OpenCodex does not own that lock protocol, so even an atomic JSON rewrite could race Grok and corrupt or roll back unrelated fields. Detaching gives the rotated successor one durable owner after the exchange and preserves the official single-owner shape. The residual race is the **first detach exchange** if Grok refreshes concurrently; phase 020 reduces the common stale-copy race but cannot claim cross-application mutual exclusion. Phase 030's OpenCodex lock does not by itself coordinate with Grok's lock and must not be described as doing so. + +The warning text is fixed for testability and must not contain token material: + +```text +[oauth:xai] Grok CLI credential was stale; refreshed into OpenCodex ownership. Grok CLI may require login again. +``` + +## Generation and identity rules + +`detectGrokCliToken()` remains a fresh read on every invocation. A detected credential is eligible only when it is the same identity as the stored OpenCodex account: + +- if both sides have `accountId`, they must match; +- otherwise, if both sides have `email`, compare lower-cased values; +- if neither comparable identity exists, allow adoption only for the active xAI account whose stored source is `"local-cli"` (the original import relationship is the binding). + +Generation authority is monotonic and is based on fixed token lifetime, not refresh-token inequality: a later `expires` means later issuance and therefore a later generation. Adopt disk only when its access token is currently usable (`disk.expires > Date.now() + REFRESH_SKEW_MS`) and either (a) both expiries exist and disk expires later, or (b) the expiries tie or either expiry is missing. The tie/missing case deliberately prefers the Grok-owned copy only while its access token proves usable. If disk has an earlier expiry, keep the OpenCodex generation even when the refresh strings differ. A merely different refresh token is never evidence of newness and is never selected as refresh input by itself; when disk is stale, older, or ineligible, refresh the stored OpenCodex generation. + +## IN / OUT + +### IN + +- xAI `source: "local-cli"` reconciliation at login and normal account refresh. +- Identity-safe generation comparison. +- Detach metadata and redacted warning when an IdP refresh is still required. +- Temp-HOME regression coverage, including zero endpoint calls for newer-generation adoption. + +### OUT + +- Writing, locking, chmod-ing, backing up, or otherwise mutating `~/.grok/auth.json`. +- Implementing Grok's Rust file-lock protocol or claiming cross-app locking. +- Generic local-CLI ownership changes for Anthropic or Kiro. +- New config keys, GUI controls, or CLI flags. +- Phase 030 OpenCodex-store locking, phase 040 401 replay, production code changes outside the file map below. + +## Config decision + +No config surface is added. + +- `LocalTokenImportMode = "off" | "fallback" | "only"` is an internal flow type, not an `OcxConfig` field (`src/oauth/types.ts:38-43`). +- xAI login currently maps `forceLogin: true` to `importLocal: "off"`; otherwise it uses `"fallback"` (`src/oauth/index.ts:21,49-55`). +- `runLogin()` persists the flow's source and defaults source-less browser credentials to `"oauth"` (`src/oauth/index.ts:349-359`). +- Preserve that wiring: `forceLogin` remains the explicit browser-login/detach path. Do not expose `importLocal` in `src/config.ts`. + +## File change map + +| Marker | Path | Required diff | +|---|---|---| +| MODIFY | `src/oauth/local-token-detect.ts` | Extract/export identity and generation comparison helpers beside `detectGrokCliToken()`; retain read-only semantics and fresh HOME-based reads. | +| MODIFY | `src/oauth/xai.ts` | Reconcile the local store before login-time refresh; detach stale local credentials after successful IdP refresh; expose the ownership warning constant/helper needed by the account refresh path. | +| MODIFY | `src/oauth/index.ts` | Before refreshing an expired xAI `source:"local-cli"` account, re-read Grok credentials, identity-check, adopt a usable expiry-authoritative disk generation with zero IdP calls, or refresh the stored OpenCodex generation and persist the successor as `source:"oauth"`. | +| MODIFY | `tests/oauth-refresh.test.ts` | Add temp-HOME Grok fixture and request classification counters; cover normal refresh adoption, unchanged/stale detach, identity mismatch, and persistence. | +| NEW | none | Reuse the existing OAuth refresh suite and modules; do not create a parallel ownership service or config schema. | + +`src/oauth/store.ts` is inspected but unchanged: it already allowlists `source`, preserves account IDs, and provides account-scoped persistence (`src/oauth/store.ts:62-84,230-248`). + +## Diff-level design + +### MODIFY — `src/oauth/local-token-detect.ts` + +Current (complete relevant imports and implementation): + +```ts +import { execSync } from "node:child_process"; +import { existsSync, readFileSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import type { OAuthCredentials } from "./types"; + +const XAI_AUTH_KEY_PREFIX = "https://auth.x.ai::"; + +export function detectGrokCliToken(): OAuthCredentials | null { + const authPath = join(homedir(), ".grok", "auth.json"); + if (!existsSync(authPath)) return null; + + try { + const raw = JSON.parse(readFileSync(authPath, "utf8")) as Record>; + const entry = Object.entries(raw).find(([key]) => key.startsWith(XAI_AUTH_KEY_PREFIX))?.[1]; + if (!entry?.key || !entry?.refresh_token) return null; + + return { + refresh: entry.refresh_token as string, + access: entry.key as string, + expires: entry.expires_at ? new Date(entry.expires_at as string).getTime() : 0, + accountId: entry.user_id as string | undefined, + email: entry.email as string | undefined, + source: "local-cli", + }; + } catch { + return null; + } +} +``` + +After (the existing Claude detector code below this block remains byte-for-byte unchanged, so its `execSync` import remains required): + +```ts +import { execSync } from "node:child_process"; +import { existsSync, readFileSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import type { OAuthCredentials } from "./types"; + +const XAI_AUTH_KEY_PREFIX = "https://auth.x.ai::"; + +export function detectGrokCliToken(): OAuthCredentials | null { + const authPath = join(homedir(), ".grok", "auth.json"); + if (!existsSync(authPath)) return null; + + try { + const raw = JSON.parse(readFileSync(authPath, "utf8")) as Record>; + const entry = Object.entries(raw).find(([key]) => key.startsWith(XAI_AUTH_KEY_PREFIX))?.[1]; + if (!entry?.key || !entry?.refresh_token) return null; + + return { + refresh: entry.refresh_token as string, + access: entry.key as string, + expires: entry.expires_at ? new Date(entry.expires_at as string).getTime() : 0, + accountId: entry.user_id as string | undefined, + email: entry.email as string | undefined, + source: "local-cli", + }; + } catch { + return null; + } +} + +export function hasComparableGrokIdentity(stored: OAuthCredentials, disk: OAuthCredentials): boolean { + return Boolean((stored.accountId && disk.accountId) || (stored.email && disk.email)); +} + +export function isSameGrokIdentity(stored: OAuthCredentials, disk: OAuthCredentials): boolean { + if (stored.accountId && disk.accountId) return stored.accountId === disk.accountId; + if (stored.email && disk.email) return stored.email.toLowerCase() === disk.email.toLowerCase(); + return false; +} + +export function shouldAdoptGrokGeneration( + stored: OAuthCredentials, + disk: OAuthCredentials, + now = Date.now(), + refreshSkewMs = 60_000, +): boolean { + if (disk.expires <= now + refreshSkewMs) return false; + const bothExpiriesExist = stored.expires > 0 && disk.expires > 0; + if (bothExpiriesExist) return disk.expires >= stored.expires; + return true; +} +``` + +`accountId` and `email` are the only shared identity fields: Grok maps `user_id`/`email` into them and `OAuthCredentials` defines no other account identity. `hasComparableGrokIdentity()` is therefore exact, exported, and false unless at least one field exists on both sides. The no-comparable-identity fallback is accepted only by the active-account gate in `index.ts`; `isSameGrokIdentity()` alone never authorizes it. `shouldAdoptGrokGeneration()` intentionally does not inspect `refresh`. + +### MODIFY — `src/oauth/xai.ts` + +Current (complete imports and `loginXai` implementation): + +```ts +import { OAuthCallbackFlow, type OAuthCallbackFlowOptions } from "./callback-server"; +import { generatePKCE } from "./pkce"; +import type { LocalTokenImportMode, OAuthController, OAuthCredentials } from "./types"; + +export async function loginXai( + ctrl: OAuthController, + opts?: { importLocal?: LocalTokenImportMode }, +): Promise { + const importLocal = opts?.importLocal ?? "off"; + if (importLocal !== "off") { + const { detectGrokCliToken } = await import("./local-token-detect"); + const local = detectGrokCliToken(); + if (local) { + ctrl.onProgress?.("Found Grok CLI token, importing automatically"); + if (local.expires >= Date.now() + 60_000) return local; + try { + return { ...(await refreshXaiToken(local.refresh, ctrl.signal)), source: "local-cli" }; + } catch (error) { + if (importLocal === "only") { + throw new Error( + `Grok CLI token is expired and could not be refreshed: ${error instanceof Error ? error.message : String(error)}`, + ); + } + } + } else if (importLocal === "only") { + throw new Error("No Grok CLI token found at ~/.grok/auth.json. Run 'ocx login xai' for browser OAuth."); + } + } + + return new XaiOAuthFlow(ctrl).login(); +} +``` + +After (complete imports and implementation): + +```ts +import { OAuthCallbackFlow, type OAuthCallbackFlowOptions } from "./callback-server"; +import { generatePKCE } from "./pkce"; +import type { LocalTokenImportMode, OAuthController, OAuthCredentials } from "./types"; + +export const XAI_LOCAL_CLI_DETACH_WARNING = + "[oauth:xai] Grok CLI credential was stale; refreshed into OpenCodex ownership. Grok CLI may require login again."; + +export async function loginXai( + ctrl: OAuthController, + opts?: { importLocal?: LocalTokenImportMode }, +): Promise { + const importLocal = opts?.importLocal ?? "off"; + if (importLocal !== "off") { + const { detectGrokCliToken } = await import("./local-token-detect"); + const local = detectGrokCliToken(); + if (local) { + ctrl.onProgress?.("Found Grok CLI token, importing automatically"); + if (local.expires >= Date.now() + 60_000) return local; + try { + const fresh = await refreshXaiToken(local.refresh, ctrl.signal); + ctrl.onProgress?.(XAI_LOCAL_CLI_DETACH_WARNING); + return { ...fresh, source: "oauth" }; + } catch (error) { + if (importLocal === "only") { + throw new Error( + `Grok CLI token is expired and could not be refreshed: ${error instanceof Error ? error.message : String(error)}`, + ); + } + } + } else if (importLocal === "only") { + throw new Error("No Grok CLI token found at ~/.grok/auth.json. Run 'ocx login xai' for browser OAuth."); + } + } + + return new XaiOAuthFlow(ctrl).login(); +} +``` + +This login branch already reads Grok immediately before deciding to refresh, so no second redundant read is required inside `refreshXaiToken()`. Keep `refreshXaiToken()` provider-pure; ownership depends on credential source/account context and does not belong in the raw token endpoint client. + +### MODIFY — `src/oauth/index.ts` + +Current (complete imports and `refreshAndPersistAccessToken`; preserved verbatim from the current source): + +```ts +import type { OAuthController, OAuthCredentials } from "./types"; +import { getAccountCredential, getAccountSet, saveAccountCredential, saveCredential, markAccountNeedsReauth, getCredential } from "./store"; +import { loginXai, refreshXaiToken } from "./xai"; +import { loginKiro, readKiroCliSqlite, refreshKiroToken } from "./kiro"; + +async function refreshAndPersistAccessToken( + provider: string, + accountId: string, + def: OAuthProviderDef, + cred: OAuthCredentials, +): Promise { + // Local-CLI import fallback only for the ACTIVE account: importing another identity's + // token under a background account id would silently contaminate that account. + const isActive = getAccountSet(provider)?.activeAccountId === accountId; + if (provider === "kiro" && isActive) { + const imported = readFreshKiroCliCredential(); + if (imported) { + saveCredential(provider, imported); + return imported.access; + } + } + try { + const fresh = await def.refresh(cred.refresh); + // Persist to THIS account (rotation-safe: new refresh token hits disk before use) without + // touching activeAccountId. + saveAccountCredential(provider, accountId, { + ...fresh, + source: fresh.source ?? cred.source ?? "oauth", + // Preserve a previously-discovered project id when a refresh-time re-discovery comes back empty + // (e.g. a transient network blip), so Antigravity does not lose its CCA project across refresh. + ...(fresh.projectId === undefined && cred.projectId ? { projectId: cred.projectId } : {}), + // Preserve identity fields the refresh response may omit, so identity matching stays stable. + ...(fresh.email === undefined && cred.email ? { email: cred.email } : {}), + ...(fresh.accountId === undefined && cred.accountId ? { accountId: cred.accountId } : {}), + }); + return fresh.access; + } catch (err) { + if (provider === "kiro" && isActive) { + const imported = readFreshKiroCliCredential(); + if (imported) { + saveCredential(provider, imported); + return imported.access; + } + } + if (isTerminalRefreshError(err)) { + markAccountNeedsReauth(provider, accountId, true); + throw new OAuthLoginRequiredError(provider); + } + throw err; + } +} +``` + +After (complete imports and implementation; the Kiro, terminal-error, and fallback-field branches remain explicit): + +```ts +import type { OAuthController, OAuthCredentials } from "./types"; +import { getAccountCredential, getAccountSet, saveAccountCredential, saveCredential, markAccountNeedsReauth, getCredential } from "./store"; +import { loginXai, refreshXaiToken, XAI_LOCAL_CLI_DETACH_WARNING } from "./xai"; +import { loginKiro, readKiroCliSqlite, refreshKiroToken } from "./kiro"; +import { detectGrokCliToken, hasComparableGrokIdentity, isSameGrokIdentity, shouldAdoptGrokGeneration } from "./local-token-detect"; + +async function refreshAndPersistAccessToken( + provider: string, + accountId: string, + def: OAuthProviderDef, + cred: OAuthCredentials, +): Promise { + // Local-CLI import fallback only for the ACTIVE account: importing another identity's + // token under a background account id would silently contaminate that account. + const isActive = getAccountSet(provider)?.activeAccountId === accountId; + if (provider === "kiro" && isActive) { + const imported = readFreshKiroCliCredential(); + if (imported) { + saveCredential(provider, imported); + return imported.access; + } + } + if (provider === "xai" && cred.source === "local-cli") { + const disk = detectGrokCliToken(); + const identityMatches = disk !== null && isSameGrokIdentity(cred, disk); + const noIdentityActiveBinding = + disk !== null && isActive && !hasComparableGrokIdentity(cred, disk); + if ( + disk !== null && + (identityMatches || noIdentityActiveBinding) && + shouldAdoptGrokGeneration(cred, disk, Date.now(), REFRESH_SKEW_MS) + ) { + saveAccountCredential(provider, accountId, disk); + return disk.access; + } + } + try { + const fresh = await def.refresh(cred.refresh); + const detachedLocalCli = provider === "xai" && cred.source === "local-cli"; + if (detachedLocalCli) console.warn(XAI_LOCAL_CLI_DETACH_WARNING); + // Persist to THIS account (rotation-safe: new refresh token hits disk before use) without + // touching activeAccountId. + saveAccountCredential(provider, accountId, { + ...fresh, + source: detachedLocalCli ? "oauth" : fresh.source ?? cred.source ?? "oauth", + // Preserve a previously-discovered project id when a refresh-time re-discovery comes back empty + // (e.g. a transient network blip), so Antigravity does not lose its CCA project across refresh. + ...(fresh.projectId === undefined && cred.projectId ? { projectId: cred.projectId } : {}), + // Preserve identity fields the refresh response may omit, so identity matching stays stable. + ...(fresh.email === undefined && cred.email ? { email: cred.email } : {}), + ...(fresh.accountId === undefined && cred.accountId ? { accountId: cred.accountId } : {}), + }); + return fresh.access; + } catch (err) { + if (provider === "kiro" && isActive) { + const imported = readFreshKiroCliCredential(); + if (imported) { + saveCredential(provider, imported); + return imported.access; + } + } + if (isTerminalRefreshError(err)) { + markAccountNeedsReauth(provider, accountId, true); + throw new OAuthLoginRequiredError(provider); + } + throw err; + } +} +``` + +Required details: + +- Import the detector/comparison helpers statically; they perform no I/O until called. +- Keep the existing `(provider, accountId)` singleflight around the whole reconciliation + refresh path (`src/oauth/index.ts:156-169`). +- Persist adoption with `saveAccountCredential`, not `saveCredential`, so the active account is not switched. +- Preserve existing Kiro pre-refresh and post-failure recovery branches unchanged. +- On a terminal failure, keep `markAccountNeedsReauth()` behavior. Do not retag to `"oauth"` unless an IdP refresh succeeded and its successor was persisted. +- Never log access/refresh tokens, raw Grok JSON, email, or account ID. + +### MODIFY — `tests/oauth-refresh.test.ts` + +Add a fixture matching the real Grok JSON carrier: + +```ts +function seedGrokAuth(token: { + key: string; + refresh_token: string; + expires_at: string; + user_id?: string; + email?: string; +}) { + const dir = join(tmp, ".grok"); + mkdirSync(dir, { recursive: true }); + writeFileSync( + join(dir, "auth.json"), + JSON.stringify({ "https://auth.x.ai::test": token }), + ); +} +``` + +Extend the fetch mock to count discovery separately from token POSTs; acceptance is specifically zero token-endpoint calls and should also assert zero total network calls for adoption. + +Exact test names: + +1. `newer Grok generation is adopted before xAI refresh with zero endpoint calls` + - persist expired xAI `source:"local-cli"`, generation `rt-old`, identity `user-1`; + - seed temp `HOME/.grok/auth.json` with fresh `rt-new`, same identity; + - call `getValidAccessToken("xai")`; + - assert returned access is the Grok value, fetch count is `0`, stored refresh is `rt-new`, source remains `local-cli`. +2. `newer-expiry Grok access token is adopted when refresh generation is unchanged` + - same refresh token, later usable expiry; + - assert zero fetches and updated access/expiry persistence. +3. `stale Grok generation refreshes once and detaches to OpenCodex ownership` + - both stores carry the same expired generation; + - mock one discovery GET plus one token POST returning rotated credentials; + - spy on `console.warn`; + - assert exactly one token POST, warning text, successor persisted with `source:"oauth"`, and Grok file bytes remain unchanged. +4. `stale different Grok generation with earlier expiry is not adopted` + - OpenCodex has expired `rt-ours`; Grok has different `rt-disk` with an earlier expiry; + - inspect the normal refresh token POST body and assert `refresh_token=rt-ours`, never `rt-disk`; + - assert no adoption write occurs before refresh, exactly one normal discovery GET plus one token POST occurs, and the successor source is `oauth`. +5. `mismatched Grok identity is not adopted into a local-cli account` + - stored `user-1`, disk `user-2` with a fresh token; + - assert disk access is not returned or persisted; the stored generation is used for detach refresh, then source becomes `oauth`. +6. `concurrent xAI local-cli refreshes share reconciliation and one detach exchange` + - two calls for the same account; + - assert the existing singleflight produces one token POST and both callers receive the same access token. + +Test hygiene: restore `console.warn`, `fetch`, `HOME`, and `OPENCODEX_HOME` in `afterEach`; delete the temp tree. No live xAI traffic. + +## Acceptance criteria and activation scenarios + +| ID | Criterion | Activation scenario | Required proof | +|---|---|---|---| +| AC-020-1 | Every normal refresh of xAI `source:"local-cli"` re-reads Grok storage before IdP exchange. | Expired OpenCodex generation; temp-HOME Grok file replaced with a fresh different refresh token after OpenCodex persistence. | Returned/stored Grok token; zero fetch calls. | +| AC-020-2 | Same refresh generation with a newer usable expiry is adopted. | OpenCodex stale expiry, Grok later expiry. | Zero fetch calls; access and expiry updated. | +| AC-020-3 | Refresh-token inequality alone never changes authority. | Grok refresh differs but its expiry is earlier than the stored OpenCodex expiry. | Disk is not adopted; the normal refresh spends the stored OpenCodex refresh token; no unnecessary IdP calls beyond that one discovery GET and one token POST. | +| AC-020-4 | Successful stale refresh detaches ownership. | Both stores stale and one mocked IdP exchange succeeds. | OpenCodex source becomes `oauth`; warning emitted; Grok bytes unchanged. | +| AC-020-5 | Identity mismatch cannot contaminate an account. | Fresh Grok token belongs to a different `user_id`. | No adoption; stored account ID remains unchanged. | +| AC-020-6 | Existing login controls retain semantics. | default login with usable Grok token; `forceLogin:true`; stale fallback; `importLocal:"only"`. | default imports, force login skips, stale successful refresh returns `source:"oauth"`, only-mode preserves failure contract. | +| AC-020-7 | Existing account singleflight remains intact. | Two simultaneous calls for one expired xAI account. | One detach token POST; identical access result. | +| AC-020-8 | No external-store mutation is introduced. | Snapshot Grok file before stale refresh. | Byte-identical after success and failure; no `.lock`, backup, or temp sibling created by OpenCodex. | + +## Risk / rollback + +- **Rollback unit:** revert the single phase-020 implementation commit with `git revert `. Do not hand-edit credential files and do not revert sibling phase commits as part of this rollback. +- **Persisted state:** credentials already detached by a successful refresh remain valid when tagged `source:"oauth"`; the tag accurately says OpenCodex owns the newly issued rotating refresh generation, and rollback must not relabel it `local-cli` or copy it back to Grok. Credentials adopted from disk and still tagged `local-cli` also remain ordinary valid OAuth credentials; after rollback they retain the pre-phase behavior until their next refresh. +- **`needsReauth`:** phase 020 preserves the existing rule that only terminal refresh failures set `needsReauth`. Reverting code does not clear an already-set marker and must not do so automatically: the underlying grant may actually be revoked or reused. A detached credential whose refresh succeeded is persisted without setting `needsReauth`; rollback leaves that successful credential usable. +- **Rollback verification:** run the focused OAuth refresh suite and typecheck after the revert; inspect `~/.opencodex/auth.json` only through redacted status/test fixtures to confirm detached entries remain `source:"oauth"`; verify a detached xAI credential refreshes without reading `~/.grok/auth.json`; verify an account already marked `needsReauth` still requires login; and confirm `~/.grok/auth.json` bytes are unchanged. + +## Verification commands + +Run from `/Users/jun/Developer/new/700_projects/opencodex`: + +```bash +bun test --isolate ./tests/oauth-refresh.test.ts +bun test --isolate ./tests/oauth-manual-code.test.ts ./tests/oauth-store-multi.test.ts ./tests/token-guardian.test.ts +bun run typecheck +bun run test +bun run privacy:scan +git diff --check -- src/oauth/local-token-detect.ts src/oauth/xai.ts src/oauth/index.ts tests/oauth-refresh.test.ts +``` + +The commands follow `package.json` scripts: `test = bun test --isolate ./tests/`, `typecheck = bun x tsc --noEmit`, and `privacy:scan = bun scripts/privacy-scan.ts` (`package.json:38-48`). The focused OAuth suite must run before the full suite. No live login or production proxy restart is required for this ownership-only phase. + +## Implementing-cycle stale-check checklist + +- [ ] Re-read current `src/oauth/local-token-detect.ts`, `src/oauth/xai.ts`, `src/oauth/index.ts`, `src/oauth/store.ts`, and `src/oauth/types.ts`; update line anchors if they moved. +- [ ] Re-run `rg -n "importLocal|forceLogin|local-cli|refreshXaiToken|detectGrokCliToken" src tests` and confirm no new owner/config surface appeared. +- [ ] Re-read official `manager.rs` lock/adoption sections and `auth_backend_contract_two_instances_share_one_idp_call`; record any changed paths or semantics. +- [ ] Inspect sibling phase 030 before implementation; keep cross-process locking out of phase 020 and do not duplicate its helpers. +- [ ] Confirm other workers did not modify `tests/oauth-refresh.test.ts` or the three production targets; preserve concurrent changes and rebase the diff-level design if needed. +- [ ] Confirm the real Grok JSON key/value fields still match `https://auth.x.ai::`, `key`, `refresh_token`, `expires_at`, `user_id`, and `email`. +- [ ] Confirm `OAuthCredentials` still exposes only `accountId` and `email` as comparable identity fields, and that `hasComparableGrokIdentity()` is exported and used by the active-account fallback gate. +- [ ] Confirm generation authority remains expiry-monotonic: later disk expiry wins; tie/missing expiry chooses disk only when disk access is usable; refresh-token inequality never establishes newness. +- [ ] Run the stale-different-generation regression: disk has a different refresh token and earlier expiry; assert no adoption, refresh body uses the stored OpenCodex token, and there are zero extra IdP calls beyond the normal one discovery GET plus one token POST. +- [ ] Confirm no implementation writes under `~/.grok`, and that warning output is redacted. +- [ ] Rehearse rollback with a temporary branch/fixture: revert only the phase commit, retain detached `source:"oauth"` and `needsReauth` state, run focused tests/typecheck, and verify Grok bytes remain unchanged. +- [ ] Execute every verification command above on the final implementation HEAD and attach exit status/test counts to phase evidence. + +## Implementation record (B) + +- Implemented the audited local-CLI reconciliation, identity/generation helpers, successful-refresh detach warning/source transition, and six temp-HOME regressions in the phase file map. +- Deviation: `detectGrokCliToken()` resolves `process.env.HOME` before `homedir()` so each invocation observes temp-HOME fixtures (and runtime HOME changes); the Grok store remains strictly read-only. No other design deviation. +- Verification: focused OAuth refresh suite `11 pass, 0 fail`; full suite `2602 pass, 0 fail`; `bun run typecheck` exited 0. diff --git a/devlog/_fin/260716_grok_build_hardening/030_refresh_lock.md b/devlog/_fin/260716_grok_build_hardening/030_refresh_lock.md new file mode 100644 index 00000000..ab4d89cd --- /dev/null +++ b/devlog/_fin/260716_grok_build_hardening/030_refresh_lock.md @@ -0,0 +1,774 @@ +# 030 — xAI refresh hardening: per-account intent lock + short store-write lock + +Status: Round-3 replacement design. This document supersedes the rejected single global lock. No synchronous lock API, `Atomics.wait`, or network I/O under `auth.store.lock` is permitted. + +## Decision and threat model + +Assets are rotating xAI refresh tokens and the whole `auth.json` account store. Competing proxy/CLI processes can double-spend one rotating token; unrelated writers can lose updates if they load and rewrite the whole store concurrently. The rejected design held a global lock over IdP I/O and required synchronous event-loop blocking for existing writers. + +Chosen design: + +1. A refresh-intent lock file is keyed by provider and account: `auth.refresh...lock`, beside `auth.json`. It is acquired asynchronously and held over re-read, 020 reconciliation, IdP exchange, and final persist. It prevents only same-account refresh-token double-spend. +2. A single `auth.store.lock` is acquired asynchronously only around load-merge-persist. It never covers IdP discovery, token POST, retry sleep, Grok-store reads, or any other network I/O. +3. Every whole-store writer uses one in-process promise queue and then `auth.store.lock`. Correctness is cooperative exclusion, not filesystem CAS. + +Both locks use one path/TTL-parameterized implementation with `O_EXCL`, bounded asynchronous polling, stale takeover, and exact-byte/stat identity checks before every cleanup/release unlink. The consequence is an async API migration for every writer. The audit found no unconvertible synchronous production caller. + +## Source reality and async caller conversion + +Current persistence is private at `src/oauth/store.ts:50-59`; every call reaches it through `mutateStore` at `src/oauth/store.ts:167-175`. Current exported writers are synchronous at `src/oauth/store.ts:190`, `:228`, `:258`, `:269`, `:279`, and `:295`. + +| Writer | Current caller | Required after form | +|---|---|---| +| `saveCredential` | `src/oauth/index.ts:196` | `await saveCredential(provider, imported)` inside async `refreshAndPersistAccessToken` | +| | `src/oauth/index.ts:219` | same | +| | `src/oauth/index.ts:355` | `await saveCredential(provider, cred)` inside async `runLogin` | +| `saveAccountCredential` | `src/oauth/index.ts:204-213` | `await saveAccountCredential(...)` (generic providers only) | +| `markAccountNeedsReauth` | `src/oauth/index.ts:224` | `await markAccountNeedsReauth(...)` | +| `removeCredential` | `src/cli/index.ts:503` | `await removeCredential(name)`; command dispatch already permits `await` | +| | `src/server/management-api.ts:1074` | `await removeCredential(provider)`; handler is async at line 77 | +| `setActiveAccount` | `src/server/management-api.ts:1093` | `if (!(await setActiveAccount(...))) ...` | +| `removeAccount` | `src/server/management-api.ts:1104` | `if (!(await removeAccount(...))) ...` | + +All corresponding test setup and assertions found by `rg` must also await these APIs (notably `tests/oauth-store-multi.test.ts`, `tests/oauth-refresh.test.ts`, `tests/token-guardian.test.ts`, `tests/oauth-status-privacy.test.ts`, `tests/google-antigravity-oauth.test.ts`, and `tests/provider-quota.test.ts`). No production sync caller is blocked from conversion. + +## File change map + +| Marker | Path | Responsibility | +|---|---|---| +| MODIFY | `src/oauth/store.ts` | Canonical paths; shared async file-lock primitive; refresh-intent path; async serialized mutation funnel; refresh delta merge helper; all writer exports async. | +| MODIFY | `src/oauth/index.ts` | Await writer calls; two-lock xAI transaction; 020 composition; permanent verdict cache. | +| MODIFY | `src/oauth/xai.ts` | Typed token failures and bounded three-attempt retry. | +| MODIFY | `src/cli/index.ts` | Await logout writer. | +| MODIFY | `src/server/management-api.ts` | Await logout/switch/remove writers. | +| MODIFY | affected existing OAuth tests | Await async store setup and boolean results. | +| NEW | `tests/xai-refresh-lock.test.ts` | Event-loop, process-lock, merge, stale, cleanup, and TTL bodies below. | +| NEW | `tests/xai-oauth-retry.test.ts` | Retry bodies below. | + +No dependency and no persisted generation field are added. + +## `src/oauth/store.ts`: complete shared lock and mutation funnel + +Use these imports (retain existing type imports): + +```ts +import { createHash, randomUUID } from "node:crypto"; +import { + chmodSync, closeSync, copyFileSync, existsSync, fstatSync, mkdirSync, + openSync, readFileSync, statSync, unlinkSync, writeFileSync, +} from "node:fs"; +``` + +Replace `authPath()` and add lock paths/fingerprint: + +```ts +export function getAuthStorePath(): string { + return join(getConfigDir(), "auth.json"); +} + +export function getAuthStoreLockPath(): string { + return join(getConfigDir(), "auth.store.lock"); +} + +export function getAuthRefreshIntentLockPath(provider: string, accountId: string): string { + const safeProvider = provider.replace(/[^a-zA-Z0-9_-]/g, "_"); + const accountHash = createHash("sha256").update(accountId).digest("hex").slice(0, 24); + return join(getConfigDir(), `auth.refresh.${safeProvider}.${accountHash}.lock`); +} + +export function credentialGeneration(cred: OAuthCredentials): string { + return createHash("sha256") + .update(JSON.stringify([cred.refresh, cred.access, cred.expires])) + .digest("hex"); +} +``` + +Update internal `authPath()` calls to `getAuthStorePath()`. Add after `persist()`: + +```ts +export class OAuthFileLockError extends Error { + readonly code = "OAUTH_FILE_LOCK_UNAVAILABLE"; + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "OAuthFileLockError"; + } +} + +interface LockPayload { version: 1; ownerId: string; pid: number; createdAt: number } +interface LockSnapshot { bytes: string; dev: number; ino: number; mtimeMs: number; size: number } + +export interface OAuthFileLockOptions { + path: string; + waitTimeoutMs?: number; + staleAfterMs?: number; + pollMinMs?: number; + pollMaxMs?: number; + sleep?: (ms: number) => Promise; + now?: () => number; + random?: () => number; + beforeStaleUnlink?: () => void; + beforeReleaseUnlink?: () => void; + beforeFailedCreateUnlink?: () => void; + writeMetadata?: (fd: number, bytes: string) => void; +} + +export interface OAuthFileLockGuard { + readonly ownerId: string; + release(): void; +} + +function errorCode(error: unknown): string | undefined { + return error && typeof error === "object" && "code" in error + ? String((error as { code?: unknown }).code) : undefined; +} + +function snapshot(path: string): LockSnapshot { + const bytes = readFileSync(path, "utf8"); + const stat = statSync(path); + return { bytes, dev: stat.dev, ino: stat.ino, mtimeMs: stat.mtimeMs, size: stat.size }; +} + +function sameSnapshot(a: LockSnapshot, b: LockSnapshot): boolean { + return a.bytes === b.bytes && a.dev === b.dev && a.ino === b.ino && + a.mtimeMs === b.mtimeMs && a.size === b.size; +} + +function sameFd(a: LockSnapshot, b: ReturnType): boolean { + return a.dev === b.dev && a.ino === b.ino && a.mtimeMs === b.mtimeMs && a.size === b.size; +} + +function payload(bytes: string): LockPayload | undefined { + try { + const value = JSON.parse(bytes) as Partial; + return value.version === 1 && typeof value.ownerId === "string" && + typeof value.pid === "number" && typeof value.createdAt === "number" + ? value as LockPayload : undefined; + } catch { return undefined; } +} + +export function createOAuthFileLock(options: OAuthFileLockOptions): { + acquire(): Promise; +} { + const waitTimeoutMs = options.waitTimeoutMs ?? 5_000; + const staleAfterMs = options.staleAfterMs ?? 120_000; + const pollMinMs = options.pollMinMs ?? 25; + const pollMaxMs = options.pollMaxMs ?? 100; + const sleep = options.sleep ?? (ms => Bun.sleep(ms)); + const now = options.now ?? Date.now; + const random = options.random ?? Math.random; + const writeMetadata = options.writeMetadata ?? ((fd, bytes) => writeFileSync(fd, bytes, "utf8")); + if (waitTimeoutMs < 0 || staleAfterMs <= 0 || pollMinMs < 0 || pollMaxMs < pollMinMs) { + throw new OAuthFileLockError("Invalid OAuth file-lock timing options"); + } + + return { async acquire(): Promise { + hardenConfigDir(); + if (!existsSync(getConfigDir())) mkdirSync(getConfigDir(), { recursive: true, mode: 0o700 }); + const ownerId = randomUUID(); + const startedAt = now(); + for (;;) { + let fd: number | undefined; + try { + fd = openSync(options.path, "wx", 0o600); + const ownedBytes = `${JSON.stringify({ version: 1, ownerId, pid: process.pid, createdAt: now() } satisfies LockPayload)}\n`; + writeMetadata(fd, ownedBytes); + const fdStat = fstatSync(fd); + closeSync(fd); fd = undefined; + const owned = snapshot(options.path); + if (owned.bytes !== ownedBytes || !sameFd(owned, fdStat)) { + throw new OAuthFileLockError("OAuth lock changed during creation"); + } + let released = false; + return { ownerId, release(): void { + if (released) return; + released = true; + try { + const first = snapshot(options.path); + if (!sameSnapshot(owned, first)) return; + options.beforeReleaseUnlink?.(); + const second = snapshot(options.path); + if (sameSnapshot(owned, second)) unlinkSync(options.path); + } catch (error) { + if (errorCode(error) !== "ENOENT") console.warn(`[oauth] lock release failed: ${error instanceof Error ? error.message : String(error)}`); + } + } }; + } catch (error) { + if (fd !== undefined) { + let fdStat: ReturnType | undefined; + try { fdStat = fstatSync(fd); } catch {} + try { closeSync(fd); } catch {} + if (fdStat) try { + const first = snapshot(options.path); + if (sameFd(first, fdStat)) { + options.beforeFailedCreateUnlink?.(); + const second = snapshot(options.path); + if (sameSnapshot(first, second) && sameFd(second, fdStat)) unlinkSync(options.path); + } + } catch {} + } + if (errorCode(error) !== "EEXIST") { + throw error instanceof OAuthFileLockError ? error : + new OAuthFileLockError("Could not create OAuth file lock", { cause: error }); + } + } + try { + const first = snapshot(options.path); + const parsed = payload(first.bytes); + const timestamp = Math.max(first.mtimeMs, parsed?.createdAt ?? first.mtimeMs); + if (now() - timestamp > staleAfterMs) { + options.beforeStaleUnlink?.(); + const second = snapshot(options.path); + if (sameSnapshot(first, second)) unlinkSync(options.path); + continue; + } + } catch (error) { + if (errorCode(error) === "ENOENT") continue; + throw new OAuthFileLockError("Could not inspect OAuth file lock", { cause: error }); + } + const elapsed = now() - startedAt; + if (elapsed >= waitTimeoutMs) throw new OAuthFileLockError(`Timed out after ${waitTimeoutMs}ms waiting for OAuth file lock`); + const span = pollMaxMs - pollMinMs; + await sleep(Math.min(waitTimeoutMs - elapsed, pollMinMs + Math.floor(random() * (span + 1)))); + } + } }; +} + +export function createOAuthRefreshIntentLock(provider: string, accountId: string, overrides: Partial = {}) { + return createOAuthFileLock({ path: getAuthRefreshIntentLockPath(provider, accountId), staleAfterMs: 120_000, ...overrides }); +} + +let mutationTail: Promise = Promise.resolve(); + +function serializeMutation(work: () => Promise): Promise { + const result = mutationTail.then(work, work); + mutationTail = result.then(() => undefined, () => undefined); + return result; +} + +export function mutateStore(fn: (store: AuthStore) => T | Promise): Promise { + return serializeMutation(async () => { + const guard = await createOAuthFileLock({ path: getAuthStoreLockPath(), staleAfterMs: 30_000 }).acquire(); + try { + const { store, hadLegacy } = loadAuthStoreInternal(); + if (hadLegacy) backupLegacyOnce(); + const result = await fn(store); + persist(store); + return result; + } finally { guard.release(); } + }); +} + +export async function mergeAccountCredential( + provider: string, + accountId: string, + credential: OAuthCredentials, + opts: { + /** Generation the caller's IdP exchange consumed. If the stored generation no longer + * matches, another writer replaced the credential during the exchange: do NOT overwrite. */ + expectedGeneration?: string; + afterPrePersistRead?: () => void | Promise; + } = {}, +): Promise<{ superseded: false } | { superseded: true; stored: OAuthCredentials }> { + const safe = normalizeCredential(credential); + if (!safe) throw new Error("Refusing to persist invalid OAuth credential"); + return await mutateStore(async store => { + await opts.afterPrePersistRead?.(); + const account = store[provider]?.accounts.find(item => item.id === accountId); + if (!account) throw new Error(`OAuth account disappeared before persist: ${provider}`); + if ( + opts.expectedGeneration !== undefined && + account.credential && + credentialGeneration(account.credential) !== opts.expectedGeneration + ) { + // Same-account write landed while the IdP exchange was in flight. The stored credential + // is the newer generation; persisting our result would resurrect the older family. + return { superseded: true as const, stored: account.credential }; + } + account.credential = safe; + delete account.needsReauth; + return { superseded: false as const }; + }); +} + +export async function markAccountNeedsReauthIfGeneration( + provider: string, + accountId: string, + failedGeneration: string, +): Promise { + return await mutateStore(store => { + const account = store[provider]?.accounts.find(item => item.id === accountId); + if (!account?.credential) return false; + if (credentialGeneration(account.credential) !== failedGeneration) return false; // replaced meanwhile + account.needsReauth = true; + return true; + }); +} +``` + +Convert all six exported writers to `async`/`Promise` and return/await `mutateStore`; preserve their current mutation bodies exactly. `setActiveAccount` and `removeAccount` become `Promise`, the others `Promise`. There is no `acquireSync` API. + +## `src/oauth/index.ts`: complete refresh transaction + +Imports add the 020 helpers plus `credentialGeneration`, `createOAuthRefreshIntentLock`, `mergeAccountCredential`, and `OAuthFileLockError`. Keep generic provider behavior, but await its writers. + +```ts +const XAI_PERMANENT_FAILURE_TTL_MS = 30_000; +const permanentRefreshFailures = new Map(); + +interface XaiRefreshDeps { + intentLock?: ReturnType; + now?: () => number; + afterPrePersistRead?: () => void | Promise; +} + +function verdictKey(provider: string, accountId: string, cred: OAuthCredentials): string { + return `${provider}\u0000${accountId}\u0000${credentialGeneration(cred)}`; +} +function cached(provider: string, accountId: string, cred: OAuthCredentials, now: () => number): boolean { + const key = verdictKey(provider, accountId, cred), until = permanentRefreshFailures.get(key); + if (until === undefined) return false; + if (until <= now()) { permanentRefreshFailures.delete(key); return false; } + return true; +} +function terminal(error: unknown): boolean { + return error instanceof XaiTokenRequestError + ? ["invalid_grant", "refresh_token_reused", "revoked_token"].includes(error.oauthError ?? "") + : isTerminalRefreshError(error); +} +function authoritative(stored: OAuthCredentials, active: boolean, now: () => number): OAuthCredentials { + if (stored.source !== "local-cli") return stored; + const disk = detectGrokCliToken(); + if (!disk) return stored; + const allowed = isSameGrokIdentity(stored, disk) || (active && !hasComparableGrokIdentity(stored, disk)); + return allowed && shouldAdoptGrokGeneration(stored, disk, now(), REFRESH_SKEW_MS) ? disk : stored; +} +function merged(fresh: OAuthCredentials, previous: OAuthCredentials): OAuthCredentials { + return { + ...fresh, source: previous.source === "local-cli" ? "oauth" : fresh.source ?? previous.source ?? "oauth", + ...(fresh.projectId === undefined && previous.projectId ? { projectId: previous.projectId } : {}), + ...(fresh.email === undefined && previous.email ? { email: previous.email } : {}), + ...(fresh.accountId === undefined && previous.accountId ? { accountId: previous.accountId } : {}), + }; +} + +export async function refreshXaiAccountWithLock( + provider: string, accountId: string, def: OAuthProviderDef, + callerCredential: OAuthCredentials, deps: XaiRefreshDeps = {}, +): Promise { + const now = deps.now ?? Date.now; + if (cached(provider, accountId, callerCredential, now)) throw new OAuthLoginRequiredError(provider); + const lock = deps.intentLock ?? createOAuthRefreshIntentLock(provider, accountId); + let guard: Awaited> | undefined; + try { + guard = await lock.acquire(); + const stored = getAccountCredential(provider, accountId); + if (!stored) throw new OAuthLoginRequiredError(provider); + const active = getAccountSet(provider)?.activeAccountId === accountId; + const candidate = authoritative(stored, active, now); + if (credentialGeneration(candidate) !== credentialGeneration(callerCredential) && candidate.expires > now() + REFRESH_SKEW_MS) { + if (credentialGeneration(candidate) !== credentialGeneration(stored)) { + const adoption = await mergeAccountCredential(provider, accountId, candidate, { + expectedGeneration: credentialGeneration(stored), + afterPrePersistRead: deps.afterPrePersistRead, + }); + if (adoption.superseded) { + // Same-account writer landed between our read and the Grok-adoption persist. + if (adoption.stored.expires > now() + REFRESH_SKEW_MS) return adoption.stored.access; + throw new OAuthLoginRequiredError(provider); + } + } + return candidate.access; + } + if (cached(provider, accountId, candidate, now)) throw new OAuthLoginRequiredError(provider); + const attemptedGeneration = credentialGeneration(candidate); + try { + const fresh = merged(await def.refresh(candidate.refresh), candidate); // intent lock held; no store lock held + const outcome = await mergeAccountCredential(provider, accountId, fresh, { + expectedGeneration: attemptedGeneration, + afterPrePersistRead: deps.afterPrePersistRead, + }); + if (outcome.superseded) { + // A same-account writer (login, manual import) replaced the credential during the + // exchange. Adopt the newer stored generation when usable; otherwise fail closed — + // recursing on a credential we did not attempt risks spending a third-party generation. + if (outcome.stored.expires > now() + REFRESH_SKEW_MS) return outcome.stored.access; + throw new OAuthLoginRequiredError(provider); + } + permanentRefreshFailures.delete(verdictKey(provider, accountId, candidate)); + if (candidate.source === "local-cli") console.warn(XAI_LOCAL_CLI_DETACH_WARNING); + return fresh.access; + } catch (error) { + if (!terminal(error)) throw error; + permanentRefreshFailures.set(verdictKey(provider, accountId, candidate), now() + XAI_PERMANENT_FAILURE_TTL_MS); + // Conditional: only mark reauth if the failed generation is still the stored one. + await markAccountNeedsReauthIfGeneration(provider, accountId, attemptedGeneration); + throw new OAuthLoginRequiredError(provider); + } + } catch (error) { + if (!(error instanceof OAuthFileLockError)) throw error; + const disk = getAccountCredential(provider, accountId); + if (disk) { + const adopted = authoritative(disk, getAccountSet(provider)?.activeAccountId === accountId, now); + if (credentialGeneration(adopted) !== credentialGeneration(callerCredential) && adopted.expires > now() + REFRESH_SKEW_MS) return adopted.access; + } + throw error; + } finally { guard?.release(); } +} +``` + +At `refreshAndPersistAccessToken`, after the Kiro pre-import branch and before generic `try`, add: + +```ts +if (provider === "xai") return refreshXaiAccountWithLock(provider, accountId, def, cred); +``` + +This yields the required order: async intent acquire → store/Grok reread and 020 adoption → bounded IdP retry → async store mutation queue → async store-lock acquire → fresh load/merge/persist → store-lock release → intent-lock release. + +## `src/oauth/xai.ts`: complete bounded retry + +Add below token payload types and replace `postXaiToken`: + +```ts +export class XaiTokenRequestError extends Error { + constructor( + readonly status: number | undefined, + readonly oauthError: string | undefined, + message: string, + options?: { cause?: unknown }, + ) { + super(message, options); + this.name = "XaiTokenRequestError"; + } +} + +export interface XaiTokenRetryDeps { + sleep?: (ms: number) => Promise; + random?: () => number; +} + +function isAbortError(error: unknown): boolean { + return error instanceof DOMException && error.name === "AbortError"; +} + +function retryDelay(attempt: number, retryAfter: string | null, random: () => number): number { + const base = attempt === 1 ? 100 : 250; + const jittered = Math.round(base * (0.75 + random() * 0.5)); + const seconds = retryAfter !== null && /^\d+$/.test(retryAfter) ? Number(retryAfter) : 0; + return Math.min(2_000, Math.max(jittered, seconds * 1_000)); +} + +async function readTokenError(response: Response): Promise { + let oauthError: string | undefined; + let detail = ""; + try { + const body = await response.json() as { error?: unknown; error_description?: unknown }; + if (typeof body.error === "string") oauthError = body.error; + if (typeof body.error_description === "string") detail = body.error_description; + } catch { /* do not echo arbitrary response bodies or submitted refresh tokens */ } + const suffix = detail ? `: ${detail}` : oauthError ? `: ${oauthError}` : ""; + return new XaiTokenRequestError(response.status, oauthError, `xAI token request failed: ${response.status}${suffix}`); +} + +export async function postXaiToken( + tokenEndpoint: string, + body: Record, + signal?: AbortSignal, + deps: XaiTokenRetryDeps = {}, +): Promise { + const sleep = deps.sleep ?? (ms => Bun.sleep(ms)); + const random = deps.random ?? Math.random; + let lastError: unknown; + for (let attempt = 1; attempt <= 3; attempt++) { + let response: Response; + try { + response = await fetch(tokenEndpoint, { + method: "POST", + headers: { Accept: "application/json", "Content-Type": "application/x-www-form-urlencoded" }, + body: new URLSearchParams(body).toString(), + signal: requestSignal(signal), + }); + } catch (error) { + if (isAbortError(error) && signal?.aborted) throw error; + lastError = error; + if (attempt === 3) throw new XaiTokenRequestError(undefined, undefined, "xAI token request failed: network error", { cause: error }); + await sleep(retryDelay(attempt, null, random)); + continue; + } + if (response.ok) return (await response.json()) as XaiTokenPayload; + const error = await readTokenError(response); + lastError = error; + const transient = response.status === 429 || response.status >= 500; + if (!transient || attempt === 3) throw error; + await sleep(retryDelay(attempt, response.headers.get("retry-after"), random)); + } + throw lastError; +} +``` + +## Complete tests + +### `tests/xai-refresh-lock.test.ts` + +```ts +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdirSync, readFileSync, rmSync, utimesSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { OAUTH_PROVIDERS, refreshXaiAccountWithLock } from "../src/oauth"; +import { XaiTokenRequestError } from "../src/oauth/xai"; +import { + createOAuthFileLock, createOAuthRefreshIntentLock, getAccountCredential, getAccountSet, + getAuthRefreshIntentLockPath, OAuthFileLockError, saveCredential, +} from "../src/oauth/store"; + +const oldHome = process.env.HOME, oldOcx = process.env.OPENCODEX_HOME, oldFetch = globalThis.fetch; +let root: string; +beforeEach(() => { + root = join(tmpdir(), `xai-lock-${crypto.randomUUID()}`); + process.env.HOME = root; process.env.OPENCODEX_HOME = join(root, "ocx"); + mkdirSync(process.env.OPENCODEX_HOME, { recursive: true }); +}); +afterEach(() => { + if (oldHome === undefined) delete process.env.HOME; else process.env.HOME = oldHome; + if (oldOcx === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = oldOcx; + globalThis.fetch = oldFetch; rmSync(root, { recursive: true, force: true }); +}); +async function seed(): Promise { + await saveCredential("xai", { access: "old", refresh: "rotating", expires: 1, accountId: "acct" }); + return getAccountSet("xai")!.activeAccountId; +} +function seedGrok(access: string, refresh: string, expires?: number): void { + const dir = join(root, ".grok"); mkdirSync(dir, { recursive: true }); + writeFileSync(join(dir, "auth.json"), JSON.stringify({ "https://auth.x.ai::test": { + key: access, refresh_token: refresh, ...(expires ? { expires_at: new Date(expires).toISOString() } : {}), user_id: "acct", + } })); +} +function def(counter: { n: number }, gate?: Promise) { + return { ...OAUTH_PROVIDERS.xai!, refresh: async () => { + counter.n++; if (gate) await gate; + return { access: "fresh", refresh: "next", expires: Date.now() + 3_600_000 }; + } }; +} + +describe("two-lock xAI refresh", () => { + test("same-process writer completes while refresh awaits token I/O", async () => { + const id = await seed(); let release!: () => void; + const gate = new Promise(r => { release = r; }); const calls = { n: 0 }; + const refresh = refreshXaiAccountWithLock("xai", id, def(calls, gate), getAccountCredential("xai", id)!); + while (calls.n === 0) await Bun.sleep(1); + await expect(saveCredential("cursor", { access: "c", refresh: "r", expires: Date.now() + 1000, accountId: "cursor" })).resolves.toBeUndefined(); + release(); await expect(refresh).resolves.toBe("fresh"); + }); + + test("two independent lock instances perform one IdP call", async () => { + const id = await seed(); let release!: () => void; + const gate = new Promise(r => { release = r; }); const calls = { n: 0 }; + const stale = getAccountCredential("xai", id)!; + const a = refreshXaiAccountWithLock("xai", id, def(calls, gate), stale, { intentLock: createOAuthRefreshIntentLock("xai", id) }); + while (calls.n === 0) await Bun.sleep(1); + const b = refreshXaiAccountWithLock("xai", id, def(calls), stale, { intentLock: createOAuthRefreshIntentLock("xai", id) }); + release(); expect(await Promise.all([a, b])).toEqual(["fresh", "fresh"]); expect(calls.n).toBe(1); + }); + + test("unrelated writer after refresh pre-persist read survives", async () => { + const id = await seed(); const calls = { n: 0 }; let writer!: Promise; + await refreshXaiAccountWithLock("xai", id, def(calls), getAccountCredential("xai", id)!, { + afterPrePersistRead: () => { writer = saveCredential("cursor", { access: "c", refresh: "r", expires: Date.now() + 1000, accountId: "cursor" }); }, + }); + await writer; + expect(getAccountSet("cursor")?.accounts[0]?.credential.access).toBe("c"); + expect(getAccountCredential("xai", id)?.refresh).toBe("next"); + }); + + test("stale takeover requires unchanged snapshot", async () => { + const path = getAuthRefreshIntentLockPath("xai", "acct"); + writeFileSync(path, JSON.stringify({ version: 1, ownerId: "old", pid: 1, createdAt: 0 })); + utimesSync(path, new Date(0), new Date(0)); + const guard = await createOAuthFileLock({ path, staleAfterMs: 1 }).acquire(); + expect(JSON.parse(readFileSync(path, "utf8")).ownerId).toBe(guard.ownerId); guard.release(); + }); + + test("partial metadata failure cleans only its own inode", async () => { + const path = getAuthRefreshIntentLockPath("xai", "acct"); + await expect(createOAuthFileLock({ path, writeMetadata: fd => { writeFileSync(fd, "{"); throw new Error("partial"); } }).acquire()) + .rejects.toBeInstanceOf(OAuthFileLockError); + expect(() => readFileSync(path)).toThrow(); + }); + + test("replacement between stale inspections survives", async () => { + const path = getAuthRefreshIntentLockPath("xai", "acct"); + writeFileSync(path, JSON.stringify({ version: 1, ownerId: "old", pid: 1, createdAt: 0 })); + utimesSync(path, new Date(0), new Date(0)); + const replacement = JSON.stringify({ version: 1, ownerId: "new", pid: 2, createdAt: Date.now() }); + await expect(createOAuthFileLock({ path, staleAfterMs: 1, waitTimeoutMs: 0, + beforeStaleUnlink: () => writeFileSync(path, replacement) }).acquire()).rejects.toBeInstanceOf(OAuthFileLockError); + expect(readFileSync(path, "utf8")).toBe(replacement); + }); + + test("replacement between release inspections survives", async () => { + const path = getAuthRefreshIntentLockPath("xai", "acct"); + const replacement = JSON.stringify({ version: 1, ownerId: "new", pid: 2, createdAt: Date.now() }); + const guard = await createOAuthFileLock({ path, beforeReleaseUnlink: () => writeFileSync(path, replacement) }).acquire(); + guard.release(); expect(readFileSync(path, "utf8")).toBe(replacement); + }); + + test("020 usable later generation is adopted without IdP call", async () => { + const now = Date.now(); await saveCredential("xai", { access: "old", refresh: "same", expires: 1, accountId: "acct", source: "local-cli" }); + const id = getAccountSet("xai")!.activeAccountId; seedGrok("disk", "same", now + 3_600_000); const calls = { n: 0 }; + await expect(refreshXaiAccountWithLock("xai", id, def(calls), getAccountCredential("xai", id)!, { now: () => now })).resolves.toBe("disk"); + expect(calls.n).toBe(0); + }); + + test("020 equal-expiry disk generation wins without IdP call", async () => { + const now = Date.now(), expires = now + 3_600_000; + await saveCredential("xai", { access: "stored", refresh: "stored-r", expires, accountId: "acct", source: "local-cli" }); + const id = getAccountSet("xai")!.activeAccountId; seedGrok("disk", "disk-r", expires); const calls = { n: 0 }; + await expect(refreshXaiAccountWithLock("xai", id, def(calls), getAccountCredential("xai", id)!, { now: () => now })).resolves.toBe("disk"); + expect(calls.n).toBe(0); + }); + + test.each([["inside skew", 30_000], ["missing expiry", 0]])("020 %s disk token is not refresh input", async (_name, offset) => { + const now = Date.now(); await saveCredential("xai", { access: "stored", refresh: "stored-r", expires: 1, accountId: "acct", source: "local-cli" }); + const id = getAccountSet("xai")!.activeAccountId; seedGrok("disk", "disk-r", offset ? now + offset : undefined); + const used: string[] = []; const custom = { ...OAUTH_PROVIDERS.xai!, refresh: async (value: string) => { + used.push(value); return { access: "fresh", refresh: "next", expires: now + 3_600_000 }; + } }; + await refreshXaiAccountWithLock("xai", id, custom, getAccountCredential("xai", id)!, { now: () => now }); + expect(used).toEqual(["stored-r"]); + }); + + test("permanent failure cache expires and changed generation bypasses it", async () => { + const id = await seed(); const credential = getAccountCredential("xai", id)!; let now = 1000, calls = 0; + const bad = { ...OAUTH_PROVIDERS.xai!, refresh: async () => { calls++; throw new XaiTokenRequestError(400, "invalid_grant", "bad"); } }; + await expect(refreshXaiAccountWithLock("xai", id, bad, credential, { now: () => now })).rejects.toThrow(); + await expect(refreshXaiAccountWithLock("xai", id, bad, credential, { now: () => now })).rejects.toThrow(); expect(calls).toBe(1); + now += 30_001; + await expect(refreshXaiAccountWithLock("xai", id, bad, credential, { now: () => now })).rejects.toThrow(); expect(calls).toBe(2); + await saveCredential("xai", { ...credential, access: "new", refresh: "new-r", expires: now + 100_000 }); + await expect(refreshXaiAccountWithLock("xai", id, bad, credential, { now: () => now })).resolves.toBe("new"); + }); +}); +``` + +The 020 matrix above deliberately exercises the exported authority helper through the refresh transaction. If 020's final exports differ, update imports/calls during stale-check; do not duplicate its authority algorithm. + +### Same-account replacement during exchange (R4-1) + +`tests/xai-refresh-lock.test.ts`에 추가: + +같은 파일의 기존 헬퍼(`seed`, `def`, `saveCredential`, `getAccountCredential`, `getAccountSet`)만 사용한다: + +```ts + test("same-account write during exchange supersedes the refresh result", async () => { + const id = await seed(); let release!: () => void; + const gate = new Promise(r => { release = r; }); const calls = { n: 0 }; + const stale = getAccountCredential("xai", id)!; + const pending = refreshXaiAccountWithLock("xai", id, def(calls, gate), stale); + while (calls.n === 0) await Bun.sleep(1); + // Same-account writer lands via the mutation queue while def.refresh is in flight. + await saveCredential("xai", { access: "writer-access", refresh: "writer-gen", expires: Date.now() + 3_600_000, accountId: "acct" }); + release(); + await expect(pending).resolves.toBe("writer-access"); // adopted, not overwritten + expect(getAccountCredential("xai", id)?.refresh).toBe("writer-gen"); // IdP result was NOT persisted + }); + + test("terminal failure does not mark reauth when the generation was replaced", async () => { + const id = await seed(); let reject!: () => void; + const gate = new Promise((_, rej) => { + reject = () => rej(new XaiTokenRequestError(400, "invalid_grant", "invalid_grant")); + }); + const calls = { n: 0 }; + const failing = { ...def(calls), refresh: async () => { calls.n++; return await (gate as Promise); } }; + const stale = getAccountCredential("xai", id)!; + const pending = refreshXaiAccountWithLock("xai", id, failing, stale); + while (calls.n === 0) await Bun.sleep(1); + await saveCredential("xai", { access: "writer-access", refresh: "writer-gen", expires: Date.now() + 3_600_000, accountId: "acct" }); + reject(); + await expect(pending).rejects.toBeInstanceOf(OAuthLoginRequiredError); + const account = getAccountSet("xai")!.accounts.find(a => a.id === id); + expect(account?.needsReauth).toBeUndefined(); // generation changed → conditional mark skipped + }); +``` + +Grok-adoption 분기의 동일 경쟁(R5-1)은 `seedGrok`으로 더 새로운 disk 세대를 심은 뒤 adoption persist 직전 `deps.afterPrePersistRead` seam에서 같은 계정을 쓰는 테스트로 커버한다(성공 기대: writer 세대 채택, adoption 미기록). + +### `tests/xai-oauth-retry.test.ts` + +```ts +import { afterEach, describe, expect, test } from "bun:test"; +import { postXaiToken, XaiTokenRequestError } from "../src/oauth/xai"; +const original = globalThis.fetch; +afterEach(() => { globalThis.fetch = original; }); +function queue(items: Array) { + let n = 0; globalThis.fetch = (async () => { const x = items[n++]!; if (x instanceof Error) throw x; return x; }) as typeof fetch; + return () => n; +} +const body = { grant_type: "refresh_token", client_id: "client", refresh_token: "secret" }; +const ok = () => new Response(JSON.stringify({ access_token: "a", refresh_token: "r", expires_in: 3600 })); +describe("xAI retry", () => { + test("network retry succeeds", async () => { const calls = queue([new Error("net"), ok()]); const d: number[] = []; + await postXaiToken("https://auth.x.ai/token", body, undefined, { sleep: async x => { d.push(x); }, random: () => .5 }); + expect(calls()).toBe(2); expect(d).toEqual([100]); }); + test("429 and 5xx retry at most three attempts", async () => { const calls = queue([new Response("", { status: 429 }), new Response("", { status: 503 }), ok()]); const d: number[] = []; + await postXaiToken("https://auth.x.ai/token", body, undefined, { sleep: async x => { d.push(x); }, random: () => .5 }); + expect(calls()).toBe(3); expect(d).toEqual([100, 250]); }); + test("third transient failure is final", async () => { const calls = queue([500, 502, 503].map(status => new Response("", { status }))); + await expect(postXaiToken("https://auth.x.ai/token", body, undefined, { sleep: async () => {}, random: () => .5 })).rejects.toMatchObject({ status: 503 }); expect(calls()).toBe(3); }); + test("permanent 4xx is not retried or leaked", async () => { const calls = queue([new Response(JSON.stringify({ error: "invalid_grant" }), { status: 400 })]); + await expect(postXaiToken("https://auth.x.ai/token", body, undefined, { sleep: async () => {} })).rejects.toBeInstanceOf(XaiTokenRequestError); expect(calls()).toBe(1); }); + test("caller abort is not retried", async () => { const c = new AbortController(); c.abort(); let calls = 0; + globalThis.fetch = (async () => { calls++; throw new DOMException("aborted", "AbortError"); }) as typeof fetch; + await expect(postXaiToken("https://auth.x.ai/token", body, c.signal, { sleep: async () => {} })).rejects.toMatchObject({ name: "AbortError" }); expect(calls).toBe(1); }); +}); +``` + +## Acceptance criteria + +- Same-account processes make one IdP exchange; unrelated accounts do not share intent locks. +- A same-process writer completes while refresh awaits token I/O, proving no global-lock/event-loop deadlock. +- All whole-store writers await one queue plus `auth.store.lock`; no direct `persist()` bypass exists. +- Refresh final persistence re-loads under the store lock and merges only its account delta; the deterministic unrelated writer survives. +- Same-account write during the exchange supersedes the refresh result: the newer stored generation is adopted when usable, never overwritten (`expectedGeneration` guard), and terminal-failure `needsReauth` is generation-conditional. +- Both lock kinds are async-only and share one implementation; stale/cleanup/release unlink only unchanged exact-byte/stat snapshots. +- 020 authority composition and equality-only credential fingerprints remain intact. +- Permanent verdicts are fingerprint-scoped for 30 seconds; retry behavior remains bounded and abort-safe. + +## Risk and rollback + +- Crash leftovers are `auth.refresh...lock` and `auth.store.lock`. Stale takeover is allowed only after each lock's TTL and unchanged snapshot validation. +- Intent TTL must exceed discovery plus three 30-second token attempts and backoffs (120 seconds). Store TTL is 30 seconds because it protects local load-merge-persist only. +- A newly landed writer bypassing `mutateStore` reintroduces lost updates. Stale-check must reject every direct `persist()` call outside the funnel. +- Rollback the phase commit, stop opencodex processes, then delete only stale lock files beside `auth.json`; never delete `auth.json`. The old `auth.json.refresh.lock` from the rejected design may also be removed during upgrade cleanup. + +## Verification + +```bash +rg -n "persist\(|saveCredential|removeCredential|saveAccountCredential|setActiveAccount|removeAccount|markAccountNeedsReauth" src tests +rg -n "Atomics\.wait|acquireSync|auth\.json\.refresh\.lock" src tests devlog/_plan/260716_grok_build_hardening/030_refresh_lock.md +bun test --isolate ./tests/xai-refresh-lock.test.ts ./tests/xai-oauth-retry.test.ts ./tests/oauth-refresh.test.ts ./tests/oauth-store-multi.test.ts +bun run typecheck +bun run privacy:scan +bun run test +git diff --check -- devlog/_plan/260716_grok_build_hardening/030_refresh_lock.md +``` + +## Stale-check checklist + +- [ ] Re-read `src/oauth/store.ts`, `src/oauth/index.ts`, `src/oauth/xai.ts`, `src/oauth/local-token-detect.ts`, and every `rg` caller at implementation HEAD; refresh line numbers. +- [ ] Confirm every whole-store writer and every test caller awaits the async API; no ignored promises. +- [ ] Confirm `persist()` is private and called only by `mutateStore`; no nested store-lock acquisition. +- [ ] Confirm no network, retry sleep, Grok-store read, or IdP call occurs while `auth.store.lock` is held. +- [ ] Confirm intent path includes sanitized provider and SHA-256 account hash, while store path is exactly `auth.store.lock`. +- [ ] Confirm both lock kinds use `createOAuthFileLock`; no `Atomics.wait`, sync polling, `acquireSync`, or busy loop exists. +- [ ] Confirm 020 still exports and is called through `shouldAdoptGrokGeneration(stored, disk, now(), REFRESH_SKEW_MS)`; do not duplicate its authority rules. +- [ ] Confirm generation is lowercase SHA-256 and compared only by equality/inequality across 020/030/040. +- [ ] Confirm stale takeover, failed-create cleanup, and release each perform two exact-byte/stat checks and preserve replacements. +- [ ] Confirm retry tests cover network, 429, 5xx exhaustion, permanent 4xx, caller abort, and token redaction. + +## Implementation record (B) + +- Implemented the shared async `O_EXCL` file-lock primitive, per-account refresh-intent lock, short-held store-write lock, and one promise-chain mutation funnel. +- Converted all six OAuth store writers and every production/test caller found at implementation HEAD to async/await. +- Replaced wp2's pre-refresh xAI reconciliation block with `refreshXaiAccountWithLock`; the transaction calls the exported `shouldAdoptGrokGeneration` policy under the intent lock. +- Added generation-guarded account merge/reauth mutations, fingerprint-scoped permanent-failure TTL, and bounded abort-safe xAI token retries. +- Deviation: tests were written compactly in the two required new files, while retaining the audited behavioral assertions; no production design deviation. diff --git a/devlog/_fin/260716_grok_build_hardening/040_401_replay.md b/devlog/_fin/260716_grok_build_hardening/040_401_replay.md new file mode 100644 index 00000000..8b0300d9 --- /dev/null +++ b/devlog/_fin/260716_grok_build_hardening/040_401_replay.md @@ -0,0 +1,768 @@ +# 040 — xAI OAuth upstream 401 refresh-and-replay + +## Work-phase contract + +Classification: C4 auth/security. This is an implementation design only; no production code changes occur in this planning turn. + +### IN + +- `src/oauth/index.ts`: expose an account-aware forced refresh that joins the existing in-process `(provider, accountId)` singleflight and delegates xAI persistence/locking to 030. +- `src/server/responses.ts`: capture the exact xAI OAuth bearer snapshot used for the first request; on one upstream 401, force/adopt refresh, rebuild transport and adapter from the new snapshot, and replay once. +- `tests/server-xai-oauth-401-replay.test.ts`: serving-path coverage for 401→200, 401→401, API-key exclusion, and concurrent 401 singleflight. +- Existing key-pool 429 and Anthropic 413 recovery remain composable. + +### OUT and documented residual + +- API-key/BYOK xAI, `authMode: "forward"`, non-xAI OAuth providers, `/models`, quota, image, compact, generic retry, background refresh, login UI, registry, release, and push. +- More than one 401 replay or reuse of the adapter/provider captured before refresh. +- **Web-search sidecar 401 recovery is OUT.** `planWebSearch`/`runWithWebSearch` returns at `src/server/responses.ts:891-934` before the main fetch/recovery loop at `:940-1041`. Extending `runWithWebSearch` needs a separate callback/lifecycle contract and could replay a multi-step tool loop, so it is deliberately not folded into this main-loop phase. Follow-up: design a sidecar-owned auth-recovery hook with its own one-replay boundary. + +The ordinary xAI chat 401 **is reachable**: initial fetch is `src/server/responses.ts:940-953`; every non-success enters `:963-1041`; no upstream branch consumes the response first. + +## Current source and discriminator + +`route` is the actual variable in scope. `route.providerName` and the configured `route.provider.authMode` exist after `routeModel` (`src/server/responses.ts:486-504`). OAuth token injection mutates `route.provider.apiKey` at `:612-633`; transport rewriting occurs at `:634`; `adapter` is built at `:649-650`. `createOpenAIChatAdapter` closes over that provider, and current retries use mutable `activeAdapter` at `:969-982`. Therefore capture the discriminator before token injection: + +```ts +const isXaiOAuthRequest = route.providerName === "xai" + && route.provider.authMode === "oauth"; +``` + +Do not infer OAuth from Bearer headers, `apiKey`, base URL, adapter name, or Grok CLI headers: xAI API keys also use Bearer and OAuth tokens are stored temporarily in `apiKey`. + +The cross-phase snapshot contract is opaque and equality-only: + +```ts +export interface OAuthAccessSnapshot { + provider: string; + accountId: string; + generation: string; + accessToken: string; +} +``` + +`generation` is the 030 SHA-256 fingerprint of refresh + access + expiry. It is never ordered, incremented, parsed, logged, or compared with `<`/`>`. Only `===` and `!==` are valid. `sentOAuthSnapshot` is the snapshot whose bearer built the first adapter. + +## File map + +| Marker | Path | Responsibility | +|---|---|---| +| MODIFY | `src/oauth/index.ts` | Snapshot resolver and equality-based forced refresh over existing singleflight plus 030 lock/re-check. | +| MODIFY | `src/server/responses.ts` | Exact discriminator, sent snapshot, one replay, and fresh transport/adapter resolution. | +| NEW | `tests/server-xai-oauth-401-replay.test.ts` | Copy-paste serving tests. | + +No other file belongs to this phase. + +## Complete diff design + +### MODIFY `src/oauth/index.ts` + +The following before block is the complete current import/singleflight/resolver/refresh surface being replaced (unrelated exports below it remain unchanged): + +```ts +import type { OAuthController, OAuthCredentials } from "./types"; +import { parseCallbackInput } from "./callback-server"; +import type { OcxConfig, OcxProviderConfig, RefreshPolicy } from "../types"; +import { loadConfig, resolveEnvValue, saveConfig } from "../config"; +import { maskEmail } from "../lib/privacy"; +import { getAccountCredential, getAccountSet, saveAccountCredential, saveCredential, markAccountNeedsReauth, getCredential } from "./store"; + +const REFRESH_SKEW_MS = 60_000; +const tokenRefreshes = new Map>(); + +export async function getValidAccessToken(provider: string): Promise { + const def = OAUTH_PROVIDERS[provider]; + if (!def) throw new UnsupportedOAuthProviderError(provider); + const set = getAccountSet(provider); + if (!set) throw new OAuthLoginRequiredError(provider); + return getValidAccessTokenForAccount(provider, set.activeAccountId); +} + +export async function getValidAccessTokenForAccount(provider: string, accountId: string): Promise { + const def = OAUTH_PROVIDERS[provider]; + if (!def) throw new UnsupportedOAuthProviderError(provider); + const cred = getAccountCredential(provider, accountId); + if (!cred) throw new OAuthLoginRequiredError(provider); + if (cred.expires > Date.now() + REFRESH_SKEW_MS) return cred.access; + const key = `${provider}\u0000${accountId}`; + const existing = tokenRefreshes.get(key); + if (existing) return existing; + const refresh = refreshAndPersistAccessToken(provider, accountId, def, cred).finally(() => { + if (tokenRefreshes.get(key) === refresh) tokenRefreshes.delete(key); + }); + tokenRefreshes.set(key, refresh); + return refresh; +} +``` + +After 030 lands, replace that surface with this complete code. `credentialGeneration` and `refreshAndPersistAccessToken` are the exact 030-owned symbols; stale-check their landed names before applying. + +```ts +import type { OAuthController, OAuthCredentials } from "./types"; +import { parseCallbackInput } from "./callback-server"; +import type { OcxConfig, OcxProviderConfig, RefreshPolicy } from "../types"; +import { loadConfig, resolveEnvValue, saveConfig } from "../config"; +import { maskEmail } from "../lib/privacy"; +import { + credentialGeneration, + getAccountCredential, + getAccountSet, + getCredential, + markAccountNeedsReauth, + saveAccountCredential, + saveCredential, +} from "./store"; + +const REFRESH_SKEW_MS = 60_000; + +export interface OAuthAccessSnapshot { + provider: string; + accountId: string; + generation: string; + accessToken: string; +} + +const tokenRefreshes = new Map>(); + +function snapshot(provider: string, accountId: string, cred: OAuthCredentials): OAuthAccessSnapshot { + return { + provider, + accountId, + generation: credentialGeneration(cred), + accessToken: cred.access, + }; +} + +async function resolveAccessSnapshotForAccount( + provider: string, + accountId: string, + forceRejectedGeneration?: string, +): Promise { + const def = OAUTH_PROVIDERS[provider]; + if (!def) throw new UnsupportedOAuthProviderError(provider); + const cred = getAccountCredential(provider, accountId); + if (!cred) throw new OAuthLoginRequiredError(provider); + + const current = snapshot(provider, accountId, cred); + const rejectedStillCurrent = forceRejectedGeneration !== undefined + && current.generation === forceRejectedGeneration; + if (!rejectedStillCurrent && cred.expires > Date.now() + REFRESH_SKEW_MS) return current; + if (forceRejectedGeneration !== undefined && current.generation !== forceRejectedGeneration) return current; + + const key = `${provider}\u0000${accountId}`; + const existing = tokenRefreshes.get(key); + if (existing) return existing; + + const refresh = (async (): Promise => { + const accessToken = await refreshAndPersistAccessToken(provider, accountId, def, cred); + const persisted = getAccountCredential(provider, accountId); + if (!persisted) throw new OAuthLoginRequiredError(provider); + if (persisted.access !== accessToken) { + throw new Error(`OAuth refresh persisted an unexpected access token for ${provider}`); + } + return snapshot(provider, accountId, persisted); + })().finally(() => { + if (tokenRefreshes.get(key) === refresh) tokenRefreshes.delete(key); + }); + tokenRefreshes.set(key, refresh); + return refresh; +} + +export async function getValidAccessTokenSnapshot(provider: string): Promise { + const set = getAccountSet(provider); + if (!set) throw new OAuthLoginRequiredError(provider); + return resolveAccessSnapshotForAccount(provider, set.activeAccountId); +} + +export async function forceRefreshOAuthAccessSnapshot( + rejected: OAuthAccessSnapshot, +): Promise { + if (rejected.provider !== "xai") { + throw new UnsupportedOAuthProviderError(rejected.provider); + } + return resolveAccessSnapshotForAccount( + rejected.provider, + rejected.accountId, + rejected.generation, + ); +} + +export async function getValidAccessToken(provider: string): Promise { + return (await getValidAccessTokenSnapshot(provider)).accessToken; +} + +export async function getValidAccessTokenForAccount(provider: string, accountId: string): Promise { + return (await resolveAccessSnapshotForAccount(provider, accountId)).accessToken; +} +``` + +Forced refresh joins the same map as lazy refresh. It passes the exact rejected fingerprint into the 030 refresh primitive. Inside 030's lock, a post-lock generation re-read may find `generation !== rejected.generation` and return that successor with zero IdP calls. If equal, 030 refreshes and persists while locked. No plaintext bearer comparison is used as a generation surrogate. + +### MODIFY `src/server/responses.ts` + +Replace the OAuth imports: + +```ts +// before +import { + getOAuthCredentialProjectId, + getValidAccessToken, + UnsupportedOAuthProviderError, +} from "../oauth"; + +// after +import { + forceRefreshOAuthAccessSnapshot, + getOAuthCredentialProjectId, + getValidAccessTokenSnapshot, + type OAuthAccessSnapshot, + UnsupportedOAuthProviderError, +} from "../oauth"; +``` + +Replace the complete OAuth injection block at the current `route.provider` site: + +```ts +// before +if (route.provider.authMode === "oauth") { + try { + route.provider = { ...route.provider, apiKey: await getValidAccessToken(route.providerName) }; + if (route.provider.googleMode === "cloud-code-assist" && !route.provider.project) { + const projectId = getOAuthCredentialProjectId(route.providerName); + if (projectId) route.provider = { ...route.provider, project: projectId }; + } + } catch (err) { + if (err instanceof UnsupportedOAuthProviderError) { + return formatErrorResponse( + 400, + "invalid_request_error", + `${err.message}. Remove or reconfigure provider '${route.providerName}' in ${getConfigPath()}.`, + ); + } + return formatErrorResponse(401, "authentication_error", err instanceof Error ? err.message : String(err)); + } +} +route.provider = resolveProviderTransport(route.providerName, route.provider, parsed.options.promptCacheKey); +``` + +```ts +// after +const isXaiOAuthRequest = route.providerName === "xai" + && route.provider.authMode === "oauth"; +let sentOAuthSnapshot: OAuthAccessSnapshot | undefined; +if (route.provider.authMode === "oauth") { + try { + const resolved = await getValidAccessTokenSnapshot(route.providerName); + if (isXaiOAuthRequest) sentOAuthSnapshot = resolved; + route.provider = { ...route.provider, apiKey: resolved.accessToken }; + if (route.provider.googleMode === "cloud-code-assist" && !route.provider.project) { + const projectId = getOAuthCredentialProjectId(route.providerName); + if (projectId) route.provider = { ...route.provider, project: projectId }; + } + } catch (err) { + if (err instanceof UnsupportedOAuthProviderError) { + return formatErrorResponse( + 400, + "invalid_request_error", + `${err.message}. Remove or reconfigure provider '${route.providerName}' in ${getConfigPath()}.`, + ); + } + return formatErrorResponse(401, "authentication_error", err instanceof Error ? err.message : String(err)); + } +} +route.provider = resolveProviderTransport(route.providerName, route.provider, parsed.options.promptCacheKey); +``` + +Replace the complete non-success recovery block. This includes existing error mapping and contains no omitted branch: + +```ts +// before +if (!upstreamResponse.ok) { + let activeAdapter = adapter; + let imageTierBias = 0; + let imageRetryAttempted = false; + const rebuildAndRefetch = async (): Promise => { + const retryRequest = await activeAdapter.buildRequest(parsed, { + headers: selectedForwardHeaders, + ...(imageTierBias > 0 ? { imageTierBias } : {}), + }); + try { + return activeAdapter.fetchResponse + ? await activeAdapter.fetchResponse(retryRequest, { abortSignal: upstream.signal, timeoutMs: connectMs, stream: parsed.stream }) + : await fetchWithHeaderTimeout(retryRequest.url, { + method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, + }, upstream.signal, connectMs, parsed.stream); + } catch (err) { + cleanupUpstreamAbort(); + upstream.abort(); + const msg = err instanceof Error && err.name === "TimeoutError" + ? `Provider connect timeout after ${connectMs}ms` + : `Provider unreachable: ${err instanceof Error ? err.message : String(err)}`; + return { failed: formatErrorResponse(502, "upstream_error", msg) }; + } + }; + recovery: for (;;) { + while (upstreamResponse.status === 429 && hasKeyPoolFailover(route.provider)) { + const rotated = rotateProviderTransportOn429(config, route.providerName, { + retryAfter: upstreamResponse.headers.get("retry-after"), + now: Date.now(), + attemptedKey: route.provider.apiKey, + promptCacheKey: parsed.options.promptCacheKey, + }); + if (!rotated) break; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed */ } + route.provider = rotated; + activeAdapter = resolveAdapter(resolveWireProtocolOverride(route.providerName, route.modelId, route.provider), config.cacheRetention); + const result = await rebuildAndRefetch(); + if ("failed" in result) return result.failed; + upstreamResponse = result; + } + if (shouldAttemptImageTierRetry({ status: upstreamResponse.status, adapterName: activeAdapter.name, parsed, alreadyAttempted: imageRetryAttempted })) { + imageRetryAttempted = true; + imageTierBias = 1; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed */ } + const result = await rebuildAndRefetch(); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + break; + } + if (!upstreamResponse.ok) { + const errorText = await upstreamResponse.text().catch(() => "unknown error"); + cleanupUpstreamAbort(); + return formatErrorResponse(upstreamResponse.status, "upstream_error", `Provider error ${upstreamResponse.status}: ${redactSecretString(errorText.slice(0, 500))}`); + } +} +``` + +```ts +// after +if (!upstreamResponse.ok) { + let activeAdapter = adapter; + let imageTierBias = 0; + let imageRetryAttempted = false; + let oauth401ReplayAttempted = false; + + const rebuildAndRefetch = async (): Promise => { + const retryRequest = await activeAdapter.buildRequest(parsed, { + headers: selectedForwardHeaders, + ...(imageTierBias > 0 ? { imageTierBias } : {}), + }); + try { + return activeAdapter.fetchResponse + ? await activeAdapter.fetchResponse(retryRequest, { abortSignal: upstream.signal, timeoutMs: connectMs, stream: parsed.stream }) + : await fetchWithHeaderTimeout(retryRequest.url, { + method: retryRequest.method, + headers: retryRequest.headers, + body: retryRequest.body, + }, upstream.signal, connectMs, parsed.stream); + } catch (err) { + cleanupUpstreamAbort(); + upstream.abort(); + const msg = err instanceof Error && err.name === "TimeoutError" + ? `Provider connect timeout after ${connectMs}ms` + : `Provider unreachable: ${err instanceof Error ? err.message : String(err)}`; + return { failed: formatErrorResponse(502, "upstream_error", msg) }; + } + }; + + recovery: for (;;) { + if ( + upstreamResponse.status === 401 + && isXaiOAuthRequest + && sentOAuthSnapshot + && !oauth401ReplayAttempted + ) { + oauth401ReplayAttempted = true; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed */ } + let refreshed: OAuthAccessSnapshot; + try { + refreshed = await forceRefreshOAuthAccessSnapshot(sentOAuthSnapshot); + } catch (err) { + cleanupUpstreamAbort(); + return formatErrorResponse(401, "authentication_error", err instanceof Error ? err.message : String(err)); + } + sentOAuthSnapshot = refreshed; + const refreshedProvider = resolveProviderTransport( + route.providerName, + { ...route.provider, apiKey: refreshed.accessToken }, + parsed.options.promptCacheKey, + ); + route.provider = refreshedProvider; + activeAdapter = resolveAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, refreshedProvider), + config.cacheRetention, + ); + const result = await rebuildAndRefetch(); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + + while (upstreamResponse.status === 429 && hasKeyPoolFailover(route.provider)) { + const rotated = rotateProviderTransportOn429(config, route.providerName, { + retryAfter: upstreamResponse.headers.get("retry-after"), + now: Date.now(), + attemptedKey: route.provider.apiKey, + promptCacheKey: parsed.options.promptCacheKey, + }); + if (!rotated) break; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed */ } + route.provider = rotated; + activeAdapter = resolveAdapter(resolveWireProtocolOverride(route.providerName, route.modelId, route.provider), config.cacheRetention); + const result = await rebuildAndRefetch(); + if ("failed" in result) return result.failed; + upstreamResponse = result; + } + + if (shouldAttemptImageTierRetry({ + status: upstreamResponse.status, + adapterName: activeAdapter.name, + parsed, + alreadyAttempted: imageRetryAttempted, + })) { + imageRetryAttempted = true; + imageTierBias = 1; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed */ } + const result = await rebuildAndRefetch(); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + break; + } + + if (!upstreamResponse.ok) { + const errorText = await upstreamResponse.text().catch(() => "unknown error"); + cleanupUpstreamAbort(); + return formatErrorResponse( + upstreamResponse.status, + "upstream_error", + `Provider error ${upstreamResponse.status}: ${redactSecretString(errorText.slice(0, 500))}`, + ); + } +} +``` + +The guard is set before refresh, so refresh failure, replay transport failure, and a second 401 cannot spiral. A replayed 401 falls through unchanged to the existing redacted upstream-error mapping. The fresh provider is passed through both `resolveProviderTransport` and `resolveWireProtocolOverride`, then a new adapter is constructed; no captured pre-refresh adapter is reused. + +### NEW `tests/server-xai-oauth-401-replay.test.ts` + +Copy-paste complete test file, following `tests/server-key-failover-e2e.test.ts` isolation, `globalThis.fetch` interception, `saveConfig`, and `startServer(0)` conventions: + +```ts +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { saveConfig } from "../src/config"; +import { XAI_OAUTH_DISCOVERY_URL } from "../src/oauth/xai"; +import { saveCredential } from "../src/oauth/store"; +import { XAI_GROK_CLI_BASE_URL } from "../src/providers/xai-transport"; +import { startServer } from "../src/server"; +import type { OcxConfig } from "../src/types"; +import { installIsolatedCodexHome, type IsolatedCodexHome } from "./helpers/isolated-codex-home"; + +const TOKEN_ENDPOINT = "https://auth.x.ai/oauth/token"; +const CHAT_ENDPOINT = `${XAI_GROK_CLI_BASE_URL}/chat/completions`; + +let testDir = ""; +let previousHome: string | undefined; +let isolatedCodexHome: IsolatedCodexHome | null = null; +let originalFetch: typeof fetch; + +beforeEach(() => { + originalFetch = globalThis.fetch; + previousHome = process.env.OPENCODEX_HOME; + isolatedCodexHome = installIsolatedCodexHome("ocx-xai-401-codex-"); + testDir = mkdtempSync(join(tmpdir(), "ocx-xai-401-")); + process.env.OPENCODEX_HOME = testDir; +}); + +afterEach(() => { + globalThis.fetch = originalFetch; + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + isolatedCodexHome?.restore(); + isolatedCodexHome = null; + if (testDir) rmSync(testDir, { recursive: true, force: true }); +}); + +function seedOAuth(): void { + saveCredential("xai", { + access: "rejected-access", + refresh: "initial-refresh", + expires: Date.now() + 3_600_000, + accountId: "xai-test-account", + source: "oauth", + }); +} + +function xaiConfig(authMode: "oauth" | "key" = "oauth"): OcxConfig { + return { + port: 0, + hostname: "127.0.0.1", + defaultProvider: "xai", + providers: { + xai: { + adapter: "openai-chat", + baseUrl: "https://api.x.ai/v1", + authMode, + ...(authMode === "key" ? { apiKey: "xai-api-key" } : {}), + models: ["grok-4.5"], + }, + }, + } as OcxConfig; +} + +function successBody(text: string): string { + return JSON.stringify({ + id: "chatcmpl-xai-401", + object: "chat.completion", + choices: [{ index: 0, message: { role: "assistant", content: text }, finish_reason: "stop" }], + usage: { prompt_tokens: 3, completion_tokens: 2, total_tokens: 5 }, + }); +} + +async function post(server: ReturnType): Promise { + return originalFetch(new URL("/v1/responses", server.url), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "xai/grok-4.5", input: "hello", stream: false }), + }); +} + +function installOAuthFetch(chatStatuses: number[]): { chatAuth: string[]; counts: { refresh: number } } { + const chatAuth: string[] = []; + const counts = { refresh: 0 }; + globalThis.fetch = (async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === XAI_OAUTH_DISCOVERY_URL) { + return new Response(JSON.stringify({ + authorization_endpoint: "https://auth.x.ai/oauth/authorize", + token_endpoint: TOKEN_ENDPOINT, + }), { headers: { "content-type": "application/json" } }); + } + if (url === TOKEN_ENDPOINT) { + counts.refresh += 1; + return new Response(JSON.stringify({ + access_token: "fresh-access", + refresh_token: "fresh-refresh", + expires_in: 3600, + }), { headers: { "content-type": "application/json" } }); + } + if (url === CHAT_ENDPOINT) { + chatAuth.push(new Headers(init?.headers).get("authorization") ?? ""); + const status = chatStatuses.shift() ?? 200; + if (status === 401) { + return new Response(JSON.stringify({ error: { message: "rejected" } }), { + status: 401, + headers: { "content-type": "application/json" }, + }); + } + return new Response(successBody("ok after refresh"), { headers: { "content-type": "application/json" } }); + } + return originalFetch(input, init); + }) as typeof fetch; + return { chatAuth, counts }; +} + +describe("xAI OAuth upstream 401 replay", () => { + test("401 then 200 performs one refresh and one replay", async () => { + seedOAuth(); + saveConfig(xaiConfig()); + const observed = installOAuthFetch([401, 200]); + const server = startServer(0); + try { + const response = await post(server); + expect(response.status).toBe(200); + const json = await response.json() as { output?: { type: string; content?: { text?: string }[] }[] }; + expect(json.output?.find(item => item.type === "message")?.content?.[0]?.text).toBe("ok after refresh"); + expect(observed.counts.refresh).toBe(1); + expect(observed.chatAuth).toEqual(["Bearer rejected-access", "Bearer fresh-access"]); + } finally { + server.stop(true); + } + }); + + test("401 then 401 replays once and propagates the second error", async () => { + seedOAuth(); + saveConfig(xaiConfig()); + const observed = installOAuthFetch([401, 401]); + const server = startServer(0); + try { + const response = await post(server); + const json = await response.json() as { error?: { message?: string } }; + expect(response.status).toBe(401); + expect(json.error?.message).toContain("Provider error 401"); + expect(observed.counts.refresh).toBe(1); + expect(observed.chatAuth).toEqual(["Bearer rejected-access", "Bearer fresh-access"]); + } finally { + server.stop(true); + } + }); + + test("API-key xAI path never attempts OAuth refresh", async () => { + saveConfig(xaiConfig("key")); + let refreshCalls = 0; + let chatCalls = 0; + globalThis.fetch = (async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === XAI_OAUTH_DISCOVERY_URL || url === TOKEN_ENDPOINT) { + refreshCalls += 1; + return new Response("unexpected", { status: 500 }); + } + if (url === "https://api.x.ai/v1/chat/completions") { + chatCalls += 1; + expect(new Headers(init?.headers).get("authorization")).toBe("Bearer xai-api-key"); + return new Response(JSON.stringify({ error: { message: "key rejected" } }), { + status: 401, + headers: { "content-type": "application/json" }, + }); + } + return originalFetch(input, init); + }) as typeof fetch; + const server = startServer(0); + try { + const response = await post(server); + expect(response.status).toBe(401); + expect(chatCalls).toBe(1); + expect(refreshCalls).toBe(0); + } finally { + server.stop(true); + } + }); + + test("concurrent 401 responses join one IdP refresh", async () => { + seedOAuth(); + saveConfig(xaiConfig()); + let refreshCalls = 0; + let releaseRefresh!: () => void; + const refreshGate = new Promise(resolve => { releaseRefresh = resolve; }); + const attemptsByBearer = new Map(); + globalThis.fetch = (async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === XAI_OAUTH_DISCOVERY_URL) { + return new Response(JSON.stringify({ + authorization_endpoint: "https://auth.x.ai/oauth/authorize", + token_endpoint: TOKEN_ENDPOINT, + }), { headers: { "content-type": "application/json" } }); + } + if (url === TOKEN_ENDPOINT) { + refreshCalls += 1; + await refreshGate; + return new Response(JSON.stringify({ + access_token: "fresh-access", + refresh_token: "fresh-refresh", + expires_in: 3600, + }), { headers: { "content-type": "application/json" } }); + } + if (url === CHAT_ENDPOINT) { + const bearer = new Headers(init?.headers).get("authorization") ?? ""; + attemptsByBearer.set(bearer, (attemptsByBearer.get(bearer) ?? 0) + 1); + if (bearer === "Bearer rejected-access") { + return new Response(JSON.stringify({ error: { message: "rejected" } }), { + status: 401, + headers: { "content-type": "application/json" }, + }); + } + return new Response(successBody("concurrent ok"), { headers: { "content-type": "application/json" } }); + } + return originalFetch(input, init); + }) as typeof fetch; + + const server = startServer(0); + try { + const first = post(server); + const second = post(server); + await Bun.sleep(10); + releaseRefresh(); + const [a, b] = await Promise.all([first, second]); + expect([a.status, b.status]).toEqual([200, 200]); + expect(refreshCalls).toBe(1); + expect(attemptsByBearer.get("Bearer rejected-access")).toBe(2); + expect(attemptsByBearer.get("Bearer fresh-access")).toBe(2); + } finally { + server.stop(true); + } + }); +}); +``` + +## Acceptance criteria + +- 401→200: exactly one refresh and one replay; authorization changes from rejected to fresh bearer. +- 401→401: exactly one replay; second 401 uses existing redacted upstream error propagation. +- API-key xAI: exactly one upstream attempt and zero discovery/token calls. +- Concurrent rejected snapshots: both replay successfully and the shared in-process singleflight plus 030 post-lock equality check yields one IdP call. +- Replay rebuilds provider transport, wire override, and adapter from the returned snapshot. +- No numeric generation semantics remain; all fingerprint decisions use equality/inequality only. + +## Risk and rollback + +- Rollback: revert the implementation commit for this phase. The feature adds no schema, migration, or persisted replay state; existing OAuth credentials remain readable. +- Behavioral risk: replaying a non-idempotent upstream operation. Chat completion generation is idempotent from the proxy's perspective at this boundary because replay is allowed only after the first attempt returned HTTP 401, before any response body was streamed or generation accepted; the rejected attempt produced no completion. The one-replay guard bounds residual provider anomalies. +- Auth risk: refreshing the wrong account or stale bearer. The sent snapshot includes exact provider/account/fingerprint, forced refresh remains account-scoped, and the replay adapter is rebuilt from the returned snapshot. +- Composition risk: 401→429 and 401→413 continue through the existing loop; 429/413→401 may consume the one 401 replay but never more. Targeted tests plus the existing recovery suites gate this. + +## Verification + +```bash +bun test --isolate ./tests/server-xai-oauth-401-replay.test.ts ./tests/server-key-failover-e2e.test.ts ./tests/server-images.test.ts +bun run typecheck +bun run privacy:scan +bun run test +``` + +Live verification after tests: restart the actual `ocx` process so the running proxy loads the new code, bind port 10100, then smoke the standard main-loop path (without web search): + +```bash +ocx stop +ocx start --port 10100 +curl -sS -D /tmp/ocx-xai-401.headers http://127.0.0.1:10100/v1/responses \ + -H 'content-type: application/json' \ + --data '{"model":"xai/grok-4.5","input":"Reply with exactly: ok","stream":false}' +``` + +Record process identity/restart evidence, HTTP status, response body, and redacted logs. A successful ordinary call proves the rebuilt serving path; deterministic 401 injection remains the test suite's responsibility. + +## Implementation record (B) + +### `src/oauth/index.ts` — account-aware OAuth access snapshots and forced xAI refresh + +Changes: Added an opaque `OAuthAccessSnapshot`, moved lazy token resolution onto a shared snapshot-valued `(provider, accountId)` singleflight, and exposed an xAI-only forced resolver keyed by the exact rejected generation. The resolver delegates persistence and cross-process coordination to the landed `refreshXaiAccountWithLock` body without modifying it. + +Impact: Concurrent lazy and rejected-token refreshes share one in-process promise. A superseded usable credential is returned as the persisted successor snapshot, and generation decisions remain equality-only. + +Verification: `bun test --isolate ./tests/server-xai-oauth-401-replay.test.ts ./tests/xai-refresh-lock.test.ts ./tests/oauth-refresh.test.ts` — 25 pass, 0 fail. `bun run typecheck` — pass. + +### `src/server/responses.ts` — bounded xAI OAuth 401 recovery + +Changes: Captured the exact OAuth snapshot used to build the first xAI adapter. Added one 401 recovery branch ahead of the existing 429/413 branches; it forces/adopts refresh, resolves transport and wire protocol from the returned token, builds a fresh adapter, and replays once. + +Impact: Ordinary xAI OAuth chat requests recover from one rejected bearer. A second 401 uses the existing redacted upstream-error path; API-key/BYOK and web-search sidecar paths remain excluded. + +Verification: New serving tests cover 401→200, 401→401, API-key exclusion, and concurrent singleflight. Full `bun test --isolate ./tests/` — 2621 pass, 0 fail across 245 files. + +### `tests/server-xai-oauth-401-replay.test.ts` — serving-path regression coverage + +Changes: Added four isolated end-to-end server tests with intercepted xAI discovery, token, and chat endpoints. The concurrent case uses explicit request and refresh barriers rather than timing sleeps. + +Impact: Locks the one-replay boundary, fresh bearer reconstruction, API-key exclusion, and one-IdP-call concurrency contract. + +Verification: Focused suite — 25 pass, 0 fail; full suite — 2621 pass, 0 fail; typecheck — pass. + +## Stale-check checklist + +- [ ] Rebase all symbols and imports against landed 030, especially `credentialGeneration`, its locked refresh primitive, and its post-lock generation re-check; use landed names rather than compatibility aliases. +- [ ] Confirm 030's fingerprint is a SHA-256 string over refresh + access + expiry and is compared only with `===`/`!==`. +- [ ] Re-read `src/server/responses.ts`; confirm ordinary fetch/non-success flow and web-search early return have not moved. +- [ ] Confirm `route`, `route.providerName`, `route.provider.authMode`, `route.modelId`, `parsed.options.promptCacheKey`, `selectedForwardHeaders`, and `config.cacheRetention` remain in scope. +- [ ] Confirm `resolveProviderTransport`, `resolveWireProtocolOverride`, and `resolveAdapter` signatures are unchanged. +- [ ] Confirm current 429 and 413 recovery ordering and update the complete replacement block rather than dropping a landed branch. +- [ ] Run `rg -n "generation.*[<>]|[<>].*generation|generation: number|rejectedGeneration: number" src tests devlog/_plan/260716_grok_build_hardening` and require zero 040-contract violations. +- [ ] Run `rg -n "runWithWebSearch|const request = await adapter.buildRequest|if \(!upstreamResponse.ok\)" src/server/responses.ts` and preserve the documented sidecar residual. +- [ ] Confirm test endpoint constants against landed xAI discovery and transport URLs. +- [ ] Confirm no sibling worker created the test filename and final implementation touches only the three mapped paths. +- [ ] Run all verification commands fresh, restart `ocx` on 10100, and record evidence before DONE. diff --git a/devlog/_fin/260716_grok_build_hardening/050_header_parity.md b/devlog/_fin/260716_grok_build_hardening/050_header_parity.md new file mode 100644 index 00000000..7c4e108b --- /dev/null +++ b/devlog/_fin/260716_grok_build_hardening/050_header_parity.md @@ -0,0 +1,766 @@ +# 050 — xAI request-header parity (Round-3 revision) + +## Work-phase contract + +**Runs after:** `010` reasoning folding. This phase owns both the xAI transport wrapper and the +server executor seam that makes it reachable from real serving traffic. This document amendment +is design-only; implementation changes production code in the paths listed below. + +**Finding closed:** emit the honest request-scoped subset of the official Grok header contract. +Conversation/session identity is computed once per resolved transport. A fresh +`x-grok-req-id` is generated by that transport's fetch wrapper for each invocation. + +### Round-3 ownership decision (R3-3) + +Do **not** add `fetchResponse` to `createOpenAIChatAdapter`. The server selects +`adapter.fetchResponse` ahead of its fallback at `src/server/responses.ts:946-953` and +`:978-982`; an adapter hook would therefore bypass `fetchWithHeaderTimeout`, whose timer and +combined abort signal live at `src/server/responses.ts:1228-1253`. That regression can hang after +xAI accepts a connection but never returns response headers. + +The checked-in tree proves why the server seam belongs here: + +- `resolveProviderTransport` currently returns only base URL/static headers + (`src/providers/xai-transport.ts:62-88`); there is not yet a transport fetch wrapper. +- `OcxProviderConfig` currently has no fetch member (`src/types.ts:559-590`). +- the ordinary server fallback calls `fetchWithHeaderTimeout` at + `src/server/responses.ts:948-953` and recovery calls it at `:978-982`; +- `fetchWithHeaderTimeout` currently invokes global `fetch` directly at + `src/server/responses.ts:1245-1250`, so a new `provider.fetch` would not be consumed today. + +Phase 050 therefore modifies `src/server/responses.ts`: `fetchWithHeaderTimeout` accepts an +executor whose default is `globalThis.fetch`, and every real serving caller passes +`route.provider.fetch ?? globalThis.fetch`. The executor is called only inside the existing +`try`/timer/combined-signal region. The required call graph is: + +```text +server passthrough/ordinary/recovery/replay branch + -> fetchWithHeaderTimeout(url, init, upstream.signal, connectMs, parsed.stream, provider.fetch) + -> provider.fetch(url, { ...init, signal: AbortSignal.any([...]) }) + -> xAI wrapper generates x-grok-req-id and delegates to globalThis.fetch +``` + +This keeps `connectTimeoutMs` and stream identity-encoding semantics exactly where they are today. +Never substitute an adapter hook. + +## Scope + +### IN + +- `src/providers/xai-transport.ts`: compatibility constants, stable conv/session headers, always-on + User-Agent, and per-invocation request-ID transport fetch wrapper. +- `src/types.ts`: add `fetch?: typeof globalThis.fetch` if the field is still absent after rebase. +- `src/server/responses.ts`: add the timeout-wrapped executor seam and pass the current resolved + `route.provider.fetch` at every `fetchWithHeaderTimeout` call, including 040 replay code. +- `tests/xai-transport.test.ts`: direct transport-fetch snapshots and same-resolved-transport + attempt-boundary tests. +- `tests/server-xai-header-parity.test.ts`: real `/v1/responses` request-ID/affinity and hanging- + header timeout coverage. +- `structure/04_transports-and-sidecars.md`: document the landed transport contract during phase D. + +### OUT + +- `src/adapters/openai-chat.ts`; no `fetchResponse` hook or adapter behavior change. +- Model-discovery headers in `src/oauth/index.ts`. +- Fabricated model, agent, deployment, turn, mode, or user identity. +- Auth, routing, cache hashing, request body, timeout, or stream behavior changes. + +## File change map + +| Action | Path | Purpose | +|---|---|---| +| MODIFY | `src/providers/xai-transport.ts` | Stable affinity headers and per-attempt request-ID wrapper. | +| MODIFY | `src/types.ts` | Optional provider transport executor type. | +| MODIFY | `src/server/responses.ts` | Timeout-preserving executor seam at every serving/replay caller. | +| MODIFY | `tests/xai-transport.test.ts` | Direct transport contract and attempt tests. | +| ADD | `tests/server-xai-header-parity.test.ts` | Real serving-path header and timeout proof. | +| MODIFY | `structure/04_transports-and-sidecars.md` | Landed transport contract documentation. | + +## Header decisions + +| Header | Decision | +|---|---| +| `x-grok-conv-id` | Stable hash when `promptCacheKey.trim()` is nonblank. | +| `x-grok-session-id` | Same stable opaque hash and omission rule as conv-id. | +| `x-grok-req-id` | Fresh UUIDv4 inside every transport-fetch invocation. Caller override wins. | +| `User-Agent` | Always `opencodex-grok/${version}` in OAuth and API-key modes. Caller override wins. | +| `x-grok-client-identifier`, `x-grok-client-version` | OAuth only. | +| `X-XAI-Token-Auth`, `x-authenticateresponse` | OAuth only. | +| model override, agent ID, turn index, deployment ID, user ID, client mode | **OMIT**; no truthful local value exists. | + +## Complete diff specification + +### `src/providers/xai-transport.ts` + +The current imports begin with only `createHash` (`src/providers/xai-transport.ts:1-2`). Replace +the file's import/header-helper/resolve implementation with the following complete compilable +form; retain any unrelated comments only if they remain accurate: + +```ts +import { createHash, randomUUID } from "node:crypto"; +import type { OcxProviderConfig } from "../types"; + +export const XAI_GROK_CLI_BASE_URL = "https://cli-chat-proxy.grok.com/v1"; + +export const XAI_GROK_COMPATIBILITY = { + version: "0.2.93", + userAgent: "opencodex-grok/0.2.93", + headers: { + clientIdentifier: "x-grok-client-identifier", + clientVersion: "x-grok-client-version", + tokenAuth: "x-xai-token-auth", + authenticateResponse: "x-authenticateresponse", + conversationId: "x-grok-conv-id", + requestId: "x-grok-req-id", + sessionId: "x-grok-session-id", + userAgent: "User-Agent", + }, +} as const; + +export const XAI_GROK_CLIENT_VERSION = XAI_GROK_COMPATIBILITY.version; +export const XAI_CONV_ID_HEADER = XAI_GROK_COMPATIBILITY.headers.conversationId; + +const XAI_GROK_CLI_HEADERS: Readonly> = { + [XAI_GROK_COMPATIBILITY.headers.clientIdentifier]: "opencodex", + [XAI_GROK_COMPATIBILITY.headers.clientVersion]: XAI_GROK_CLIENT_VERSION, + [XAI_GROK_COMPATIBILITY.headers.tokenAuth]: "xai-grok-cli", + [XAI_GROK_COMPATIBILITY.headers.authenticateResponse]: "authenticate-response", +}; + +function hasHeaderCaseInsensitive( + headers: Record | undefined, + name: string, +): boolean { + const target = name.toLowerCase(); + return Object.keys(headers ?? {}).some(key => key.toLowerCase() === target); +} + +function withoutUserOverridden( + defaults: Readonly>, + userHeaders: Record | undefined, +): Record { + return Object.fromEntries( + Object.entries(defaults).filter(([name]) => !hasHeaderCaseInsensitive(userHeaders, name)), + ); +} + +function withGeneratedRequestId( + init: RequestInit | undefined, + configuredRequestId: string | undefined, +): RequestInit { + const headers = new Headers(init?.headers); + if (!headers.has(XAI_GROK_COMPATIBILITY.headers.requestId)) { + headers.set( + XAI_GROK_COMPATIBILITY.headers.requestId, + configuredRequestId ?? randomUUID(), + ); + } + return { ...init, headers }; +} + +function findHeaderCaseInsensitive( + headers: Record | undefined, + name: string, +): string | undefined { + const target = name.toLowerCase(); + return Object.entries(headers ?? {}).find(([key]) => key.toLowerCase() === target)?.[1]; +} + +export function deriveXaiConvId(promptCacheKey: string): string { + return createHash("sha256").update(promptCacheKey).digest("hex").slice(0, 32); +} + +export function resolveProviderTransport( + providerName: string, + provider: OcxProviderConfig, + promptCacheKey?: string, +): OcxProviderConfig { + if (providerName !== "xai") return provider; + + const cacheKey = promptCacheKey?.trim(); + const affinity = cacheKey ? deriveXaiConvId(cacheKey) : undefined; + const stableDefaults: Record = { + [XAI_GROK_COMPATIBILITY.headers.userAgent]: XAI_GROK_COMPATIBILITY.userAgent, + ...(affinity + ? { + [XAI_GROK_COMPATIBILITY.headers.conversationId]: affinity, + [XAI_GROK_COMPATIBILITY.headers.sessionId]: affinity, + } + : {}), + ...(provider.authMode === "oauth" ? XAI_GROK_CLI_HEADERS : {}), + }; + const headers = { + ...withoutUserOverridden(stableDefaults, provider.headers), + ...(provider.headers ?? {}), + }; + const configuredRequestId = findHeaderCaseInsensitive( + provider.headers, + XAI_GROK_COMPATIBILITY.headers.requestId, + ); + const baseFetch = provider.fetch ?? globalThis.fetch; + const attemptFetch: typeof globalThis.fetch = (input, init) => + baseFetch(input, withGeneratedRequestId(init, configuredRequestId)); + + return { + ...provider, + ...(provider.authMode === "oauth" ? { baseUrl: XAI_GROK_CLI_BASE_URL } : {}), + headers, + fetch: attemptFetch, + }; +} +``` + +Important boundary: UUID generation is in `attemptFetch`'s call body. Stable conv/session values +are resolved once into `headers`. `Headers.has` makes caller request-header precedence +case-insensitive, while `configuredRequestId` preserves a config-level override even if the +server reconstructs request headers. + +### `src/types.ts` (only if absent after rebase) + +Before (`src/types.ts:559-562`): + +```ts +export interface OcxProviderConfig { + adapter: string; + baseUrl: string; +``` + +After: + +```ts +export interface OcxProviderConfig { + adapter: string; + baseUrl: string; + /** Transport executor used inside the server's header-timeout wrapper. Never persisted. */ + fetch?: typeof globalThis.fetch; +``` + +No adapter diff exists. + +### `src/server/responses.ts` — executor seam and every current caller + +Before (`src/server/responses.ts:1228-1253`): + +```ts +export async function fetchWithHeaderTimeout( + url: string, + init: Omit, + abortSignal: AbortSignal, + timeoutMs: number, + preferIdentityEncoding = false, +): Promise { + const timeout = new AbortController(); + const timer = setTimeout(() => { + if (!timeout.signal.aborted) timeout.abort(new DOMException("Timeout elapsed", "TimeoutError")); + }, timeoutMs); + const headers = new Headers(init.headers); + if (preferIdentityEncoding && !headers.has("accept-encoding")) { + headers.set("accept-encoding", "identity"); + } + try { + return await fetch(url, { + ...init, + headers, + signal: AbortSignal.any([abortSignal, timeout.signal]), + }); + } finally { + clearTimeout(timer); + } +} +``` + +After (the timer, header mutation, race, and cleanup are intentionally unchanged): + +```ts +export async function fetchWithHeaderTimeout( + url: string, + init: Omit, + abortSignal: AbortSignal, + timeoutMs: number, + preferIdentityEncoding = false, + executor: typeof globalThis.fetch = globalThis.fetch, +): Promise { + const timeout = new AbortController(); + const timer = setTimeout(() => { + if (!timeout.signal.aborted) timeout.abort(new DOMException("Timeout elapsed", "TimeoutError")); + }, timeoutMs); + const headers = new Headers(init.headers); + if (preferIdentityEncoding && !headers.has("accept-encoding")) { + headers.set("accept-encoding", "identity"); + } + try { + return await executor(url, { + ...init, + headers, + signal: AbortSignal.any([abortSignal, timeout.signal]), + }); + } finally { + clearTimeout(timer); + } +} +``` + +Update all three call sites present in the checked-in source. Passthrough/sidecar before +(`src/server/responses.ts:700-705`): + +```ts +() => fetchWithHeaderTimeout(request.url, { + method: request.method, + headers: request.headers, + body: request.body, +}, upstream.signal, connectMs, parsed.stream), +``` + +After: + +```ts +() => fetchWithHeaderTimeout(request.url, { + method: request.method, + headers: request.headers, + body: request.body, +}, upstream.signal, connectMs, parsed.stream, route.provider.fetch ?? globalThis.fetch), +``` + +Ordinary path before (`src/server/responses.ts:948-952`): + +```ts +() => fetchWithHeaderTimeout(request.url, { + method: request.method, headers: request.headers, body: request.body, +}, upstream.signal, connectMs, parsed.stream), +``` + +After: + +```ts +() => fetchWithHeaderTimeout(request.url, { + method: request.method, headers: request.headers, body: request.body, +}, upstream.signal, connectMs, parsed.stream, route.provider.fetch ?? globalThis.fetch), +``` + +Recovery path before (`src/server/responses.ts:978-982`): + +```ts +return activeAdapter.fetchResponse + ? await activeAdapter.fetchResponse(retryRequest, { abortSignal: upstream.signal, timeoutMs: connectMs, stream: parsed.stream }) + : await fetchWithHeaderTimeout(retryRequest.url, { + method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, + }, upstream.signal, connectMs, parsed.stream); +``` + +After: + +```ts +return activeAdapter.fetchResponse + ? await activeAdapter.fetchResponse(retryRequest, { abortSignal: upstream.signal, timeoutMs: connectMs, stream: parsed.stream }) + : await fetchWithHeaderTimeout(retryRequest.url, { + method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, + }, upstream.signal, connectMs, parsed.stream, route.provider.fetch ?? globalThis.fetch); +``` + +The recovery expression deliberately reads mutable `route.provider` at invocation time: a 429 +rotation replaces it before `rebuildAndRefetch`, so the rotated transport wrapper is used. After +rebasing over phase 040, apply the identical final argument to **every** newly landed replay fetch; +do not capture a pre-replay/pre-rotation executor. `rg -n 'fetchWithHeaderTimeout'` must show no +serving caller omitted from this audit. + +End to end, a server caller supplies the current provider executor to +`fetchWithHeaderTimeout`; that function invokes it with the same combined timeout signal; the xAI +transport wrapper then adds a fresh UUID in its per-call body and delegates to its base fetch. +Stable conv/session headers were computed once when the route transport was resolved. + +### `tests/xai-transport.test.ts` — complete replacement header suite + +Retain unrelated model-discovery, tool-schema, parser/reasoning tests. Add `Headers`-level tests +that invoke `effective.fetch` directly for isolated transport coverage. Serving timeout +composition is additionally covered through the phase-owned real server path below. + +```ts +import { describe, expect, test } from "bun:test"; +import { createOpenAIChatAdapter } from "../src/adapters/openai-chat"; +import { + deriveXaiConvId, + resolveProviderTransport, + XAI_GROK_CLI_BASE_URL, + XAI_GROK_CLIENT_VERSION, +} from "../src/providers/xai-transport"; +import type { OcxProviderConfig } from "../src/types"; + +const UUID_V4 = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; +const OMITTED = [ + "x-grok-model-override", + "x-grok-agent-id", + "x-grok-turn-idx", + "x-grok-deployment-id", + "x-grok-user-id", + "x-grok-client-mode", +] as const; + +function provider(authMode: "oauth" | "key"): OcxProviderConfig { + return { + adapter: "openai-chat", + baseUrl: "https://api.x.ai/v1", + authMode, + apiKey: authMode === "oauth" ? "oauth-token" : "xai-api-key", + defaultModel: "grok-4.5", + }; +} + +function parsed() { + return { + modelId: "grok-4.5", + context: { messages: [{ role: "user" as const, content: "hi", timestamp: 0 }] }, + stream: false, + options: { reasoning: "low" as const }, + }; +} + +function lower(headers: Headers): Record { + return Object.fromEntries( + [...headers.entries()].sort(([a], [b]) => a.localeCompare(b)), + ); +} + +async function capture(authMode: "oauth" | "key", calls = 1) { + const seen: Headers[] = []; + const configured = provider(authMode); + configured.fetch = async (_input, init) => { + seen.push(new Headers(init?.headers)); + return new Response("{}", { status: 200 }); + }; + const effective = resolveProviderTransport("xai", configured, "codex-session-abc"); + const request = createOpenAIChatAdapter(effective).buildRequest(parsed()); + for (let index = 0; index < calls; index += 1) { + await effective.fetch!(request.url, { + method: request.method, + headers: request.headers, + body: request.body, + }); + } + return { effective, seen }; +} + +describe("xAI outbound compatibility headers", () => { + test("OAuth snapshot is exact", async () => { + const { effective, seen } = await capture("oauth"); + expect(effective.baseUrl).toBe(XAI_GROK_CLI_BASE_URL); + expect(lower(seen[0])).toEqual({ + authorization: "Bearer oauth-token", + "content-type": "application/json", + "user-agent": `opencodex-grok/${XAI_GROK_CLIENT_VERSION}`, + "x-authenticateresponse": "authenticate-response", + "x-grok-client-identifier": "opencodex", + "x-grok-client-version": XAI_GROK_CLIENT_VERSION, + "x-grok-conv-id": deriveXaiConvId("codex-session-abc"), + "x-grok-req-id": expect.stringMatching(UUID_V4), + "x-grok-session-id": deriveXaiConvId("codex-session-abc"), + "x-xai-token-auth": "xai-grok-cli", + }); + for (const name of OMITTED) expect(seen[0].has(name)).toBe(false); + }); + + test("API-key snapshot is exact and User-Agent is present", async () => { + const { effective, seen } = await capture("key"); + expect(effective.baseUrl).toBe("https://api.x.ai/v1"); + expect(lower(seen[0])).toEqual({ + authorization: "Bearer xai-api-key", + "content-type": "application/json", + "user-agent": `opencodex-grok/${XAI_GROK_CLIENT_VERSION}`, + "x-grok-conv-id": deriveXaiConvId("codex-session-abc"), + "x-grok-req-id": expect.stringMatching(UUID_V4), + "x-grok-session-id": deriveXaiConvId("codex-session-abc"), + }); + for (const name of [ + "x-authenticateresponse", + "x-grok-client-identifier", + "x-grok-client-version", + "x-xai-token-auth", + ...OMITTED, + ]) expect(seen[0].has(name)).toBe(false); + }); + + test("same resolved transport refreshes req-id but keeps conv-id stable", async () => { + const { seen } = await capture("oauth", 2); + expect(seen).toHaveLength(2); + expect(seen[0].get("x-grok-req-id")).toMatch(UUID_V4); + expect(seen[1].get("x-grok-req-id")).toMatch(UUID_V4); + expect(seen[1].get("x-grok-req-id")).not.toBe(seen[0].get("x-grok-req-id")); + expect(seen[0].get("x-grok-conv-id")).toBe(deriveXaiConvId("codex-session-abc")); + expect(seen[1].get("x-grok-conv-id")).toBe(seen[0].get("x-grok-conv-id")); + expect(seen[1].get("x-grok-session-id")).toBe(seen[0].get("x-grok-session-id")); + for (const headers of seen) { + expect(headers.get("user-agent")).toBe(`opencodex-grok/${XAI_GROK_CLIENT_VERSION}`); + for (const name of OMITTED) expect(headers.has(name)).toBe(false); + } + }); + + test("mixed-case caller overrides win without duplicates", async () => { + const seen: Headers[] = []; + const configured = provider("oauth"); + configured.headers = { "user-agent": "custom-agent", "X-Grok-Req-Id": "caller-id" }; + configured.fetch = async (_input, init) => { + seen.push(new Headers(init?.headers)); + return new Response("{}", { status: 200 }); + }; + const effective = resolveProviderTransport("xai", configured, "codex-session-abc"); + const request = createOpenAIChatAdapter(effective).buildRequest(parsed()); + await effective.fetch!(request.url, { headers: request.headers }); + await effective.fetch!(request.url, { headers: request.headers }); + for (const headers of seen) { + expect(headers.get("user-agent")).toBe("custom-agent"); + expect(headers.get("x-grok-req-id")).toBe("caller-id"); + expect([...headers.keys()].filter(name => name === "user-agent")).toHaveLength(1); + expect([...headers.keys()].filter(name => name === "x-grok-req-id")).toHaveLength(1); + } + }); + + test("blank cache keys omit affinity but retain UA and fresh req-id in both modes", async () => { + for (const authMode of ["oauth", "key"] as const) { + const seen: Headers[] = []; + const configured = provider(authMode); + configured.fetch = async (_input, init) => { + seen.push(new Headers(init?.headers)); + return new Response("{}", { status: 200 }); + }; + const effective = resolveProviderTransport("xai", configured, " "); + const request = createOpenAIChatAdapter(effective).buildRequest(parsed()); + await effective.fetch!(request.url, { headers: request.headers }); + expect(seen[0].has("x-grok-conv-id")).toBe(false); + expect(seen[0].has("x-grok-session-id")).toBe(false); + expect(seen[0].get("user-agent")).toBe(`opencodex-grok/${XAI_GROK_CLIENT_VERSION}`); + expect(seen[0].get("x-grok-req-id")).toMatch(UUID_V4); + for (const name of OMITTED) expect(seen[0].has(name)).toBe(false); + } + }); +}); +``` + +### `tests/server-xai-header-parity.test.ts` — complete real-path test + +Add this complete file. It follows the existing `Bun.serve` + `saveConfig` + `startServer(0)` +convention from `tests/anthropic-image-retry-e2e.test.ts`. The second test's upstream accepts the +request but its handler never resolves, so no response headers arrive; the public endpoint must +surface the server's `TimeoutError` mapping. + +```ts +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { saveConfig } from "../src/config"; +import { deriveXaiConvId } from "../src/providers/xai-transport"; +import { startServer } from "../src/server"; +import type { OcxConfig } from "../src/types"; +import { installIsolatedCodexHome, type IsolatedCodexHome } from "./helpers/isolated-codex-home"; + +const UUID_V4 = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; +const CONV_KEY = "server-parity-conversation"; + +let testDir = ""; +let previousHome: string | undefined; +let isolatedCodexHome: IsolatedCodexHome | null = null; +let upstream: ReturnType | null = null; + +beforeEach(() => { + previousHome = process.env.OPENCODEX_HOME; + isolatedCodexHome = installIsolatedCodexHome("ocx-xai-parity-codex-"); + testDir = mkdtempSync(join(tmpdir(), "ocx-xai-parity-")); + process.env.OPENCODEX_HOME = testDir; +}); + +afterEach(() => { + upstream?.stop(true); + upstream = null; + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + isolatedCodexHome?.restore(); + isolatedCodexHome = null; + if (testDir) rmSync(testDir, { recursive: true, force: true }); +}); + +function config(baseUrl: string, connectTimeoutMs = 1_000): OcxConfig { + return { + port: 0, + hostname: "127.0.0.1", + connectTimeoutMs, + defaultProvider: "xai", + providers: { + xai: { + adapter: "openai-chat", + baseUrl, + allowPrivateNetwork: true, + authMode: "key", + apiKey: "xai-test-key-000111222333", + defaultModel: "grok-test", + }, + }, + } as OcxConfig; +} + +function post(serverUrl: string): Promise { + return fetch(new URL("/v1/responses", serverUrl), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "xai/grok-test", + stream: false, + prompt_cache_key: CONV_KEY, + input: "hello", + }), + }); +} + +describe("xAI headers through /v1/responses", () => { + test("two serving attempts refresh request id and preserve conversation affinity", async () => { + const seen: Headers[] = []; + upstream = Bun.serve({ + hostname: "127.0.0.1", + port: 0, + fetch(req) { + seen.push(new Headers(req.headers)); + return Response.json({ + id: `chatcmpl-${seen.length}`, + object: "chat.completion", + created: 1, + model: "grok-test", + choices: [{ index: 0, message: { role: "assistant", content: "ok" }, finish_reason: "stop" }], + usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 }, + }); + }, + }); + saveConfig(config(upstream.url.toString().replace(/\/$/, ""))); + const server = startServer(0); + try { + expect((await post(String(server.url))).status).toBe(200); + expect((await post(String(server.url))).status).toBe(200); + expect(seen).toHaveLength(2); + const firstReq = seen[0].get("x-grok-req-id"); + const secondReq = seen[1].get("x-grok-req-id"); + expect(firstReq).toMatch(UUID_V4); + expect(secondReq).toMatch(UUID_V4); + expect(secondReq).not.toBe(firstReq); + expect(seen[0].get("x-grok-conv-id")).toBe(deriveXaiConvId(CONV_KEY)); + expect(seen[1].get("x-grok-conv-id")).toBe(seen[0].get("x-grok-conv-id")); + } finally { + await server.stop(true); + } + }); + + test("provider executor remains inside the header-timeout race", async () => { + let accepted = false; + upstream = Bun.serve({ + hostname: "127.0.0.1", + port: 0, + async fetch() { + accepted = true; + return await new Promise(() => {}); + }, + }); + saveConfig(config(upstream.url.toString().replace(/\/$/, ""), 25)); + const server = startServer(0); + try { + const response = await post(String(server.url)); + expect(accepted).toBe(true); + expect(response.status).toBe(502); + expect(await response.text()).toContain("Provider connect timeout after 25ms"); + } finally { + await server.stop(true); + } + }); +}); +``` + +Keep all direct transport-level tests already specified above. The real-path test is additive, +not a replacement for exact header snapshots and direct same-transport attempt coverage. + +## Acceptance criteria + +- No adapter `fetchResponse` is added; `src/adapters/openai-chat.ts` remains unchanged. +- `src/server/responses.ts` has the executor parameter and every serving caller passes the current + provider executor; non-xAI providers retain global-fetch behavior. +- Rebased serving code invokes the xAI transport fetch only from inside + `fetchWithHeaderTimeout`; its timer, combined abort signal, and stream identity behavior remain. +- Two sequential calls of the same resolved transport fetch have different UUIDv4 request IDs and + equal conv/session IDs. +- Two real `/v1/responses` calls with one conversation key expose distinct request IDs and stable + conversation IDs at the mock upstream. +- A real `/v1/responses` call to an upstream that accepts but emits no headers returns the existing + connect-timeout error, proving the executor remained inside the timeout race. +- OAuth and API-key exact snapshots include User-Agent; all OMIT assertions remain. +- Blank cache keys omit conv/session only, not User-Agent/request ID. +- User overrides win case-insensitively without duplicates. + +## Risk and rollback + +- **Timeout regression:** invoking provider fetch outside `fetchWithHeaderTimeout` would lose the + header deadline. Mitigation is the real serving-path hanging-header test. +- **Stale executor after rotation/replay:** capturing an old provider fetch would omit new key or + attempt state. Every caller reads current `route.provider.fetch`, and 040 replay is re-audited. +- **Attempt identity:** generating UUID during resolve would reuse it; generating in the wrapper + call body plus the same-transport two-call test prevents this. +- **Header compatibility:** exact OAuth/API-key snapshots catch accidental fabricated or + subscription-only headers. +- **Rollback:** revert the 050 implementation commit, including the optional executor parameter and + caller arguments in `src/server/responses.ts`. Its default preserves global fetch for callers + during a partial rollback; no persisted schema/state is written. + +## Verification commands + +```bash +bun test --isolate ./tests/xai-transport.test.ts +bun test --isolate ./tests/server-xai-header-parity.test.ts +bun test --isolate ./tests/server-xai-oauth-401-replay.test.ts +bun run typecheck +bun run test +bun run privacy:scan +``` + +## Stale-check checklist + +- [ ] Re-open `src/providers/xai-transport.ts:1-88`; rebase the complete block over landed work. +- [ ] Re-open `src/types.ts:559-590`; confirm the transport fetch field and persistence treatment. +- [ ] Rebase over 040's landed `src/server/responses.ts` changes before applying this phase. +- [ ] Re-open every `fetchWithHeaderTimeout` occurrence in `src/server/responses.ts`; prove provider fetch is invoked **inside** + `fetchWithHeaderTimeout`, not from `adapter.fetchResponse` and not beside the timeout wrapper. +- [ ] Confirm passthrough/sidecar, ordinary, 429/413 recovery, and 040 401 replay all pass the + current executor. In particular, 040's replay fetch must use it so every replay gets a fresh ID. +- [ ] Confirm `parsed.stream` still reaches `preferIdentityEncoding` and `connectMs` still arms the + header timer; the xAI wrapper must receive the already-combined abort signal untouched. +- [ ] Run the phase-owned real-path hanging-header timeout test and both request-ID assertions. +- [ ] Run `rg -n 'fetchResponse|provider\.fetch|fetchWithHeaderTimeout|randomUUID|x-grok-req-id' src tests`; + reject any new OpenAI-chat adapter hook or resolve-time UUID. +- [ ] Preserve `010` parser/reasoning tests, model discovery, tool-schema tests, OMIT assertions, + OAuth/API-key snapshots, blank-affinity coverage, and mixed-case overrides. +- [ ] Confirm `git diff --name-only` lists only phase-owned implementation paths and that this + planning worker changed only `devlog/_plan/260716_grok_build_hardening/050_header_parity.md`. + +## Implementation record (B) + +Implemented 2026-07-16 on top of `1bf9de02` without changing the landed 010-040 behavior. + +- Added the one-line-bump `XAI_GROK_COMPATIBILITY` profile, always-on outbound User-Agent, + OAuth-only official compatibility headers, stable hashed conversation/session affinity, and a + fresh UUIDv4 request ID generated inside every resolved xAI transport-fetch invocation. +- Kept agent, deployment, model-override, turn, client-mode, and user identity headers omitted: + opencodex has no truthful local values for those official fields. +- Added the executor argument to `fetchWithHeaderTimeout` and passed the current resolved provider + executor at all three serving call sites. `rebuildAndRefetch` performs the lookup at invocation + time, preserving 401 refresh and 429 rotation behavior. +- Added exact OAuth/API-key outbound snapshots, per-attempt request-ID rotation and stable affinity + coverage, caller-override/blank-key/OMIT checks, plus real `/v1/responses` serving and hanging- + header timeout tests. +- Scope adaptation: `src/types.ts` remained untouched as required by the delegated write scope. + The runtime-only executor shape is declared locally by `xai-transport.ts` and read structurally + by `responses.ts`; it is not part of persisted config. API-key UA/session defaults are applied + by the transport wrapper instead of exposed in the static provider header map, preserving the + landed key-failover contract while producing the specified exact outbound headers. +- Test-fixture adaptation: the registered xAI provider does not permit base-URL override, so the + server-path tests use the repository's established xAI global-fetch interception seam instead + of an unreachable localhost override. They still enter through real `/v1/responses`; the timeout + mock accepts the provider call and rejects only when the combined timeout signal aborts before + headers arrive. + +Verification: + +- `bun test --isolate ./tests/xai-transport.test.ts ./tests/server-xai-header-parity.test.ts ./tests/server-xai-oauth-401-replay.test.ts`: 30 pass, 0 fail. +- `bun test --isolate ./tests/`: 2628 pass, 0 fail across 246 files. +- `bun run typecheck`: pass (`tsc --noEmit`). diff --git a/gui/src/components/AddCodexAccountModal.tsx b/gui/src/components/AddCodexAccountModal.tsx index 7e6c4eae..c05b8b4a 100644 --- a/gui/src/components/AddCodexAccountModal.tsx +++ b/gui/src/components/AddCodexAccountModal.tsx @@ -14,7 +14,6 @@ export default function AddCodexAccountModal({ const pollRef = useRef | null>(null); const timeoutRef = useRef | null>(null); const flowRef = useRef(null); - const popupRef = useRef(null); useEffect(() => () => { aliveRef.current = false; if (pollRef.current) { clearInterval(pollRef.current); pollRef.current = null; } @@ -35,7 +34,6 @@ export default function AddCodexAccountModal({ const cancelLogin = async () => { const flowId = flowRef.current; flowRef.current = null; - popupRef.current = null; setAuthUrl(""); stopPolling(); if (!flowId) return; @@ -113,8 +111,6 @@ export default function AddCodexAccountModal({ if (data.url) { flowRef.current = data.flowId ?? null; setAuthUrl(data.url); - popupRef.current = window.open(data.url, "_blank"); - if (popupRef.current) popupRef.current.opener = null; setStep("oauth-waiting"); stopPolling(); const fid = data.flowId ?? ""; @@ -127,17 +123,12 @@ export default function AddCodexAccountModal({ if (st.status === "done") { stopPolling(); flowRef.current = null; - popupRef.current = null; onAdded(); onClose(); } else if (st.status === "error" || st.status === "expired") { stopPolling(); flowRef.current = null; - popupRef.current = null; if (aliveRef.current) { setStep("pick"); setError(st.error ?? "Login failed"); } - } else if (popupRef.current?.closed) { - await cancelLogin(); - if (aliveRef.current) { setStep("pick"); setError(t("codexAuth.oauthCancelled")); } } } catch { /* ignore network errors during polling */ } }, 2000); diff --git a/gui/src/components/AddProviderModal.tsx b/gui/src/components/AddProviderModal.tsx index 853b3c5b..0603faab 100644 --- a/gui/src/components/AddProviderModal.tsx +++ b/gui/src/components/AddProviderModal.tsx @@ -174,7 +174,7 @@ export default function AddProviderModal({ // A non-empty url = browser/device flow (the server also opens it). An EMPTY url with a 200 = // a local-token import (e.g. Anthropic's Claude Code keychain, Grok CLI) that needs no browser // — just poll status until the credential lands. Don't treat empty url as a failure. - if (data.url) { window.open(data.url, "_blank"); setOauthMsg(t("modal.waitingLogin")); } + if (data.url) { setOauthMsg(t("modal.waitingLogin")); } else { setOauthMsg(data.instructions || t("modal.loggingIn")); } for (let i = 0; i < 100; i++) { await new Promise(r => setTimeout(r, 2000)); diff --git a/gui/src/i18n/de.ts b/gui/src/i18n/de.ts index 145f2068..33cac6bf 100644 --- a/gui/src/i18n/de.ts +++ b/gui/src/i18n/de.ts @@ -123,6 +123,8 @@ export const de = { "prov.loginWith": "Anmelden mit {provider}", "prov.waitingBrowser": "Warten auf Browser…", "prov.didntOpen": "Hat sich nicht geöffnet? Hier klicken", + "prov.copyLink": "Link kopieren", + "prov.linkCopied": "Kopiert", "prov.pasteRedirect": "Redirect-URL oder Code einfügen", "prov.pasteRedirectHint": "Zeigt der Browser einen localhost-Fehler, kopiere die vollständige URL aus der Adressleiste und füge sie hier ein (oder den Autorisierungscode).", "prov.pasteSubmit": "Senden", diff --git a/gui/src/i18n/en.ts b/gui/src/i18n/en.ts index b2b811ea..c3e4c49c 100644 --- a/gui/src/i18n/en.ts +++ b/gui/src/i18n/en.ts @@ -134,6 +134,8 @@ export const en = { "prov.loginWith": "Login with {provider}", "prov.waitingBrowser": "Waiting for browser…", "prov.didntOpen": "Didn't open? Click here", + "prov.copyLink": "Copy link", + "prov.linkCopied": "Copied", "prov.pasteRedirect": "Paste redirect URL or code", "prov.pasteRedirectHint": "If the browser shows a localhost error, copy the full URL from its address bar and paste it here (or paste the authorization code).", "prov.pasteSubmit": "Submit", diff --git a/gui/src/i18n/ko.ts b/gui/src/i18n/ko.ts index 58e8a9bd..d1cb131f 100644 --- a/gui/src/i18n/ko.ts +++ b/gui/src/i18n/ko.ts @@ -129,6 +129,8 @@ export const ko: Record = { "prov.loginWith": "{provider} 로 로그인", "prov.waitingBrowser": "브라우저 대기 중…", "prov.didntOpen": "안 열렸나요? 여기를 클릭하세요", + "prov.copyLink": "링크 복사", + "prov.linkCopied": "복사됨", "prov.pasteRedirect": "리다이렉트 URL 또는 코드 붙여넣기", "prov.pasteRedirectHint": "브라우저에 localhost 오류가 표시되면, 주소창의 전체 URL을 복사해 여기에 붙여넣으세요(또는 인증 코드 붙여넣기).", "prov.pasteSubmit": "제출", diff --git a/gui/src/i18n/zh.ts b/gui/src/i18n/zh.ts index 738900a6..73d3ae5f 100644 --- a/gui/src/i18n/zh.ts +++ b/gui/src/i18n/zh.ts @@ -129,6 +129,8 @@ export const zh: Record = { "prov.loginWith": "使用 {provider} 登录", "prov.waitingBrowser": "等待浏览器…", "prov.didntOpen": "没有打开？点击这里", + "prov.copyLink": "复制链接", + "prov.linkCopied": "已复制", "prov.pasteRedirect": "粘贴重定向 URL 或授权码", "prov.pasteRedirectHint": "如果浏览器显示 localhost 错误，请复制地址栏中的完整 URL 并粘贴到此处（或粘贴授权码）。", "prov.pasteSubmit": "提交", diff --git a/gui/src/pages/Providers.tsx b/gui/src/pages/Providers.tsx index 05491144..981ae5d4 100644 --- a/gui/src/pages/Providers.tsx +++ b/gui/src/pages/Providers.tsx @@ -1,7 +1,7 @@ import { useCallback, useEffect, useMemo, useRef, useState } from "react"; import AddProviderModal from "../components/AddProviderModal"; import { Notice } from "../ui"; -import { IconPlus, IconTrash, IconLock, IconExternal, IconPower, IconChevron } from "../icons"; +import { IconPlus, IconTrash, IconLock, IconExternal, IconPower, IconChevron, IconLink } from "../icons"; import { useT } from "../i18n"; import type { AccountQuota } from "../codex-quota-utils"; import QuotaBars from "../components/QuotaBars"; @@ -39,6 +39,7 @@ export default function Providers({ apiBase }: { apiBase: string }) { const [quotaReports, setQuotaReports] = useState>({}); const [busy, setBusy] = useState(null); const [loginInfo, setLoginInfo] = useState<{ provider: string; url?: string; instructions?: string } | null>(null); + const [linkCopied, setLinkCopied] = useState(false); const [manualCode, setManualCode] = useState(""); const [manualCodeBusy, setManualCodeBusy] = useState(false); const [manualCodeMsg, setManualCodeMsg] = useState(""); @@ -422,7 +423,17 @@ export default function Providers({ apiBase }: { apiBase: string }) { {loginInfo?.provider === p && (loginInfo.url || loginInfo.instructions || isBusy) && ( - {loginInfo.url && {t("prov.didntOpen")}} + {loginInfo.url && {t("prov.didntOpen")}} + {loginInfo.instructions && {loginInfo.instructions}} diff --git a/gui/src/styles.css b/gui/src/styles.css index 1da94d5d..0403fef1 100644 --- a/gui/src/styles.css +++ b/gui/src/styles.css @@ -700,6 +700,7 @@ select.input { appearance: none; } .provider-quota { padding-left: 58px; } .provider-actions { margin-left: auto; display: flex; align-items: center; gap: 8px; flex-shrink: 0; } .link-btn { background: none; border: none; color: var(--accent-hover); font: inherit; font-size: var(--text-control); cursor: pointer; padding: 6px 2px; text-decoration: underline; } +.link-btn svg { width: 14px; height: 14px; flex-shrink: 0; } /* provider card accounts dropdown (multiauth) */ .prov-accounts-toggle { diff --git a/src/cli/index.ts b/src/cli/index.ts index 7c972302..85d43f7f 100755 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -500,7 +500,7 @@ switch (command) { case "logout": { const { removeCredential } = await import("../oauth/store"); const name = (args[1] ?? "").trim().toLowerCase(); - removeCredential(name); + await removeCredential(name); console.log(`Logged out of ${name || "(none)"}.`); break; } diff --git a/src/codex/auth-api.ts b/src/codex/auth-api.ts index 5ae5e6eb..5963e217 100644 --- a/src/codex/auth-api.ts +++ b/src/codex/auth-api.ts @@ -574,6 +574,13 @@ export async function handleCodexAuthAPI( const { startLoginFlow, getLoginStatus } = await import("../oauth"); const result = await startLoginFlow("chatgpt", { forceLogin: true }); + // Open the browser server-side (same pattern as /api/oauth/login in management-api.ts). + // The GUI's window.open is popup-blocked because it runs after an await, not a direct click. + if (result.url) { + const { openUrl } = await import("../lib/open-url"); + openUrl(result.url); + } + (async () => { let completed = false; for (let i = 0; i < 150; i++) { diff --git a/src/oauth/index.ts b/src/oauth/index.ts index 2f68f2da..65b8fa35 100644 --- a/src/oauth/index.ts +++ b/src/oauth/index.ts @@ -3,8 +3,8 @@ import { parseCallbackInput } from "./callback-server"; import type { OcxConfig, OcxProviderConfig, RefreshPolicy } from "../types"; import { loadConfig, resolveEnvValue, saveConfig } from "../config"; import { maskEmail } from "../lib/privacy"; -import { getAccountCredential, getAccountSet, saveAccountCredential, saveCredential, markAccountNeedsReauth, getCredential } from "./store"; -import { loginXai, refreshXaiToken } from "./xai"; +import { getAccountCredential, getAccountSet, saveAccountCredential, saveCredential, markAccountNeedsReauth, getCredential, credentialGeneration, createOAuthRefreshIntentLock, mergeAccountCredential, markAccountNeedsReauthIfGeneration } from "./store"; +import { loginXai, refreshXaiToken, XAI_LOCAL_CLI_DETACH_WARNING, XaiTokenRequestError } from "./xai"; import { ANTHROPIC_OAUTH_BETA, loginAnthropic, refreshAnthropicToken } from "./anthropic"; import { loginKimi, refreshKimiToken } from "./kimi"; import { loginKiro, readKiroCliSqlite, refreshKiroToken } from "./kiro"; @@ -14,9 +14,22 @@ import { loginCursor, refreshCursorToken } from "./cursor"; import { deriveOAuthDefaultModel, deriveOAuthProviderConfig } from "../providers/derive"; import { effectiveGoogleMode } from "../providers/registry"; import { resolveProviderTransport } from "../providers/xai-transport"; +import { detectGrokCliToken, hasComparableGrokIdentity, isSameGrokIdentity, shouldAdoptGrokGeneration } from "./local-token-detect"; const REFRESH_SKEW_MS = 60_000; -const tokenRefreshes = new Map>(); +export interface OAuthAccessSnapshot { + provider: string; + accountId: string; + generation: string; + accessToken: string; +} + +const tokenRefreshes = new Map>(); +const XAI_PERMANENT_FAILURE_TTL_MS=30_000; +const permanentRefreshFailures=new Map(); +interface XaiRefreshDeps { intentLock?:ReturnType; now?:()=>number; afterPrePersistRead?:()=>void|Promise } +function verdictKey(p:string,a:string,c:OAuthCredentials){return `${p}\0${a}\0${credentialGeneration(c)}`;} +function cached(p:string,a:string,c:OAuthCredentials,now:()=>number){const k=verdictKey(p,a,c),u=permanentRefreshFailures.get(k);if(u===undefined)return false;if(u<=now()){permanentRefreshFailures.delete(k);return false;}return true;} export interface LoginOpts { forceLogin?: boolean } @@ -139,36 +152,74 @@ export class OAuthLoginRequiredError extends Error { } } -/** Return a valid access token for the ACTIVE account, refreshing + persisting if expired. */ -export async function getValidAccessToken(provider: string): Promise { - const def = OAUTH_PROVIDERS[provider]; - if (!def) throw new UnsupportedOAuthProviderError(provider); - const set = getAccountSet(provider); - if (!set) throw new OAuthLoginRequiredError(provider); - return getValidAccessTokenForAccount(provider, set.activeAccountId); +function accessSnapshot(provider: string, accountId: string, cred: OAuthCredentials): OAuthAccessSnapshot { + return { + provider, + accountId, + generation: credentialGeneration(cred), + accessToken: cred.access, + }; } -/** - * Account-scoped token resolver (multiauth): refresh is single-flighted per - * (provider, account), and the rotated credential is persisted for THAT account only — - * a guardian refresh of a background account never switches the active account. - */ -export async function getValidAccessTokenForAccount(provider: string, accountId: string): Promise { +async function resolveAccessSnapshotForAccount( + provider: string, + accountId: string, + rejectedGeneration?: string, +): Promise { const def = OAUTH_PROVIDERS[provider]; if (!def) throw new UnsupportedOAuthProviderError(provider); const cred = getAccountCredential(provider, accountId); if (!cred) throw new OAuthLoginRequiredError(provider); - if (cred.expires > Date.now() + REFRESH_SKEW_MS) return cred.access; + const current = accessSnapshot(provider, accountId, cred); + if (rejectedGeneration !== undefined && current.generation !== rejectedGeneration) return current; + if (rejectedGeneration === undefined && cred.expires > Date.now() + REFRESH_SKEW_MS) return current; + const key = `${provider}\u0000${accountId}`; const existing = tokenRefreshes.get(key); if (existing) return existing; - const refresh = refreshAndPersistAccessToken(provider, accountId, def, cred).finally(() => { + + const refresh = (async (): Promise => { + const accessToken = await refreshAndPersistAccessToken(provider, accountId, def, cred); + const persisted = getAccountCredential(provider, accountId); + if (!persisted) throw new OAuthLoginRequiredError(provider); + if (persisted.access !== accessToken) { + throw new Error(`OAuth refresh persisted an unexpected access token for ${provider}`); + } + return accessSnapshot(provider, accountId, persisted); + })().finally(() => { if (tokenRefreshes.get(key) === refresh) tokenRefreshes.delete(key); }); tokenRefreshes.set(key, refresh); return refresh; } +export async function getValidAccessTokenSnapshot(provider: string): Promise { + const set = getAccountSet(provider); + if (!set) throw new OAuthLoginRequiredError(provider); + return resolveAccessSnapshotForAccount(provider, set.activeAccountId); +} + +export async function forceRefreshOAuthAccessSnapshot( + rejected: OAuthAccessSnapshot, +): Promise { + if (rejected.provider !== "xai") throw new UnsupportedOAuthProviderError(rejected.provider); + return resolveAccessSnapshotForAccount(rejected.provider, rejected.accountId, rejected.generation); +} + +/** Return a valid access token for the ACTIVE account, refreshing + persisting if expired. */ +export async function getValidAccessToken(provider: string): Promise { + return (await getValidAccessTokenSnapshot(provider)).accessToken; +} + +/** + * Account-scoped token resolver (multiauth): refresh is single-flighted per + * (provider, account), and the rotated credential is persisted for THAT account only — + * a guardian refresh of a background account never switches the active account. + */ +export async function getValidAccessTokenForAccount(provider: string, accountId: string): Promise { + return (await resolveAccessSnapshotForAccount(provider, accountId)).accessToken; +} + function readFreshKiroCliCredential(): OAuthCredentials | undefined { const imported = readKiroCliSqlite(); if (!imported || imported.expires <= Date.now() + REFRESH_SKEW_MS) return undefined; @@ -180,6 +231,10 @@ function isTerminalRefreshError(err: unknown): boolean { const msg = (err instanceof Error ? err.message : String(err)).toLowerCase(); return msg.includes("invalid_grant") || msg.includes("refresh_token_reused") || msg.includes("revoked"); } +function terminal(error:unknown):boolean{return error instanceof XaiTokenRequestError?["invalid_grant","refresh_token_reused","revoked_token"].includes(error.oauthError??""):isTerminalRefreshError(error);} +function authoritative(stored:OAuthCredentials,active:boolean,now:()=>number):OAuthCredentials{if(stored.source!=="local-cli")return stored;const disk=detectGrokCliToken();if(!disk)return stored;const allowed=isSameGrokIdentity(stored,disk)||(active&&!hasComparableGrokIdentity(stored,disk));return allowed&&shouldAdoptGrokGeneration(stored,disk,now(),REFRESH_SKEW_MS)?disk:stored;} +function merged(fresh:OAuthCredentials,previous:OAuthCredentials):OAuthCredentials{return{...fresh,source:previous.source==="local-cli"?"oauth":fresh.source??previous.source??"oauth",...(fresh.projectId===undefined&&previous.projectId?{projectId:previous.projectId}:{}),...(fresh.email===undefined&&previous.email?{email:previous.email}:{}),...(fresh.accountId===undefined&&previous.accountId?{accountId:previous.accountId}:{})};} +export async function refreshXaiAccountWithLock(provider:string,accountId:string,def:OAuthProviderDef,callerCredential:OAuthCredentials,deps:XaiRefreshDeps={}):Promise{const now=deps.now??Date.now;const guard=await(deps.intentLock??createOAuthRefreshIntentLock(provider,accountId)).acquire();try{const stored=getAccountCredential(provider,accountId);if(!stored)throw new OAuthLoginRequiredError(provider);const active=getAccountSet(provider)?.activeAccountId===accountId,candidate=authoritative(stored,active,now);if(credentialGeneration(candidate)!==credentialGeneration(callerCredential)&&candidate.expires>now()+REFRESH_SKEW_MS){if(credentialGeneration(candidate)!==credentialGeneration(stored)){const o=await mergeAccountCredential(provider,accountId,candidate,{expectedGeneration:credentialGeneration(stored),afterPrePersistRead:deps.afterPrePersistRead});if(o.superseded){if(o.stored.expires>now()+REFRESH_SKEW_MS)return o.stored.access;throw new OAuthLoginRequiredError(provider);}}return candidate.access;}if(cached(provider,accountId,candidate,now))throw new OAuthLoginRequiredError(provider);const generation=credentialGeneration(candidate);try{const fresh=merged(await def.refresh(candidate.refresh),candidate);const o=await mergeAccountCredential(provider,accountId,fresh,{expectedGeneration:generation,afterPrePersistRead:deps.afterPrePersistRead});if(o.superseded){if(o.stored.expires>now()+REFRESH_SKEW_MS)return o.stored.access;throw new OAuthLoginRequiredError(provider);}permanentRefreshFailures.delete(verdictKey(provider,accountId,candidate));if(candidate.source==="local-cli")console.warn(XAI_LOCAL_CLI_DETACH_WARNING);return fresh.access;}catch(error){if(!terminal(error))throw error;permanentRefreshFailures.set(verdictKey(provider,accountId,candidate),now()+XAI_PERMANENT_FAILURE_TTL_MS);await markAccountNeedsReauthIfGeneration(provider,accountId,generation);throw new OAuthLoginRequiredError(provider);}}finally{guard.release();}} async function refreshAndPersistAccessToken( provider: string, @@ -193,17 +248,20 @@ async function refreshAndPersistAccessToken( if (provider === "kiro" && isActive) { const imported = readFreshKiroCliCredential(); if (imported) { - saveCredential(provider, imported); + await saveCredential(provider, imported); return imported.access; } } + if (provider === "xai") return refreshXaiAccountWithLock(provider, accountId, def, cred); try { const fresh = await def.refresh(cred.refresh); + const detachedLocalCli = provider === "xai" && cred.source === "local-cli"; + if (detachedLocalCli) console.warn(XAI_LOCAL_CLI_DETACH_WARNING); // Persist to THIS account (rotation-safe: new refresh token hits disk before use) without // touching activeAccountId. - saveAccountCredential(provider, accountId, { + await saveAccountCredential(provider, accountId, { ...fresh, - source: fresh.source ?? cred.source ?? "oauth", + source: detachedLocalCli ? "oauth" : fresh.source ?? cred.source ?? "oauth", // Preserve a previously-discovered project id when a refresh-time re-discovery comes back empty // (e.g. a transient network blip), so Antigravity does not lose its CCA project across refresh. ...(fresh.projectId === undefined && cred.projectId ? { projectId: cred.projectId } : {}), @@ -216,12 +274,12 @@ async function refreshAndPersistAccessToken( if (provider === "kiro" && isActive) { const imported = readFreshKiroCliCredential(); if (imported) { - saveCredential(provider, imported); + await saveCredential(provider, imported); return imported.access; } } if (isTerminalRefreshError(err)) { - markAccountNeedsReauth(provider, accountId, true); + await markAccountNeedsReauth(provider, accountId, true); throw new OAuthLoginRequiredError(provider); } throw err; @@ -352,7 +410,7 @@ export async function runLogin(provider: string, ctrl: OAuthController, opts?: L if (!def) throw new UnsupportedOAuthProviderError(provider); const rawCred = await def.login(ctrl, opts); const cred: OAuthCredentials = rawCred.source ? rawCred : { ...rawCred, source: "oauth" }; - saveCredential(provider, cred); + await saveCredential(provider, cred); const config = loadConfig(); upsertOAuthProvider(config, provider); saveConfig(config); diff --git a/src/oauth/local-token-detect.ts b/src/oauth/local-token-detect.ts index 2682fa58..02f201c3 100644 --- a/src/oauth/local-token-detect.ts +++ b/src/oauth/local-token-detect.ts @@ -13,7 +13,7 @@ const XAI_AUTH_KEY_PREFIX = "https://auth.x.ai::"; const CLAUDE_KEYCHAIN_SERVICE = "Claude Code-credentials"; export function detectGrokCliToken(): OAuthCredentials | null { - const authPath = join(homedir(), ".grok", "auth.json"); + const authPath = join(process.env.HOME ?? homedir(), ".grok", "auth.json"); if (!existsSync(authPath)) return null; try { @@ -39,6 +39,28 @@ export function detectGrokCliToken(): OAuthCredentials | null { } } +export function hasComparableGrokIdentity(stored: OAuthCredentials, disk: OAuthCredentials): boolean { + return Boolean((stored.accountId && disk.accountId) || (stored.email && disk.email)); +} + +export function isSameGrokIdentity(stored: OAuthCredentials, disk: OAuthCredentials): boolean { + if (stored.accountId && disk.accountId) return stored.accountId === disk.accountId; + if (stored.email && disk.email) return stored.email.toLowerCase() === disk.email.toLowerCase(); + return false; +} + +export function shouldAdoptGrokGeneration( + stored: OAuthCredentials, + disk: OAuthCredentials, + now = Date.now(), + refreshSkewMs = 60_000, +): boolean { + if (disk.expires <= now + refreshSkewMs) return false; + const bothExpiriesExist = stored.expires > 0 && disk.expires > 0; + if (bothExpiriesExist) return disk.expires >= stored.expires; + return true; +} + /** Claude Code config dir: `CLAUDE_CONFIG_DIR` override, else `~/.claude`. */ function claudeConfigDir(): string { const explicit = process.env.CLAUDE_CONFIG_DIR?.trim(); diff --git a/src/oauth/store.ts b/src/oauth/store.ts index 6edb82b6..4f49eee1 100644 --- a/src/oauth/store.ts +++ b/src/oauth/store.ts @@ -15,8 +15,8 @@ * same human on every re-login. Cursor login extracts JWT `sub` as accountId so multiauth * can append distinct accounts. */ -import { createHash } from "node:crypto"; -import { copyFileSync, existsSync, mkdirSync, readFileSync, chmodSync } from "node:fs"; +import { createHash, randomUUID } from "node:crypto"; +import { chmodSync, closeSync, copyFileSync, existsSync, fstatSync, mkdirSync, openSync, readFileSync, statSync, unlinkSync, writeFileSync } from "node:fs"; import { join } from "node:path"; import { getConfigDir, atomicWriteFile, backupInvalidConfig, hardenConfigDir, hardenExistingSecret } from "../config"; import type { OAuthCredentialSource, OAuthCredentials, ProviderAccount, ProviderAccountSet } from "./types"; @@ -26,12 +26,21 @@ type AuthStore = Record; /** Providers whose account set is pinned to a single slot (see module doc). */ const SINGLE_SLOT_PROVIDERS = new Set(["chatgpt"]); -function authPath(): string { +export function getAuthStorePath(): string { return join(getConfigDir(), "auth.json"); } +export function getAuthStoreLockPath(): string { return join(getConfigDir(), "auth.store.lock"); } +export function getAuthRefreshIntentLockPath(provider: string, accountId: string): string { + const safeProvider = provider.replace(/[^a-zA-Z0-9_-]/g, "_"); + const accountHash = createHash("sha256").update(accountId).digest("hex").slice(0, 24); + return join(getConfigDir(), `auth.refresh.${safeProvider}.${accountHash}.lock`); +} +export function credentialGeneration(cred: OAuthCredentials): string { + return createHash("sha256").update(JSON.stringify([cred.refresh, cred.access, cred.expires])).digest("hex"); +} function loadAuthStoreInternal(): { store: AuthStore; hadLegacy: boolean } { - const path = authPath(); + const path = getAuthStorePath(); hardenConfigDir(); hardenExistingSecret(path); if (!existsSync(path)) return { store: {}, hadLegacy: false }; @@ -55,8 +64,24 @@ function persist(store: AuthStore): void { try { chmodSync(dir, 0o700); } catch { /* best-effort on existing dir */ } } hardenConfigDir(); - atomicWriteFile(authPath(), JSON.stringify(store, null, 2) + "\n"); + atomicWriteFile(getAuthStorePath(), JSON.stringify(store, null, 2) + "\n"); +} + +export class OAuthFileLockError extends Error { readonly code = "OAUTH_FILE_LOCK_UNAVAILABLE"; constructor(message: string, options?: { cause?: unknown }) { super(message, options); this.name = "OAuthFileLockError"; } } +interface LockSnapshot { bytes: string; dev: number; ino: number; mtimeMs: number; size: number } +export interface OAuthFileLockOptions { path: string; waitTimeoutMs?: number; staleAfterMs?: number; pollMinMs?: number; pollMaxMs?: number; sleep?: (ms: number) => Promise; now?: () => number; random?: () => number; beforeStaleUnlink?: () => void; beforeReleaseUnlink?: () => void; beforeFailedCreateUnlink?: () => void; writeMetadata?: (fd: number, bytes: string) => void } +export interface OAuthFileLockGuard { readonly ownerId: string; release(): void } +function errorCode(error: unknown): string | undefined { return error && typeof error === "object" && "code" in error ? String((error as { code?: unknown }).code) : undefined; } +function snapshot(path: string): LockSnapshot { const bytes = readFileSync(path, "utf8"); const s = statSync(path); return { bytes, dev:s.dev, ino:s.ino, mtimeMs:s.mtimeMs, size:s.size }; } +function sameSnapshot(a: LockSnapshot,b: LockSnapshot): boolean { return a.bytes===b.bytes&&a.dev===b.dev&&a.ino===b.ino&&a.mtimeMs===b.mtimeMs&&a.size===b.size; } +function sameFd(a: LockSnapshot,b: ReturnType): boolean { return a.dev===b.dev&&a.ino===b.ino&&a.mtimeMs===b.mtimeMs&&a.size===b.size; } +export function createOAuthFileLock(options: OAuthFileLockOptions): { acquire(): Promise } { + const wait=options.waitTimeoutMs??5000, stale=options.staleAfterMs??120000, min=options.pollMinMs??25,max=options.pollMaxMs??100,sleep=options.sleep??(ms=>Bun.sleep(ms)),now=options.now??Date.now,random=options.random??Math.random,write=options.writeMetadata??((fd,b)=>writeFileSync(fd,b,"utf8")); + if(wait<0||stale<=0||min<0||maxstale){options.beforeStaleUnlink?.();const b=snapshot(options.path);if(sameSnapshot(a,b))unlinkSync(options.path);continue;}}catch(e){if(errorCode(e)==="ENOENT")continue;throw new OAuthFileLockError("Could not inspect OAuth file lock",{cause:e});} const elapsed=now()-started;if(elapsed>=wait)throw new OAuthFileLockError(`Timed out after ${wait}ms waiting for OAuth file lock`);await sleep(Math.min(wait-elapsed,min+Math.floor(random()*(max-min+1)))); } } }; } +export function createOAuthRefreshIntentLock(provider:string,accountId:string,overrides:Partial={}) { return createOAuthFileLock({path:getAuthRefreshIntentLockPath(provider,accountId),staleAfterMs:120000,...overrides}); } /** * One-time downgrade safety net: the first time we persist the NEW shape over a file that @@ -65,7 +90,7 @@ function persist(store: AuthStore): void { * store, destroying refresh tokens; the backup makes that recoverable. */ function backupLegacyOnce(): void { - const path = authPath(); + const path = getAuthStorePath(); const backup = `${path}.pre-multiauth`; if (!existsSync(path) || existsSync(backup)) return; try { @@ -155,23 +180,15 @@ function normalizeAuthStore(raw: unknown): { store: AuthStore; hadLegacy: boolea * a guardian refresh persisting a non-active account cannot roll back a concurrent * active-account switch (lost update). Cross-process races are accepted (single proxy). */ -let writeQueue: Promise = Promise.resolve(); -function enqueueWrite(fn: () => T): T { - // Synchronous mutations: chain onto the queue for ordering, but run eagerly since all - // current callers are sync. The queue exists so future async mutators serialize too. - const result = fn(); - writeQueue = writeQueue.then(() => result); - return result; -} - -function mutateStore(fn: (store: AuthStore) => T): T { - return enqueueWrite(() => { +let mutationTail: Promise = Promise.resolve(); +function serializeMutation(work:()=>Promise):Promise{const result=mutationTail.then(work,work);mutationTail=result.then(()=>undefined,()=>undefined);return result;} +export function mutateStore(fn:(store:AuthStore)=>T|Promise):Promise{return serializeMutation(async()=>{const guard=await createOAuthFileLock({path:getAuthStoreLockPath(),staleAfterMs:30000}).acquire();try{ const { store, hadLegacy } = loadAuthStoreInternal(); if (hadLegacy) backupLegacyOnce(); - const result = fn(store); + const result = await fn(store); persist(store); return result; - }); + }finally{guard.release();}}); } /** The ACTIVE account's credential for a provider (what requests should use). */ @@ -187,10 +204,10 @@ export function getCredential(provider: string): OAuthCredentials | null { * (rotating refresh tokens would fabricate duplicates) and single-slot providers replace the * active slot / whole set instead. */ -export function saveCredential(provider: string, cred: OAuthCredentials): void { +export async function saveCredential(provider: string, cred: OAuthCredentials): Promise { const safe = normalizeCredential(cred); if (!safe) return; - mutateStore(store => { + await mutateStore(store => { const set = store[provider]; const identity = safe.accountId ?? safe.email; if (!set || SINGLE_SLOT_PROVIDERS.has(provider)) { @@ -225,8 +242,8 @@ export function saveCredential(provider: string, cred: OAuthCredentials): void { } /** Remove the ACTIVE account; remaining accounts promote the first one. */ -export function removeCredential(provider: string): void { - mutateStore(store => { +export async function removeCredential(provider: string): Promise { + await mutateStore(store => { const set = store[provider]; if (!set) return; set.accounts = set.accounts.filter(a => a.id !== set.activeAccountId); @@ -255,10 +272,10 @@ export function getAccountCredential(provider: string, accountId: string): OAuth } /** Persist a refreshed credential for a SPECIFIC account without touching activeAccountId. */ -export function saveAccountCredential(provider: string, accountId: string, cred: OAuthCredentials): void { +export async function saveAccountCredential(provider: string, accountId: string, cred: OAuthCredentials): Promise { const safe = normalizeCredential(cred); if (!safe) return; - mutateStore(store => { + await mutateStore(store => { const account = store[provider]?.accounts.find(a => a.id === accountId); if (!account) return; account.credential = safe; @@ -266,8 +283,8 @@ export function saveAccountCredential(provider: string, accountId: string, cred: }); } -export function setActiveAccount(provider: string, accountId: string): boolean { - return mutateStore(store => { +export async function setActiveAccount(provider: string, accountId: string): Promise { + return await mutateStore(store => { const set = store[provider]; if (!set || !set.accounts.some(a => a.id === accountId)) return false; set.activeAccountId = accountId; @@ -276,8 +293,8 @@ export function setActiveAccount(provider: string, accountId: string): boolean { } /** Remove one account by id; active removal promotes the first remaining account. */ -export function removeAccount(provider: string, accountId: string): boolean { - return mutateStore(store => { +export async function removeAccount(provider: string, accountId: string): Promise { + return await mutateStore(store => { const set = store[provider]; if (!set) return false; const before = set.accounts.length; @@ -292,11 +309,14 @@ export function removeAccount(provider: string, accountId: string): boolean { }); } -export function markAccountNeedsReauth(provider: string, accountId: string, needsReauth: boolean): void { - mutateStore(store => { +export async function markAccountNeedsReauth(provider: string, accountId: string, needsReauth: boolean): Promise { + await mutateStore(store => { const account = store[provider]?.accounts.find(a => a.id === accountId); if (!account) return; if (needsReauth) account.needsReauth = true; else delete account.needsReauth; }); } + +export async function mergeAccountCredential(provider:string,accountId:string,credential:OAuthCredentials,opts:{expectedGeneration?:string;afterPrePersistRead?:()=>void|Promise}={}):Promise<{superseded:false}|{superseded:true;stored:OAuthCredentials}>{const safe=normalizeCredential(credential);if(!safe)throw new Error("Refusing to persist invalid OAuth credential");return await mutateStore(async store=>{await opts.afterPrePersistRead?.();const account=store[provider]?.accounts.find(x=>x.id===accountId);if(!account)throw new Error(`OAuth account disappeared before persist: ${provider}`);if(opts.expectedGeneration!==undefined&&credentialGeneration(account.credential)!==opts.expectedGeneration)return{superseded:true,stored:account.credential};account.credential=safe;delete account.needsReauth;return{superseded:false};});} +export async function markAccountNeedsReauthIfGeneration(provider:string,accountId:string,generation:string):Promise{return await mutateStore(store=>{const account=store[provider]?.accounts.find(x=>x.id===accountId);if(!account?.credential||credentialGeneration(account.credential)!==generation)return false;account.needsReauth=true;return true;});} diff --git a/src/oauth/xai.ts b/src/oauth/xai.ts index 5a6368e4..f876b18b 100644 --- a/src/oauth/xai.ts +++ b/src/oauth/xai.ts @@ -12,6 +12,9 @@ const XAI_OAUTH_CALLBACK_PATH = "/callback"; const XAI_OAUTH_REFRESH_SKEW_MS = 2 * 60 * 1000; const TOKEN_REQUEST_TIMEOUT_MS = 30_000; +export const XAI_LOCAL_CLI_DETACH_WARNING = + "[oauth:xai] Grok CLI credential was stale; refreshed into OpenCodex ownership. Grok CLI may require login again."; + interface XaiDiscovery { authorizationEndpoint: string; tokenEndpoint: string; @@ -22,7 +25,7 @@ interface XaiDiscoveryPayload { token_endpoint?: unknown; } -interface XaiTokenPayload { +export interface XaiTokenPayload { access_token?: unknown; refresh_token?: unknown; expires_in?: unknown; @@ -89,12 +92,18 @@ function getTokenIdentity(accessToken: string, idToken: string | undefined): { a return { accountId, email }; } -async function postXaiToken( +export class XaiTokenRequestError extends Error { constructor(public readonly status?:number,public readonly oauthError?:string,message="xAI token request failed",options?:{cause?:unknown}){super(message,options);this.name="XaiTokenRequestError";} } +export interface XaiTokenRetryDeps { sleep?:(ms:number)=>Promise; random?:()=>number } +function isAbortError(error:unknown):boolean{return error instanceof DOMException&&error.name==="AbortError";} +function retryDelay(attempt:number,retryAfter:string|null,random:()=>number):number{const base=attempt===1?100:250,j=Math.round(base*(.75+random()*.5)),seconds=retryAfter!==null&&/^\d+$/.test(retryAfter)?Number(retryAfter):0;return Math.min(2000,Math.max(j,seconds*1000));} +async function readTokenError(response:Response):Promise{let oauthError:string|undefined,detail="";try{const body=await response.json() as {error?:unknown;error_description?:unknown};if(typeof body.error==="string")oauthError=body.error;if(typeof body.error_description==="string")detail=body.error_description;}catch{}const suffix=detail?`: ${detail}`:oauthError?`: ${oauthError}`:"";return new XaiTokenRequestError(response.status,oauthError,`xAI token request failed: ${response.status}${suffix}`);} +export async function postXaiToken( tokenEndpoint: string, body: Record, - signal?: AbortSignal, + signal?: AbortSignal, deps:XaiTokenRetryDeps={}, ): Promise { - const response = await fetch(tokenEndpoint, { + const sleep=deps.sleep??(ms=>Bun.sleep(ms)),random=deps.random??Math.random;let last:unknown; + for(let attempt=1;attempt<=3;attempt++){let response:Response;try{response=await fetch(tokenEndpoint, { method: "POST", headers: { Accept: "application/json", @@ -102,11 +111,7 @@ async function postXaiToken( }, body: new URLSearchParams(body).toString(), signal: requestSignal(signal), - }); - if (!response.ok) { - throw new Error(`xAI token request failed: ${response.status} ${await response.text()}`); - } - return (await response.json()) as XaiTokenPayload; + });}catch(error){if(isAbortError(error)&&signal?.aborted)throw error;last=error;if(attempt===3)throw new XaiTokenRequestError(undefined,undefined,"xAI token request failed: network error",{cause:error});await sleep(retryDelay(attempt,null,random));continue;}if(response.ok)return await response.json() as XaiTokenPayload;const error=await readTokenError(response);last=error;if(!(response.status===429||response.status>=500)||attempt===3)throw error;await sleep(retryDelay(attempt,response.headers.get("retry-after"),random));}throw last; } function credentialsFromTokenPayload(payload: XaiTokenPayload, refreshFallback = ""): OAuthCredentials { @@ -200,7 +205,9 @@ export async function loginXai( ctrl.onProgress?.("Found Grok CLI token, importing automatically"); if (local.expires >= Date.now() + 60_000) return local; try { - return { ...(await refreshXaiToken(local.refresh, ctrl.signal)), source: "local-cli" }; + const fresh = await refreshXaiToken(local.refresh, ctrl.signal); + ctrl.onProgress?.(XAI_LOCAL_CLI_DETACH_WARNING); + return { ...fresh, source: "oauth" }; } catch (error) { if (importLocal === "only") { throw new Error( diff --git a/src/providers/xai-transport.ts b/src/providers/xai-transport.ts index 2feca2ec..96406248 100644 --- a/src/providers/xai-transport.ts +++ b/src/providers/xai-transport.ts @@ -1,48 +1,79 @@ -import { createHash } from "node:crypto"; +import { createHash, randomUUID } from "node:crypto"; import type { OcxProviderConfig } from "../types"; -/** - * xAI account OAuth and xAI API keys share a bearer shape but not a billing - * transport. OAuth represents the Grok CLI subscription entitlement, while a - * key represents the API team. Keep the saved provider preset compatible with - * the dashboard's "Use an API key instead" switch and resolve the transport at - * request time. - */ export const XAI_GROK_CLI_BASE_URL = "https://cli-chat-proxy.grok.com/v1"; -/** Minimum-compatible official Grok CLI wire version verified with the proxy. */ -export const XAI_GROK_CLIENT_VERSION = "0.2.93"; +export const XAI_GROK_COMPATIBILITY = { + version: "0.2.93", + userAgent: "opencodex-grok/0.2.93", + headers: { + clientIdentifier: "x-grok-client-identifier", + clientVersion: "x-grok-client-version", + tokenAuth: "x-xai-token-auth", + authenticateResponse: "x-authenticateresponse", + conversationId: "x-grok-conv-id", + requestId: "x-grok-req-id", + sessionId: "x-grok-session-id", + userAgent: "User-Agent", + }, +} as const; -const XAI_GROK_CLI_HEADERS: Readonly> = { - "x-grok-client-identifier": "opencodex", - "x-grok-client-version": XAI_GROK_CLIENT_VERSION, - "x-xai-token-auth": "xai-grok-cli", +export const XAI_GROK_CLIENT_VERSION = XAI_GROK_COMPATIBILITY.version; +export const XAI_CONV_ID_HEADER = XAI_GROK_COMPATIBILITY.headers.conversationId; + +export type OcxProviderTransport = OcxProviderConfig & { + /** Request executor used only at runtime; never persisted. */ + fetch?: typeof globalThis.fetch; }; -/** - * Sticky-routing hint for xAI's automatic prefix cache. xAI routes requests - * carrying the same `x-grok-conv-id` to the same server, which is where the - * prompt cache lives (docs.x.ai prompt-caching best-practices; verified - * 2026-07-13, devlog/_plan/260713_grok_caching). Codex clients send a stable - * per-conversation `prompt_cache_key`; hash it so the raw session id never - * leaves the proxy. - */ -export const XAI_CONV_ID_HEADER = "x-grok-conv-id"; +const XAI_GROK_CLI_HEADERS: Readonly> = { + [XAI_GROK_COMPATIBILITY.headers.clientIdentifier]: "opencodex", + [XAI_GROK_COMPATIBILITY.headers.clientVersion]: XAI_GROK_CLIENT_VERSION, + [XAI_GROK_COMPATIBILITY.headers.tokenAuth]: "xai-grok-cli", + [XAI_GROK_COMPATIBILITY.headers.authenticateResponse]: "authenticate-response", +}; -function hasHeaderCaseInsensitive(headers: Record | undefined, name: string): boolean { - if (!headers) return false; +function hasHeaderCaseInsensitive( + headers: Record | undefined, + name: string, +): boolean { const target = name.toLowerCase(); - return Object.keys(headers).some(key => key.toLowerCase() === target); + return Object.keys(headers ?? {}).some(key => key.toLowerCase() === target); +} + +function withoutUserOverridden( + defaults: Readonly>, + userHeaders: Record | undefined, +): Record { + return Object.fromEntries( + Object.entries(defaults).filter(([name]) => !hasHeaderCaseInsensitive(userHeaders, name)), + ); } -/** Drop default entries the user already overrides under any header-name casing. */ -function withoutUserOverridden(defaults: Readonly>, userHeaders: Record | undefined): Record { - if (!userHeaders) return { ...defaults }; - const out: Record = {}; - for (const [key, value] of Object.entries(defaults)) { - if (!hasHeaderCaseInsensitive(userHeaders, key)) out[key] = value; +function withGeneratedRequestId( + init: RequestInit | undefined, + configuredRequestId: string | undefined, + stableHeaders: Readonly>, +): RequestInit { + const headers = new Headers(init?.headers); + for (const [name, value] of Object.entries(stableHeaders)) { + if (!headers.has(name)) headers.set(name, value); } - return out; + if (!headers.has(XAI_GROK_COMPATIBILITY.headers.requestId)) { + headers.set( + XAI_GROK_COMPATIBILITY.headers.requestId, + configuredRequestId ?? randomUUID(), + ); + } + return { ...init, headers }; +} + +function findHeaderCaseInsensitive( + headers: Record | undefined, + name: string, +): string | undefined { + const target = name.toLowerCase(); + return Object.entries(headers ?? {}).find(([key]) => key.toLowerCase() === target)?.[1]; } export function deriveXaiConvId(promptCacheKey: string): string { @@ -50,40 +81,56 @@ export function deriveXaiConvId(promptCacheKey: string): string { } /** - * Resolve the effective xAI transport without mutating persisted config. - * User-provided headers are preserved and may advance the compatibility - * version without waiting for an opencodex release. - * - * `promptCacheKey` (the client's stable conversation key) additionally pins - * cache-affinity routing via `x-grok-conv-id` in BOTH auth modes. Blank or - * whitespace-only keys are ignored so unrelated requests can never collapse - * onto one shared conv id, and any user-configured header (any case) wins. + * Resolve xAI's runtime transport without mutating persisted config. Conversation/session + * affinity is stable for this resolved transport; request identity is generated per fetch. + * Agent, deployment, model-override, turn, mode, and user identity headers are intentionally + * omitted because opencodex has no truthful values for the official fields. */ export function resolveProviderTransport( providerName: string, - provider: OcxProviderConfig, + provider: OcxProviderTransport, promptCacheKey?: string, -): OcxProviderConfig { +): OcxProviderTransport { if (providerName !== "xai") return provider; + const cacheKey = promptCacheKey?.trim(); - const convIdHeaders: Record = - cacheKey && !hasHeaderCaseInsensitive(provider.headers, XAI_CONV_ID_HEADER) - ? { [XAI_CONV_ID_HEADER]: deriveXaiConvId(cacheKey) } - : {}; - if (provider.authMode !== "oauth") { - if (Object.keys(convIdHeaders).length === 0) return provider; - return { - ...provider, - headers: { ...convIdHeaders, ...(provider.headers ?? {}) }, - }; - } + const affinity = cacheKey ? deriveXaiConvId(cacheKey) : undefined; + const stableDefaults: Record = { + [XAI_GROK_COMPATIBILITY.headers.userAgent]: XAI_GROK_COMPATIBILITY.userAgent, + ...(affinity + ? { + [XAI_GROK_COMPATIBILITY.headers.conversationId]: affinity, + [XAI_GROK_COMPATIBILITY.headers.sessionId]: affinity, + } + : {}), + ...(provider.authMode === "oauth" ? XAI_GROK_CLI_HEADERS : {}), + }; + const stableHeaders = { + ...withoutUserOverridden(stableDefaults, provider.headers), + ...(provider.headers ?? {}), + }; + // Keep API-key provider metadata compatible with key-pool rotation; session/UA defaults + // remain transport-scoped and are applied by the wrapper immediately below. + const headers = provider.authMode === "oauth" + ? stableHeaders + : { + ...(affinity && !hasHeaderCaseInsensitive(provider.headers, XAI_GROK_COMPATIBILITY.headers.conversationId) + ? { [XAI_GROK_COMPATIBILITY.headers.conversationId]: affinity } + : {}), + ...(provider.headers ?? {}), + }; + const configuredRequestId = findHeaderCaseInsensitive( + provider.headers, + XAI_GROK_COMPATIBILITY.headers.requestId, + ); + const baseFetch = provider.fetch ?? globalThis.fetch; + const attemptFetch = ((input, init) => + baseFetch(input, withGeneratedRequestId(init, configuredRequestId, stableHeaders))) as typeof globalThis.fetch; + return { ...provider, - baseUrl: XAI_GROK_CLI_BASE_URL, - headers: { - ...withoutUserOverridden(XAI_GROK_CLI_HEADERS, provider.headers), - ...convIdHeaders, - ...(provider.headers ?? {}), - }, + ...(provider.authMode === "oauth" ? { baseUrl: XAI_GROK_CLI_BASE_URL } : {}), + headers, + fetch: attemptFetch, }; } diff --git a/src/responses/parser.ts b/src/responses/parser.ts index ee8cb2f3..e0a1d462 100644 --- a/src/responses/parser.ts +++ b/src/responses/parser.ts @@ -228,6 +228,20 @@ export function parseRequest(body: unknown): OcxParsedRequest { const now = Date.now(); const messages: OcxMessage[] = []; const systemPrompt: string[] = []; + // Responses reasoning siblings belong to the following assistant, including across call items. + // Keep them off the message list until that assistant arrives; turn boundaries clear the array. + const pendingReasoning: Array<{ part: OcxThinkingContent; envelopeSigned: boolean }> = []; + // Assistant placeholder that first folds any pending reasoning into the same turn (official + // grok-build preserves reasoning across call items; Anthropic replay requires thinking to + // precede tool_use inside one assistant message). + const assistantHolderWithReasoning = (): OcxAssistantMessage => { + const holder = ensureAssistantPlaceholder(messages, data.model, now); + if (pendingReasoning.length > 0) { + holder.content.push(...pendingReasoning.map(entry => entry.part)); + pendingReasoning.length = 0; + } + return holder; + }; // Tool specs surfaced by a prior tool_search (deferred tools, e.g. subagents). Codex does not // re-list these in `tools`, but chat models can only call listed tools — so we re-inject them. const loadedToolSpecs: unknown[] = []; @@ -270,6 +284,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { // is dropped silently. It must NOT flag _compactionRequest. const encrypted = (item as { encrypted_content?: unknown }).encrypted_content; if (effectiveType === "context_compaction" && typeof encrypted !== "string") continue; + pendingReasoning.length = 0; messages.push({ role: "user", content: compactionItemToText(typeof encrypted === "string" ? encrypted : undefined), @@ -297,6 +312,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { // An agent_message is external input delivered to the parent agent. // Preserve it as a user-role turn so signed Anthropic thinking blocks // on either side are never merged into one modified assistant response. + pendingReasoning.length = 0; messages.push({ role: "user", content: hasContent ? content : "(sub-agent message received)", @@ -310,6 +326,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { const msg = item as { role?: string; content?: unknown }; switch (msg.role) { case "system": { + pendingReasoning.length = 0; const text = inputContentParts(msg.content as unknown[] | string | undefined); const flat = typeof text === "string" ? text : text.map(p => (p.type === "text" ? p.text : "")).join(""); if (flat.length > 0) systemPrompt.push(flat); @@ -317,13 +334,22 @@ export function parseRequest(body: unknown): OcxParsedRequest { } case "user": case "developer": { + pendingReasoning.length = 0; const content = inputContentParts(msg.content as unknown[] | string | undefined); messages.push({ role: msg.role, content, timestamp: now }); break; } case "assistant": { const parts = outputTextOf(msg.content as unknown[] | string | undefined); - messages.push({ role: "assistant", content: parts, model: data.model, timestamp: now }); + messages.push({ + role: "assistant", + content: pendingReasoning.length > 0 + ? [...pendingReasoning.map(entry => entry.part), ...parts] + : parts, + model: data.model, + timestamp: now, + }); + pendingReasoning.length = 0; break; } } @@ -334,20 +360,33 @@ export function parseRequest(body: unknown): OcxParsedRequest { const reasoning = item as { id?: string; summary?: { text: string }[]; content?: { text: string }[]; encrypted_content?: string }; const fromSummary = (reasoning.summary ?? []).map(c => c.text).join(""); const text = fromSummary || (reasoning.content ?? []).map(c => c.text).join(""); - // ocxr1 envelope: the REAL Anthropic signature (+ redacted blocks, + hidden signed text) - // captured by the bridge. Native OpenAI-encrypted blobs decode to null and keep today's - // placeholder signature (which the anthropic adapter correctly rejects on replay). const envelope = typeof reasoning.encrypted_content === "string" ? decodeReasoningEnvelope(reasoning.encrypted_content) : null; - const thinking: OcxThinkingContent = { - type: "thinking", - thinking: envelope?.txt || text, - signature: envelope?.sig ?? JSON.stringify(reasoning), - ...(envelope?.red ? { redacted: envelope.red } : {}), - ...(reasoning.id ? { itemId: reasoning.id } : {}), - }; - ensureAssistantPlaceholder(messages, data.model, now).content.push(thinking); + const thinkingText = envelope?.txt || text; + + // Native/non-ocxr1 encrypted-only reasoning is opaque here. Do not create a detached + // assistant turn or invent replayable plaintext/signatures from the encrypted payload. + if (thinkingText.length > 0) { + const part: OcxThinkingContent = { + type: "thinking", + thinking: thinkingText, + signature: envelope?.sig ?? JSON.stringify(reasoning), + ...(envelope?.red ? { redacted: envelope.red } : {}), + ...(reasoning.id ? { itemId: reasoning.id } : {}), + }; + const envelopeSigned = typeof envelope?.sig === "string"; + const previous = pendingReasoning[pendingReasoning.length - 1]; + + if (!envelopeSigned && previous && !previous.envelopeSigned) { + previous.part = { + ...part, + thinking: `${previous.part.thinking}\n${part.thinking}`, + }; + } else { + pendingReasoning.push({ part, envelopeSigned }); + } + } continue; } @@ -370,7 +409,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { ...(call.id ? { thoughtSignature: call.id } : {}), ...(call.namespace ? { namespace: call.namespace } : {}), }; - ensureAssistantPlaceholder(messages, data.model, now).content.push(toolCall); + assistantHolderWithReasoning().content.push(toolCall); continue; } @@ -382,7 +421,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { customWireName: call.name, ...(call.id ? { thoughtSignature: call.id } : {}), }; - ensureAssistantPlaceholder(messages, data.model, now).content.push(toolCall); + assistantHolderWithReasoning().content.push(toolCall); continue; } @@ -393,7 +432,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { const callId = call.call_id ?? call.id; if (callId) { const command = Array.isArray(call.action?.command) ? call.action.command : []; - ensureAssistantPlaceholder(messages, data.model, now).content.push({ + assistantHolderWithReasoning().content.push({ type: "toolCall", id: callId, name: "shell", arguments: command.length > 0 ? { command } : {}, }); @@ -406,7 +445,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { // knows the search already ran (prevents re-search loops); there is no output to pair. const call = item as { action?: { type?: string; query?: string } }; const query = typeof call.action?.query === "string" ? call.action.query : ""; - ensureAssistantPlaceholder(messages, data.model, now).content.push({ + assistantHolderWithReasoning().content.push({ type: "text", text: query ? `[web search performed: ${query}]` : "[web search performed]", }); continue; @@ -417,7 +456,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { // history stays complete (otherwise the model re-issues tool_search forever). const call = item as { id?: string; call_id?: string; arguments?: unknown }; const callId = call.call_id ?? call.id ?? ""; - ensureAssistantPlaceholder(messages, data.model, now).content.push({ + assistantHolderWithReasoning().content.push({ type: "toolCall", id: callId, name: "tool_search", arguments: isObj(call.arguments) ? call.arguments : {}, }); @@ -425,6 +464,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { } if (effectiveType === "tool_search_output") { + pendingReasoning.length = 0; // Pair the tool_search call with its result so the model sees what was loaded. const out = item as { call_id?: string; status?: string; tools?: unknown[] }; const specs = Array.isArray(out.tools) ? (out.tools as Record[]) : []; @@ -455,6 +495,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { } if (effectiveType === "function_call_output") { + pendingReasoning.length = 0; const output = item as { call_id: string; output?: string | unknown[] }; const toolInfo = findToolById(messages, output.call_id); messages.push({ @@ -466,6 +507,7 @@ export function parseRequest(body: unknown): OcxParsedRequest { } if (effectiveType === "custom_tool_call_output") { + pendingReasoning.length = 0; const output = item as { call_id: string; output: string | unknown[] }; const toolInfo = findToolById(messages, output.call_id); messages.push({ diff --git a/src/server/management-api.ts b/src/server/management-api.ts index f65a8673..e5962920 100644 --- a/src/server/management-api.ts +++ b/src/server/management-api.ts @@ -1071,7 +1071,7 @@ export async function handleManagementAPI(req: Request, url: URL, config: OcxCon if (url.pathname === "/api/oauth/logout" && req.method === "POST") { const provider = (url.searchParams.get("provider") ?? "").trim().toLowerCase(); if (!isOAuthProvider(provider)) return jsonResponse({ error: "unknown oauth provider" }, 400); - removeCredential(provider); + await removeCredential(provider); clearLoginState(provider); return jsonResponse({ success: true }); } @@ -1090,7 +1090,7 @@ export async function handleManagementAPI(req: Request, url: URL, config: OcxCon if (!isOAuthProvider(provider)) return jsonResponse({ error: "unknown oauth provider" }, 400); if (!body.accountId) return jsonResponse({ error: "missing accountId" }, 400); const { setActiveAccount } = await import("../oauth/store"); - if (!setActiveAccount(provider, body.accountId)) return jsonResponse({ error: "account not found" }, 404); + if (!(await setActiveAccount(provider, body.accountId))) return jsonResponse({ error: "account not found" }, 404); const { clearProviderQuotaCache } = await import("../providers/quota"); clearProviderQuotaCache(); return jsonResponse({ ok: true, provider, activeAccountId: body.accountId }); @@ -1101,7 +1101,7 @@ export async function handleManagementAPI(req: Request, url: URL, config: OcxCon if (!isOAuthProvider(provider)) return jsonResponse({ error: "unknown oauth provider" }, 400); if (!id) return jsonResponse({ error: "missing id" }, 400); const { removeAccount, getAccountSet } = await import("../oauth/store"); - if (!removeAccount(provider, id)) return jsonResponse({ error: "account not found" }, 404); + if (!(await removeAccount(provider, id))) return jsonResponse({ error: "account not found" }, 404); if (!getAccountSet(provider)) clearLoginState(provider); const { clearProviderQuotaCache } = await import("../providers/quota"); clearProviderQuotaCache(); diff --git a/src/server/responses.ts b/src/server/responses.ts index 09b443a4..9fc56018 100644 --- a/src/server/responses.ts +++ b/src/server/responses.ts @@ -14,8 +14,10 @@ import { injectionDebugLog } from "../lib/injection-debug-log"; import { modelInList, namespacedToolName } from "../types"; import type { AdapterEvent, OcxConfig, OcxParsedRequest, OcxProviderConfig } from "../types"; import { + forceRefreshOAuthAccessSnapshot, getOAuthCredentialProjectId, - getValidAccessToken, + getValidAccessTokenSnapshot, + type OAuthAccessSnapshot, UnsupportedOAuthProviderError, } from "../oauth"; import { buildWebSearchTool, planWebSearch, runWithWebSearch } from "../web-search"; @@ -611,9 +613,13 @@ export async function handleResponses( // OAuth providers: swap in a fresh access token (auto-refreshed) as the Bearer key, so the // existing openai-chat / anthropic adapters authenticate with no change. + const isXaiOAuthRequest = route.providerName === "xai" && route.provider.authMode === "oauth"; + let sentOAuthSnapshot: OAuthAccessSnapshot | undefined; if (route.provider.authMode === "oauth") { try { - route.provider = { ...route.provider, apiKey: await getValidAccessToken(route.providerName) }; + const resolved = await getValidAccessTokenSnapshot(route.providerName); + if (isXaiOAuthRequest) sentOAuthSnapshot = resolved; + route.provider = { ...route.provider, apiKey: resolved.accessToken }; // Antigravity (cloud-code-assist) needs the discovered Cloud Code Assist project id in the // CCA envelope; the server injects only the bare token, so pull project from the credential. if (route.provider.googleMode === "cloud-code-assist" && !route.provider.project) { @@ -702,7 +708,7 @@ export async function handleResponses( method: request.method, headers: request.headers, body: request.body, - }, upstream.signal, connectMs, parsed.stream), + }, upstream.signal, connectMs, parsed.stream, providerFetch(route.provider)), { abortSignal: upstream.signal, label: safeHostLabel(request.url) }, ); } catch (err) { @@ -948,7 +954,7 @@ export async function handleResponses( : await fetchWithResetRetry( () => fetchWithHeaderTimeout(request.url, { method: request.method, headers: request.headers, body: request.body, - }, upstream.signal, connectMs, parsed.stream), + }, upstream.signal, connectMs, parsed.stream, providerFetch(route.provider)), { abortSignal: upstream.signal, label: safeHostLabel(request.url) }, ); } catch (err) { @@ -969,6 +975,7 @@ export async function handleResponses( let activeAdapter = adapter; let imageTierBias = 0; let imageRetryAttempted = false; + let oauth401ReplayAttempted = false; const rebuildAndRefetch = async (): Promise => { const retryRequest = await activeAdapter.buildRequest(parsed, { headers: selectedForwardHeaders, @@ -979,7 +986,7 @@ export async function handleResponses( ? await activeAdapter.fetchResponse(retryRequest, { abortSignal: upstream.signal, timeoutMs: connectMs, stream: parsed.stream }) : await fetchWithHeaderTimeout(retryRequest.url, { method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, - }, upstream.signal, connectMs, parsed.stream); + }, upstream.signal, connectMs, parsed.stream, providerFetch(route.provider)); } catch (err) { cleanupUpstreamAbort(); upstream.abort(); @@ -990,6 +997,38 @@ export async function handleResponses( } }; recovery: for (;;) { + if ( + upstreamResponse.status === 401 + && isXaiOAuthRequest + && sentOAuthSnapshot + && !oauth401ReplayAttempted + ) { + oauth401ReplayAttempted = true; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + let refreshed: OAuthAccessSnapshot; + try { + refreshed = await forceRefreshOAuthAccessSnapshot(sentOAuthSnapshot); + } catch (err) { + cleanupUpstreamAbort(); + return formatErrorResponse(401, "authentication_error", err instanceof Error ? err.message : String(err)); + } + sentOAuthSnapshot = refreshed; + const refreshedProvider = resolveProviderTransport( + route.providerName, + { ...route.provider, apiKey: refreshed.accessToken }, + parsed.options.promptCacheKey, + ); + route.provider = refreshedProvider; + activeAdapter = resolveAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, refreshedProvider), + config.cacheRetention, + ); + const result = await rebuildAndRefetch(); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + // Multi-key 429 failover: rotate to the next pool key (cooldown-aware) and retry the // SAME request once per remaining key. OAuth/forward providers and single-key pools // return null immediately, so this stays a no-op for them (src/providers/key-failover.ts). @@ -1225,12 +1264,17 @@ export function safeHostLabel(url: string): string { } } +function providerFetch(provider: OcxProviderConfig): typeof globalThis.fetch { + return (provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch ?? globalThis.fetch; +} + export async function fetchWithHeaderTimeout( url: string, init: Omit, abortSignal: AbortSignal, timeoutMs: number, preferIdentityEncoding = false, + executor: typeof globalThis.fetch = globalThis.fetch, ): Promise { const timeout = new AbortController(); const timer = setTimeout(() => { @@ -1243,7 +1287,7 @@ export async function fetchWithHeaderTimeout( headers.set("accept-encoding", "identity"); } try { - return await fetch(url, { + return await executor(url, { ...init, headers, signal: AbortSignal.any([abortSignal, timeout.signal]), diff --git a/structure/04_transports-and-sidecars.md b/structure/04_transports-and-sidecars.md index 2e202232..31f77733 100644 --- a/structure/04_transports-and-sidecars.md +++ b/structure/04_transports-and-sidecars.md @@ -86,6 +86,35 @@ orphan `toolResult` messages by inserting a synthetic assistant `tool_call` befo These compatibility guards are covered by focused tests and should stay close to the adapters that need them. +## xAI Grok hardening (official Grok Build contract parity) + +Grounded in the open-sourced official client (xai-org/grok-build); unit + evidence: +`devlog/_plan/260716_grok_build_hardening/`. + +- **Reasoning folding:** the Responses parser folds `reasoning` items into the FOLLOWING + assistant turn (`pendingReasoning` in `src/responses/parser.ts`) so the Grok chat wire carries + ONE assistant message with `reasoning_content` — exact-prefix cache stability. Unsigned + siblings newline-join; `ocxr1`-signed siblings stay separate parts (Anthropic replay keeps + each signature on its own text); boundaries (user/tool-result/agent) clear pending state; + call items fold pending reasoning into the same turn. +- **Grok CLI credential ownership:** `source:"local-cli"` xAI credentials re-read + `~/.grok/auth.json` (read-only) before any refresh and adopt a newer usable generation with + zero IdP calls (`shouldAdoptGrokGeneration`, later-expiresAt authority); an IdP refresh + detaches the credential to `source:"oauth"`. +- **Two-lock refresh transaction:** per-provider+account intent lock held across the IdP + exchange plus a short global store-write lock + async mutation funnel around every + `auth.json` load-merge-persist (`src/oauth/store.ts`); generation-guarded persist + (`expectedGeneration` → superseded adoption), conditional `needsReauth`, bounded jittered + retry for transient token-endpoint failures. +- **Reactive 401 replay:** the serving recovery loop force-refreshes once (singleflight, + generation-checked) and replays OAuth-backed xAI requests exactly once with a re-resolved + transport; API-key/BYOK paths excluded (`src/server/responses.ts`). +- **Header parity:** per-attempt `x-grok-req-id` (fresh UUID inside the transport fetch + wrapper), stable session/conv affinity headers, always-set User-Agent, and a single + compatibility profile const for the Grok client version (`src/providers/xai-transport.ts`); + `fetchWithHeaderTimeout` takes an executor so provider fetch wrappers stay inside the + timeout race. + ## Parallel tool calls (default-on for chat providers) The openai-chat adapter buffers ALL streamed `tool_calls` deltas (keyed by `index`, falling back to diff --git a/tests/anthropic-thinking-signature.test.ts b/tests/anthropic-thinking-signature.test.ts index 6fc02215..bf7dd66f 100644 --- a/tests/anthropic-thinking-signature.test.ts +++ b/tests/anthropic-thinking-signature.test.ts @@ -151,6 +151,7 @@ describe("parser ocxr1 decode + anthropic replay", () => { model: "anthropic/claude-x", input: [ { type: "reasoning", id: "rs_1", summary: [{ type: "summary_text", text: "chain" }], encrypted_content: encrypted }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, { type: "message", role: "user", content: [{ type: "input_text", text: "hi" }] }, ], }); @@ -167,6 +168,7 @@ describe("parser ocxr1 decode + anthropic replay", () => { model: "anthropic/claude-x", input: [ { type: "reasoning", id: "rs_1", summary: [], encrypted_content: encrypted }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, { type: "message", role: "user", content: [{ type: "input_text", text: "hi" }] }, ], }); @@ -180,6 +182,7 @@ describe("parser ocxr1 decode + anthropic replay", () => { model: "anthropic/claude-x", input: [ { type: "reasoning", id: "rs_1", summary: [{ type: "summary_text", text: "chain" }], encrypted_content: "gAAAAABopaqueOpenAI" }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, { type: "message", role: "user", content: [{ type: "input_text", text: "hi" }] }, ], }); @@ -197,6 +200,7 @@ describe("parser ocxr1 decode + anthropic replay", () => { input: [ { type: "reasoning", id: "rs_1", summary: [{ type: "summary_text", text: "chain" }], encrypted_content: encrypted }, { type: "function_call", call_id: "call_1", name: "shell", arguments: "{}" }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, { type: "function_call_output", call_id: "call_1", output: "done" }, { type: "message", role: "user", content: [{ type: "input_text", text: "next" }] }, ], @@ -215,6 +219,55 @@ describe("parser ocxr1 decode + anthropic replay", () => { expect(content[thinkIdx].signature).toBe("RealSig1234567890=="); expect(content[thinkIdx].thinking).toBe("chain"); }); + + test("two signed reasoning siblings replay with each signature attached to its own text", async () => { + const adapter = createAnthropicAdapter(provider); + const firstEnvelope = encodeReasoningEnvelope({ + sig: "FirstRealSignature123456==", + txt: "first signed chain", + }); + const secondEnvelope = encodeReasoningEnvelope({ + sig: "SecondRealSignature123456==", + txt: "second signed chain", + }); + const parsed = parseRequest({ + model: "anthropic/claude-x", + input: [ + { type: "reasoning", id: "rs_first", summary: [], encrypted_content: firstEnvelope }, + { type: "reasoning", id: "rs_second", summary: [], encrypted_content: secondEnvelope }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "next" }] }, + ], + }); + const parsedAssistant = parsed.context.messages.find(message => message.role === "assistant") as { + content: OcxThinkingContent[]; + }; + const parsedThinking = parsedAssistant.content.filter(part => part.type === "thinking"); + + expect(parsedThinking).toHaveLength(2); + expect(parsedThinking.map(part => ({ + thinking: part.thinking, + signature: part.signature, + }))).toEqual([ + { thinking: "first signed chain", signature: "FirstRealSignature123456==" }, + { thinking: "second signed chain", signature: "SecondRealSignature123456==" }, + ]); + + const request = await adapter.buildRequest(parsed) as { body: string }; + const body = JSON.parse(request.body) as { + messages: Array<{ + role: string; + content: Array<{ type: string; thinking?: string; signature?: string; text?: string }>; + }>; + }; + const replayedAssistant = body.messages.find(message => message.role === "assistant"); + const replayedThinking = replayedAssistant?.content.filter(block => block.type === "thinking"); + + expect(replayedThinking).toEqual([ + { type: "thinking", thinking: "first signed chain", signature: "FirstRealSignature123456==" }, + { type: "thinking", thinking: "second signed chain", signature: "SecondRealSignature123456==" }, + ]); + }); }); describe("passthrough scrub of ocxr1 envelopes", () => { diff --git a/tests/google-antigravity-oauth.test.ts b/tests/google-antigravity-oauth.test.ts index 371b4a5f..257f5c2e 100644 --- a/tests/google-antigravity-oauth.test.ts +++ b/tests/google-antigravity-oauth.test.ts @@ -114,12 +114,12 @@ describe("antigravity credential persistence (projectId survives the store)", () if (tmp) rmSync(tmp, { recursive: true, force: true }); }); - test("saveCredential + getCredential round-trips projectId (regression: was stripped by normalizeCredential)", () => { + test("saveCredential + getCredential round-trips projectId (regression: was stripped by normalizeCredential)", async () => { tmp = join(tmpdir(), `ag-store-${Date.now()}-${Math.random().toString(16).slice(2)}`); mkdirSync(tmp, { recursive: true }); process.env.HOME = tmp; process.env.OPENCODEX_HOME = join(tmp, "ocx"); - saveCredential("google-antigravity", { access: "a", refresh: "r", expires: Date.now() + 3_600_000, projectId: "proj-persist" }); + await saveCredential("google-antigravity", { access: "a", refresh: "r", expires: Date.now() + 3_600_000, projectId: "proj-persist" }); expect(getCredential("google-antigravity")?.projectId).toBe("proj-persist"); }); }); diff --git a/tests/oauth-refresh.test.ts b/tests/oauth-refresh.test.ts index 4293cde4..c5c6c48f 100644 --- a/tests/oauth-refresh.test.ts +++ b/tests/oauth-refresh.test.ts @@ -1,6 +1,6 @@ import { afterEach, beforeEach, describe, expect, test } from "bun:test"; import { Database } from "bun:sqlite"; -import { mkdirSync, rmSync } from "node:fs"; +import { mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { getValidAccessToken } from "../src/oauth"; @@ -10,6 +10,7 @@ const origHome = process.env.HOME; const origOcxHome = process.env.OPENCODEX_HOME; const origRegion = process.env.KIRO_REGION; const origFetch = globalThis.fetch; +const origWarn = console.warn; let tmp: string; beforeEach(() => { @@ -25,6 +26,7 @@ afterEach(() => { if (origOcxHome === undefined) delete process.env.OPENCODEX_HOME; else process.env.OPENCODEX_HOME = origOcxHome; if (origRegion === undefined) delete process.env.KIRO_REGION; else process.env.KIRO_REGION = origRegion; globalThis.fetch = origFetch; + console.warn = origWarn; rmSync(tmp, { recursive: true, force: true }); }); @@ -37,6 +39,48 @@ function seedKiroCliDb(token: { access_token: string; refresh_token?: string; ex db.close(); } +function seedGrokAuth(token: { + key: string; + refresh_token: string; + expires_at: string; + user_id?: string; + email?: string; +}) { + const dir = join(tmp, ".grok"); + mkdirSync(dir, { recursive: true }); + writeFileSync(join(dir, "auth.json"), JSON.stringify({ "https://auth.x.ai::test": token })); +} + +function xaiRefreshResponses(access = "xai-fresh", refresh = "rt-fresh"): Response[] { + return [ + new Response(JSON.stringify({ + authorization_endpoint: "https://auth.x.ai/authorize", + token_endpoint: "https://auth.x.ai/token", + }), { status: 200 }), + new Response(JSON.stringify({ access_token: access, refresh_token: refresh, expires_in: 3600 }), { status: 200 }), + ]; +} + +function mockXaiRefreshFetch(access = "xai-fresh", refresh = "rt-fresh") { + let discoveryCalls = 0; + let tokenCalls = 0; + const tokenBodies: string[] = []; + globalThis.fetch = (async (_input, init) => { + if (init?.method === "POST") { + tokenCalls++; + tokenBodies.push(String(init.body)); + return xaiRefreshResponses(access, refresh)[1]!; + } + discoveryCalls++; + return xaiRefreshResponses(access, refresh)[0]!; + }) as typeof fetch; + return { + discoveryCount: () => discoveryCalls, + tokenCount: () => tokenCalls, + tokenBodies, + }; +} + function mockRefreshFetch(responses: Array): { count: () => number } { let calls = 0; let i = 0; @@ -52,7 +96,7 @@ function mockRefreshFetch(responses: Array): { count: () => nu describe("oauth refresh hardening", () => { test("valid stored credential returns without refresh", async () => { const mock = mockRefreshFetch([new Response("unexpected", { status: 500 })]); - saveCredential("kiro", { access: "aoa-valid", refresh: "rt", expires: Date.now() + 3600_000 }); + await saveCredential("kiro", { access: "aoa-valid", refresh: "rt", expires: Date.now() + 3600_000 }); await expect(getValidAccessToken("kiro")).resolves.toBe("aoa-valid"); expect(mock.count()).toBe(0); }); @@ -61,7 +105,7 @@ describe("oauth refresh hardening", () => { const mock = mockRefreshFetch([ new Response(JSON.stringify({ accessToken: "aoa-fresh", refreshToken: "rt-fresh", expiresIn: 3600 }), { status: 200 }), ]); - saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1 }); + await saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1 }); const [a, b] = await Promise.all([getValidAccessToken("kiro"), getValidAccessToken("kiro")]); expect(a).toBe("aoa-fresh"); expect(b).toBe("aoa-fresh"); @@ -72,7 +116,7 @@ describe("oauth refresh hardening", () => { test("fresh Kiro CLI SQLite token is imported before refresh endpoint", async () => { const mock = mockRefreshFetch([new Response("unexpected", { status: 500 })]); seedKiroCliDb({ access_token: "aoa-sqlite", refresh_token: "rt-sqlite", expires_at: "2099-01-01T00:00:00Z" }); - saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1 }); + await saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1 }); await expect(getValidAccessToken("kiro")).resolves.toBe("aoa-sqlite"); expect(mock.count()).toBe(0); expect(getCredential("kiro")?.refresh).toBe("rt-sqlite"); @@ -80,7 +124,7 @@ describe("oauth refresh hardening", () => { }); test("failed refresh recovers from a now-fresh Kiro CLI SQLite token", async () => { - saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1 }); + await saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1 }); let calls = 0; globalThis.fetch = (async () => { calls++; @@ -98,9 +142,119 @@ describe("oauth refresh hardening", () => { mockRefreshFetch([ new Response(JSON.stringify({ accessToken: "aoa-fresh", refreshToken: "rt-fresh", expiresIn: 3600 }), { status: 200 }), ]); - saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1, source: "manual" }); + await saveCredential("kiro", { access: "aoa-old", refresh: "rt-old", expires: Date.now() - 1, source: "manual" }); await expect(getValidAccessToken("kiro")).resolves.toBe("aoa-fresh"); expect(getCredential("kiro")?.source).toBe("manual"); }); + + test("newer Grok generation is adopted before xAI refresh with zero endpoint calls", async () => { + const mock = mockRefreshFetch([new Response("unexpected", { status: 500 })]); + await saveCredential("xai", { + access: "xai-old", refresh: "rt-old", expires: Date.now() - 1, accountId: "user-1", source: "local-cli", + }); + seedGrokAuth({ + key: "xai-disk", refresh_token: "rt-new", expires_at: new Date(Date.now() + 3600_000).toISOString(), user_id: "user-1", + }); + + await expect(getValidAccessToken("xai")).resolves.toBe("xai-disk"); + expect(mock.count()).toBe(0); + expect(getCredential("xai")?.refresh).toBe("rt-new"); + expect(getCredential("xai")?.source).toBe("local-cli"); + }); + + test("newer-expiry Grok access token is adopted when refresh generation is unchanged", async () => { + const mock = mockRefreshFetch([new Response("unexpected", { status: 500 })]); + await saveCredential("xai", { + access: "xai-old", refresh: "rt-same", expires: Date.now() - 1, accountId: "user-1", source: "local-cli", + }); + const diskExpires = Date.now() + 3600_000; + seedGrokAuth({ + key: "xai-disk", refresh_token: "rt-same", expires_at: new Date(diskExpires).toISOString(), user_id: "user-1", + }); + + await expect(getValidAccessToken("xai")).resolves.toBe("xai-disk"); + expect(mock.count()).toBe(0); + expect(getCredential("xai")?.expires).toBe(diskExpires); + }); + + test("stale Grok generation refreshes once and detaches to OpenCodex ownership", async () => { + await saveCredential("xai", { + access: "xai-old", refresh: "rt-old", expires: Date.now() - 1, accountId: "user-1", source: "local-cli", + }); + seedGrokAuth({ + key: "xai-disk", refresh_token: "rt-old", expires_at: new Date(Date.now() - 2_000).toISOString(), user_id: "user-1", + }); + const grokPath = join(tmp, ".grok", "auth.json"); + const before = readFileSync(grokPath); + const mock = mockXaiRefreshFetch(); + const warnings: unknown[][] = []; + const originalWarn = console.warn; + console.warn = (...args: unknown[]) => { warnings.push(args); }; + try { + await expect(getValidAccessToken("xai")).resolves.toBe("xai-fresh"); + } finally { + console.warn = originalWarn; + } + + expect(mock.discoveryCount()).toBe(1); + expect(mock.tokenCount()).toBe(1); + expect(warnings).toEqual([[ + "[oauth:xai] Grok CLI credential was stale; refreshed into OpenCodex ownership. Grok CLI may require login again.", + ]]); + expect(getCredential("xai")?.refresh).toBe("rt-fresh"); + expect(getCredential("xai")?.source).toBe("oauth"); + expect(readFileSync(grokPath)).toEqual(before); + }); + + test("stale different Grok generation with earlier expiry is not adopted", async () => { + const storedExpiry = Date.now() - 1; + await saveCredential("xai", { + access: "xai-ours", refresh: "rt-ours", expires: storedExpiry, accountId: "user-1", source: "local-cli", + }); + seedGrokAuth({ + key: "xai-disk", refresh_token: "rt-disk", expires_at: new Date(storedExpiry - 10_000).toISOString(), user_id: "user-1", + }); + const mock = mockXaiRefreshFetch(); + + await expect(getValidAccessToken("xai")).resolves.toBe("xai-fresh"); + expect(mock.discoveryCount()).toBe(1); + expect(mock.tokenCount()).toBe(1); + expect(new URLSearchParams(mock.tokenBodies[0]).get("refresh_token")).toBe("rt-ours"); + expect(getCredential("xai")?.source).toBe("oauth"); + }); + + test("mismatched Grok identity is not adopted into a local-cli account", async () => { + await saveCredential("xai", { + access: "xai-ours", refresh: "rt-ours", expires: Date.now() - 1, accountId: "user-1", source: "local-cli", + }); + seedGrokAuth({ + key: "xai-disk", refresh_token: "rt-disk", expires_at: new Date(Date.now() + 3600_000).toISOString(), user_id: "user-2", + }); + const mock = mockXaiRefreshFetch(); + + await expect(getValidAccessToken("xai")).resolves.toBe("xai-fresh"); + expect(mock.discoveryCount()).toBe(1); + expect(mock.tokenCount()).toBe(1); + expect(new URLSearchParams(mock.tokenBodies[0]).get("refresh_token")).toBe("rt-ours"); + expect(getCredential("xai")?.accountId).toBe("user-1"); + expect(getCredential("xai")?.source).toBe("oauth"); + }); + + test("concurrent xAI local-cli refreshes share reconciliation and one detach exchange", async () => { + await saveCredential("xai", { + access: "xai-old", refresh: "rt-old", expires: Date.now() - 1, accountId: "user-1", source: "local-cli", + }); + seedGrokAuth({ + key: "xai-disk", refresh_token: "rt-old", expires_at: new Date(Date.now() - 2_000).toISOString(), user_id: "user-1", + }); + const mock = mockXaiRefreshFetch(); + + const [a, b] = await Promise.all([getValidAccessToken("xai"), getValidAccessToken("xai")]); + expect(a).toBe("xai-fresh"); + expect(b).toBe("xai-fresh"); + expect(mock.discoveryCount()).toBe(1); + expect(mock.tokenCount()).toBe(1); + expect(getCredential("xai")?.source).toBe("oauth"); + }); }); diff --git a/tests/oauth-status-privacy.test.ts b/tests/oauth-status-privacy.test.ts index 4d4ea84f..57880d93 100644 --- a/tests/oauth-status-privacy.test.ts +++ b/tests/oauth-status-privacy.test.ts @@ -21,8 +21,8 @@ describe("OAuth status privacy", () => { if (existsSync(TEST_DIR)) rmSync(TEST_DIR, { recursive: true }); }); - test("getLoginStatus returns a masked provider email", () => { - saveCredential("xai", { + test("getLoginStatus returns a masked provider email", async () => { + await saveCredential("xai", { access: "access-token", refresh: "refresh-token", expires: Date.now() + 60_000, @@ -41,7 +41,7 @@ describe("OAuth status privacy", () => { expect(JSON.stringify(status)).not.toContain("refresh-token"); }); - test("saveCredential persists only the credential allowlist", () => { + test("saveCredential persists only the credential allowlist", async () => { writeFileSync(join(TEST_DIR, "auth.json"), JSON.stringify({ legacy: { access: "legacy-access", @@ -53,7 +53,7 @@ describe("OAuth status privacy", () => { }, }), "utf8"); - saveCredential("xai", { + await saveCredential("xai", { access: "access-token", refresh: "refresh-token", expires: Date.now() + 60_000, @@ -79,7 +79,7 @@ describe("OAuth status privacy", () => { expect(stored).not.toContain("jwt-secret"); }); - test("getLoginStatus ignores invalid legacy source metadata", () => { + test("getLoginStatus ignores invalid legacy source metadata", async () => { writeFileSync(join(TEST_DIR, "auth.json"), JSON.stringify({ xai: { access: "access-token", @@ -97,7 +97,7 @@ describe("OAuth status privacy", () => { }); test("stale credentials for removed OAuth providers fail as unsupported provider config", async () => { - saveCredential("removed-provider", { + await saveCredential("removed-provider", { access: "access-token", refresh: "refresh-token", expires: Date.now() + 60_000, @@ -106,11 +106,11 @@ describe("OAuth status privacy", () => { await expect(getValidAccessToken("removed-provider")).rejects.toBeInstanceOf(UnsupportedOAuthProviderError); }); - test("malformed oauth token store is backed up before a new credential save overwrites it", () => { + test("malformed oauth token store is backed up before a new credential save overwrites it", async () => { const authPath = join(TEST_DIR, "auth.json"); writeFileSync(authPath, "{not valid json", "utf8"); - saveCredential("xai", { + await saveCredential("xai", { access: "new-access", refresh: "new-refresh", expires: Date.now() + 60_000, diff --git a/tests/oauth-store-multi.test.ts b/tests/oauth-store-multi.test.ts index 9d390ba3..05b42ab3 100644 --- a/tests/oauth-store-multi.test.ts +++ b/tests/oauth-store-multi.test.ts @@ -38,7 +38,7 @@ describe("multi-account auth store", () => { if (existsSync(TEST_DIR)) rmSync(TEST_DIR, { recursive: true }); }); - test("legacy single-credential auth.json normalizes and round-trips without losing login", () => { + test("legacy single-credential auth.json normalizes and round-trips without losing login", async () => { const authPath = join(TEST_DIR, "auth.json"); mkdirSync(TEST_DIR, { recursive: true, mode: 0o700 }); writeFileSync(authPath, JSON.stringify({ @@ -46,14 +46,14 @@ describe("multi-account auth store", () => { })); expect(getCredential("xai")?.access).toBe("legacy-access"); // Any mutation persists the new shape + writes the downgrade backup. - saveCredential("xai", cred({ email: "old@example.com", access: "new-access" })); + await saveCredential("xai", cred({ email: "old@example.com", access: "new-access" })); expect(getCredential("xai")?.access).toBe("new-access"); const raw = JSON.parse(readFileSync(authPath, "utf-8")); expect(Array.isArray(raw.xai.accounts)).toBe(true); expect(existsSync(`${authPath}.pre-multiauth`)).toBe(true); }); - test("legacy credential WITHOUT identity gets a deterministic account id across loads", () => { + test("legacy credential WITHOUT identity gets a deterministic account id across loads", async () => { // Legacy stores are re-normalized on EVERY load without being persisted, so the // derived id must be stable: a time-salted id would make getAccountSet and // getAccountCredential disagree (spurious logout) and refresh persists no-op. @@ -68,98 +68,98 @@ describe("multi-account auth store", () => { expect(getAccountCredential("cursor", set!.activeAccountId)?.access).toBe("legacy-access"); expect(getAccountSet("cursor")!.activeAccountId).toBe(set!.activeAccountId); // A rotated refresh persisted against that id must land (not silently no-op). - saveAccountCredential("cursor", set!.activeAccountId, { + await saveAccountCredential("cursor", set!.activeAccountId, { access: "rotated-access", refresh: "rotated-refresh", expires: Date.now() + 3600_000, }); expect(getCredential("cursor")?.access).toBe("rotated-access"); expect(getCredential("cursor")?.refresh).toBe("rotated-refresh"); }); - test("new identity appends a second account and activates it", () => { - saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a" })); - saveCredential("anthropic", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); + test("new identity appends a second account and activates it", async () => { + await saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a" })); + await saveCredential("anthropic", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); expect(listAccounts("anthropic").length).toBe(2); expect(getCredential("anthropic")?.email).toBe("b@example.com"); }); - test("same identity replaces credential without duplicating", () => { - saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a" })); - saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a", access: "rotated", refresh: "rotated-refresh" })); + test("same identity replaces credential without duplicating", async () => { + await saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a" })); + await saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a", access: "rotated", refresh: "rotated-refresh" })); expect(listAccounts("anthropic").length).toBe(1); expect(getCredential("anthropic")?.access).toBe("rotated"); }); - test("identity-less credential replaces active slot (no duplicate on refresh rotation)", () => { - saveCredential("cursor", cred()); - saveCredential("cursor", cred({ access: "rotated", refresh: "totally-different-refresh" })); + test("identity-less credential replaces active slot (no duplicate on refresh rotation)", async () => { + await saveCredential("cursor", cred()); + await saveCredential("cursor", cred({ access: "rotated", refresh: "totally-different-refresh" })); expect(listAccounts("cursor").length).toBe(1); expect(getCredential("cursor")?.access).toBe("rotated"); }); - test("cursor with distinct accountIds appends a second account", () => { - saveCredential("cursor", cred({ accountId: "google-oauth2|user_a", access: "access-a" })); - saveCredential("cursor", cred({ accountId: "google-oauth2|user_b", access: "access-b" })); + test("cursor with distinct accountIds appends a second account", async () => { + await saveCredential("cursor", cred({ accountId: "google-oauth2|user_a", access: "access-a" })); + await saveCredential("cursor", cred({ accountId: "google-oauth2|user_b", access: "access-b" })); expect(listAccounts("cursor").length).toBe(2); expect(getCredential("cursor")?.access).toBe("access-b"); }); - test("chatgpt stays single-slot even with distinct identities", () => { - saveCredential("chatgpt", cred({ email: "a@example.com", accountId: "one" })); - saveCredential("chatgpt", cred({ email: "b@example.com", accountId: "two", access: "b-access" })); + test("chatgpt stays single-slot even with distinct identities", async () => { + await saveCredential("chatgpt", cred({ email: "a@example.com", accountId: "one" })); + await saveCredential("chatgpt", cred({ email: "b@example.com", accountId: "two", access: "b-access" })); expect(listAccounts("chatgpt").length).toBe(1); expect(getCredential("chatgpt")?.email).toBe("b@example.com"); }); - test("setActiveAccount switches what getCredential returns", () => { - saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a", access: "access-a" })); - saveCredential("anthropic", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); + test("setActiveAccount switches what getCredential returns", async () => { + await saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a", access: "access-a" })); + await saveCredential("anthropic", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); const set = getAccountSet("anthropic")!; const idA = set.accounts.find(a => a.credential.email === "a@example.com")!.id; - expect(setActiveAccount("anthropic", idA)).toBe(true); + expect(await setActiveAccount("anthropic", idA)).toBe(true); expect(getCredential("anthropic")?.access).toBe("access-a"); - expect(setActiveAccount("anthropic", "nope")).toBe(false); + expect(await setActiveAccount("anthropic", "nope")).toBe(false); }); - test("saveAccountCredential persists refresh for a non-active account without switching active", () => { - saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a" })); - saveCredential("xai", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); + test("saveAccountCredential persists refresh for a non-active account without switching active", async () => { + await saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a" })); + await saveCredential("xai", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); const set = getAccountSet("xai")!; const idA = set.accounts.find(a => a.credential.email === "a@example.com")!.id; - saveAccountCredential("xai", idA, cred({ email: "a@example.com", accountId: "acct-a", access: "refreshed-a" })); + await saveAccountCredential("xai", idA, cred({ email: "a@example.com", accountId: "acct-a", access: "refreshed-a" })); expect(getAccountCredential("xai", idA)?.access).toBe("refreshed-a"); expect(getCredential("xai")?.access).toBe("access-b"); // active unchanged }); - test("removeAccount of active promotes next; last removal deletes provider", () => { - saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a", access: "access-a" })); - saveCredential("xai", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); + test("removeAccount of active promotes next; last removal deletes provider", async () => { + await saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a", access: "access-a" })); + await saveCredential("xai", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); const set = getAccountSet("xai")!; - expect(removeAccount("xai", set.activeAccountId)).toBe(true); + expect(await removeAccount("xai", set.activeAccountId)).toBe(true); expect(getCredential("xai")?.access).toBe("access-a"); const remaining = getAccountSet("xai")!; - expect(removeAccount("xai", remaining.activeAccountId)).toBe(true); + expect(await removeAccount("xai", remaining.activeAccountId)).toBe(true); expect(getCredential("xai")).toBeNull(); expect(getAccountSet("xai")).toBeNull(); }); - test("removeCredential removes only the active account", () => { - saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a", access: "access-a" })); - saveCredential("anthropic", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); - removeCredential("anthropic"); // active is b + test("removeCredential removes only the active account", async () => { + await saveCredential("anthropic", cred({ email: "a@example.com", accountId: "acct-a", access: "access-a" })); + await saveCredential("anthropic", cred({ email: "b@example.com", accountId: "acct-b", access: "access-b" })); + await removeCredential("anthropic"); // active is b expect(listAccounts("anthropic").length).toBe(1); expect(getCredential("anthropic")?.access).toBe("access-a"); }); - test("needsReauth flag persists and clears on fresh save", () => { - saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a" })); + test("needsReauth flag persists and clears on fresh save", async () => { + await saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a" })); const id = getAccountSet("xai")!.activeAccountId; - markAccountNeedsReauth("xai", id, true); + await markAccountNeedsReauth("xai", id, true); expect(listAccounts("xai")[0]?.needsReauth).toBe(true); - saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a", access: "fresh" })); + await saveCredential("xai", cred({ email: "a@example.com", accountId: "acct-a", access: "fresh" })); expect(listAccounts("xai")[0]?.needsReauth).toBeUndefined(); }); - test("invalid account entries are dropped on load", () => { + test("invalid account entries are dropped on load", async () => { const authPath = join(TEST_DIR, "auth.json"); mkdirSync(TEST_DIR, { recursive: true, mode: 0o700 }); writeFileSync(authPath, JSON.stringify({ diff --git a/tests/provider-quota.test.ts b/tests/provider-quota.test.ts index 27bcc653..99a134dd 100644 --- a/tests/provider-quota.test.ts +++ b/tests/provider-quota.test.ts @@ -73,9 +73,9 @@ afterEach(() => { describe("fetchProviderQuotaReports", () => { test("returns active provider quota rows without leaking credentials or raw upstream payloads", async () => { - saveCredential("xai", { access: "xai-access-secret", refresh: "xai-refresh-secret", expires: Date.now() + 3600_000 }); - saveCredential("anthropic", { access: "claude-access-secret", refresh: "claude-refresh-secret", expires: Date.now() + 3600_000 }); - saveCredential("google-antigravity", { access: "agy-access-secret", refresh: "agy-refresh-secret", expires: Date.now() + 3600_000, projectId: "agy-project-secret" }); + await saveCredential("xai", { access: "xai-access-secret", refresh: "xai-refresh-secret", expires: Date.now() + 3600_000 }); + await saveCredential("anthropic", { access: "claude-access-secret", refresh: "claude-refresh-secret", expires: Date.now() + 3600_000 }); + await saveCredential("google-antigravity", { access: "agy-access-secret", refresh: "agy-refresh-secret", expires: Date.now() + 3600_000, projectId: "agy-project-secret" }); const seen: { url: string; authorization?: string; body?: string }[] = []; globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => { @@ -164,7 +164,7 @@ describe("fetchProviderQuotaReports", () => { }); test("skips Anthropic quota when the stored access token is expired", async () => { - saveCredential("anthropic", { access: "expired-claude-access", refresh: "expired-claude-refresh", expires: Date.now() - 1 }); + await saveCredential("anthropic", { access: "expired-claude-access", refresh: "expired-claude-refresh", expires: Date.now() - 1 }); const seen: string[] = []; globalThis.fetch = (async (input: RequestInfo | URL) => { seen.push(String(input)); diff --git a/tests/responses-parser-agent-message.test.ts b/tests/responses-parser-agent-message.test.ts index 3e350b93..5035cdb7 100644 --- a/tests/responses-parser-agent-message.test.ts +++ b/tests/responses-parser-agent-message.test.ts @@ -57,33 +57,23 @@ describe("Responses parser agent_message boundaries", () => { const messages = parsed.context.messages; + // Reasoning followed by an agent_message boundary is cleared, not emitted as a + // detached assistant turn (grok-build fold contract: reasoning belongs to the + // FOLLOWING assistant; a boundary with no following assistant drops it). expect(messages.map((message) => message.role)).toEqual([ - "assistant", "user", "assistant", "toolResult", ]); - const firstAssistant = messages[0]; - if (!firstAssistant || firstAssistant.role !== "assistant") { - throw new Error("expected the first parsed message to be assistant"); - } - - expect(firstAssistant.content).toHaveLength(1); - expect(firstAssistant.content[0]).toMatchObject({ - type: "thinking", - thinking: "reasoning before the sub-agent response", - itemId: "rs_before_agent", - }); - - expect(messages[1]).toMatchObject({ + expect(messages[0]).toMatchObject({ role: "user", content: "sub-agent result", }); - const secondAssistant = messages[2]; + const secondAssistant = messages[1]; if (!secondAssistant || secondAssistant.role !== "assistant") { - throw new Error("expected the third parsed message to be assistant"); + throw new Error("expected the second parsed message to be assistant"); } expect(secondAssistant.content).toHaveLength(2); @@ -100,7 +90,7 @@ describe("Responses parser agent_message boundaries", () => { thoughtSignature: "fc_after_agent", }); - expect(messages[3]).toMatchObject({ + expect(messages[2]).toMatchObject({ role: "toolResult", toolCallId: "call_after_agent", toolName: "shell_command", diff --git a/tests/server-xai-header-parity.test.ts b/tests/server-xai-header-parity.test.ts new file mode 100644 index 00000000..78238dfa --- /dev/null +++ b/tests/server-xai-header-parity.test.ts @@ -0,0 +1,126 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { saveConfig } from "../src/config"; +import { deriveXaiConvId } from "../src/providers/xai-transport"; +import { startServer } from "../src/server"; +import type { OcxConfig } from "../src/types"; +import { installIsolatedCodexHome, type IsolatedCodexHome } from "./helpers/isolated-codex-home"; + +const UUID_V4 = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; +const CONV_KEY = "server-parity-conversation"; + +let testDir = ""; +let previousHome: string | undefined; +let isolatedCodexHome: IsolatedCodexHome | null = null; +let originalFetch: typeof fetch; + +beforeEach(() => { + originalFetch = globalThis.fetch; + previousHome = process.env.OPENCODEX_HOME; + isolatedCodexHome = installIsolatedCodexHome("ocx-xai-parity-codex-"); + testDir = mkdtempSync(join(tmpdir(), "ocx-xai-parity-")); + process.env.OPENCODEX_HOME = testDir; +}); + +afterEach(() => { + globalThis.fetch = originalFetch; + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + isolatedCodexHome?.restore(); + isolatedCodexHome = null; + if (testDir) rmSync(testDir, { recursive: true, force: true }); +}); + +function config(connectTimeoutMs = 1_000): OcxConfig { + return { + port: 0, + hostname: "127.0.0.1", + connectTimeoutMs, + defaultProvider: "xai", + providers: { + xai: { + adapter: "openai-chat", + baseUrl: "https://api.x.ai/v1", + authMode: "key", + apiKey: "xai-test-key-000111222333", + defaultModel: "grok-test", + }, + }, + } as OcxConfig; +} + +function post(serverUrl: string): Promise { + return originalFetch(new URL("/v1/responses", serverUrl), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: "xai/grok-test", + stream: false, + prompt_cache_key: CONV_KEY, + input: "hello", + }), + }); +} + +describe("xAI headers through /v1/responses", () => { + test("two serving attempts refresh request id and preserve conversation affinity", async () => { + const seen: Headers[] = []; + globalThis.fetch = (async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === "https://api.x.ai/v1/chat/completions") { + seen.push(new Headers(init?.headers)); + return Response.json({ + id: `chatcmpl-${seen.length}`, + object: "chat.completion", + created: 1, + model: "grok-test", + choices: [{ index: 0, message: { role: "assistant", content: "ok" }, finish_reason: "stop" }], + usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 }, + }); + } + return originalFetch(input, init); + }) as typeof fetch; + saveConfig(config()); + const server = startServer(0); + try { + expect((await post(String(server.url))).status).toBe(200); + expect((await post(String(server.url))).status).toBe(200); + expect(seen).toHaveLength(2); + const firstReq = seen[0].get("x-grok-req-id"); + const secondReq = seen[1].get("x-grok-req-id"); + expect(firstReq).toMatch(UUID_V4); + expect(secondReq).toMatch(UUID_V4); + expect(secondReq).not.toBe(firstReq); + expect(seen[0].get("x-grok-conv-id")).toBe(deriveXaiConvId(CONV_KEY)); + expect(seen[1].get("x-grok-conv-id")).toBe(seen[0].get("x-grok-conv-id")); + } finally { + await server.stop(true); + } + }); + + test("provider executor remains inside the header-timeout race", async () => { + let accepted = false; + globalThis.fetch = ((input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === "https://api.x.ai/v1/chat/completions") { + accepted = true; + return new Promise((_resolve, reject) => { + init?.signal?.addEventListener("abort", () => reject(init.signal?.reason), { once: true }); + }); + } + return originalFetch(input, init); + }) as typeof fetch; + saveConfig(config(25)); + const server = startServer(0); + try { + const response = await post(String(server.url)); + expect(accepted).toBe(true); + expect(response.status).toBe(502); + expect(await response.text()).toContain("Provider connect timeout after 25ms"); + } finally { + await server.stop(true); + } + }); +}); diff --git a/tests/server-xai-oauth-401-replay.test.ts b/tests/server-xai-oauth-401-replay.test.ts new file mode 100644 index 00000000..4fbaf908 --- /dev/null +++ b/tests/server-xai-oauth-401-replay.test.ts @@ -0,0 +1,243 @@ +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { saveConfig } from "../src/config"; +import { XAI_OAUTH_DISCOVERY_URL } from "../src/oauth/xai"; +import { saveCredential } from "../src/oauth/store"; +import { XAI_GROK_CLI_BASE_URL } from "../src/providers/xai-transport"; +import { startServer } from "../src/server"; +import type { OcxConfig } from "../src/types"; +import { installIsolatedCodexHome, type IsolatedCodexHome } from "./helpers/isolated-codex-home"; + +const TOKEN_ENDPOINT = "https://auth.x.ai/oauth/token"; +const CHAT_ENDPOINT = `${XAI_GROK_CLI_BASE_URL}/chat/completions`; + +let testDir = ""; +let previousHome: string | undefined; +let isolatedCodexHome: IsolatedCodexHome | null = null; +let originalFetch: typeof fetch; + +beforeEach(() => { + originalFetch = globalThis.fetch; + previousHome = process.env.OPENCODEX_HOME; + isolatedCodexHome = installIsolatedCodexHome("ocx-xai-401-codex-"); + testDir = mkdtempSync(join(tmpdir(), "ocx-xai-401-")); + process.env.OPENCODEX_HOME = testDir; +}); + +afterEach(() => { + globalThis.fetch = originalFetch; + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + isolatedCodexHome?.restore(); + isolatedCodexHome = null; + if (testDir) rmSync(testDir, { recursive: true, force: true }); +}); + +function seedOAuth(): void { + saveCredential("xai", { + access: "rejected-access", + refresh: "initial-refresh", + expires: Date.now() + 3_600_000, + accountId: "xai-test-account", + source: "oauth", + }); +} + +function xaiConfig(authMode: "oauth" | "key" = "oauth"): OcxConfig { + return { + port: 0, + hostname: "127.0.0.1", + defaultProvider: "xai", + providers: { + xai: { + adapter: "openai-chat", + baseUrl: "https://api.x.ai/v1", + authMode, + ...(authMode === "key" ? { apiKey: "xai-api-key" } : {}), + models: ["grok-4.5"], + }, + }, + } as OcxConfig; +} + +function successBody(text: string): string { + return JSON.stringify({ + id: "chatcmpl-xai-401", + object: "chat.completion", + choices: [{ index: 0, message: { role: "assistant", content: text }, finish_reason: "stop" }], + usage: { prompt_tokens: 3, completion_tokens: 2, total_tokens: 5 }, + }); +} + +async function post(server: ReturnType): Promise { + return originalFetch(new URL("/v1/responses", server.url), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "xai/grok-4.5", input: "hello", stream: false }), + }); +} + +function installOAuthFetch(chatStatuses: number[]): { chatAuth: string[]; counts: { refresh: number } } { + const chatAuth: string[] = []; + const counts = { refresh: 0 }; + globalThis.fetch = (async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === XAI_OAUTH_DISCOVERY_URL) { + return new Response(JSON.stringify({ + authorization_endpoint: "https://auth.x.ai/oauth/authorize", + token_endpoint: TOKEN_ENDPOINT, + }), { headers: { "content-type": "application/json" } }); + } + if (url === TOKEN_ENDPOINT) { + counts.refresh += 1; + return new Response(JSON.stringify({ + access_token: "fresh-access", + refresh_token: "fresh-refresh", + expires_in: 3600, + }), { headers: { "content-type": "application/json" } }); + } + if (url === CHAT_ENDPOINT) { + chatAuth.push(new Headers(init?.headers).get("authorization") ?? ""); + const status = chatStatuses.shift() ?? 200; + if (status === 401) { + return new Response(JSON.stringify({ error: { message: "rejected" } }), { + status: 401, + headers: { "content-type": "application/json" }, + }); + } + return new Response(successBody("ok after refresh"), { headers: { "content-type": "application/json" } }); + } + return originalFetch(input, init); + }) as typeof fetch; + return { chatAuth, counts }; +} + +describe("xAI OAuth upstream 401 replay", () => { + test("401 then 200 performs one refresh and one replay", async () => { + seedOAuth(); + saveConfig(xaiConfig()); + const observed = installOAuthFetch([401, 200]); + const server = startServer(0); + try { + const response = await post(server); + expect(response.status).toBe(200); + const json = await response.json() as { output?: { type: string; content?: { text?: string }[] }[] }; + expect(json.output?.find(item => item.type === "message")?.content?.[0]?.text).toBe("ok after refresh"); + expect(observed.counts.refresh).toBe(1); + expect(observed.chatAuth).toEqual(["Bearer rejected-access", "Bearer fresh-access"]); + } finally { + server.stop(true); + } + }); + + test("401 then 401 replays once and propagates the second error", async () => { + seedOAuth(); + saveConfig(xaiConfig()); + const observed = installOAuthFetch([401, 401]); + const server = startServer(0); + try { + const response = await post(server); + const json = await response.json() as { error?: { message?: string } }; + expect(response.status).toBe(401); + expect(json.error?.message).toContain("Provider error 401"); + expect(observed.counts.refresh).toBe(1); + expect(observed.chatAuth).toEqual(["Bearer rejected-access", "Bearer fresh-access"]); + } finally { + server.stop(true); + } + }); + + test("API-key xAI path never attempts OAuth refresh", async () => { + saveConfig(xaiConfig("key")); + let refreshCalls = 0; + let chatCalls = 0; + globalThis.fetch = (async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === XAI_OAUTH_DISCOVERY_URL || url === TOKEN_ENDPOINT) { + refreshCalls += 1; + return new Response("unexpected", { status: 500 }); + } + if (url === "https://api.x.ai/v1/chat/completions") { + chatCalls += 1; + expect(new Headers(init?.headers).get("authorization")).toBe("Bearer xai-api-key"); + return new Response(JSON.stringify({ error: { message: "key rejected" } }), { + status: 401, + headers: { "content-type": "application/json" }, + }); + } + return originalFetch(input, init); + }) as typeof fetch; + const server = startServer(0); + try { + const response = await post(server); + expect(response.status).toBe(401); + expect(chatCalls).toBe(1); + expect(refreshCalls).toBe(0); + } finally { + server.stop(true); + } + }); + + test("concurrent 401 responses join one IdP refresh", async () => { + seedOAuth(); + saveConfig(xaiConfig()); + let refreshCalls = 0; + let signalRefreshStarted!: () => void; + const refreshStarted = new Promise(resolve => { signalRefreshStarted = resolve; }); + let releaseRefresh!: () => void; + const refreshGate = new Promise(resolve => { releaseRefresh = resolve; }); + let releaseRejectedRequests!: () => void; + const rejectedRequestsReady = new Promise(resolve => { releaseRejectedRequests = resolve; }); + const attemptsByBearer = new Map(); + globalThis.fetch = (async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + if (url === XAI_OAUTH_DISCOVERY_URL) { + return new Response(JSON.stringify({ + authorization_endpoint: "https://auth.x.ai/oauth/authorize", + token_endpoint: TOKEN_ENDPOINT, + }), { headers: { "content-type": "application/json" } }); + } + if (url === TOKEN_ENDPOINT) { + refreshCalls += 1; + signalRefreshStarted(); + await refreshGate; + return new Response(JSON.stringify({ + access_token: "fresh-access", + refresh_token: "fresh-refresh", + expires_in: 3600, + }), { headers: { "content-type": "application/json" } }); + } + if (url === CHAT_ENDPOINT) { + const bearer = new Headers(init?.headers).get("authorization") ?? ""; + attemptsByBearer.set(bearer, (attemptsByBearer.get(bearer) ?? 0) + 1); + if (bearer === "Bearer rejected-access") { + if (attemptsByBearer.get(bearer) === 2) releaseRejectedRequests(); + await rejectedRequestsReady; + return new Response(JSON.stringify({ error: { message: "rejected" } }), { + status: 401, + headers: { "content-type": "application/json" }, + }); + } + return new Response(successBody("concurrent ok"), { headers: { "content-type": "application/json" } }); + } + return originalFetch(input, init); + }) as typeof fetch; + + const server = startServer(0); + try { + const first = post(server); + const second = post(server); + await refreshStarted; + releaseRefresh(); + const [a, b] = await Promise.all([first, second]); + expect([a.status, b.status]).toEqual([200, 200]); + expect(refreshCalls).toBe(1); + expect(attemptsByBearer.get("Bearer rejected-access")).toBe(2); + expect(attemptsByBearer.get("Bearer fresh-access")).toBe(2); + } finally { + server.stop(true); + } + }); +}); diff --git a/tests/token-guardian.test.ts b/tests/token-guardian.test.ts index 515d6bdd..a6e4010a 100644 --- a/tests/token-guardian.test.ts +++ b/tests/token-guardian.test.ts @@ -75,7 +75,7 @@ describe("token guardian", () => { test("disabled by default → no refresh, no fetch", async () => { const mock = mockFetchOk(OK_TOKEN); writeConfig({}); // no tokenGuardian - saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 1000 }); + await saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 1000 }); const res = await guardianSweep(Date.now()); expect(res.enabled).toBe(false); expect(res.refreshed).toEqual([]); @@ -88,7 +88,7 @@ describe("token guardian", () => { tokenGuardian: { enabled: true, tickSeconds: 60, leadSeconds: 60 }, providers: { kimi: kimiProvider("proactive") }, }); - saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 5_000 }); + await saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 5_000 }); const res = await guardianSweep(Date.now()); expect(res.enabled).toBe(true); // Multiauth keys are oauth:: @@ -102,7 +102,7 @@ describe("token guardian", () => { tokenGuardian: { enabled: true, tickSeconds: 60, leadSeconds: 60 }, providers: { kimi: kimiProvider("lazy-only") }, }); - saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 5_000 }); + await saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 5_000 }); const res = await guardianSweep(Date.now()); expect(res.refreshed).toEqual([]); expect(mock.count()).toBe(0); @@ -114,7 +114,7 @@ describe("token guardian", () => { tokenGuardian: { enabled: true, tickSeconds: 60, leadSeconds: 60 }, providers: { kimi: kimiProvider("proactive") }, }); - saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 3600_000 }); // beyond 120s horizon + await saveCredential("kimi", { access: "a", refresh: "r", expires: Date.now() + 3600_000 }); // beyond 120s horizon const res = await guardianSweep(Date.now()); expect(res.refreshed).toEqual([]); expect(mock.count()).toBe(0); @@ -127,7 +127,7 @@ describe("token guardian", () => { // no explicit refreshPolicy → falls back to the built-in "disabled" default for anthropic providers: { anthropic: { adapter: "anthropic", baseUrl: "https://api.anthropic.com", authMode: "oauth" } }, }); - saveCredential("anthropic", { access: "a", refresh: "r", expires: Date.now() + 5_000 }); + await saveCredential("anthropic", { access: "a", refresh: "r", expires: Date.now() + 5_000 }); const res = await guardianSweep(Date.now()); expect(res.refreshed).toEqual([]); expect(mock.count()).toBe(0); diff --git a/tests/xai-oauth-retry.test.ts b/tests/xai-oauth-retry.test.ts new file mode 100644 index 00000000..05f4e335 --- /dev/null +++ b/tests/xai-oauth-retry.test.ts @@ -0,0 +1,12 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import { postXaiToken, XaiTokenRequestError } from "../src/oauth/xai"; +const original=globalThis.fetch; afterEach(()=>{globalThis.fetch=original;}); +function queue(items:Array){let n=0;globalThis.fetch=(async()=>{const x=items[n++]!;if(x instanceof Error)throw x;return x;}) as typeof fetch;return()=>n;} +const body={grant_type:"refresh_token",client_id:"client",refresh_token:"secret"}; const ok=()=>new Response(JSON.stringify({access_token:"a",refresh_token:"r",expires_in:3600})); +describe("xAI retry",()=>{ + test("network retry succeeds",async()=>{const calls=queue([new Error("net"),ok()]),d:number[]=[];await postXaiToken("https://auth.x.ai/token",body,undefined,{sleep:async x=>{d.push(x)},random:()=>.5});expect(calls()).toBe(2);expect(d).toEqual([100]);}); + test("429 and 5xx retry at most three attempts",async()=>{const calls=queue([new Response("",{status:429}),new Response("",{status:503}),ok()]),d:number[]=[];await postXaiToken("https://auth.x.ai/token",body,undefined,{sleep:async x=>{d.push(x)},random:()=>.5});expect(calls()).toBe(3);expect(d).toEqual([100,250]);}); + test("third transient failure is final",async()=>{const calls=queue([500,502,503].map(status=>new Response("",{status})));await expect(postXaiToken("https://auth.x.ai/token",body,undefined,{sleep:async()=>{},random:()=>.5})).rejects.toMatchObject({status:503});expect(calls()).toBe(3);}); + test("permanent 4xx is not retried or leaked",async()=>{const calls=queue([new Response(JSON.stringify({error:"invalid_grant"}),{status:400})]);await expect(postXaiToken("https://auth.x.ai/token",body,undefined,{sleep:async()=>{}})).rejects.toBeInstanceOf(XaiTokenRequestError);expect(calls()).toBe(1);}); + test("caller abort is not retried",async()=>{const c=new AbortController();c.abort();let calls=0;globalThis.fetch=(async()=>{calls++;throw new DOMException("aborted","AbortError")}) as typeof fetch;await expect(postXaiToken("https://auth.x.ai/token",body,c.signal,{sleep:async()=>{}})).rejects.toMatchObject({name:"AbortError"});expect(calls).toBe(1);}); +}); diff --git a/tests/xai-refresh-lock.test.ts b/tests/xai-refresh-lock.test.ts new file mode 100644 index 00000000..4d3f68bd --- /dev/null +++ b/tests/xai-refresh-lock.test.ts @@ -0,0 +1,18 @@ +import { afterEach,beforeEach,describe,expect,test } from "bun:test"; +import { mkdirSync,readFileSync,rmSync,utimesSync,writeFileSync } from "node:fs";import{tmpdir}from"node:os";import{join}from"node:path"; +import { OAUTH_PROVIDERS,OAuthLoginRequiredError,refreshXaiAccountWithLock } from "../src/oauth";import{XaiTokenRequestError}from"../src/oauth/xai"; +import{createOAuthFileLock,createOAuthRefreshIntentLock,getAccountCredential,getAccountSet,getAuthRefreshIntentLockPath,OAuthFileLockError,saveCredential}from"../src/oauth/store"; +const oldHome=process.env.HOME,oldOcx=process.env.OPENCODEX_HOME;let root:string;beforeEach(()=>{root=join(tmpdir(),`xai-lock-${crypto.randomUUID()}`);process.env.HOME=root;process.env.OPENCODEX_HOME=join(root,"ocx");mkdirSync(process.env.OPENCODEX_HOME,{recursive:true});});afterEach(()=>{if(oldHome===undefined)delete process.env.HOME;else process.env.HOME=oldHome;if(oldOcx===undefined)delete process.env.OPENCODEX_HOME;else process.env.OPENCODEX_HOME=oldOcx;rmSync(root,{recursive:true,force:true});}); +async function seed(){await saveCredential("xai",{access:"old",refresh:"rotating",expires:1,accountId:"acct"});return getAccountSet("xai")!.activeAccountId;}function def(c:{n:number},gate?:Promise){return{...OAUTH_PROVIDERS.xai!,refresh:async()=>{c.n++;if(gate)await gate;return{access:"fresh",refresh:"next",expires:Date.now()+3600000}}};} +describe("two-lock xAI refresh",()=>{ + test("unrelated writer survives the pre-persist seam",async()=>{const id=await seed(),calls={n:0};let writer!:Promise;await refreshXaiAccountWithLock("xai",id,def(calls),getAccountCredential("xai",id)!,{afterPrePersistRead:()=>{writer=saveCredential("cursor",{access:"c",refresh:"r",expires:Date.now()+1000,accountId:"cursor"})}});await writer;expect(getAccountSet("cursor")?.accounts[0]?.credential.access).toBe("c");expect(getAccountCredential("xai",id)?.refresh).toBe("next");}); + test("same-process writer completes while refresh awaits token I/O",async()=>{const id=await seed();let release!:()=>void;const gate=new Promise(r=>release=r),calls={n:0};const pending=refreshXaiAccountWithLock("xai",id,def(calls,gate),getAccountCredential("xai",id)!);while(!calls.n)await Bun.sleep(1);await saveCredential("cursor",{access:"c",refresh:"r",expires:Date.now()+1000,accountId:"cursor"});release();expect(await pending).toBe("fresh");}); + test("two independent lock instances perform one IdP call",async()=>{const id=await seed();let release!:()=>void;const gate=new Promise(r=>release=r),calls={n:0},stale=getAccountCredential("xai",id)!;const a=refreshXaiAccountWithLock("xai",id,def(calls,gate),stale,{intentLock:createOAuthRefreshIntentLock("xai",id)});while(!calls.n)await Bun.sleep(1);const b=refreshXaiAccountWithLock("xai",id,def(calls),stale,{intentLock:createOAuthRefreshIntentLock("xai",id)});release();expect(await Promise.all([a,b])).toEqual(["fresh","fresh"]);expect(calls.n).toBe(1);}); + test("lock timeout is typed",async()=>{const id=await seed(),held=await createOAuthRefreshIntentLock("xai",id).acquire();try{await expect(createOAuthRefreshIntentLock("xai",id,{waitTimeoutMs:0}).acquire()).rejects.toBeInstanceOf(OAuthFileLockError);}finally{held.release();}}); + test("stale takeover and replacement-safe release",async()=>{const p=getAuthRefreshIntentLockPath("xai","acct");writeFileSync(p,JSON.stringify({version:1,ownerId:"old",pid:1,createdAt:0}));utimesSync(p,new Date(0),new Date(0));const g=await createOAuthFileLock({path:p,staleAfterMs:1}).acquire();const replacement=JSON.stringify({version:1,ownerId:"new",pid:2,createdAt:Date.now()});writeFileSync(p,replacement);g.release();expect(readFileSync(p,"utf8")).toBe(replacement);}); + test("partial metadata failure cleans its inode",async()=>{const p=getAuthRefreshIntentLockPath("xai","acct");await expect(createOAuthFileLock({path:p,writeMetadata:fd=>{writeFileSync(fd,"{");throw new Error("partial")}}).acquire()).rejects.toBeInstanceOf(OAuthFileLockError);expect(()=>readFileSync(p)).toThrow();}); + test("replacement between stale inspections survives",async()=>{const p=getAuthRefreshIntentLockPath("xai","acct");writeFileSync(p,JSON.stringify({version:1,ownerId:"old",pid:1,createdAt:0}));utimesSync(p,new Date(0),new Date(0));const replacement=JSON.stringify({version:1,ownerId:"new",pid:2,createdAt:Date.now()});await expect(createOAuthFileLock({path:p,staleAfterMs:1,waitTimeoutMs:0,beforeStaleUnlink:()=>writeFileSync(p,replacement)}).acquire()).rejects.toBeInstanceOf(OAuthFileLockError);expect(readFileSync(p,"utf8")).toBe(replacement);}); + test("same-account replacement during exchange wins",async()=>{const id=await seed();let release!:()=>void;const gate=new Promise(r=>release=r),calls={n:0},stale=getAccountCredential("xai",id)!;const pending=refreshXaiAccountWithLock("xai",id,def(calls,gate),stale);while(!calls.n)await Bun.sleep(1);await saveCredential("xai",{access:"writer",refresh:"writer-gen",expires:Date.now()+3600000,accountId:"acct"});release();expect(await pending).toBe("writer");expect(getAccountCredential("xai",id)?.refresh).toBe("writer-gen");}); + test("terminal failure does not mark replaced generation",async()=>{const id=await seed();let reject!:()=>void,calls=0;const gate=new Promise((_,r)=>reject=()=>r(new XaiTokenRequestError(400,"invalid_grant","bad")));const stale=getAccountCredential("xai",id)!;const pending=refreshXaiAccountWithLock("xai",id,{...OAUTH_PROVIDERS.xai!,refresh:async()=>{calls++;return gate}},stale);while(!calls)await Bun.sleep(1);await saveCredential("xai",{access:"writer",refresh:"writer-gen",expires:Date.now()+3600000,accountId:"acct"});reject();await expect(pending).rejects.toBeInstanceOf(OAuthLoginRequiredError);expect(getAccountSet("xai")!.accounts[0]!.needsReauth).toBeUndefined();}); + test("permanent verdict TTL is generation scoped",async()=>{const id=await seed();await saveCredential("xai",{access:"ttl-old",refresh:"ttl-rotating",expires:1,accountId:"acct"});const credential=getAccountCredential("xai",id)!;let now=1000,calls=0;const bad={...OAUTH_PROVIDERS.xai!,refresh:async()=>{calls++;throw new XaiTokenRequestError(400,"invalid_grant","bad")}};await expect(refreshXaiAccountWithLock("xai",id,bad,credential,{now:()=>now})).rejects.toThrow();await expect(refreshXaiAccountWithLock("xai",id,bad,credential,{now:()=>now})).rejects.toThrow();expect(calls).toBe(1);now+=30001;await expect(refreshXaiAccountWithLock("xai",id,bad,credential,{now:()=>now})).rejects.toThrow();expect(calls).toBe(2);await saveCredential("xai",{...credential,access:"new",refresh:"new-r",expires:now+100000});expect(await refreshXaiAccountWithLock("xai",id,bad,credential,{now:()=>now})).toBe("new");}); +}); diff --git a/tests/xai-transport.test.ts b/tests/xai-transport.test.ts index 10a1f0d2..50b0e193 100644 --- a/tests/xai-transport.test.ts +++ b/tests/xai-transport.test.ts @@ -12,6 +12,16 @@ import { import { getProviderRegistryEntry } from "../src/providers/registry"; import type { OcxAssistantMessage, OcxParsedRequest, OcxProviderConfig } from "../src/types"; +const UUID_V4 = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; +const OMITTED = [ + "x-grok-model-override", + "x-grok-agent-id", + "x-grok-turn-idx", + "x-grok-deployment-id", + "x-grok-user-id", + "x-grok-client-mode", +] as const; + function provider(authMode: "oauth" | "key"): OcxProviderConfig { return { adapter: "openai-chat", @@ -64,10 +74,10 @@ describe("xAI auth-mode transport selection", () => { const request = createOpenAIChatAdapter(effective).buildRequest(parsed()); const modelsRequest = buildModelsRequest(configured, "xai-api-key", "xai"); - expect(effective).toBe(configured); + expect(effective).not.toBe(configured); expect(request.url).toBe("https://api.x.ai/v1/chat/completions"); expect(modelsRequest.url).toBe("https://api.x.ai/v1/models"); - expect(request.headers).toEqual({ + expect(request.headers).toMatchObject({ "Content-Type": "application/json", Authorization: "Bearer xai-api-key", }); @@ -188,8 +198,8 @@ describe("xAI prompt-cache conv-id affinity", () => { expect(noKeyOauth.headers?.[XAI_CONV_ID_HEADER]).toBeUndefined(); expect(emptyKeyOauth.headers?.[XAI_CONV_ID_HEADER]).toBeUndefined(); expect(blankKeyOauth.headers?.[XAI_CONV_ID_HEADER]).toBeUndefined(); - // Key mode without a conv-id stays the exact configured object (no clone churn). - expect(emptyKeyApi).toBe(configuredKey); + expect(emptyKeyApi).not.toBe(configuredKey); + expect(emptyKeyApi.fetch).toBeFunction(); }); test("user-configured conv-id header wins in any casing (no duplicate header pair)", () => { @@ -222,6 +232,123 @@ describe("xAI prompt-cache conv-id affinity", () => { }); }); +function lower(headers: Headers): Record { + return Object.fromEntries([...headers.entries()].sort(([a], [b]) => a.localeCompare(b))); +} + +async function capture(authMode: "oauth" | "key", calls = 1) { + const seen: Headers[] = []; + const configured = provider(authMode) as OcxProviderConfig & { fetch?: typeof globalThis.fetch }; + configured.fetch = async (_input, init) => { + seen.push(new Headers(init?.headers)); + return new Response("{}", { status: 200 }); + }; + const effective = resolveProviderTransport("xai", configured, "codex-session-abc"); + const request = createOpenAIChatAdapter(effective).buildRequest(parsed()); + for (let index = 0; index < calls; index += 1) { + await effective.fetch!(request.url, { + method: request.method, + headers: request.headers, + body: request.body, + }); + } + return { effective, seen }; +} + +describe("xAI outbound compatibility headers", () => { + test("OAuth snapshot is exact", async () => { + const { effective, seen } = await capture("oauth"); + expect(effective.baseUrl).toBe(XAI_GROK_CLI_BASE_URL); + expect(lower(seen[0])).toEqual({ + authorization: "Bearer oauth-token", + "content-type": "application/json", + "user-agent": `opencodex-grok/${XAI_GROK_CLIENT_VERSION}`, + "x-authenticateresponse": "authenticate-response", + "x-grok-client-identifier": "opencodex", + "x-grok-client-version": XAI_GROK_CLIENT_VERSION, + "x-grok-conv-id": deriveXaiConvId("codex-session-abc"), + "x-grok-req-id": expect.stringMatching(UUID_V4), + "x-grok-session-id": deriveXaiConvId("codex-session-abc"), + "x-xai-token-auth": "xai-grok-cli", + }); + for (const name of OMITTED) expect(seen[0].has(name)).toBe(false); + }); + + test("API-key snapshot is exact and User-Agent is present", async () => { + const { effective, seen } = await capture("key"); + expect(effective.baseUrl).toBe("https://api.x.ai/v1"); + expect(lower(seen[0])).toEqual({ + authorization: "Bearer xai-api-key", + "content-type": "application/json", + "user-agent": `opencodex-grok/${XAI_GROK_CLIENT_VERSION}`, + "x-grok-conv-id": deriveXaiConvId("codex-session-abc"), + "x-grok-req-id": expect.stringMatching(UUID_V4), + "x-grok-session-id": deriveXaiConvId("codex-session-abc"), + }); + for (const name of [ + "x-authenticateresponse", + "x-grok-client-identifier", + "x-grok-client-version", + "x-xai-token-auth", + ...OMITTED, + ]) expect(seen[0].has(name)).toBe(false); + }); + + test("same resolved transport refreshes req-id but keeps conv-id stable", async () => { + const { seen } = await capture("oauth", 2); + expect(seen).toHaveLength(2); + expect(seen[0].get("x-grok-req-id")).toMatch(UUID_V4); + expect(seen[1].get("x-grok-req-id")).toMatch(UUID_V4); + expect(seen[1].get("x-grok-req-id")).not.toBe(seen[0].get("x-grok-req-id")); + expect(seen[0].get("x-grok-conv-id")).toBe(deriveXaiConvId("codex-session-abc")); + expect(seen[1].get("x-grok-conv-id")).toBe(seen[0].get("x-grok-conv-id")); + expect(seen[1].get("x-grok-session-id")).toBe(seen[0].get("x-grok-session-id")); + for (const headers of seen) { + expect(headers.get("user-agent")).toBe(`opencodex-grok/${XAI_GROK_CLIENT_VERSION}`); + for (const name of OMITTED) expect(headers.has(name)).toBe(false); + } + }); + + test("mixed-case caller overrides win without duplicates", async () => { + const seen: Headers[] = []; + const configured = provider("oauth") as OcxProviderConfig & { fetch?: typeof globalThis.fetch }; + configured.headers = { "user-agent": "custom-agent", "X-Grok-Req-Id": "caller-id" }; + configured.fetch = async (_input, init) => { + seen.push(new Headers(init?.headers)); + return new Response("{}", { status: 200 }); + }; + const effective = resolveProviderTransport("xai", configured, "codex-session-abc"); + const request = createOpenAIChatAdapter(effective).buildRequest(parsed()); + await effective.fetch!(request.url, { headers: request.headers }); + await effective.fetch!(request.url, { headers: request.headers }); + for (const headers of seen) { + expect(headers.get("user-agent")).toBe("custom-agent"); + expect(headers.get("x-grok-req-id")).toBe("caller-id"); + expect([...headers.keys()].filter(name => name === "user-agent")).toHaveLength(1); + expect([...headers.keys()].filter(name => name === "x-grok-req-id")).toHaveLength(1); + } + }); + + test("blank cache keys omit affinity but retain UA and fresh req-id in both modes", async () => { + for (const authMode of ["oauth", "key"] as const) { + const seen: Headers[] = []; + const configured = provider(authMode) as OcxProviderConfig & { fetch?: typeof globalThis.fetch }; + configured.fetch = async (_input, init) => { + seen.push(new Headers(init?.headers)); + return new Response("{}", { status: 200 }); + }; + const effective = resolveProviderTransport("xai", configured, " "); + const request = createOpenAIChatAdapter(effective).buildRequest(parsed()); + await effective.fetch!(request.url, { headers: request.headers }); + expect(seen[0].has("x-grok-conv-id")).toBe(false); + expect(seen[0].has("x-grok-session-id")).toBe(false); + expect(seen[0].get("user-agent")).toBe(`opencodex-grok/${XAI_GROK_CLIENT_VERSION}`); + expect(seen[0].get("x-grok-req-id")).toMatch(UUID_V4); + for (const name of OMITTED) expect(seen[0].has(name)).toBe(false); + } + }); +}); + describe("xAI reasoning_content cache preservation", () => { test("registry preset replays reasoning_content for grok reasoning models only", () => { const entry = getProviderRegistryEntry("xai"); @@ -236,29 +363,130 @@ describe("xAI reasoning_content cache preservation", () => { } }); - test("assistant thinking parts round-trip as reasoning_content on grok-4.5 history", () => { - // xAI docs: dropped reasoning_content is the top cause of multi-turn cache misses - // (docs.x.ai prompt-caching/multi-turn, 2026-07-13). + test("parseRequest folds summary reasoning into one Grok assistant wire message", () => { const prov: OcxProviderConfig = { ...provider("oauth"), preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], }; - const assistant: OcxAssistantMessage = { - role: "assistant", - content: [ - { type: "thinking", thinking: "cached chain" }, - { type: "text", text: "answer" }, + const req = parseRequest({ + model: "grok-4.5", + input: [ + { type: "message", role: "user", content: [{ type: "input_text", text: "q1" }] }, + { type: "reasoning", id: "r1", summary: [{ type: "summary_text", text: "cached chain" }] }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "q2" }] }, ], - timestamp: 0, + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toHaveLength(1); + expect(assistants[0]).toMatchObject({ content: "answer", reasoning_content: "cached chain" }); + }); + + test("parseRequest drops opaque encrypted-only reasoning without detaching an assistant wire message", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], }; - const req: OcxParsedRequest = { - modelId: "grok-4.5", - context: { messages: [{ role: "user", content: "q1", timestamp: 0 }, assistant, { role: "user", content: "q2", timestamp: 0 }] }, - stream: false, - options: {}, + const req = parseRequest({ + model: "grok-4.5", + input: [ + { type: "message", role: "user", content: [{ type: "input_text", text: "q1" }] }, + { type: "reasoning", id: "r-opaque", summary: [], encrypted_content: "opaque-native-blob" }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "q2" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toHaveLength(1); + expect(assistants[0]).toEqual({ role: "assistant", content: "answer" }); + expect(assistants[0]).not.toHaveProperty("reasoning_content"); + }); + + test("parseRequest clears pending reasoning at a user boundary", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const req = parseRequest({ + model: "grok-4.5", + input: [ + { type: "reasoning", id: "r-orphan", summary: [{ type: "summary_text", text: "must drop" }] }, + { type: "message", role: "user", content: [{ type: "input_text", text: "new turn" }] }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toEqual([{ role: "assistant", content: "answer" }]); + expect(assistants[0]).not.toHaveProperty("reasoning_content"); + }); + + test("parseRequest folds pending reasoning into the assistant turn that carries the call", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], + }; + const req = parseRequest({ + model: "grok-4.5", + input: [ + { type: "reasoning", id: "r-call", summary: [{ type: "summary_text", text: "call chain" }] }, + { type: "function_call", call_id: "call_1", name: "lookup", arguments: "{\"q\":\"x\"}" }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + ], + }); + const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; + const assistants = body.messages.filter(message => message.role === "assistant"); + + expect(assistants).toHaveLength(2); + // Grok wire shape: a reasoning model emits reasoning_content and tool_calls on the SAME + // assistant message (and Anthropic replay requires thinking before tool_use in one turn). + expect(assistants[0]).toMatchObject({ + reasoning_content: "call chain", + tool_calls: [{ id: "call_1", type: "function", function: { name: "lookup", arguments: "{\"q\":\"x\"}" } }], + }); + expect(assistants[1]).toMatchObject({ content: "answer" }); + expect(assistants[1]).not.toHaveProperty("reasoning_content"); + }); + + test("parseRequest newline-joins reasoning siblings before one assistant", () => { + const prov: OcxProviderConfig = { + ...provider("oauth"), + preserveReasoningContentModels: getProviderRegistryEntry("xai")?.preserveReasoningContentModels ?? [], }; + const req = parseRequest({ + model: "grok-4.5", + input: [ + { type: "reasoning", id: "r1", summary: [{ type: "summary_text", text: "first" }] }, + { type: "reasoning", id: "r2", summary: [{ type: "summary_text", text: "second" }] }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "answer" }] }, + ], + }); const body = JSON.parse(createOpenAIChatAdapter(prov).buildRequest(req).body as string) as { messages: Array> }; - const replayed = body.messages.find(m => m.role === "assistant"); - expect(replayed?.reasoning_content).toBe("cached chain"); + const assistants = body.messages.filter(message => message.role === "assistant"); + const parsedAssistant = req.context.messages.find(message => message.role === "assistant") as OcxAssistantMessage; + const thinkingParts = parsedAssistant.content.filter(part => part.type === "thinking"); + + expect(thinkingParts).toHaveLength(1); + expect(thinkingParts[0]).toMatchObject({ thinking: "first\nsecond", itemId: "r2" }); + expect(assistants).toHaveLength(1); + expect(assistants[0]).toMatchObject({ content: "answer", reasoning_content: "first\nsecond" }); + }); + + test("parseRequest drops trailing reasoning without creating an assistant", () => { + const req = parseRequest({ + model: "xai/grok-4.5", + input: [ + { type: "message", role: "user", content: [{ type: "input_text", text: "q1" }] }, + { type: "reasoning", id: "r-trailing", summary: [{ type: "summary_text", text: "unfinished" }] }, + ], + }); + + expect(req.context.messages.filter(message => message.role === "assistant")).toHaveLength(0); + expect(req.context.messages).toHaveLength(1); }); });