Repository navigation
fix: recover retired and unavailable models with compatible choices - #10763
Conversation
Prefer compatible policy-permitted retirement replacements and bounded same-backend catalog alternatives. Preserve explicit fallback ordering, request features, guards, enclave attempt budgets, pricing snapshots, and routing evidence. Keep opaque WebSocket sessions fail-closed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 58f067de-fd39-4522-aed4-c6bbc3f07e61
Documentation PreviewDocumentation has been built for this PR. To view locally:
Built from commit 9db8b1d |
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (2 files)
Coverage comparison generated by |
There was a problem hiding this comment.
🟡 Changes recommended
Automatic model recovery can reject valid requests, reduce output capacity, and bypass alias fallback opt-outs because of compatibility checks and inconsistent catalogue snapshots.
8 open issues
What changed in this PR
Extends AWF’s API proxy to recover retired and unavailable models using compatible alternatives from the same backend.
Changes:
- Adds catalogue-based HTTP recovery before dispatch and after model-specific upstream rejections.
- Adds compatibility filtering, bounded substitutions, recovery telemetry, and opt-out handling while keeping WebSocket sessions opaque.
- Packages the recovery module and adds documentation and regression coverage.
| File | Description |
|---|---|
docs/api-proxy-sidecar.md |
Documents recovery behavior, safeguards, and WebSocket limits. |
containers/api-proxy/websocket-proxy.js |
Rejects unavailable handshake models without rewriting sessions. |
containers/api-proxy/upstream-retry.js |
Prioritizes model fallback after eligible upstream rejections. |
containers/api-proxy/upstream-http.js |
Combines configured fallbacks with bounded automatic candidates. |
containers/api-proxy/server.websocket.test.js |
Covers unavailable-model upgrade rejection. |
containers/api-proxy/server.model-recovery.test.js |
Adds recovery, compatibility, policy, budget, and retry regressions. |
containers/api-proxy/routing-observation.test.js |
Tests separate original-selection and recovery evidence. |
containers/api-proxy/routing-observation.js |
Records recovery details in routing telemetry. |
containers/api-proxy/proxy-request.js |
Identifies authoritative native Copilot catalogues. |
containers/api-proxy/providers/copilot-auth.js |
Clarifies alias fallback versus request recovery in comments. |
containers/api-proxy/model-recovery.js |
Implements availability checks, candidate filtering, and recovery records. |
containers/api-proxy/model-config.js |
Marks strict-alias requests to disable automatic recovery. |
containers/api-proxy/guards/retired-model-guard.js |
Updates comments to explain recovery-before-rejection behavior. |
containers/api-proxy/Dockerfile |
Includes the recovery module in the proxy image. |
containers/api-proxy/body-handler.js |
Invokes pre-dispatch recovery in the body-transform pipeline. |
Review effort: Max
| const recovered = recoverRequestModel(body, provider, req, requestId, logRequest); | ||
| if (recovered) body = recovered; |
| if (isWritableMethod) { | ||
| const recovered = recoverRequestModel(body, provider, req, requestId, logRequest); | ||
| if (recovered) body = recovered; | ||
| } |
| const requested = parseBodyAsObject(body)?.model; | ||
| const alias = typeof requested === 'string' && Object.entries(MODEL_ALIASES?.models || {}) | ||
| .find(([key]) => key.toLowerCase() === requested.split('?')[0].toLowerCase())?.[1]; | ||
| if (alias?.fallback === false && req) req.awfModelRecoveryDisabled = true; |
| const records = recoveryRecords(provider, req); | ||
| if (provider === 'copilot' && req?.awfModelCatalogAuthoritative && records.length > 0 && | ||
| nativeModel(model, provider).toLowerCase() !== 'auto' && | ||
| !records.some(record => record.id.toLowerCase() === nativeModel(model, provider).toLowerCase())) { | ||
| return { reason: 'model_unavailable' }; | ||
| } |
| const outputLimit = record.capabilities?.limits?.max_output_tokens; | ||
| const requestedOutput = parsed.max_tokens ?? parsed.max_completion_tokens ?? parsed.max_output_tokens; | ||
| if (requestedOutput !== undefined && | ||
| (!Number.isFinite(outputLimit) || requestedOutput > outputLimit)) return false; |
| for (const child of Object.values(value)) { | ||
| if (child && typeof child === 'object') pendingContent.push(child); | ||
| } |
| const automaticRecovery = isAutomaticRecoveryEnabled(provider, req) && | ||
| recoveryRecords(provider, req).length > 0; |
| ? getAutomaticRecoveryCandidates({ | ||
| model: attempted[0].model, provider, | ||
| body: origin.sourceBody, url: req.url, req, isPermitted, | ||
| }).map(model => `${provider}/${model}`) |
|
🔌 Smoke Services — All services reachable! ✅
|
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
🔑 Smoke Copilot BYOK AOAI (api-key) is testing Azure OpenAI BYOK (api-key) mode on this pull request... |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟
|
|
💎 Smoke Gemini is crystallizing results on this pull request... |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓
|
|
Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.
|
|
✅ Smoke Claude passed
|
|
📡 Smoke OTel Tracing reports failed. OTel tracing regression detected.
|
|
✅ Security Guard completed successfully! Security GuardOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅
|
|
✅ Build Test Suite completed successfully!
|
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.
|
|
Services smoke test: FAIL
|
Smoke Test: Copilot BYOK (Direct) Mode✅ GitHub MCP connectivity (merged PRs retrieved) Status: PASS — Direct BYOK mode (COPILOT_PROVIDER_API_KEY) working correctly.
|
github-actions
Bot
added
the
smoke-copilot-network-isolation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.
|
|
🪪 Smoke Copilot BYOK AOAI (Entra) is testing Azure OpenAI BYOK (Entra / GitHub OIDC) mode on this pull request... |
|
🔌 Smoke Services is testing service connectivity for this pull request... |
|
📡 Smoke OTel Tracing is starting OTel tracing validation... |
|
🔑 Smoke Copilot BYOK is testing direct BYOK mode on this pull request... |
|
✅ Security Guard completed successfully! Security GuardOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.
|
|
✅ Build Test Suite completed successfully!
|
|
💎 Smoke Gemini is crystallizing results on this pull request... |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟
|
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤
|
|
✅ Smoke Claude passed
|
|
Smoke test:
Overall: FAIL
|
|
Cloud Hypervisor + Copilot smoke test
Overall: PASS
|
|
EGRESS_RESULT allow=pass deny=pass
|
|
fix: warn instead of rejecting models without pricing ✅
|
Smoke Test: Claude Engine Validation
Overall result: PASS
|
|
Smoke Copilot (@lpcox)
Overall: PASS
|
🏗️ Build Test Suite Results
Overall: 7/8 ecosystems passed — FAIL Java failures:
Notes:
|
Chroot Version Comparison
Result: Not all tests passed (Node.js mismatch), so the
|
Summary
400/404rejections on HTTP Responses, Chat Completions, and Messages paths.WebSocket boundary
WebSocket tunnels are opaque, so this intentionally does not rewrite only the handshake or replay a live session: later frames may contain incompatible features or another model. Known-retired/catalog-unavailable handshake models fail explicitly; clients can use HTTP inference for compatible automatic recovery or choose an available model before upgrading.
Validation
npm test --prefix containers/api-proxy -- --runInBand).npm run build, Markdown lint ondocs/api-proxy-sidecar.md, andgit diff --checkpassed.No pricing catalog/source changes are included. Separate pricing PR #10757 is not duplicated or bundled. No live credentialed-provider or container integration run was performed.