feat(bedrock): support responses API auth refresh#10123
feat(bedrock): support responses API auth refresh#10123jameshiester-oai wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #10123 +/- ##
==========================================
+ Coverage 80.28% 80.30% +0.01%
==========================================
Files 926 927 +1
Lines 74670 74760 +90
Branches 24102 24128 +26
==========================================
+ Hits 59951 60035 +84
- Misses 14719 14725 +6
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
![promptfoo-scanner[bot]](https://avatars.githubusercontent.com/in/2172945?s=60&v=4){kind=small}
There was a problem hiding this comment.
👍 All Clear
Reviewed the additions for Amazon Bedrock Responses API support, including token generation from AWS credentials, provider routing, UI env configuration, and example configs. The changes adjust authentication and endpoint selection for Bedrock models without altering execution sinks or adding agent tools. No medium-or-higher LLM security issues were identified in these diffs.
Minimum severity threshold: 🟡 Medium | To re-scan after changes, comment @promptfoo-scanner
Learn more
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| try { | ||
| return await generation; | ||
| } finally { | ||
| if (this.generationLock === generation) { |
There was a problem hiding this comment.
Pull request overview
Adds a dedicated bedrock:responses:<model> provider route for Amazon Bedrock’s OpenAI-compatible Responses API (including GPT OSS mantle ids), and introduces request-scoped bearer-token resolution that can refresh safely during long, high-concurrency runs by generating short-lived tokens from standard AWS credentials when an explicit Bedrock bearer token is not provided.
Changes:
- Introduces a Bedrock token provider that prefers configured bearer tokens but can generate short-lived tokens from AWS credentials/profile/default chain with in-flight deduplication.
- Adds explicit
bedrock:responses:routing and GPT OSS mantle/v1/responsesbase URL handling while preserving legacy InvokeModel routing for versioned ids. - Exposes related env vars in the UI and updates docs/examples/tests for the new auth + routing behavior.
Reviewed changes
Copilot reviewed 20 out of 21 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| test/providers/registry.test.ts | Adds coverage for explicit bedrock:responses: provider resolution. |
| test/providers/families/aws.test.ts | Updates routing/auth expectations to allow AWS credential chain for Bedrock Responses. |
| test/providers/bedrock/tokenProvider.test.ts | Adds unit tests for token selection/generation, env/profile handling, and concurrency deduping. |
| test/providers/bedrock/openaiResponses.test.ts | Adds GPT OSS Responses-specific classification/base URL/routing tests and updates auth expectations. |
| test/contracts/index.test.ts | Verifies newly supported AWS credential env overrides are accepted by contracts schema. |
| src/providers/openai/responses.ts | Adds request-scoped async API key resolution to support short-lived credentials. |
| src/providers/families/aws.ts | Adds explicit bedrock:responses: route to Bedrock provider factory. |
| src/providers/bedrock/tokenProvider.ts | Implements short-lived Bedrock bearer token resolution with concurrency deduping. |
| src/providers/bedrock/openaiResponses.ts | Adds GPT OSS Responses provider class, /v1 mantle base URL support, and token-provider-backed auth. |
| src/providers/bedrock/mantle.ts | Adds GPT OSS Responses id classifier and keeps key/region resolution utilities aligned. |
| src/envars.ts | Expands recognized AWS env vars (credentials/profile/region) for Bedrock auth flows. |
| src/contracts/env.ts | Allows additional AWS credential env vars in provider env overrides schema. |
| src/app/src/pages/redteam/setup/components/Targets/FoundationModelConfiguration.tsx | Adds “Responses API” mode selection and adjusts Bedrock settings UI for Responses. |
| src/app/src/pages/redteam/setup/components/Targets/FoundationModelConfiguration.test.tsx | Adds UI tests for Bedrock Responses mode behaviors and settings. |
| src/app/src/pages/eval-creator/components/ConfigureEnvButton.tsx | Adds Bedrock-related credential fields in the env configuration dialog. |
| src/app/src/pages/eval-creator/components/ConfigureEnvButton.test.tsx | Adds tests ensuring Bedrock/AWS credential env values persist via UI flow. |
| site/docs/providers/aws-bedrock.md | Documents Responses API auth via bearer token or AWS credential chain and adds GPT OSS bedrock:responses: example. |
| package.json | Adds @aws/bedrock-token-generator as an optional dependency (also present in devDependencies). |
| package-lock.json | Lockfile updates for the new optional dependency and transitive AWS SDK graph changes. |
| examples/amazon-bedrock/models/README.md | Adds a Responses API example reference and run instructions. |
| examples/amazon-bedrock/models/promptfooconfig.openai-responses.yaml | Adds a new example config using bedrock:responses:openai.gpt-oss-120b. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| expect(mockUpdateCustomTarget).toHaveBeenCalledWith( | ||
| 'id', | ||
| 'bedrock:responses:openai.gpt-oss-120b-1:0', | ||
| ); | ||
| expect(screen.queryByText('MCP Servers')).not.toBeInTheDocument(); |
| const config: Record<string, any> = providerOptions.config ?? {}; | ||
| const isGrok = isBedrockGrokModel(modelName); | ||
| const isGptOss = isBedrockGptOssResponsesModel(modelName); | ||
| const region = resolveBedrockMantleRegion( |
| For the OpenAI-compatible Responses API variant, use | ||
| `promptfooconfig.openai-responses.yaml`. It targets the shorter mantle model id | ||
| `openai.gpt-oss-120b` through `bedrock:responses:` and requires | ||
| `AWS_BEARER_TOKEN_BEDROCK`. |
| # Export an Amazon Bedrock API key before running: | ||
| # export AWS_BEARER_TOKEN_BEDROCK="..." | ||
|
|
| const buildBedrockProviderId = (apiMode: BedrockApiMode, modelId: string): string => { | ||
| if (apiMode === 'responses') { | ||
| return `bedrock:responses:${modelId}`; | ||
| } | ||
| return apiMode === 'converse' ? `bedrock:converse:${modelId}` : `bedrock:${modelId}`; | ||
| }; |
|
|
||
| - [`promptfooconfig.claude.yaml`](promptfooconfig.claude.yaml) - Claude 4.6 Opus, Claude 4.1 Opus, Claude 4 Opus/Sonnet, Claude Haiku 4.5 | ||
| - [`promptfooconfig.openai.yaml`](promptfooconfig.openai.yaml) - OpenAI GPT-OSS models (120B and 20B) with reasoning effort | ||
| - [`promptfooconfig.openai-responses.yaml`](promptfooconfig.openai-responses.yaml) - OpenAI GPT-OSS 120B through the Bedrock Responses API (requires `AWS_BEARER_TOKEN_BEDROCK`) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c15f09334f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return modelName.startsWith('openai.') && !isBedrockGptOssResponsesModel(modelName) | ||
| ? getBedrockMantleBaseUrl(region) | ||
| : isBedrockGrokModel(modelName) | ||
| ? getBedrockMantleBaseUrl(region) | ||
| : getBedrockMantleResponsesBaseUrl(region); |
There was a problem hiding this comment.
When a Responses provider is created with the legacy GPT OSS InvokeModel id (bedrock:responses:openai.gpt-oss-120b-1:0), which the new UI can emit when switching an existing bedrock:openai.gpt-oss-120b-1:0 target to Responses, this predicate is false and routes the call to /openai/v1/responses with frontier-model semantics. The new docs/examples require the short mantle id (openai.gpt-oss-120b) on /v1/responses, so migrated configs fail instead of using the new GPT OSS Responses path; strip the -1:0 suffix or reject it before selecting the endpoint.
Useful? React with 👍 / 👎.
| ANTHROPIC_API_KEY: z.string().optional(), | ||
| ANTHROPIC_BASE_URL: z.string().optional(), | ||
| ATLASCLOUD_API_KEY: z.string().optional(), | ||
| AWS_ACCESS_KEY_ID: z.string().optional(), |
There was a problem hiding this comment.
These new ProviderEnvOverridesSchema keys change the public config/env schema, but site/static/config-schema.json in this commit still contains none of AWS_ACCESS_KEY_ID, AWS_PROFILE, AWS_SECRET_ACCESS_KEY, or AWS_SESSION_TOKEN, so configs using the documented env: overrides continue to get stale YAML-schema validation until the generated schema is updated. Please run npm run jsonSchema:generate and commit the result.
Useful? React with 👍 / 👎.
| <EnvField | ||
| label="AWS secret access key" | ||
| envKey="AWS_SECRET_ACCESS_KEY" | ||
| value={env.AWS_SECRET_ACCESS_KEY || ''} |
There was a problem hiding this comment.
When a user enters AWS_SECRET_ACCESS_KEY in this new Bedrock env field, it is saved under config.env and later included in JSON outputs/exports. The existing sanitizer redacts AWS_BEARER_TOKEN_BEDROCK and generic secret-shaped keys, but AWS_SECRET_ACCESS_KEY normalizes to awssecretaccesskey and typical 40-character AWS secret keys do not match the 64+ character value heuristic, so exported results can disclose the credential; add this env key to the sanitizer/redaction tests before exposing it in the UI.
Useful? React with 👍 / 👎.
| const accessKeyId = this.getConfiguredValue('accessKeyId', 'AWS_ACCESS_KEY_ID'); | ||
| const secretAccessKey = this.getConfiguredValue('secretAccessKey', 'AWS_SECRET_ACCESS_KEY'); | ||
| const sessionToken = this.getConfiguredValue('sessionToken', 'AWS_SESSION_TOKEN'); |
There was a problem hiding this comment.
When a Responses provider is configured with config.profile/AWS_PROFILE but the shell also has AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for another account, these lookups select the ambient static credentials and return before the profile is considered below. That makes an explicit profile ineffective and can generate tokens for the wrong account; resolve the configured profile before falling back to process/env static credentials, or distinguish explicit config credentials from ambient ones.
Useful? React with 👍 / 👎.
| protected async getApiKeyForRequest(): Promise<string | undefined> { | ||
| return this.bedrockTokenProvider.getToken(); |
There was a problem hiding this comment.
When a Bedrock Responses target points apiBaseUrl at a local/proxy Responses fixture and sets apiKeyRequired: false, this override still calls getToken() before the base class checks whether a key is required. That makes no-auth custom endpoints fail on AWS credential/token-generator setup instead of sending the request without an Authorization header; return undefined when requiresApiKey() is false or defer token generation until after that check.
Useful? React with 👍 / 👎.
| * reasoning effort plus explicit temperature/top_p, so preserve those controls while still | ||
| * sending the real Bedrock model id. | ||
| */ | ||
| export class BedrockGptOssResponsesProvider extends BedrockOpenAiResponsesProvider { |
There was a problem hiding this comment.
For bedrock:responses:openai.gpt-oss-*, this new provider inherits OpenAI Responses billing, but the OpenAI rate table has no gpt-oss-* entries after getCapabilityModelName() strips openai., so successful GPT OSS Responses calls with usage return no cost. The existing Bedrock pricing table already has regional openai.gpt-oss-* rates, so this new route loses cost reporting compared with the InvokeModel GPT OSS path; override billing for this subclass to use the Bedrock pricing/region.
Useful? React with 👍 / 👎.
| default credential chain when unset. | ||
| </p> | ||
| </div> | ||
| {bedrockApiMode !== 'responses' && ( |
There was a problem hiding this comment.
When configuring a Bedrock Responses target in the redteam setup UI, this condition hides the only config.profile control even though the new token provider accepts profile for generated Bedrock bearer tokens. Users who need a per-target SSO/shared-config profile cannot add, edit, or remove it here and are forced to use global AWS_PROFILE for every target; keep the profile field visible for Responses or add an equivalent Responses credential setting.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Release audit on c15f093: this is not a candidate for the current release. The branch conflicts with main, CI has multiple failures, and the current threads include a credential-export blocker (AWS_SECRET_ACCESS_KEY is exposed through the new env UI without sanitizer coverage), missing await/token-generation issues, and incorrect GPT-OSS Responses routing. Please resolve the security and routing/auth findings, regenerate the public schema, sync with current main, and rerun the affected Bedrock/UI suites before re-review.
Summary
bedrock:responses:<model>route for Bedrock Mantle Responses models, including GPT OSS, while preserving existing InvokeModel routingAWS_BEARER_TOKEN_BEDROCKor short-lived bearer tokens generated from standard AWS credentials, profiles, or the default credential chainWhy
On-prem customers often use Amazon Bedrock as their inference provider and need the OpenAI-compatible Responses API surface. Static Bedrock bearer tokens are short-lived; requiring operators to refresh them manually can break scans mid-run. Using the standard AWS credential chain lets promptfoo refresh safely while still supporting explicitly supplied bearer tokens and team-secret environment overrides.
Validation
npm run tscnpm exec -- vitest run test/providers/bedrock/tokenProvider.test.ts test/providers/bedrock/openaiResponses.test.ts test/providers/families/aws.test.ts test/providers/registry.test.ts test/contracts/index.test.tsnpm exec -- vitest run test/providers/openai/responses/request.test.ts test/providers/openai/responses/errors.test.ts test/providers/openai/responses/tracing.test.tsnpm run test:app -- src/pages/redteam/setup/components/Targets/FoundationModelConfiguration.test.tsx src/pages/eval-creator/components/ConfigureEnvButton.test.tsx --runnpm run lint:src,npm run lint:tests, andnpm run lint:site(existing complexity warnings only)