Skip to content

bugfix: don't cache empty inference-profile sets in AWSBedrock#6373

Open
Ankit5467 wants to merge 2 commits into
FlowiseAI:mainfrom
Ankit5467:bugfix/bedrock-inference-profile-cache
Open

bugfix: don't cache empty inference-profile sets in AWSBedrock#6373
Ankit5467 wants to merge 2 commits into
FlowiseAI:mainfrom
Ankit5467:bugfix/bedrock-inference-profile-cache

Conversation

@Ankit5467
Copy link
Copy Markdown
Contributor

Summary

Fixes a process-lifetime cache poisoning bug in the AWS Bedrock chat model node.

discoverInferenceProfiles() in packages/components/nodes/chatmodels/AWSBedrock/utils.ts previously cached an empty Set<string> on both thrown errors and empty successful responses. Once poisoned, every subsequent call for that region in the same process would short-circuit and return the empty set. resolveBedrockModel() then skipped auto-applying a cross-region inference profile, Flowise rejected the bare model ID with:

"This model requires a cross-region inference profile. The system attempted to auto-apply one but none was available in this region."
Bedrock API response: "Invocation of model ID anthropic.claude-sonnet-4-6 with on-demand throughput isn't supported. Retry your request with the ID or ARN of an inference profile that contains this model."

This reproduces when the first Bedrock invocation for a region happens before AWS credentials are available in the Node process — e.g. a server startup race where AWS_PROFILE / SSO creds refresh after Flowise starts. The ListInferenceProfiles call throws, the bare catch swallows it, an empty set is cached, and the region never recovers until the server is restarted.

Fix

  • Only cache when ListInferenceProfiles returns at least one profile.
  • On exception, return an empty set without writing to the cache, so the next invocation retries.
  • Log the underlying error via console.error so operators can diagnose credential or permission issues instead of seeing a silent failure that manifests only later as a confusing Bedrock validation error.

Test plan

  • Verified locally in us-east-1 with anthropic.claude-sonnet-4-6: before the fix, the AgentFlow Agent node failed with the "requires inference profile" error on every invocation until server restart; after the fix, discovery succeeds and us.anthropic.claude-sonnet-4-6 is auto-applied.
  • Confirmed other regions continue to work as before (they were never poisoned because their first call succeeded).
  • aws bedrock list-inference-profiles --region us-east-1 --type-equals SYSTEM_DEFINED confirms the us.anthropic.claude-sonnet-4-6 profile is active on the account under test.

  discoverInferenceProfiles() previously cached an empty Set on both failure and empty
  success, for the process lifetime. If the first call for a region happened before AWS
  credentials were available (e.g. server startup race), every subsequent call for that
  region short-circuited to the cached empty Set. resolveBedrockModel() then skipped
  auto-applying a cross-region inference profile, and Bedrock rejected the bare model ID
  with "This model requires a cross-region inference profile."

  Only cache non-empty results, and log discovery failures instead of silently swallowing
  them so operators can diagnose credential or permission issues.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the AWS Bedrock utility to prevent caching results when the inference profile discovery fails and adds error logging. The reviewer suggests that successful but empty responses should still be cached to avoid redundant API calls and potential rate limiting, noting that the updated error handling already prevents the cache from being populated with invalid data during failures.

Comment thread packages/components/nodes/chatmodels/AWSBedrock/utils.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants