[BUG] fix: CrewAI 1.12.x LLM routing - litellm does not receive api_base/api_key for multi-provider setups

[clement-brijyt]

Description

Problem

When using CrewAI 1.12.x with provider="litellm" and multiple cloud providers (Scaleway + Nebius), LLM calls fail with litellm.AuthenticationError because:

1. CrewAI's LLM class does not map base_url to api_base: The LLM(base_url=...) constructor stores the URL in base_url but litellm reads from api_base, which remains None. This causes litellm to fall back to the default OpenAI endpoint (api.openai.com), sending the Scaleway/Nebius API key to OpenAI.

2. CrewAI agent internals bypass LLM object params: When a CrewAI Agent executes (tool selection, reasoning loops), its internal litellm calls do not pass api_key or api_base from the LLM object. Instead, litellm falls back to OPENAI_API_KEY / OPENAI_API_BASE environment variables. With a multi-provider setup (Scaleway for mistral-small, Nebius for qwen3), global env vars cannot serve both providers simultaneously.

Root Causes

Issue	Impact
LLM(base_url=X) sets self.base_url=X but self.api_base=None	Direct llm.call() sends requests to api.openai.com instead of custom endpoint
CrewAI agent internal litellm calls don't inherit LLM object params	Agent execution fails when OPENAI_API_KEY/OPENAI_API_BASE env vars are not set or point to the wrong provider
Global OPENAI_API_BASE env var conflicts with multi-provider setup	Setting it to Scaleway breaks Nebius calls and vice versa

Fix

Three changes in model_service.py:

1. Pass both base_url and api_base to LLM constructors

LLM(
    model="openai/model-name",
    api_key=api_key,
    base_url=base_url,
    api_base=base_url,  # litellm reads this field
    provider="litellm",
    ...
)

2. Add provider="litellm" to all LLM constructors

Required in CrewAI 1.12.x to ensure litellm is used as the backend (without it, CrewAI may use a default OpenAI provider that ignores base_url).

3. Thread-safe litellm routing via monkey-patch

Monkey-patch litellm.completion and litellm.acompletion to inject the correct api_key and api_base per model name. This ensures CrewAI's internal agent calls (which bypass the LLM object) are routed to the correct provider.

_MODEL_PROVIDER_ROUTING: dict[str, dict[str, str]] = {}

def _install_litellm_routing():
    _orig = litellm.completion
    def _patched(*args, **kwargs):
        model = kwargs.get("model", "")
        for pattern, cfg in _MODEL_PROVIDER_ROUTING.items():
            if pattern in model:
                kwargs.setdefault("api_key", cfg["api_key"])
                kwargs.setdefault("api_base", cfg["api_base"])
                break
        return _orig(*args, **kwargs)
    litellm.completion = _patched

Thread-safety: Each litellm call resolves credentials from its own model kwarg. The routing table is read-only after initialization. No shared mutable state is modified at call time.

Environment

• crewai[litellm]~=1.12.0
• Multiple providers: Scaleway (mistral-small, llama3, deepseek-r1) + Nebius (qwen3)

Steps to Reproduce

1. Install crewai[litellm]~=1.12.0
2. Configure two cloud providers with different api_key / base_url:
   Scaleway (for mistral-small, llama3, deepseek-r1)
   Nebius (for qwen3)
3. Create an LLM instance using base_url and api_key:
   `from crewai import LLM

llm = LLM(
model="openai/mistral-small-3.2-24b-instruct-2506",
api_key="scw-xxx",
base_url="https://api.scaleway.ai/v1",
temperature=0.2,
)4. Use this LLM inside a CrewAI Agent and run a Crew:from crewai import Agent, Crew, Task, Process
agent = Agent(
role="Assistant",
goal="Answer the user's question",
backstory="You are a helpful assistant.",
llm=llm,
)
task = Task(
description="What is the capital of France?",
expected_output="A short answer.",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task], process=Process.sequential)
result = crew.kickoff()`
5. Observe a litellm.AuthenticationError because the request was sent to api.openai.com instead of the Scaleway endpoint.

Expected behavior

• LLM(base_url="https://api.scaleway.ai/v1") should route all litellm calls (both direct llm.call() and internal agent reasoning/tool-selection calls) to https://api.scaleway.ai/v1 using the provided api_key.

• In a multi-provider setup, each LLM instance should use its own api_key / base_url independently, without relying on global OPENAI_API_KEY / OPENAI_API_BASE environment variables.

Screenshots/Code snippets

Problem 1 -- base_url not mapped to api_base:
The LLM class stores base_url but litellm reads from api_base, which remains None:
# Before fix: only api_base was set, base_url was missing LLM( model="openai/mistral-small-3.2-24b-instruct-2506", api_key=self.scaleway_api_key, api_base=self.scaleway_base_url, # litellm reads this # base_url is missing -> CrewAI may not propagate api_base correctly temperature=0.2, timeout=120, )
Fix: pass both base_url and api_base, and add provider="litellm":
LLM( model="openai/mistral-small-3.2-24b-instruct-2506", api_key=self.scaleway_api_key, base_url=self.scaleway_base_url, api_base=self.scaleway_base_url, # litellm reads this field provider="litellm", temperature=0.2, timeout=120, )

Problem 2 -- Agent internals bypass LLM object params:
CrewAI agent internal litellm calls (tool selection, reasoning loops) do not pass api_key / api_base from the LLM object. They fall back to OPENAI_API_KEY / OPENAI_API_BASE env vars, which cannot serve two providers at once.
Fix: monkey-patch litellm.completion / litellm.acompletion to inject the correct credentials based on model name:
`_MODEL_PROVIDER_ROUTING: dict[str, dict[str, str]] = {}

def _install_litellm_routing():
_orig_completion = litellm.completion
_orig_acompletion = litellm.acompletion

def _inject(kwargs):
    model = kwargs.get("model", "")
    for pattern, cfg in _MODEL_PROVIDER_ROUTING.items():
        if pattern in model:
            kwargs.setdefault("api_key", cfg["api_key"])
            kwargs.setdefault("api_base", cfg["api_base"])
            break

def _patched_completion(*args, **kwargs):
    _inject(kwargs)
    return _orig_completion(*args, **kwargs)

async def _patched_acompletion(*args, **kwargs):
    _inject(kwargs)
    return await _orig_acompletion(*args, **kwargs)

litellm.completion = _patched_completion
litellm.acompletion = _patched_acompletion`

Thread-safety: each call resolves credentials from its own model kwarg. The routing table (_MODEL_PROVIDER_ROUTING) is read-only after initialization.

Operating System

Ubuntu 20.04

Python Version

3.10

crewAI Version

1.12.2

crewAI Tools Version

1.12.2

Virtual Environment

Venv

Evidence

• litellm.AuthenticationError on first LLM call -> LLM(base_url=X) sets self.base_url=X but self.api_base=None -- litellm falls back to api.openai.com
  (Requests logged against OpenAI endpoint instead of Scaleway/Nebius; the Scaleway API key is rejected by OpenAI)

• Agent tool-calling / reasoning steps fail even when llm.call() works -> CrewAI agent internals call litellm.completion() without passing api_key / api_base from the LLM object

(Adding debug logging to litellm.completion shows api_key=None, api_base=None on agent-internal calls)

• Setting OPENAI_API_BASE fixes one provider but breaks the other -> Global env var cannot serve Scaleway and Nebius simultaneously

(Setting OPENAI_API_BASE to Scaleway makes qwen3 (Nebius) fail, and vice versa)

Possible Solution

None

Additional context

None