Using litellm proxy model (OpenAI compatible endpoint)

[pradhyumna85]

Could you please help with configuring Morphik to use models on litellm proxy server (OpenAI compatible endpoint)?

[BloggerBust]

Hi @pradhyumna85,

I know this is an older thread, but I wanted to share some clarification in case it is still helpful for you or others having the same question.

Morphik forwards provider specific kwargs, including api_base, from the selected registered_models entry to LiteLLM.

Example morphik.toml:

[completion]
model = "litellm_proxy"

[registered_models]
litellm_proxy = { 
  model_name = "gpt-4o-mini",          # whatever your proxy exposes
  api_base   = "http://localhost:4000", # LiteLLM proxy base (OpenAI-compatible)
  api_key    = "..."                    # if your proxy requires it
}

With that configuration, Morphik will include api_base in the kwargs passed to litellm.acompletion(...).

If this doesn’t work in your setup, paste your [completion] and [registered_models] config sections as well as any relevant error log you are able to capture and I will help troubleshoot.