feat: Custom inference endpoints for third-party API models by dagmfactory · Pull Request #10781 · warpdotdev/warp

dagmfactory · 2026-05-12T23:53:03Z

Description

Adds support for custom inference endpoints in AI settings, allowing users to add, edit, and remove third-party OpenAI-compatible API endpoints. Custom endpoint models surface in all model selectors when the feature flag is enabled.

Changes

CustomInferenceEndpoints data model (crates/ai/src/api_keys.rs, app/src/ai/llms.rs) — stores endpoint name, URL, API key, and model list with aliases.
Feature flag (crates/warp_features/src/lib.rs) — gates the feature behind CustomInferenceEndpoints.
Agent request wiring (app/src/ai/agent/api.rs, app/src/ai/agent/api/impl_tests.rs) — passes custom_model_providers to the agent API.
AI settings UI (app/src/settings_view/ai_page.rs) — new "Custom inference" section with endpoint cards, add/edit/remove flow.
Custom endpoint modal (app/src/settings_view/custom_inference_modal.rs) — form for entering endpoint details and model names/aliases.
Remove confirmation dialog (app/src/settings_view/remove_custom_endpoint_confirmation_dialog.rs) — confirms destructive removal with model chips preview.
Model selector integration — custom endpoint models appear in agent mode, toolbar, and ambient agent model selectors.
Infrastructure — modal dismiss-on-escape keystroke, menu right-side icon support, settings info-tooltip max-width.

Screenshots / Videos

https://www.loom.com/share/0a8b6ee1ce0d43e09b698e621be8416d

Linked Issue

https://linear.app/warpdotdev/issue/QUALITY-605/bring-your-own-endpoint

Remaining work:

Disable bring your own feature for enterprise accounts
Order custom models right after auto models in the model picker
Warp TOS hyperlink in custom inference info icon

Testing

I have manually tested my changes locally with ./script/run
Added unit tests for CustomInferenceEndpoints serialization/deserialization in crates/ai/src/api_keys_tests.rs and app/src/ai/llms_tests.rs.
Added impl_tests.rs coverage for agent API wiring.

Agent Mode

Warp Agent Mode - This PR was created via Warp's AI Agent Mode

CHANGELOG-NEW-FEATURE: Add custom inference endpoints to AI settings, allowing users to bring their own OpenAI-compatible third-party API endpoints and use them across all model selectors.

…s info-tooltip max-width

…t request wiring

…oint modal

oz-for-oss · 2026-05-13T00:03:16Z

@dagmfactory

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss

Overview

This PR adds local storage, settings UI, model-selector integration, and request wiring for custom OpenAI-compatible inference endpoints.

Concerns

Custom endpoint secrets can be sent even when the workspace BYOK entitlement is disabled.
The settings UI now enables BYOK fields/custom endpoint setup based only on AI availability and the feature flag, not BYOK eligibility.
Custom endpoint URLs are accepted with only a non-empty check before being persisted and sent to the agent backend.
Custom model IDs still resolve when the feature flag is disabled, while request wiring omits the provider registry.
Secure-storage writes for API keys were changed to fire-and-forget spawned callbacks, which can persist stale endpoint/API-key state out of order.

Security

The request path bypasses the BYOK entitlement by calling custom_model_providers_for_request(true).
Arbitrary custom endpoint URLs need HTTPS/public-host validation before they can become cloud inference targets.

Verdict

Found: 0 critical, 5 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

oz-for-oss · 2026-05-13T00:09:32Z

        );
+        let custom_model_providers = FeatureFlag::CustomInferenceEndpoints
+            .is_enabled()
+            .then(|| api_key_manager.custom_model_providers_for_request(true))


⚠️ [IMPORTANT] [SECURITY] Passing true bypasses the workspace BYOK entitlement computed above, so custom endpoint API keys are sent even when BYOK is disabled. Use is_byo_enabled for this request path.

do we mean to hardcode true as the argument here? or should this be disabled when the user is an enterprise user?

oz-for-oss · 2026-05-13T00:09:32Z

+        let ai_settings = AISettings::as_ref(app);
+        let is_any_ai_enabled = ai_settings.is_any_ai_enabled(app);
+        let is_byo_enabled = UserWorkspaces::as_ref(app).is_byo_api_key_enabled(app);
+        let is_enabled = is_any_ai_enabled;


⚠️ [IMPORTANT] This drops the previous is_byo_enabled check, so users without BYOK can edit provider keys and configure custom endpoints despite the upgrade CTA. Keep the input/add-button enablement tied to both is_any_ai_enabled and BYOK eligibility.

This seems valid? We want to make sure we don't regress any disabled state.

oz-for-oss · 2026-05-13T00:09:32Z

+                .is_empty()
+        });
+        !name.trim().is_empty()
+            && !url.trim().is_empty()


⚠️ [IMPORTANT] [SECURITY] The URL validation only checks for non-empty text, so arbitrary schemes or private hosts can be persisted and sent as cloud inference targets. Require HTTPS and reject local/private hosts before saving or sending an endpoint.

oz-for-oss · 2026-05-13T00:09:32Z

+    /// Resolves an `LLMId` against the user's custom-endpoint LLMs.
+    /// Returns `None` if the id isn't a known custom model `config_key`.
+    pub fn custom_llm_info_for_id(&self, id: &LLMId) -> Option<&LLMInfo> {
+        self.custom_llms.iter().find(|info| info.id == *id)


⚠️ [IMPORTANT] This lookup resolves custom models even when CustomInferenceEndpoints is disabled, while request wiring omits custom_model_providers in that state. Gate this helper too or reconcile stored custom selections back to a built-in model when the flag is off.

oz-for-oss · 2026-05-13T00:09:32Z

+        // already emitted, so the UI updates immediately while the
+        // potentially slow platform secure-storage call runs in a
+        // subsequent main-thread callback.
+        ctx.spawn(async move { json }, |_, json, ctx| {


⚠️ [IMPORTANT] These spawned secure-storage writes can complete out of order when a user saves/removes endpoints quickly, allowing an older payload to overwrite a newer delete/edit. Serialize writes or version-check the callback before writing.

Did this change get made back when we were investigating lag? If there wasn't an issue with the previous writing mechanism I'm hesitant to make changes.

danielpeng2

Doing some testing, going to leave comments about behaviour as they come up:

The "Add endpoint" button isn't enabled after I fill things in, might be another ctx.notify issue: https://www.loom.com/share/5f945cf47641482ca4d0033b11c9d615

danielpeng2 · 2026-05-13T01:43:14Z

        );
+        let custom_model_providers = FeatureFlag::CustomInferenceEndpoints
+            .is_enabled()
+            .then(|| api_key_manager.custom_model_providers_for_request(true))


do we mean to hardcode true as the argument here? or should this be disabled when the user is an enterprise user?

danielpeng2 · 2026-05-13T01:46:37Z

+    /// Returns `None` when BYOK is disabled or no endpoint has both a non-empty URL and API key.
+    pub fn custom_model_providers_for_request(
+        &self,
+        include_byo_keys: bool,


is include_byo_keys the right name for this condition, naming it something like include_custom_model_providers seems more clear? alternatively, we could lift this check outside of the method and check it before calling this method, which might be a simpler API.

danielpeng2 · 2026-05-13T01:57:27Z

            |me, _event: &ApiKeyManagerEvent, ctx| {
+                me.rebuild_custom_llms(ctx);
                me.reconcile_disabled_model_preferences(ctx);
+                ctx.emit(LLMPreferencesEvent::UpdatedAvailableLLMs);


Can we make it so that we emit the LLMPreferencesEvent::UpdatedAvailableLLMs event directly at the site where the custom endpoints are modified? I'm a little worried about this indirection.

danielpeng2 · 2026-05-13T01:58:52Z

+        self.get_computer_use_available()
+            .choices
+            .iter()
+            .chain(self.custom_llm_choices())


can we not allow custom models in computer use for now? this requires some special model setup that we aren't doing

danielpeng2 · 2026-05-13T02:18:58Z

+        },
+        description: Some(format!("Custom · {}", endpoint.name)),
+        disable_reason: None,
+        vision_supported: false,


I believe this field controls whether the user can input images when the model is selected. I think it'd be good to leave this on by default and let it error if the model doesn't actually support vision.

danielpeng2 · 2026-05-13T02:35:46Z

+        let ai_settings = AISettings::as_ref(app);
+        let is_any_ai_enabled = ai_settings.is_any_ai_enabled(app);
+        let is_byo_enabled = UserWorkspaces::as_ref(app).is_byo_api_key_enabled(app);
+        let is_enabled = is_any_ai_enabled;


This seems valid? We want to make sure we don't regress any disabled state.

danielpeng2 · 2026-05-13T05:39:59Z

        }
    }

-    /// Creates a new menu item with vertically stacked primary and secondary text.


nit: this comment got truncated

danielpeng2 · 2026-05-13T05:52:09Z

+        // Lazy migration: stamp a UUID `config_key` on any pre-existing custom model that
+        // was persisted before this field landed. If any were filled, schedule a re-save so
+        // we don't repeat the work on the next launch.
+        let migrated = Self::backfill_missing_config_keys(&mut keys);
+        if migrated {
+            let json = match serde_json::to_string(&keys) {
+                Ok(json) => json,
+                Err(e) => {
+                    log::error!("Failed to re-serialize API keys after migration: {e:#}");
+                    return keys;
+                }
+            };
+            ctx.spawn(async move { json }, |_, json, ctx| {
+                if let Err(e) = ctx.secure_storage().write_value(SECURE_STORAGE_KEY, &json) {
+                    log::error!("Failed to write migrated API keys to secure storage: {e:#}");
+                }
+            });
+        }


Do we need this migration logic? Users can't add custom models right now, so I don't see how they would create one without a UUID.

danielpeng2 · 2026-05-13T05:55:02Z

+        // already emitted, so the UI updates immediately while the
+        // potentially slow platform secure-storage call runs in a
+        // subsequent main-thread callback.
+        ctx.spawn(async move { json }, |_, json, ctx| {


Did this change get made back when we were investigating lag? If there wasn't an issue with the previous writing mechanism I'm hesitant to make changes.

danielpeng2 · 2026-05-13T05:56:40Z

nit: the info icon isn't vertically centered with the text

danielpeng2 · 2026-05-13T05:58:03Z

nit: the default width of the modal feels wider than what we have in the mocks

danielpeng2 · 2026-05-13T05:58:51Z

when there's only 1 model, we create a gap for the "x" icon even though there's nothing there. I think it should fill to the width if there's only 1 model

danielpeng2 · 2026-05-13T06:31:38Z

the custom models don't have the borders around the text, and the edit button doesn't appear in the top right like the mocks

danielpeng2 · 2026-05-13T06:34:34Z

the remove button in the edit endpoints modal doesn't have the same styling as in the mocks

danielpeng2

Nice job on this! Approving to unblock.

My most important comments are the logic/behavoural ones. We don't have to fix all the UI nits in this PR. Important thing is that we get this in before the bug bash tomorrow.

dagmfactory added 4 commits May 12, 2026 19:43

chore: add modal dismiss-keystroke, menu right-side icon, and setting…

8cb7c42

…s info-tooltip max-width

feat: add CustomInferenceEndpoints data model, feature flag, and agen…

98140af

…t request wiring

feat: surface custom endpoint models in all model selectors

57bdfbd

feat: add Custom inference settings section with add/edit/remove endp…

18cbd04

…oint modal

cla-bot Bot added the cla-signed label May 12, 2026

dagmfactory marked this pull request as ready for review May 13, 2026 00:03

dagmfactory requested a review from danielpeng2 May 13, 2026 00:03

oz-for-oss Bot reviewed May 13, 2026

View reviewed changes

danielpeng2 reviewed May 13, 2026

View reviewed changes

danielpeng2 approved these changes May 13, 2026

View reviewed changes

dagmfactory added 3 commits May 13, 2026 11:58

Implement URL validation in custom inference modal

59d45b2

Refactor LLM preferences to use AppContext and feature flags

4b0abbb

Merge branch 'master' into dagm/custom-api-endpoint

8d2aed3

Conversation

dagmfactory commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Screenshots / Videos

Linked Issue

Remaining work:

Testing

Agent Mode

Uh oh!

oz-for-oss Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oz-for-oss Bot left a comment

Choose a reason for hiding this comment

Overview

Concerns

Security

Verdict

Uh oh!

oz-for-oss Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oz-for-oss Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oz-for-oss Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

oz-for-oss Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

oz-for-oss Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danielpeng2 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danielpeng2 commented May 13, 2026

Uh oh!

danielpeng2 commented May 13, 2026

Uh oh!

danielpeng2 commented May 13, 2026

Uh oh!

danielpeng2 commented May 13, 2026

Uh oh!

danielpeng2 commented May 13, 2026

Uh oh!

danielpeng2 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

dagmfactory commented May 12, 2026 •

edited

Loading

oz-for-oss Bot commented May 13, 2026 •

edited

Loading