Skip to content

feat: support llmConfig BYOK on jobs and retrieval#123

Merged
suguanYang merged 5 commits into
mainfrom
feat/suguanYang/byok-llm-credentials
Jul 15, 2026
Merged

feat: support llmConfig BYOK on jobs and retrieval#123
suguanYang merged 5 commits into
mainfrom
feat/suguanYang/byok-llm-credentials

Conversation

@suguanYang

@suguanYang suguanYang commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add LlmConfig / LlmProviderConfig / LlmModelsConfig and optional llmConfig on job create, parse, and retrieval.
  • Flat root {apiKey, model, baseUrl} for multimodal; models for same-endpoint different model ids; text / vision for different endpoints.

Depends on: Ontos-AI/knowhere#217

Usage

import { Knowhere } from "@ontos-ai/knowhere-sdk";

const client = new Knowhere({ apiKey: "..." });

// One multimodal model
await client.jobs.create({
  sourceType: "url",
  sourceUrl: "https://example.com/doc.pdf",
  llmConfig: {
    apiKey: "sk-...",
    model: "gpt-4o",
    baseUrl: "https://api.openai.com/v1",
  },
});

// Same endpoint, different models
await client.parse({
  url: "https://example.com/doc.pdf",
  llmConfig: {
    apiKey: "sk-...",
    baseUrl: "https://api.openai.com/v1",
    models: { text: "gpt-4o-mini", vision: "gpt-4o" },
  },
});

// Two different endpoints
await client.retrieval.query({
  query: "What is the refund policy?",
  useAgentic: true,
  llmConfig: {
    text: {
      apiKey: "sk-...",
      model: "gpt-4o-mini",
      baseUrl: "https://api.openai.com/v1",
    },
    vision: {
      apiKey: "sk-ali-...",
      model: "qwen-vl-max",
      baseUrl: "https://dashscope.aliyuncs.com/compatible-mode/v1",
    },
  },
});

Wire JSON uses snake_case (llm_config, api_key, base_url) via the existing HttpClient interceptor.

Test plan

  • pnpm test

Expose optional text/vision OpenAI-compatible credentials on job create,
parse, and retrieval.query so callers can drive KNOWHERE with their own keys.

Co-authored-by: Cursor <cursoragent@cursor.com>
suguanYang and others added 4 commits July 15, 2026 21:39
Document that missing text/vision slots keep server defaults instead of
acting as a unified multimodal fallback.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document and type the shared provider slot for single multimodal BYOK.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document flat multimodal shorthand and split text/vision endpoints.

Co-authored-by: Cursor <cursoragent@cursor.com>
Support shared auth with different text/vision model ids.

Co-authored-by: Cursor <cursoragent@cursor.com>
@suguanYang
suguanYang merged commit 4735052 into main Jul 15, 2026
4 checks passed
@github-actions github-actions Bot mentioned this pull request Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant