✨ feat: 分组定价、自动 fallback 与运行时分组选择 ✨ feat: group-based pricing, auto fallback, and runtime group selection#6116
Open
Wu-jiyan wants to merge 2 commits into
Open
Conversation
Contributor
WalkthroughModel-level system prompts are now stored, configured in the web UI, and applied to OpenAI, Gemini, and Claude requests. Request groups support ordered multi-group channel fallback, while model-specific group ratios flow into pricing. ChangesModel-specific group ratios
Group-aware channel selection
Model-level system prompts
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant ModelUI
participant ModelAPI
participant Distributor
participant RelayHandler
participant Upstream
ModelUI->>ModelAPI: submit system prompt and mode
ModelAPI->>Distributor: provide active model configuration
Distributor->>RelayHandler: set resolved prompt context
RelayHandler->>Upstream: send request with model-level prompt
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…m fallback - Add model_group_ratio for per-model per-group pricing overlay - Support group parameter in standard API request body for runtime group selection - Support comma-separated group list as custom fallback order - Billing uses the actual group that successfully served the request - Fully backward compatible when no new config is set
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@service/channel_select.go`:
- Around line 106-135: Update the group-selection loop around
GetRandomSatisfiedChannel to capture each returned error while preserving the
existing channel fallback behavior. Track the last non-nil error across failed
groups, continue trying groups as before, and when all groups are exhausted
return nil with that error; retain a nil error when failures only indicate no
available channel.
In `@setting/ratio_setting/group_ratio.go`:
- Around line 154-156: Add a "ModelGroupRatio" case to the options update switch
in model/option.go, routing its value to UpdateModelGroupRatioByJSONString.
Preserve the existing option handling and ensure the exposed model_group_ratio
setting is updated through the standard options path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 6af0008a-eb0c-4eed-9a50-969cfcbe95db
📒 Files selected for processing (5)
middleware/distributor.gorelay/helper/price.goservice/channel_select.goservice/group.gosetting/ratio_setting/group_ratio.go
Wu-jiyan
force-pushed
the
feat/group-pricing-and-fallback
branch
from
July 11, 2026 11:00
4527f70 to
3eddaf4
Compare
- Fix error being discarded when all groups fail in multi-group fallback - Add ModelGroupRatio option route for admin API read/write
Wu-jiyan
force-pushed
the
feat/group-pricing-and-fallback
branch
from
July 11, 2026 11:56
29fe543 to
386c029
Compare
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
middleware/distributor.go (1)
89-95: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winValidate each fallback group individually.
GroupInUserUsableGroupsperforms an exact lookup, so a valid value such as"group-a,group-b"is checked as one group and rejected with 403. Split the ordered list, validate every group against the original user group, then store the normalized list for downstream fallback.Proposed fix
if modelRequest.Group != "" { - if !service.GroupInUserUsableGroups(usingGroup, modelRequest.Group) && modelRequest.Group != usingGroup { - abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorGroupAccessDenied)) - return + requestedGroups := strings.Split(modelRequest.Group, ",") + for index, requestedGroup := range requestedGroups { + requestedGroup = strings.TrimSpace(requestedGroup) + if requestedGroup == "" || + (!service.GroupInUserUsableGroups(usingGroup, requestedGroup) && requestedGroup != usingGroup) { + abortWithOpenAiMessage(c, http.StatusForbidden, i18n.T(c, i18n.MsgDistributorGroupAccessDenied)) + return + } + requestedGroups[index] = requestedGroup } - usingGroup = modelRequest.Group + usingGroup = strings.Join(requestedGroups, ",") common.SetContextKey(c, constant.ContextKeyUsingGroup, usingGroup) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@middleware/distributor.go` around lines 89 - 95, Update the modelRequest.Group handling around GroupInUserUsableGroups to split the ordered fallback-group string into individual groups, validate each group against the original usingGroup, and reject the request if any is unauthorized. Normalize and store the validated list as usingGroup and in ContextKeyUsingGroup while preserving the existing single-group behavior and fallback order.
🧹 Nitpick comments (1)
web/default/src/features/models/components/drawers/model-mutate-drawer.tsx (1)
1327-1353: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider spreading
{...field}on the system prompt Textarea for consistency.The Textarea uses explicit
value/onChangeinstead of{...field}, omittingfield.refandfield.onBlur. Other Textarea fields in this file (e.g., the description field at line 738) use{...field}. Whilesystem_prompthas no validation constraints so no visible issues arise, includingref/onBlurensures consistent form registration and blur-state tracking.♻️ Suggested change
<FormControl> <Textarea + {...field} value={field.value || ''} - onChange={field.onChange} placeholder={ '{\n "en": "You are a helpful assistant.",\n "zh": "你是一个有用的助手。"\n}' } rows={6} disabled={form.watch('system_prompt_mode') === 0} className='font-mono text-sm' /> </FormControl>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/default/src/features/models/components/drawers/model-mutate-drawer.tsx` around lines 1327 - 1353, Update the system_prompt Textarea in the FormField render callback to spread the complete field object with {...field}, removing the separate value and onChange props while preserving the existing empty-value fallback behavior if needed. Keep the disabled, placeholder, styling, and other Textarea configuration unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@middleware/distributor.go`:
- Around line 89-95: Update the modelRequest.Group handling around
GroupInUserUsableGroups to split the ordered fallback-group string into
individual groups, validate each group against the original usingGroup, and
reject the request if any is unauthorized. Normalize and store the validated
list as usingGroup and in ContextKeyUsingGroup while preserving the existing
single-group behavior and fallback order.
---
Nitpick comments:
In `@web/default/src/features/models/components/drawers/model-mutate-drawer.tsx`:
- Around line 1327-1353: Update the system_prompt Textarea in the FormField
render callback to spread the complete field object with {...field}, removing
the separate value and onChange props while preserving the existing empty-value
fallback behavior if needed. Keep the disabled, placeholder, styling, and other
Textarea configuration unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 8b8173bb-5d20-47db-a3c1-f8f290aea27c
📒 Files selected for processing (11)
constant/context_key.gomiddleware/distributor.gomodel/model_meta.gorelay/claude_handler.gorelay/compatible_handler.gorelay/gemini_handler.gorelay/helper/system_prompt.goweb/default/src/features/models/components/drawers/model-mutate-drawer.tsxweb/default/src/features/models/components/models-columns.tsxweb/default/src/features/models/lib/model-form.tsweb/default/src/features/models/types.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
功能说明
实现了 AI API 调用的分组定价、自动 fallback 和运行时分组选择功能。
改动内容
向后兼容
不传 group 参数或不配置 model_group_ratio 时，行为与原来完全一致。
Summary by CodeRabbit
autoand comma-separated fallback priorities with cross-group retries.