feat(channel): add affinity-aware concurrency routing#6264
Open
Ericcc-Ma wants to merge 9 commits into
Open
Conversation
Enforce a per-channel concurrency cap via an in-memory semaphore (service/channel_gate.go). SelectChannelWithLimits acquires the slot at channel-selection time and excludes any channel at its cap, retrying within the same priority tier without consuming the retry counter. Multi-key channels also get per-key gating via selectHealthyKey. The relay retry loop (controller/relay.go) and the distributor's first- attempt selection (middleware/distributor.go) both release the acquired slot on success and failure paths through releaseLimitGate, so slots are never leaked. The channel cache / ability / RetryParam selection chain gains an excludeIDs filter so a full channel is skipped on re-selection. A global toggle (channel_limit_setting.enabled) gates the feature; per-channel MaxConcurrency=0 means unlimited. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a "Max Concurrency" field to the channel edit drawer and wire it through the channel form schema, defaults, and setting-JSON build/parse. Add a global "Channel Limit Enabled" toggle to the routing-reliability system-settings section. i18n entries added for en/zh/fr/ja/ru/vi. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cover the concurrency gate (acquire/release, zero-max-unlimited, in-flight never exceeds cap), the limit orchestrator (failed-concurrency acquire skips and excludes; disabled limit allows everything; GateHandle release is safe on nil/empty), the channel-cache excludeIDs filter, the distributor limit-gate wiring, and GetChannelLimits merge behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
WalkthroughChangesChannel concurrency routing
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Request as Request
participant Distributor as Distributor
participant Selector as SelectChannelWithLimits
participant Gate as ConcurrencyGate
participant Relay as Relay
participant Upstream as UpstreamChannel
Request->>Distributor: begin channel selection
Distributor->>Selector: select with affinity exclusions
Selector->>Gate: acquire channel/key slot
Gate-->>Selector: return acquisition result
Selector-->>Distributor: return channel, key index, and handle
Distributor->>Relay: configure selected channel
Relay->>Upstream: execute request
Upstream-->>Relay: return response or error
Relay->>Gate: release slot
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 13
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
web/default/src/features/channels/components/drawers/channel-mutate-drawer.tsx (1)
992-997: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winInclude
max_concurrencyin the configured state.The
max_concurrencyfield is not included in theroutingStrategyConfiguredevaluation. If a user customizes only themax_concurrencysetting and leavespriorityandweightat defaults, the "Routing Strategy" section will incorrectly appear as unconfigured in the navigation menu.Please watch the
max_concurrencyfield and include it in the boolean check.🐛 Proposed fix
First, watch the new field around Line 731:
const currentPriority = form.watch('priority') const currentWeight = form.watch('weight') + const currentMaxConcurrency = form.watch('max_concurrency') const currentTestModel = form.watch('test_model')Then, include it in the condition here:
const routingStrategyConfigured = Boolean( currentPriority || currentWeight || + currentMaxConcurrency || currentTestModel?.trim() || (currentAutoBan ?? 1) !== 1 )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/default/src/features/channels/components/drawers/channel-mutate-drawer.tsx` around lines 992 - 997, Update the routing strategy state in the channel mutate drawer to track the max_concurrency field alongside the existing priority, weight, test model, and auto-ban values, then include that watched value in routingStrategyConfigured so customizing only max_concurrency marks the section as configured.relay/relay_task.go (1)
92-100: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy liftTransfer the concurrency reservation when switching channels.
When the origin task uses another channel, this overwrites the selected channel and key but leaves the existing gate handle attached to the distributor-selected channel. The origin channel therefore bypasses its limit while an unrelated slot remains occupied. Release the current handle and acquire limits for
chbefore updating the context.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@relay/relay_task.go` around lines 92 - 100, When switching channels in the origin-task handling branch, release the existing concurrency gate handle and acquire a new reservation for ch before updating the channel context keys. Ensure the new handle is attached to the origin channel and preserve the existing GetNextEnabledKey error handling.
🧹 Nitpick comments (5)
docs/superpowers/specs/2026-07-15-glm-affinity-concurrency-routing-design.md (1)
183-193: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winDocument per-key gates and their capacity composition.
The specification documents only
channel:<channel_id>, whileservice/channel_limit_select.goalso useschannel:<id>:key:<idx>gates andGateHandlecan release both. Clarify whether a request acquires both channel-level and key-level slots, and how those limits compose, so the design, metrics, and tests do not assume a single counter.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/superpowers/specs/2026-07-15-glm-affinity-concurrency-routing-design.md` around lines 183 - 193, Update the concurrency-routing design section describing per-process semaphores to document both channel-level keys and the channel-key gates used by service/channel_limit_select.go. Specify whether requests acquire both GateHandle-associated slots, how their capacities compose, and how acquisition, release, read-only concurrency, metrics, and tests account for the two independent limits; preserve the existing single-process and atomic TryAcquireConcurrency semantics.web/default/src/features/models/components/drawers/model-mutate-drawer.tsx (1)
216-216: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove unused default setting.
channel_limit_setting.enabledwas added todefaultModelSettingshere but is not utilized anywhere else in this component (which primarily processes pricing and ratio configs). Unless it is strictly required by theModelSettingstype definition, consider removing it to reduce clutter.♻️ Proposed refactor
- 'channel_limit_setting.enabled': true,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/default/src/features/models/components/drawers/model-mutate-drawer.tsx` at line 216, Remove the unused channel_limit_setting.enabled entry from defaultModelSettings in the model mutation drawer, unless ModelSettings explicitly requires it; preserve the remaining pricing and ratio defaults unchanged.service/channel_limit_select_test.go (1)
80-83: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse
testify/assertfor non-fatal value checks.As per coding guidelines, new or substantially rewritten Go backend tests must use
testify/assertfor non-fatal value checks. Consider replacingrequire.Lenandrequire.Equalwithassert.Lenandassert.Equalfor the final test assertions. This applies to similar assertions on lines 95-96, 153-154, and 171-172.♻️ Proposed refactor
leastLoaded := leastLoadedChannels(channels) - require.Len(t, leastLoaded, 1) - require.Equal(t, 9412, leastLoaded[0].Id) + assert.Len(t, leastLoaded, 1) + assert.Equal(t, 9412, leastLoaded[0].Id) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@service/channel_limit_select_test.go` around lines 80 - 83, Update the final assertions in the channel-limit selection tests, including the checks around leastLoadedChannels and the similar assertions at the referenced test sections, to use testify/assert.Len and assert.Equal instead of require.Len and require.Equal. Preserve the existing assertion values and test behavior.Source: Coding guidelines
controller/relay.go (2)
613-618: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove redundant
releaseLimitGatecall.Similar to the standard relay loop, the
releaseLimitGate(c)call on line 616 is redundant because the gate was already released unconditionally on line 605.♻️ Proposed refactor
if !shouldRetryTaskRelay(c, channel.Id, taskErr, common.RetryTimes-retryParam.GetRetry()) { - // not retrying: release the slot held by this iteration's handle so - // we don't leak the semaphore across the rest of the process lifetime. - releaseLimitGate(c) break }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@controller/relay.go` around lines 613 - 618, Remove the redundant releaseLimitGate(c) call from the non-retry branch of the relay loop around shouldRetryTaskRelay; the gate is already released unconditionally earlier in this iteration. Preserve the existing retry decision and loop-break behavior.
229-245: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove redundant
releaseLimitGatecall.The
releaseLimitGate(c)call on line 243 is redundant because it is already called unconditionally on line 236 beforeprocessChannelError. SincereleaseLimitGateclears the context key, the second invocation acts as a no-op.♻️ Proposed refactor
if !shouldRetry(c, newAPIError, common.RetryTimes-retryParam.GetRetry()) { - // not retrying: release the slot held by this iteration's handle so - // we don't leak the semaphore across the rest of the process lifetime. - releaseLimitGate(c) break }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@controller/relay.go` around lines 229 - 245, Remove the conditional releaseLimitGate(c) call inside the !shouldRetry branch of the retry flow. Keep the unconditional releaseLimitGate(c) call before processChannelError, and preserve the existing break behavior when retries are exhausted.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@docs/superpowers/specs/2026-07-15-glm-affinity-concurrency-routing-design.md`:
- Around line 181-193: 明确并落实并发限制变更策略：在设计文档中说明渠道限制变更是否要求进程重启，或在
service/channel_gate.go 的 gate 管理逻辑中支持安全动态调整（包括 10→2 和 2→10）。若支持动态调整，避免
LoadOrStore 让旧容量永久生效，并安全淘汰旧 gate；同时为两个变更方向补充回归测试。
In `@middleware/distributor.go`:
- Around line 110-132: Update the affinity auto-group loop around
GetUserAutoGroup to capture the selected group’s index along with
preferredGroup. When setting ContextKeyAutoGroup after acquiring the preferred
channel, also store the corresponding ContextKeyAutoGroupIndex, matching the
normal selection path so retries resume from the affinity-selected group.
In `@model/ability.go`:
- Around line 162-169: Replace the per-ability DB.First call in the
channel-loading loop with a single query using all eligible ability.ChannelId
values and an IN clause. Index the fetched channels by ID, then rebuild channels
in the original abilities order, preserving error handling and the existing
result type.
In `@model/channel_cache.go`:
- Around line 213-241: Update GetSatisfiedChannels to sort the assembled
channels by descending priority before returning them, using the Channel
priority field and adding the sort import if needed. Preserve the existing
filtering, exclusion, cache lookup, and missing-channel error behavior.
In `@model/channel.go`:
- Around line 233-236: Update the polling loop that selects channel keys to
require !excludeKeyIdx[idx] in addition to the existing status check, ensuring
excluded keys are never returned even when capacity is saturated. Keep the
enabledIdx filtering and other polling behavior unchanged.
In `@service/channel_gate.go`:
- Around line 24-29: Update TryAcquireConcurrency so an existing concurrency
gate reflects the current max when the configured limit changes, rather than
permanently reusing the first channel capacity. Store a gate object with safely
updateable limit/state, or replace the existing gate only when it can be drained
without violating active acquisitions; preserve unlimited behavior for max <= 0
and ensure concurrent callers cannot bypass the configured limit.
In `@service/channel_limit_select.go`:
- Around line 81-94: Update selectHealthyKey to retain the successful
TryAcquireConcurrency result instead of releasing it before returning, and
return the selected index together with that retained acquisition. Ensure no
release occurs when MaxConcurrency <= 0, and attach the retained acquisition
directly to the caller’s GateHandle so reservation remains atomic.
In `@web/default/src/features/channels/constants.ts`:
- Line 362: Add the exact literal string used by
FIELD_DESCRIPTIONS.MAX_CONCURRENCY to the static key registry in static-keys.ts,
preserving the existing key format and placement conventions so dynamic lookup
extraction recognizes it.
In `@web/default/src/i18n/locales/en.json`:
- Line 2028: Restore the protected Unicode-escaped key
footer.new\u0061pi.projectAttributionSuffix in both
web/default/src/i18n/locales/en.json at lines 2028-2028 and
web/default/src/i18n/locales/fr.json at lines 2028-2028; leave the translation
values unchanged.
In `@web/default/src/i18n/locales/fr.json`:
- Around line 5182-5186: Translate the English values for “Channel Limit
Enabled,” “Channel Limits,” “Max Concurrency,” and “Max concurrent requests to
this channel (0 = unlimited)” in the French locale, while preserving the
existing keys and JSON structure.
In `@web/default/src/i18n/locales/ja.json`:
- Around line 5182-5186: Translate the user-facing values for “Channel Limit
Enabled,” “Channel Limits,” “Max Concurrency,” and “Max concurrent requests to
this channel (0 = unlimited)” in the Japanese locale, while preserving the
existing translation for “Master switch for per-channel concurrency limits.”
In `@web/default/src/i18n/locales/ru.json`:
- Around line 5182-5186: Translate the four newly added concurrency-limit
entries in the Russian locale—“Channel Limit Enabled”, “Channel Limits”, “Max
Concurrency”, and “Max concurrent requests to this channel (0 =
unlimited)”—while preserving the existing Russian translation for the
master-switch description.
In `@web/default/src/i18n/locales/vi.json`:
- Around line 5182-5183: Translate the Vietnamese locale values for the “Channel
Limit Enabled” and “Channel Limits” keys in the locale data, replacing the
English text while preserving the existing keys and JSON structure.
---
Outside diff comments:
In `@relay/relay_task.go`:
- Around line 92-100: When switching channels in the origin-task handling
branch, release the existing concurrency gate handle and acquire a new
reservation for ch before updating the channel context keys. Ensure the new
handle is attached to the origin channel and preserve the existing
GetNextEnabledKey error handling.
In
`@web/default/src/features/channels/components/drawers/channel-mutate-drawer.tsx`:
- Around line 992-997: Update the routing strategy state in the channel mutate
drawer to track the max_concurrency field alongside the existing priority,
weight, test model, and auto-ban values, then include that watched value in
routingStrategyConfigured so customizing only max_concurrency marks the section
as configured.
---
Nitpick comments:
In `@controller/relay.go`:
- Around line 613-618: Remove the redundant releaseLimitGate(c) call from the
non-retry branch of the relay loop around shouldRetryTaskRelay; the gate is
already released unconditionally earlier in this iteration. Preserve the
existing retry decision and loop-break behavior.
- Around line 229-245: Remove the conditional releaseLimitGate(c) call inside
the !shouldRetry branch of the retry flow. Keep the unconditional
releaseLimitGate(c) call before processChannelError, and preserve the existing
break behavior when retries are exhausted.
In
`@docs/superpowers/specs/2026-07-15-glm-affinity-concurrency-routing-design.md`:
- Around line 183-193: Update the concurrency-routing design section describing
per-process semaphores to document both channel-level keys and the channel-key
gates used by service/channel_limit_select.go. Specify whether requests acquire
both GateHandle-associated slots, how their capacities compose, and how
acquisition, release, read-only concurrency, metrics, and tests account for the
two independent limits; preserve the existing single-process and atomic
TryAcquireConcurrency semantics.
In `@service/channel_limit_select_test.go`:
- Around line 80-83: Update the final assertions in the channel-limit selection
tests, including the checks around leastLoadedChannels and the similar
assertions at the referenced test sections, to use testify/assert.Len and
assert.Equal instead of require.Len and require.Equal. Preserve the existing
assertion values and test behavior.
In `@web/default/src/features/models/components/drawers/model-mutate-drawer.tsx`:
- Line 216: Remove the unused channel_limit_setting.enabled entry from
defaultModelSettings in the model mutation drawer, unless ModelSettings
explicitly requires it; preserve the remaining pricing and ratio defaults
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 95fb1fcb-f896-485f-bc66-5d45929cab7f
📒 Files selected for processing (39)
constant/context_key.gocontroller/channel_upstream_update.gocontroller/ratio_sync.gocontroller/relay.godocs/superpowers/specs/2026-07-15-glm-affinity-concurrency-routing-design.mddto/channel_settings.gomiddleware/distributor.gomiddleware/distributor_test.gomodel/ability.gomodel/channel.gomodel/channel_cache.gomodel/channel_cache_exclude_test.gomodel/channel_candidates_test.gorelay/common/relay_info.gorelay/relay_task.goservice/channel_affinity.goservice/channel_affinity_template_test.goservice/channel_gate.goservice/channel_gate_test.goservice/channel_limit.goservice/channel_limit_select.goservice/channel_limit_select_test.goservice/channel_limit_test.goservice/channel_select.gosetting/operation_setting/channel_limit_setting.goweb/default/src/features/channels/components/drawers/channel-mutate-drawer.tsxweb/default/src/features/channels/constants.tsweb/default/src/features/channels/lib/channel-form.tsweb/default/src/features/models/components/drawers/model-mutate-drawer.tsxweb/default/src/features/system-settings/models/index.tsxweb/default/src/features/system-settings/models/routing-reliability-section.tsxweb/default/src/features/system-settings/models/section-registry.tsxweb/default/src/features/system-settings/types.tsweb/default/src/i18n/locales/en.jsonweb/default/src/i18n/locales/fr.jsonweb/default/src/i18n/locales/ja.jsonweb/default/src/i18n/locales/ru.jsonweb/default/src/i18n/locales/vi.jsonweb/default/src/i18n/locales/zh.json
Comment on lines
+181
to
+193
| ## 8. 并发状态与多实例语义 | ||
|
|
||
| 继续使用进程内信号量实现渠道并发限制： | ||
|
|
||
| ```text | ||
| channel:<channel_id> | ||
| ``` | ||
|
|
||
| Affinity 和非 Affinity 请求共享同一个 key。 | ||
|
|
||
| 该限制为单进程限制。多实例部署时，每个实例都拥有独立上限。例如渠道上限为 10，部署 3 个实例时，理论最大总并发为 30。这是现有并发 gate 的既有语义，本阶段不改为 Redis 全局计数。 | ||
|
|
||
| 需要提供安全的只读当前并发方法，供最少连接选择使用。读取值只用于排序，最终正确性仍由原子 `TryAcquireConcurrency` 保证。 |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Define the live-update policy for concurrency limits.
service/channel_gate.go creates gates with LoadOrStore(..., make(chan struct{}, max)), so the first configured limit becomes sticky for that process-local gate. Changing a channel from 10→2 or 2→10 will not resize the existing gate. Specify and implement a safe policy—such as versioned gate keys and retirement of old gates—or explicitly require process reinitialization, with regression tests for both directions.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/superpowers/specs/2026-07-15-glm-affinity-concurrency-routing-design.md`
around lines 181 - 193, 明确并落实并发限制变更策略：在设计文档中说明渠道限制变更是否要求进程重启，或在
service/channel_gate.go 的 gate 管理逻辑中支持安全动态调整（包括 10→2 和 2→10）。若支持动态调整，避免
LoadOrStore 让旧容量永久生效，并安全淘汰旧 gate；同时为两个变更方向补充回归测试。
Comment on lines
+110
to
+132
| preferredGroup := "" | ||
| if usingGroup == "auto" { | ||
| userGroup := common.GetContextKeyString(c, constant.ContextKeyUserGroup) | ||
| autoGroups := service.GetUserAutoGroup(userGroup) | ||
| for _, g := range autoGroups { | ||
| if model.IsChannelEnabledForGroupModel(g, modelRequest.Model, preferred.Id) { | ||
| selectGroup = g | ||
| common.SetContextKey(c, constant.ContextKeyAutoGroup, g) | ||
| channel = preferred | ||
| affinityUsable = true | ||
| service.MarkChannelAffinityUsed(c, g, preferred.Id) | ||
| preferredGroup = g | ||
| break | ||
| } | ||
| } | ||
| } else if model.IsChannelEnabledForGroupModel(usingGroup, modelRequest.Model, preferred.Id) { | ||
| channel = preferred | ||
| selectGroup = usingGroup | ||
| affinityUsable = true | ||
| service.MarkChannelAffinityUsed(c, usingGroup, preferred.Id) | ||
| preferredGroup = usingGroup | ||
| } | ||
| if preferredGroup != "" { | ||
| affinityCandidateValid = true | ||
| handle, acquired := service.AcquireChannelWithLimits(c, preferred) | ||
| if acquired { | ||
| channel = preferred | ||
| selectGroup = preferredGroup | ||
| common.SetContextKey(c, constant.ContextKeyChannelLimitGate, handle) | ||
| if usingGroup == "auto" { | ||
| common.SetContextKey(c, constant.ContextKeyAutoGroup, preferredGroup) | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Preserve the affinity-selected auto-group index.
This path stores ContextKeyAutoGroup but not ContextKeyAutoGroupIndex. A later SelectChannelWithLimits retry therefore restarts from the first auto group rather than the affinity channel’s group. Capture the loop index and store both values, matching the normal selection path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@middleware/distributor.go` around lines 110 - 132, Update the affinity
auto-group loop around GetUserAutoGroup to capture the selected group’s index
along with preferredGroup. When setting ContextKeyAutoGroup after acquiring the
preferred channel, also store the corresponding ContextKeyAutoGroupIndex,
matching the normal selection path so retries resume from the affinity-selected
group.
Comment on lines
+162
to
+169
| channels := make([]*Channel, 0, len(abilities)) | ||
| for _, ability := range abilities { | ||
| var channel Channel | ||
| if err := DB.First(&channel, "id = ?", ability.ChannelId).Error; err != nil { | ||
| return nil, err | ||
| } | ||
| channels = append(channels, &channel) | ||
| } |
Contributor
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟠 Major | ⚡ Quick win
Batch-load candidate channels to avoid an N+1 query.
This performs one DB.First call for every eligible ability on the routing hot path. Fetch all channel IDs with one IN query, index them by ID, then rebuild the result in ability order.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@model/ability.go` around lines 162 - 169, Replace the per-ability DB.First
call in the channel-loading loop with a single query using all eligible
ability.ChannelId values and an IN clause. Index the fetched channels by ID,
then rebuild channels in the original abilities order, preserving error handling
and the existing result type.
Comment on lines
+213
to
+241
| // GetSatisfiedChannels returns every enabled channel that can serve the | ||
| // request, ordered by descending priority. The returned slice is independent | ||
| // from the cached channel ID slices and is safe for the caller to reorder. | ||
| func GetSatisfiedChannels(group string, modelName string, requestPath string, excludeIDs []int) ([]*Channel, error) { | ||
| if !common.MemoryCacheEnabled { | ||
| return getSatisfiedChannelsFromDB(group, modelName, requestPath, excludeIDs) | ||
| } | ||
|
|
||
| channelSyncLock.RLock() | ||
| defer channelSyncLock.RUnlock() | ||
|
|
||
| channelIDs := filterChannelsByRequestPathAndModel(group2model2channels[group][modelName], requestPath, modelName) | ||
| if len(channelIDs) == 0 { | ||
| normalizedModel := ratio_setting.FormatMatchingModelName(modelName) | ||
| channelIDs = filterChannelsByRequestPathAndModel(group2model2channels[group][normalizedModel], requestPath, modelName) | ||
| } | ||
| channelIDs = excludeChannelIDs(channelIDs, excludeIDs) | ||
|
|
||
| channels := make([]*Channel, 0, len(channelIDs)) | ||
| for _, channelID := range channelIDs { | ||
| channel, ok := channelsIDM[channelID] | ||
| if !ok { | ||
| return nil, fmt.Errorf("channel #%d does not exist", channelID) | ||
| } | ||
| channels = append(channels, channel) | ||
| } | ||
| return channels, nil | ||
| } | ||
|
|
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Ensure the returned channels are ordered by descending priority.
The function documentation explicitly states that the returned channels will be "ordered by descending priority." However, the current implementation simply returns them in the order they appear in group2model2channels, which is not guaranteed to be strictly sorted by priority. If downstream logic (like leastLoadedChannels or affinity fallback) relies on this exact ordering, it could lead to incorrect channel selection.
♻️ Proposed fix
for _, channelID := range channelIDs {
channel, ok := channelsIDM[channelID]
if !ok {
return nil, fmt.Errorf("channel #%d does not exist", channelID)
}
channels = append(channels, channel)
}
+
+ sort.Slice(channels, func(i, j int) bool {
+ return channels[i].GetPriority() > channels[j].GetPriority()
+ })
+
return channels, nil
}Note: You will also need to import "sort" if it's not already imported in this block.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // GetSatisfiedChannels returns every enabled channel that can serve the | |
| // request, ordered by descending priority. The returned slice is independent | |
| // from the cached channel ID slices and is safe for the caller to reorder. | |
| func GetSatisfiedChannels(group string, modelName string, requestPath string, excludeIDs []int) ([]*Channel, error) { | |
| if !common.MemoryCacheEnabled { | |
| return getSatisfiedChannelsFromDB(group, modelName, requestPath, excludeIDs) | |
| } | |
| channelSyncLock.RLock() | |
| defer channelSyncLock.RUnlock() | |
| channelIDs := filterChannelsByRequestPathAndModel(group2model2channels[group][modelName], requestPath, modelName) | |
| if len(channelIDs) == 0 { | |
| normalizedModel := ratio_setting.FormatMatchingModelName(modelName) | |
| channelIDs = filterChannelsByRequestPathAndModel(group2model2channels[group][normalizedModel], requestPath, modelName) | |
| } | |
| channelIDs = excludeChannelIDs(channelIDs, excludeIDs) | |
| channels := make([]*Channel, 0, len(channelIDs)) | |
| for _, channelID := range channelIDs { | |
| channel, ok := channelsIDM[channelID] | |
| if !ok { | |
| return nil, fmt.Errorf("channel #%d does not exist", channelID) | |
| } | |
| channels = append(channels, channel) | |
| } | |
| return channels, nil | |
| } | |
| // GetSatisfiedChannels returns every enabled channel that can serve the | |
| // request, ordered by descending priority. The returned slice is independent | |
| // from the cached channel ID slices and is safe for the caller to reorder. | |
| func GetSatisfiedChannels(group string, modelName string, requestPath string, excludeIDs []int) ([]*Channel, error) { | |
| if !common.MemoryCacheEnabled { | |
| return getSatisfiedChannelsFromDB(group, modelName, requestPath, excludeIDs) | |
| } | |
| channelSyncLock.RLock() | |
| defer channelSyncLock.RUnlock() | |
| channelIDs := filterChannelsByRequestPathAndModel(group2model2channels[group][modelName], requestPath, modelName) | |
| if len(channelIDs) == 0 { | |
| normalizedModel := ratio_setting.FormatMatchingModelName(modelName) | |
| channelIDs = filterChannelsByRequestPathAndModel(group2model2channels[group][normalizedModel], requestPath, modelName) | |
| } | |
| channelIDs = excludeChannelIDs(channelIDs, excludeIDs) | |
| channels := make([]*Channel, 0, len(channelIDs)) | |
| for _, channelID := range channelIDs { | |
| channel, ok := channelsIDM[channelID] | |
| if !ok { | |
| return nil, fmt.Errorf("channel #%d does not exist", channelID) | |
| } | |
| channels = append(channels, channel) | |
| } | |
| sort.Slice(channels, func(i, j int) bool { | |
| return channels[i].GetPriority() > channels[j].GetPriority() | |
| }) | |
| return channels, nil | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@model/channel_cache.go` around lines 213 - 241, Update GetSatisfiedChannels
to sort the assembled channels by descending priority before returning them,
using the Channel priority field and adding the sort import if needed. Preserve
the existing filtering, exclusion, cache lookup, and missing-channel error
behavior.
Comment on lines
+233
to
+236
| // Collect indexes of enabled, non-excluded keys | ||
| enabledIdx := make([]int, 0, len(keys)) | ||
| for i := range keys { | ||
| if getStatus(i) == common.ChannelStatusEnabled { | ||
| if getStatus(i) == common.ChannelStatusEnabled && !excludeKeyIdx[i] { |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Honor exclusions in polling mode too.
The new filter only affects enabledIdx; the polling loop at Lines 274–279 checks status alone and can still return an excluded, capacity-saturated key. Add !excludeKeyIdx[idx] to that loop’s condition.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@model/channel.go` around lines 233 - 236, Update the polling loop that
selects channel keys to require !excludeKeyIdx[idx] in addition to the existing
status check, ensuring excluded keys are never returned even when capacity is
saturated. Keep the enabledIdx filtering and other polling behavior unchanged.
| "footer.columns.related.title": "Related Projects", | ||
| "footer.defaultCopyright": "All rights reserved.", | ||
| "footer.new\u0061pi.projectAttributionSuffix": "All rights reserved. Designed and developed by the project contributors.", | ||
| "footer.newapi.projectAttributionSuffix": "All rights reserved. Designed and developed by the project contributors.", |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Restore protected project identity reference.
As per coding guidelines, "Protected references to the new-api and QuantumNous project or organization identities must not be modified, deleted, replaced, or removed." Modifying the Unicode escape sequence (\u0061) in the footer.new\u0061pi.projectAttributionSuffix key violates this protection mechanism.
web/default/src/i18n/locales/en.json#L2028-L2028: revert the key back tofooter.new\u0061pi.projectAttributionSuffix.web/default/src/i18n/locales/fr.json#L2028-L2028: revert the key back tofooter.new\u0061pi.projectAttributionSuffix.
📍 Affects 2 files
web/default/src/i18n/locales/en.json#L2028-L2028(this comment)web/default/src/i18n/locales/fr.json#L2028-L2028
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/default/src/i18n/locales/en.json` at line 2028, Restore the protected
Unicode-escaped key footer.new\u0061pi.projectAttributionSuffix in both
web/default/src/i18n/locales/en.json at lines 2028-2028 and
web/default/src/i18n/locales/fr.json at lines 2028-2028; leave the translation
values unchanged.
Source: Coding guidelines
Comment on lines
+5182
to
+5186
| "Channel Limit Enabled": "Channel Limit Enabled", | ||
| "Channel Limits": "Channel Limits", | ||
| "Master switch for per-channel concurrency limits": "Interrupteur principal pour les limites de concurrence par canal", | ||
| "Max Concurrency": "Max Concurrency", | ||
| "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)" |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Translate English text to French.
The newly added keys have English values in the French locale file. They should be translated into French.
📝 Proposed translations
- "Channel Limit Enabled": "Channel Limit Enabled",
- "Channel Limits": "Channel Limits",
- "Master switch for per-channel concurrency limits": "Master switch for per-channel concurrency limits",
- "Max Concurrency": "Max Concurrency",
- "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)"
+ "Channel Limit Enabled": "Limite de canal activée",
+ "Channel Limits": "Limites de canal",
+ "Master switch for per-channel concurrency limits": "Interrupteur principal pour les limites de simultanéité par canal",
+ "Max Concurrency": "Simultanéité maximale",
+ "Max concurrent requests to this channel (0 = unlimited)": "Nombre maximal de requêtes simultanées vers ce canal (0 = illimité)"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| "Channel Limit Enabled": "Channel Limit Enabled", | |
| "Channel Limits": "Channel Limits", | |
| "Master switch for per-channel concurrency limits": "Interrupteur principal pour les limites de concurrence par canal", | |
| "Max Concurrency": "Max Concurrency", | |
| "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)" | |
| "Channel Limit Enabled": "Limite de canal activée", | |
| "Channel Limits": "Limites de canal", | |
| "Master switch for per-channel concurrency limits": "Interrupteur principal pour les limites de simultanéité par canal", | |
| "Max Concurrency": "Simultanéité maximale", | |
| "Max concurrent requests to this channel (0 = unlimited)": "Nombre maximal de requêtes simultanées vers ce canal (0 = illimité)" |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/default/src/i18n/locales/fr.json` around lines 5182 - 5186, Translate the
English values for “Channel Limit Enabled,” “Channel Limits,” “Max Concurrency,”
and “Max concurrent requests to this channel (0 = unlimited)” in the French
locale, while preserving the existing keys and JSON structure.
Comment on lines
+5182
to
+5186
| "Channel Limit Enabled": "Channel Limit Enabled", | ||
| "Channel Limits": "Channel Limits", | ||
| "Master switch for per-channel concurrency limits": "チャネルごとの同時実行制限のメインスイッチ", | ||
| "Max Concurrency": "Max Concurrency", | ||
| "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)" |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Translate the new Japanese locale values.
These entries are user-facing, but the labels and description remain in English, so Japanese users will see untranslated concurrency settings.
Proposed fix
- "Channel Limit Enabled": "Channel Limit Enabled",
- "Channel Limits": "Channel Limits",
+ "Channel Limit Enabled": "チャネル制限を有効化",
+ "Channel Limits": "チャネル制限",
"Master switch for per-channel concurrency limits": "チャネルごとの同時実行制限のメインスイッチ",
- "Max Concurrency": "Max Concurrency",
- "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)"
+ "Max Concurrency": "最大同時実行数",
+ "Max concurrent requests to this channel (0 = unlimited)": "このチャネルへの最大同時リクエスト数（0 = 無制限）"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| "Channel Limit Enabled": "Channel Limit Enabled", | |
| "Channel Limits": "Channel Limits", | |
| "Master switch for per-channel concurrency limits": "チャネルごとの同時実行制限のメインスイッチ", | |
| "Max Concurrency": "Max Concurrency", | |
| "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)" | |
| "Channel Limit Enabled": "チャネル制限を有効化", | |
| "Channel Limits": "チャネル制限", | |
| "Master switch for per-channel concurrency limits": "チャネルごとの同時実行制限のメインスイッチ", | |
| "Max Concurrency": "最大同時実行数", | |
| "Max concurrent requests to this channel (0 = unlimited)": "このチャネルへの最大同時リクエスト数（0 = 無制限）" |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/default/src/i18n/locales/ja.json` around lines 5182 - 5186, Translate the
user-facing values for “Channel Limit Enabled,” “Channel Limits,” “Max
Concurrency,” and “Max concurrent requests to this channel (0 = unlimited)” in
the Japanese locale, while preserving the existing translation for “Master
switch for per-channel concurrency limits.”
Sources: Coding guidelines, Learnings
Comment on lines
+5182
to
+5186
| "Channel Limit Enabled": "Channel Limit Enabled", | ||
| "Channel Limits": "Channel Limits", | ||
| "Master switch for per-channel concurrency limits": "Главный переключатель ограничений параллелизма канала", | ||
| "Max Concurrency": "Max Concurrency", | ||
| "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)" |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Translate the new concurrency-limit values.
Channel Limit Enabled, Channel Limits, Max Concurrency, and the maximum-request description remain English in ru.json, so Russian users will see incomplete localization.
Suggested translations
- "Channel Limit Enabled": "Channel Limit Enabled",
- "Channel Limits": "Channel Limits",
+ "Channel Limit Enabled": "Ограничение каналов включено",
+ "Channel Limits": "Лимиты каналов",
"Master switch for per-channel concurrency limits": "Главный переключатель ограничений параллелизма канала",
- "Max Concurrency": "Max Concurrency",
- "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)"
+ "Max Concurrency": "Максимальное число одновременных запросов",
+ "Max concurrent requests to this channel (0 = unlimited)": "Максимальное число одновременных запросов к этому каналу (0 = без ограничений)"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| "Channel Limit Enabled": "Channel Limit Enabled", | |
| "Channel Limits": "Channel Limits", | |
| "Master switch for per-channel concurrency limits": "Главный переключатель ограничений параллелизма канала", | |
| "Max Concurrency": "Max Concurrency", | |
| "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)" | |
| "Channel Limit Enabled": "Ограничение каналов включено", | |
| "Channel Limits": "Лимиты каналов", | |
| "Master switch for per-channel concurrency limits": "Главный переключатель ограничений параллелизма канала", | |
| "Max Concurrency": "Максимальное число одновременных запросов", | |
| "Max concurrent requests to this channel (0 = unlimited)": "Максимальное число одновременных запросов к этому каналу (0 = без ограничений)" |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/default/src/i18n/locales/ru.json` around lines 5182 - 5186, Translate the
four newly added concurrency-limit entries in the Russian locale—“Channel Limit
Enabled”, “Channel Limits”, “Max Concurrency”, and “Max concurrent requests to
this channel (0 = unlimited)”—while preserving the existing Russian translation
for the master-switch description.
Comment on lines
+5182
to
+5183
| "Channel Limit Enabled": "Channel Limit Enabled", | ||
| "Channel Limits": "Channel Limits", |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Translate the new Vietnamese locale values.
These entries currently render English text in the Vietnamese UI. Replace them with Vietnamese translations.
Proposed fix
- "Channel Limit Enabled": "Channel Limit Enabled",
- "Channel Limits": "Channel Limits",
+ "Channel Limit Enabled": "Bật giới hạn kênh",
+ "Channel Limits": "Giới hạn kênh",
"Master switch for per-channel concurrency limits": "Công tắc chính cho giới hạn đồng thời mỗi kênh",
- "Max Concurrency": "Max Concurrency",
- "Max concurrent requests to this channel (0 = unlimited)": "Max concurrent requests to this channel (0 = unlimited)"
+ "Max Concurrency": "Số lượng đồng thời tối đa",
+ "Max concurrent requests to this channel (0 = unlimited)": "Số yêu cầu đồng thời tối đa đến kênh này (0 = không giới hạn)"Also applies to: 5185-5186
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/default/src/i18n/locales/vi.json` around lines 5182 - 5183, Translate the
Vietnamese locale values for the “Channel Limit Enabled” and “Channel Limits”
keys in the locale data, replacing the English text while preserving the
existing keys and JSON structure.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
📝 变更描述 / Description
为渠道增加最大并发配置，并将并发限制接入普通分发与渠道亲和路由。候选渠道并发相同时按占用量优先选择；请求结束后释放占用，自动分组重试时保留正确的亲和状态。
同时补充默认前端配置入口、多语言文案、设计文档和相关回归测试。
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
Bug fix，我已提交或关联对应 Issue，且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
已运行
go test ./... -count=1。除上游已有的TestObserveChannelAffinityUsageCacheByRelayFormat_UnsupportedModeKeepsEmpty缓存计数隔离失败外，其余包通过。Summary by CodeRabbit
0means unlimited.