feat(relay): add RELAY_DISABLE_HTTP2 to force upstream HTTP/1.1#6143
feat(relay): add RELAY_DISABLE_HTTP2 to force upstream HTTP/1.1#6143m11y wants to merge 1 commit into
Conversation
The outbound relay transports set ForceAttemptHTTP2: true, so Go multiplexes every request to a given upstream onto a small pool of shared HTTP/2 connections (in practice ~2 per host). Under heavy concurrent streaming to the same upstream — e.g. many long-running coding-agent / reasoning streams sharing one channel — those shared connections saturate at the connection level and head-of-line-block latency-sensitive requests: a trivial call that takes ~3s on its own can stall 30-240s while queued on a connection carrying the heavy streams. A fresh, dedicated connection to the same upstream stays fast, which is what HTTP/1.1 gives (one pooled connection per in-flight request). This is the same class of problem as QuantumNous#2351 ("high concurrency issue to a single host"), whose RELAY_MAX_IDLE_CONNS* knobs only help HTTP/1.1 — under HTTP/2 Go keeps the connection count tiny regardless, so those knobs have no effect. Add an opt-in RELAY_DISABLE_HTTP2 (default false, unchanged behavior). When enabled, each relay transport (default + HTTP/HTTPS proxy + SOCKS5 proxy) is pinned to HTTP/1.1 via http.Protocols, so concurrent requests spread across dedicated connections instead of contending on shared h2 pipes. Verified: reproduced the stall with 40 concurrent heavy streams to one upstream (fast probe 30-90s on shared h2); with RELAY_DISABLE_HTTP2=true the probe stays ~2-6s and the connection count scales with concurrency. gofmt/vet/build clean.
WalkthroughAdds the ChangesRelay HTTP/2 Control
Estimated code review effort: 2 (Simple) | ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.env.example:
- Around line 61-65: Update the comments describing
RELAY_MAX_IDLE_CONNS_PER_HOST to state that it limits idle keep-alive
connections, not in-flight requests; do not claim each active HTTP/1.1 request
is constrained by it. If the intended behavior requires a hard per-host
connection cap, reference or configure the appropriate MaxConnsPerHost setting
instead.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: fcd3bf07-bb0e-4bbb-bcc7-8b23bb59e760
📒 Files selected for processing (5)
.env.examplecommon/constants.gocommon/init.gorelay/helper/stream_scanner.goservice/http_client.go
| # 强制上游 relay 客户端使用 HTTP/1.1 而非 HTTP/2。默认 false（用 HTTP/2）。 | ||
| # Go 会把发往同一上游的所有请求多路复用到少量共享 HTTP/2 连接上； | ||
| # 当大并发流式请求打到同一上游时，这些共享连接会阻塞延迟敏感的小请求。 | ||
| # 开启后每个在飞请求各占一条独立连接（受 RELAY_MAX_IDLE_CONNS_PER_HOST 约束）。 | ||
| # RELAY_DISABLE_HTTP2=false |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## Files mentioning RELAY_MAX_IDLE_CONNS_PER_HOST\n'
rg -n "RELAY_MAX_IDLE_CONNS_PER_HOST|DISABLE_HTTP2|MaxIdleConnsPerHost|MaxConnsPerHost" .
printf '\n## .env.example excerpt\n'
nl -ba .env.example | sed -n '50,80p'
printf '\n## Candidate source files (if any)\n'
fd -a -t f '.*' . | sed -n '1,120p'Repository: QuantumNous/new-api
Length of output: 1219
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## .env.example lines 55-70"
sed -n '55,70p' .env.example | cat -n
echo
echo "## service/http_client.go around the relevant transport config"
sed -n '1,260p' service/http_client.go | cat -n | sed -n '1,240p'
echo
echo "## common/init.go around env parsing"
sed -n '100,125p' common/init.go | cat -n
echo
echo "## common/constants.go around declarations"
sed -n '180,200p' common/constants.go | cat -nRepository: QuantumNous/new-api
Length of output: 12272
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '## .env.example lines 55-70\n'
python3 - <<'PY'
from pathlib import Path
p = Path('.env.example')
for i, line in enumerate(p.read_text(encoding='utf-8').splitlines(), 1):
if 55 <= i <= 70:
print(f"{i:>4}: {line}")
PY
printf '\n## service/http_client.go around the relevant transport config\n'
python3 - <<'PY'
from pathlib import Path
p = Path('service/http_client.go')
for i, line in enumerate(p.read_text(encoding='utf-8').splitlines(), 1):
if 1 <= i <= 260:
print(f"{i:>4}: {line}")
PY
printf '\n## common/init.go around env parsing\n'
python3 - <<'PY'
from pathlib import Path
p = Path('common/init.go')
for i, line in enumerate(p.read_text(encoding='utf-8').splitlines(), 1):
if 100 <= i <= 125:
print(f"{i:>4}: {line}")
PY
printf '\n## common/constants.go around declarations\n'
python3 - <<'PY'
from pathlib import Path
p = Path('common/constants.go')
for i, line in enumerate(p.read_text(encoding='utf-8').splitlines(), 1):
if 180 <= i <= 200:
print(f"{i:>4}: {line}")
PYRepository: QuantumNous/new-api
Length of output: 11982
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '## .env.example lines 55-70\n'
python3 - <<'PY'
from pathlib import Path
for i, line in enumerate(Path('.env.example').read_text(encoding='utf-8').splitlines(), 1):
if 55 <= i <= 70:
print(f"{i:>4}: {line}")
PY
printf '\n## service/http_client.go around transport config\n'
python3 - <<'PY'
from pathlib import Path
for i, line in enumerate(Path('service/http_client.go').read_text(encoding='utf-8').splitlines(), 1):
if 1 <= i <= 260:
print(f"{i:>4}: {line}")
PY
printf '\n## common/init.go around env parsing\n'
python3 - <<'PY'
from pathlib import Path
for i, line in enumerate(Path('common/init.go').read_text(encoding='utf-8').splitlines(), 1):
if 100 <= i <= 125:
print(f"{i:>4}: {line}")
PY
printf '\n## common/constants.go around declarations\n'
python3 - <<'PY'
from pathlib import Path
for i, line in enumerate(Path('common/constants.go').read_text(encoding='utf-8').splitlines(), 1):
if 180 <= i <= 200:
print(f"{i:>4}: {line}")
PYRepository: QuantumNous/new-api
Length of output: 11969
🌐 Web query:
net/http Transport MaxIdleConnsPerHost limits only idle connections MaxConnsPerHost documentation
💡 Result:
In the Go net/http package, MaxIdleConnsPerHost and MaxConnsPerHost serve different purposes regarding connection management in a Transport [1][2]. MaxIdleConnsPerHost limits only the number of idle (keep-alive) connections that can be kept in the connection pool for a specific host [1][2]. When a connection is no longer active but is kept alive for reuse, it is placed in an idle pool [1]. If the number of idle connections for a host reaches this limit, additional connections are closed rather than being added to the idle pool [1]. The default value is defined by DefaultMaxIdleConnsPerHost, which is currently 2 [1][2]. MaxConnsPerHost, by contrast, provides a broader limit on the total number of connections allowed per host [1][2]. This total includes all connections in any state: dialing, active, or idle [1][2]. If this limit is reached, any attempts to dial new connections to that host will block until an existing connection is released or closed [1][2]. A value of zero (the default) means there is no limit [1][2]. In summary: - MaxIdleConnsPerHost manages the pool of reusable, inactive connections [1][2]. - MaxConnsPerHost manages the absolute capacity of connections (active + idle + dialing) permitted for a host [1][2].
Citations:
- 1: https://go.googlesource.com/go/+/master/src/net/http/transport.go
- 2: https://github.com/golang/go/blob/release-branch.go1.12/src/net/http/transport.go
Don't describe RELAY_MAX_IDLE_CONNS_PER_HOST as an in-flight cap MaxIdleConnsPerHost only limits idle keep-alive connections; active HTTP/1.1 requests can still exceed it. If a hard per-host limit is needed, use MaxConnsPerHost instead.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.env.example around lines 61 - 65, Update the comments describing
RELAY_MAX_IDLE_CONNS_PER_HOST to state that it limits idle keep-alive
connections, not in-flight requests; do not claim each active HTTP/1.1 request
is constrained by it. If the intended behavior requires a hard per-host
connection cap, reference or configure the appropriate MaxConnsPerHost setting
instead.
|
禁用HTTP/2在Golang标准库net/http里有配置项，并不需要额外添加配置项 |
2 participants
📝 变更描述 / Description
出站 relay 的
http.Transport都设了ForceAttemptHTTP2: true，于是 Go 会把发往同一上游的所有请求多路复用到少量共享 HTTP/2 连接上（实测每个上游 host 恒定 ~2 条）。当大量长时流式请求（典型如 coding-agent / 长 reasoning）打到同一个渠道时，这几条共享连接在连接级被大流量占满，会队头阻塞延迟敏感的小请求——一个本该 ~3s 的请求，排在承载重流的连接上会被拖到 30–240s。而对同一上游新建一条独立连接始终很快，这正是 HTTP/1.1 的行为（每个在飞请求各占一条池化连接）。The outbound relay transports use
ForceAttemptHTTP2: true, so Go multiplexes every request to a given upstream onto a small shared HTTP/2 connection pool. Under heavy concurrent streaming to one upstream, those shared connections saturate at the connection level and head-of-line-block small latency-sensitive requests (a ~3s call can stall 30–240s), while a fresh dedicated connection stays fast.这与 #2351（"high concurrency issue to a single host"）是同一类问题；但 #2351 加的
RELAY_MAX_IDLE_CONNS*只对 HTTP/1.1 生效——HTTP/2 下 Go 无论如何只保持极少连接，那些旋钮不起作用。本 PR 新增一个默认关闭的开关
RELAY_DISABLE_HTTP2（默认false，行为不变）。开启后，三条 relay transport（默认 / HTTP(S) 代理 / SOCKS5 代理）都通过http.Protocols固定为 HTTP/1.1，让并发请求分散到独立连接、不再在共享 h2 管道上互相拖累。🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
RELAY_MAX_IDLE_CONNS*仅对 HTTP/1.1 有效）✅ 提交前检查项 / Checklist
RELAY_DISABLE_HTTP2/ 关闭 HTTP2 的现有实现（feat: 配置 HTTP/2 PING 保活机制，防止空闲连接被中间设备 RST #5308 是 h2 PING 保活，属另一问题）。false，不改变现有行为；仅在显式开启时降级为 HTTP/1.1。gofmt/go vet/go build ./service/... ./common/... ./relay/helper/...均通过。📸 运行证明 / Proof of Work
复现（40 条并发重流式请求打到同一上游 + 1 个小探测请求，同一时刻）：
RELAY_DISABLE_HTTP2=true（HTTP/1.1）开启后容器到上游的连接数随并发上涨（不再恒定 ~2 条），小请求不再被重流阻塞。
debug日志新增relay disable http2: %t一行便于确认生效。Summary by CodeRabbit
RELAY_DISABLE_HTTP2configuration option to force relay connections to use HTTP/1.1 instead of HTTP/2.