fix(relay): detect upstream SSE when Content-Type header is missing (fixes #6075)#6254
fix(relay): detect upstream SSE when Content-Type header is missing (fixes #6075)#6254BenLiu104 wants to merge 1 commit into
Conversation
Some backends (e.g. the ChatGPT Codex /responses endpoint) return a 200
SSE stream without a Content-Type: text/event-stream header. The existing
isResponsesEventStreamContentType() check only inspects the header, so
these responses were misclassified as non-stream and the SSE body was
handed to the JSON handler, producing:
status_code=500 invalid character 'e' looking for beginning of value
After the header check fails, sniff the body prefix one byte at a time.
A leading "event:" or "data:" (after optional BOM/whitespace) is treated
as SSE; a JSON-like start ({, [, ") is rejected early. Every consumed
byte is preserved via a bufio.Reader so downstream stream handlers still
read the full original body.
Fixes QuantumNous#6075
WalkthroughThe Responses relay now detects SSE streams when upstream responses omit ChangesResponses SSE detection
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@relay/chat_completions_via_responses.go`:
- Around line 212-229: Update the SSE probing logic around peekReadCloser to use
br.Peek(i) rather than consuming bytes with ReadByte, ensuring all inspected
bytes remain available to downstream parsers. Apply BOM and whitespace trimming
to the peeked data, check the first trimmed byte for JSON-like starts to exit
promptly on padded JSON, and preserve the existing SSE detection and fallback
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1016a84b-f98d-483e-955a-c50a6d827572
📒 Files selected for processing (1)
relay/chat_completions_via_responses.go
| buf := make([]byte, 0, 64) | ||
| for len(buf) < cap(buf) { | ||
| b, err := br.ReadByte() | ||
| if err != nil { | ||
| break // EOF or read error: not enough to confirm SSE | ||
| } | ||
| buf = append(buf, b) | ||
| trimmed := strings.TrimPrefix(string(buf), "\xef\xbb\xbf") | ||
| trimmed = strings.TrimLeft(trimmed, " \r\n") | ||
| if strings.HasPrefix(trimmed, "event:") || strings.HasPrefix(trimmed, "data:") { | ||
| *out = &peekReadCloser{Reader: br, closer: rc} | ||
| return true | ||
| } | ||
| // JSON-like start → definitely not SSE, stop probing. | ||
| if len(buf) >= 1 && (buf[0] == '{' || buf[0] == '[' || buf[0] == '"') { | ||
| break | ||
| } | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🔴 Critical | ⚡ Quick win
Restore consumed bytes and prevent blocking on padded JSON.
bufio.Reader.ReadByte() consumes bytes from the underlying buffer. Because peekReadCloser only wraps br without prepending the read bytes, the initial prefix read into buf is permanently lost to downstream parsers, resulting in stream corruption.
Additionally, checking buf[0] instead of trimmed[0] for JSON characters bypasses the early exit if the response starts with whitespace, which could cause the loop to block and wait for 64 bytes on a slow padded JSON stream.
Replacing the manual byte-read loop with br.Peek(i) solves both issues: Peek reads data without advancing the internal offset (safely preserving all bytes for peekReadCloser), and checking trimmed[0] guarantees a fast exit for padded JSON.
🛠️ Proposed fix
- buf := make([]byte, 0, 64)
- for len(buf) < cap(buf) {
- b, err := br.ReadByte()
- if err != nil {
- break // EOF or read error: not enough to confirm SSE
- }
- buf = append(buf, b)
- trimmed := strings.TrimPrefix(string(buf), "\xef\xbb\xbf")
- trimmed = strings.TrimLeft(trimmed, " \r\n")
+ for i := 1; i <= 64; i++ {
+ p, err := br.Peek(i)
+
+ trimmed := strings.TrimPrefix(string(p), "\xef\xbb\xbf")
+ trimmed = strings.TrimLeft(trimmed, " \t\r\n")
if strings.HasPrefix(trimmed, "event:") || strings.HasPrefix(trimmed, "data:") {
*out = &peekReadCloser{Reader: br, closer: rc}
return true
}
// JSON-like start → definitely not SSE, stop probing.
- if len(buf) >= 1 && (buf[0] == '{' || buf[0] == '[' || buf[0] == '"') {
+ if len(trimmed) >= 1 && (trimmed[0] == '{' || trimmed[0] == '[' || trimmed[0] == '"') {
break
}
+ if err != nil {
+ break // EOF or read error: not enough to confirm SSE
+ }
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| buf := make([]byte, 0, 64) | |
| for len(buf) < cap(buf) { | |
| b, err := br.ReadByte() | |
| if err != nil { | |
| break // EOF or read error: not enough to confirm SSE | |
| } | |
| buf = append(buf, b) | |
| trimmed := strings.TrimPrefix(string(buf), "\xef\xbb\xbf") | |
| trimmed = strings.TrimLeft(trimmed, " \r\n") | |
| if strings.HasPrefix(trimmed, "event:") || strings.HasPrefix(trimmed, "data:") { | |
| *out = &peekReadCloser{Reader: br, closer: rc} | |
| return true | |
| } | |
| // JSON-like start → definitely not SSE, stop probing. | |
| if len(buf) >= 1 && (buf[0] == '{' || buf[0] == '[' || buf[0] == '"') { | |
| break | |
| } | |
| } | |
| for i := 1; i <= 64; i++ { | |
| p, err := br.Peek(i) | |
| trimmed := strings.TrimPrefix(string(p), "\xef\xbb\xbf") | |
| trimmed = strings.TrimLeft(trimmed, " \t\r\n") | |
| if strings.HasPrefix(trimmed, "event:") || strings.HasPrefix(trimmed, "data:") { | |
| *out = &peekReadCloser{Reader: br, closer: rc} | |
| return true | |
| } | |
| // JSON-like start → definitely not SSE, stop probing. | |
| if len(trimmed) >= 1 && (trimmed[0] == '{' || trimmed[0] == '[' || trimmed[0] == '"') { | |
| break | |
| } | |
| if err != nil { | |
| break // EOF or read error: not enough to confirm SSE | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@relay/chat_completions_via_responses.go` around lines 212 - 229, Update the
SSE probing logic around peekReadCloser to use br.Peek(i) rather than consuming
bytes with ReadByte, ensuring all inspected bytes remain available to downstream
parsers. Apply BOM and whitespace trimming to the peeked data, check the first
trimmed byte for JSON-like starts to exit promptly on padded JSON, and preserve
the existing SSE detection and fallback behavior.
1 participant
Summary
/responses) channel via the/chat/completionsadaptor returnsstatus_code=500, invalid character 'e' looking for beginning of value.isResponsesEventStreamContentType()only inspects theContent-Typeheader. The ChatGPT Codex/responsesendpoint returns a 200 SSE stream without aContent-Type: text/event-streamheader, so new-api classified it as non-stream and passed the raw SSE body to the JSON handler, which failed onevent: ...→invalid character 'e'.event:/data:(after optional BOM/whitespace) is treated as SSE; a JSON-like start ({,[,") is rejected early. Every consumed byte is preserved via abufio.Reader(peekReadCloser) so downstream stream handlers still read the full original body.Why incremental peeking (not
Peek(64))A fixed
Reader.Peek(64)can block until 64 bytes arrive or EOF on a slow SSE prefix (e.g.event: ping\n\n). Reading one byte at a time recognizes the SSE prefix as soon as it appears and never blocks beyond the data actually sent.Behavior
Content-Type→ correctly routed to the SSE handler.Verification
go build ./relay/...passes onmain.event:/data:detected incl. BOM+whitespace; plain JSON{"..."}not mistaken for SSE).v1.0.0-rc.21arm64 image and deployed to a live instance; the Codex channel now streams correctly through/chat/completions(issue reporter confirmed).Test plan
relay/forisResponsesEventStreamSSEBody(SSE w/o Content-Type vs JSON)./chat/completionswith streaming enabled.Summary by CodeRabbit
Content-Typeheader.