Skip to content

Commit d0d7c28

Browse files
committed
fix(stream): proactively clean stream queue before each request to prevent stale done responses
This fixes an intermittent bug where streaming API responses would return empty (done=True, no content), and subsequent requests would also return empty responses until model switch/refresh. The root cause was the STREAM_QUEUE holding leftover done signals from previous requests due to a race condition in queue clearing. - Clear stream queue at request start in _process_request_refactored - Enhance use_stream_response to ignore and log empty stale done signals - Add detailed logging for all stream/queue edge cases for diagnosis - Reduce risk of state leakage between requests; safer under heavy load No breaking changes. Existing APIs and non-streaming mode unaffected.
1 parent 0dde544 commit d0d7c28

2 files changed

Lines changed: 325 additions & 271 deletions

File tree

0 commit comments

Comments
 (0)