upstream: reply 503 to client when forwarding the request fails#16
Open
ChristopherWirt wants to merge 1 commit into
Open
Conversation
The write handler deactivated the upstream on send() error without replying to the waiting client, so the request hung until the client's read timeout. Send a 503 like the other failure paths. Fixes the 12_routing.t flake.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
12_routing.tfails nondeterministically under concurrent load. The failures are reported asMISROUTE ... got='', but they are dropped responses, not misroutes: an occasional request receives no response at all and the client blocks until its read timeout (passing runs ~5.4s, failing runs ~10.5s — one client timeout longer).Cause
In
httplite_upstream_write_handler, whenc->send()of the request to the upstream returnsNGX_ERROR(the upstream connection failed or was refused under burst load), the handler deactivated the upstream and returned without replying to the waiting client.Every other upstream-failure path (broken connection, read error, connect timeout) sends the client an HTTP 503, which the test treats as a retriable error. This one path sent nothing, so the client received no bytes and hung until timeout.
Fix
Send the same
503 Service Unavailablethe read-error path uses before deactivating the upstream, so the client gets a retriable error instead of hanging.Verification
12_routing.t: 10/10 consecutive passes (was flaky).