Skip to content

perf: Rewrite _assign_requests_to_connections as a single-pass loop#974

Open
mbeijen wants to merge 1 commit into
pydantic:mainfrom
mbeijen:perf/assign-requests-single-pass
Open

perf: Rewrite _assign_requests_to_connections as a single-pass loop#974
mbeijen wants to merge 1 commit into
pydantic:mainfrom
mbeijen:perf/assign-requests-single-pass

Conversation

@mbeijen
Copy link
Copy Markdown
Contributor

@mbeijen mbeijen commented May 20, 2026

The previous implementation re-scanned self._connections per queued request to rebuild available_connections and idle_connections lists, giving O(N*M) behaviour when many requests are queued against a populated pool. The keepalive-eviction step in pass 1 was also quadratic, since sum(... is_idle() ...) was recomputed for every connection.

This rewrite:

  • Walks self._connections once to drop closed connections and schedule expired ones for close.
  • Counts idle connections once, then evicts surplus idle connections in a single pass to enforce max_keepalive_connections.
  • Snapshots available_connections once before the queued-request loop, and consults it (rather than re-filtering self._connections) for each request.

Behaviour is preserved exactly:

  • The same connection may still be assigned to multiple HTTP/1.1 requests in a single call. The pool's existing ConnectionNotAvailable retry loop in handle_async_request handles that case as before.
  • HTTP/2 multiplexing onto a single connection is preserved for the same reason — connections are not removed from available_connections on assignment.

Inspired by encode/httpcore#1035 by @VictorPrins.

Citing some issues related to this, including in downstream libraries relying on httpcore:

The previous implementation re-scanned `self._connections` per queued
request to rebuild `available_connections` and `idle_connections` lists,
giving O(N*M) behaviour when many requests are queued against a populated
pool. The keepalive-eviction step in pass 1 was also quadratic, since
`sum(... is_idle() ...)` was recomputed for every connection.

This rewrite:

- Walks `self._connections` once to drop closed connections and schedule
  expired ones for close.
- Counts idle connections once, then evicts surplus idle connections in
  a single pass to enforce `max_keepalive_connections`.
- Snapshots `available_connections` once before the queued-request loop,
  and consults it (rather than re-filtering `self._connections`) for each
  request.

Behaviour is preserved exactly:

- The same connection may still be assigned to multiple HTTP/1.1 requests
  in a single call. The pool's existing `ConnectionNotAvailable` retry
  loop in `handle_async_request` handles that case as before.
- HTTP/2 multiplexing onto a single connection is preserved for the same
  reason — connections are not removed from `available_connections` on
  assignment.

Inspired by encode/httpcore#1035 by @VictorPrins

Co-Authored-By: Victor Prins <32959052+VictorPrins@users.noreply.github.com>
@mbeijen mbeijen changed the title Rewrite _assign_requests_to_connections as a single-pass loop perf: Rewrite _assign_requests_to_connections as a single-pass loop May 20, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 20, 2026

Merging this PR will not alter performance

✅ 7 untouched benchmarks


Comparing mbeijen:perf/assign-requests-single-pass (80c9942) with main (b4c5940)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant