Replace httpx and httpx-sse with httpx2#2972
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
5 issues found across 65 files
Tip: instead of fixing issues one by one fix them all with cubic
Partial review: This PR has more than 50 files, so cubic reviewed the highest-priority files first. During the trial, paid plans get a higher file limit.
You can try an ultrareview to bypass the file limit, comment @cubic-dev-ai ultrareview. Learn more.
Re-trigger cubic
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
|
maxisbey
force-pushed
the
httpx2-migration
branch
from
86e257f to
d4b5611
Compare
|
is anyone working on this ? moving to httpx2 would trigger a lot of downstream repos to move to the maintained library and I think security wise it's not a bad investment. |
|
@maxisbey we need to merge this now, it's time. |
httpx2 (2.5.0) is the next-generation httpx fork with server-sent events support built in, so the separate httpx-sse dependency is no longer needed. - Swap the httpx/httpx-sse dependencies for httpx2>=2.5.0 in the SDK and the example projects. - Rewrite the SSE transports against httpx2's API: aconnect_sse(...) -> client.stream(...)/client.sse(...) wrapped in EventSource, and iterate the EventSource directly instead of .aiter_sse(). - Document the swap as a v2 breaking change in docs/migration.md and update docs/installation.md, README.v2.md, and the example sources. Verified: ruff, pyright, and the full test suite pass at 100% coverage.
The rebase onto main picked up files added since the swap (stories examples, identity-assertion client/docs, client probe, docs_src tutorials) that still imported httpx. Apply the same httpx -> httpx2 rename to them, update the migration guide's mcp-types and identity-assertion sections to name httpx2, and document the certifi -> truststore TLS verification change.
httpx-sse's aconnect_sse() always sent Accept: text/event-stream and Cache-Control: no-store; the swap to bare client.stream() dropped both. Open the legacy SSE GET and the streamable HTTP GET/resumption/ reconnection streams with AsyncClient.sse(), which injects those headers (explicit caller headers still take precedence), and update the mocked sse_client test to drive the new call. Example fixes from the same review pass: - simple-chatbot: catch httpx2.HTTPError instead of RequestError so raise_for_status() failures take the handled path (the HTTPStatusError isinstance branch was unreachable), and drop the Raises section the method never honoured. - sse-polling-client: suppress the httpcore2 logger; httpcore is no longer in the dependency tree, so the old suppression was a no-op.
httpx2's ServerSentEvent declares id as str defaulting to empty, where httpx-sse allowed str | None. The handler under test checks truthiness, so the default is behaviourally identical.
New test files and comments picked up httpx imports on main; convert them to httpx2 and re-apply the dependency swap to uv.lock (httpx, httpx-sse, and httpcore out; httpx2, httpcore2, and truststore in; idna raised to the 3.18 floor httpx2 requires). No-Verification-Needed: test-only conversions plus lockfile, no runtime surface
The migration guide and client docs were reorganized on main after the swap branched: re-add the httpx2 section under Packaging with rows in the dependency table, convert the After-v2 examples that arrived with the reorganization (Before-v1 blocks keep httpx), and surface the system-trust-store TLS note on the transports page. No-Verification-Needed: documentation-only change
The TLS notes now cover the zero-code SSL_CERT_FILE/SSL_CERT_DIR route httpx2 honors before falling back to the system trust store, and warn that verify="ca.pem" and cert= are deprecated there. A handful of docstrings still branded the client HTTPX; they now say httpx2. No-Verification-Needed: docstring- and documentation-only change
maxisbey
force-pushed
the
httpx2-migration
branch
from
53622d5 to
8a2dfbf
Compare
📚 Documentation preview
|
Old except httpx.X handlers fail silently after the swap because httpx usually stays co-installed transitively, so the migration guide now calls out exception handlers, isinstance checks, and that httpx client objects are not interchangeable at runtime. The whats-new silent-changes list gains the certifi to system-trust-store entry. No-Verification-Needed: documentation-only change
There was a problem hiding this comment.
1 issue found across 3 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="docs/whats-new.md">
<violation number="1" location="docs/whats-new.md:141">
P3: The new TLS warning describes an impossible private-CA case: certifi's bundled Mozilla root store does not include private CAs. This can misdirect users who actually need to preserve a public root that is in certifi but absent from their system store (or who need to configure their private CA separately). Consider describing it as a CA/root present in certifi's bundle but absent from the system store.</violation>
</file>
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
| * **URI templates are real RFC 6570 now.** `{+path}`, `{?query}` and friends work, matching is exact instead of regex-loose, and path traversal in extracted values is rejected by default. Stricter templates fail at decoration time, not on the first request. **[URI templates](servers/uri-templates.md)**. | ||
| * **The streamable HTTP lifespan runs once**, at startup, and its state is shared by every session and request. In v1 it ran once per session, and once per request under `stateless_http=True`. Pools and caches built in a lifespan get dramatically cheaper; anything that acquired a per-connection resource there belongs in the handler body now. **[Lifespan](handlers/lifespan.md)**. | ||
| * **`mcp dev` and `mcp install` pin the environment they spawn** to your installed SDK version. Both commands run your server in a fresh `uv run --with ...` environment, which used to resolve `mcp` to the newest stable release rather than the version you are developing against. **[Migration Guide](migration.md#mcp-dev-and-mcp-install-pin-the-spawned-environment-to-your-sdk-version)**. | ||
| * **TLS certificates verify against the operating system trust store, not certifi.** The SDK's HTTP client is now `httpx2`, which validates through `truststore` instead of certifi's bundled CA list. Most environments never notice; a minimal container with no system CA store, or a private CA that only certifi's bundle knew about, starts failing the TLS handshake. Set `SSL_CERT_FILE`/`SSL_CERT_DIR` or pass `verify=ssl_context` to your client. **[Migration Guide](migration.md#httpx-and-httpx-sse-replaced-by-httpx2)**. |
There was a problem hiding this comment.
P3: The new TLS warning describes an impossible private-CA case: certifi's bundled Mozilla root store does not include private CAs. This can misdirect users who actually need to preserve a public root that is in certifi but absent from their system store (or who need to configure their private CA separately). Consider describing it as a CA/root present in certifi's bundle but absent from the system store.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/whats-new.md, line 141:
<comment>The new TLS warning describes an impossible private-CA case: certifi's bundled Mozilla root store does not include private CAs. This can misdirect users who actually need to preserve a public root that is in certifi but absent from their system store (or who need to configure their private CA separately). Consider describing it as a CA/root present in certifi's bundle but absent from the system store.</comment>
<file context>
@@ -138,6 +138,7 @@ The renames announce themselves. These do not:
* **URI templates are real RFC 6570 now.** `{+path}`, `{?query}` and friends work, matching is exact instead of regex-loose, and path traversal in extracted values is rejected by default. Stricter templates fail at decoration time, not on the first request. **[URI templates](servers/uri-templates.md)**.
* **The streamable HTTP lifespan runs once**, at startup, and its state is shared by every session and request. In v1 it ran once per session, and once per request under `stateless_http=True`. Pools and caches built in a lifespan get dramatically cheaper; anything that acquired a per-connection resource there belongs in the handler body now. **[Lifespan](handlers/lifespan.md)**.
* **`mcp dev` and `mcp install` pin the environment they spawn** to your installed SDK version. Both commands run your server in a fresh `uv run --with ...` environment, which used to resolve `mcp` to the newest stable release rather than the version you are developing against. **[Migration Guide](migration.md#mcp-dev-and-mcp-install-pin-the-spawned-environment-to-your-sdk-version)**.
+* **TLS certificates verify against the operating system trust store, not certifi.** The SDK's HTTP client is now `httpx2`, which validates through `truststore` instead of certifi's bundled CA list. Most environments never notice; a minimal container with no system CA store, or a private CA that only certifi's bundle knew about, starts failing the TLS handshake. Set `SSL_CERT_FILE`/`SSL_CERT_DIR` or pass `verify=ssl_context` to your client. **[Migration Guide](migration.md#httpx-and-httpx-sse-replaced-by-httpx2)**.
### Removed outright
</file context>
| * **TLS certificates verify against the operating system trust store, not certifi.** The SDK's HTTP client is now `httpx2`, which validates through `truststore` instead of certifi's bundled CA list. Most environments never notice; a minimal container with no system CA store, or a private CA that only certifi's bundle knew about, starts failing the TLS handshake. Set `SSL_CERT_FILE`/`SSL_CERT_DIR` or pass `verify=ssl_context` to your client. **[Migration Guide](migration.md#httpx-and-httpx-sse-replaced-by-httpx2)**. | |
| * **TLS certificates verify against the operating system trust store, not certifi.** The SDK's HTTP client is now `httpx2`, which validates through `truststore` instead of certifi's bundled CA list. Most environments never notice; a minimal container with no system CA store, or a CA root present in certifi's bundle but absent from the system store, starts failing the TLS handshake. Set `SSL_CERT_FILE`/`SSL_CERT_DIR` or pass `verify=ssl_context` to your client. **[Migration Guide](migration.md#httpx-and-httpx-sse-replaced-by-httpx2)**. |
…ames The whats-new bullet now headlines the dependency swap itself with TLS as one of its consequences, and the migration guide covers the renamed loggers, the new User-Agent, and module-keyed telemetry integrations going quiet. No-Verification-Needed: documentation-only change
| from typing import Any | ||
|
|
||
| import httpx | ||
| import httpx2 |
There was a problem hiding this comment.
🟡 The simple-chatbot example now does import httpx2 directly (main.py line 11), but neither mcp_simple_chatbot/requirements.txt (the README's documented pip install -r requirements.txt path) nor the example's pyproject.toml declares httpx2 — with mcp>=1.0.0 resolving to the published 1.x line (which depends on httpx, not httpx2), following the README fails at import with ModuleNotFoundError: No module named 'httpx2'. Add httpx2>=2.5.0 to requirements.txt (or the example's pyproject) to match the convention this PR applied to every other example.
Extended reasoning...
What happened. The import-rename sweep converted examples/clients/simple-chatbot/mcp_simple_chatbot/main.py from import httpx to import httpx2 (and the code genuinely uses it: httpx2.Client(), httpx2.HTTPError, httpx2.HTTPStatusError in LLMClient.get_response), but the example's install metadata was not updated. mcp_simple_chatbot/requirements.txt still lists only python-dotenv>=1.0.0, requests>=2.31.0, mcp>=1.0.0, uvicorn>=0.32.1, and the example's pyproject.toml dependencies are python-dotenv, mcp, uvicorn — httpx2 appears in neither.
How it manifests. The example's README documents pip install -r requirements.txt as the setup path. Step by step: (1) a user runs that command in a fresh venv; (2) pip resolves mcp>=1.0.0 to the newest published release, currently the 1.x line, whose dependencies include httpx>=0.27.1 — not httpx2; (3) nothing else in the requirements pulls httpx2 in; (4) the user runs the chatbot and line 11 executes import httpx2 → ModuleNotFoundError: No module named 'httpx2'. The example never starts.
Why this was missed. The PR's own convention was to update dependency declarations alongside the import rename — it bumped httpx>=0.27 to httpx2>=2.5.0 in every server-example pyproject that declared httpx (simple-auth, simple-tool, simple-prompt, simple-resource, simple-pagination, both simple-streamablehttp variants, everything-server, sse-polling-demo). simple-chatbot was skipped only because it never declared httpx explicitly; its old import httpx was satisfied transitively through mcp. This is the same class of miss maxisbey flagged for .github/actions/conformance/client.py (since fixed).
Why nit rather than blocking — the refutation's points are fair mitigations. One verifier argued this doesn't regress anything, and two of its points hold up on inspection: (1) the undeclared-direct-import pattern is pre-existing — before this PR the example imported httpx without declaring it either, relying on mcp's transitive dependency; (2) the example already requires mcp v2 independent of httpx2 — main.py:115 reads tool.input_schema, a v2 snake_case attribute (v1 types use inputSchema), so the example on this main branch is broken against a pip-installed mcp 1.x regardless of this PR; (3) inside the repo's uv workspace, the example depends on the editable mcp v2, which now depends on httpx2, so CI/workspace runs work; and (4) once mcp v2 is published, mcp>=1.0.0 resolves to 2.x and httpx2 arrives transitively again.
Why it's still worth a one-line fix. A package a module imports directly should be declared directly — relying on a transitive dependency of mcp is exactly the fragility that produced this gap, and the PR fixed that same fragility in every other example. Declaring httpx2>=2.5.0 in requirements.txt (and/or the example pyproject) makes the example robust regardless of which mcp version the loose >=1.0.0 pin resolves to. While touching the file, the stale mcp>=1.0.0 pin (the example needs v2 APIs) and the unused requests entry are also worth cleaning up, though those are pre-existing.
Fix. Add httpx2>=2.5.0 to examples/clients/simple-chatbot/mcp_simple_chatbot/requirements.txt and to the example's pyproject.toml dependencies.
|
following the merge of this (thank you!), when is the new release targeted for ? |
3 participants
Summary
Swaps the
httpx+httpx-ssedependencies forhttpx2>=2.5.0.httpx2is the next-generation httpx fork with server-sent events support built in, so the separatehttpx-ssedependency is no longer needed.Changes
httpx>=0.27.1,<1.0.0+httpx-sse>=0.4withhttpx2>=2.5.0in the SDK and the example projects (lockfile regenerated).aconnect_sse(...)→client.stream(...)/client.sse(...)wrapped inEventSource, and iterate theEventSourcedirectly instead of.aiter_sse().docs/migration.mdanddocs/whats-new.md; update the docs, README, and example sources.Intentional behavior changes
Accept: application/json, text/event-stream(previously exactlytext/event-stream): httpx2'sclient.sse()lets the transport's headers win wherehttpx_sseforce-overwrote them.Cache-Control: no-storeis unchanged.truststoreinstead of certifi's bundle;SSL_CERT_FILE/SSL_CERT_DIRare honored first. Documented in the migration guide.httpx→httpx2,httpcore.*→httpcore2.*; the default User-Agent becomespython-httpx2/<version>.except httpx.RequestError→except httpx2.HTTPError, so HTTP status errors are reported instead of propagating (the old handler's status-error branch was unreachable).Notes
httpx2is API-compatible withhttpx, so most of the diff is thehttpx→httpx2import rename. Users passing their ownhttp_clientonly need to change the import.AsyncClient.sse()convenience method for the raw-client SSE call sites.httpx2requirement deliberately has no upper bound, unlike v1'shttpx<1.0.0cap (Restrict httpx version to <1.0.0 #2345): httpx2 is maintained in lockstep with this SDK, so a breaking httpx2 major would land alongside a coordinated SDK release rather than being fenced off ahead of time.Verification
ruff,pyright, and the full test suite all pass at 100% coverage (strict-no-coverclean).AI Disclaimer
This PR was developed with the assistance of either Claude or Codex. I've reviewed and verified the changes.