[None][perf] serve: always use msgspec msgpack for disagg orchestrator->worker body

[Tabrizian]

Dev Engineer Review

• Unconditionally enables msgspec MessagePack for internal orchestrator-to-worker request forwarding.
• Removes the environment flag and JSON fallback while preserving standard JSON handling for external OpenAI clients.
• Always installs the MessagePack route and retains header-based decoding behavior.
• Documentation is consistent with the new default transport.
• No configuration or test-list changes identified.

QA Engineer Review

No test changes.

Description

Follow-up to #16126, which introduced the msgspec MessagePack transport for the
disaggregated orchestrator→worker request body behind the
TRTLLM_SERVE_ENABLE_MSGSPEC opt-in flag.

This PR makes that transport unconditional and removes the environment variable.

• openai_client.py: the orchestrator always encodes the forwarded body as
  msgpack and tags it with X-TRTLLM-Msgpack: 1. The model_dump_json fallback
  branch is gone.
• openai_server.py: _MsgspecRequest / _MsgspecRoute move to module scope
  (they were nested inside the if _MSGSPEC_ENABLED: block) and the route class
  is always installed on the FastAPI app.
• Both files now import msgspec unconditionally. msgspec is already a hard
  dependency — it is in requirements.txt, which setup.py feeds into
  install_requires — so the guarded import and its ImportError shim were
  redundant.

Behavior for external clients is unchanged: the worker keys off the
X-TRTLLM-Msgpack header, so anything without that header (i.e. every
OpenAI-API client) still goes through the stdlib json.loads path. Content-Type
stays application/json so FastAPI keeps routing bodies through Request.json().

The tech blog paragraph describing this path as "opt-in" is updated to match.

Test Coverage

No new tests. This PR removes a configuration branch rather than adding a code
path — the msgpack transport itself is unchanged from #16126, it is just no
longer gated.

Existing coverage now exercises the msgpack path by default:

• tests/unittest/disaggregated/test_disagg_openai_client.py (registered in
  l0_a10.yml) covers the orchestrator send/retry/streaming paths, which now
  always serialize via msgpack.
• Disaggregated integration tests exercise the orchestrator→worker hop
  end-to-end; the previously opt-in path is the one they now run.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[Tabrizian]

/bot run --disable-fail-fast

[coderabbitai]

Walkthrough

The orchestrator-to-worker relay now always uses msgspec MessagePack encoding and decoding. The environment-variable gate was removed, FastAPI routing is always wired to the custom request handler, unmarked requests retain JSON decoding, and documentation describes the internal transport as the default.

Changes

Orchestrator-to-worker MessagePack transport

Layer / File(s)	Summary
Client MessagePack request encoding tensorrt_llm/serve/openai_client.py	The client unconditionally encodes forwarded request bodies with MessagePack and sends the X-TRTLLM-Msgpack: 1 header while retaining Content-Type: application/json.
Server MessagePack decoding and route wiring tensorrt_llm/serve/openai_server.py, docs/source/blogs/tech_blog/...	The server unconditionally installs _MsgspecRoute, decodes marked bodies with MessagePack, falls back to standard JSON for unmarked bodies, and documents the relay transport as the default internal mechanism.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: aswinvisva

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly states the main change: making msgspec msgpack transport unconditional for disaggregated orchestrator-to-worker bodies.
Description check	✅ Passed	The PR description includes Description, Test Coverage, and PR Checklist sections and explains the change and rationale.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@docs/source/blogs/tech_blog/blog26_DeepSeek_V4_on_NVIDIA_Blackwell_Model_Specific_and_Agentic_Workload_Optimizations_in_TensorRT-LLM.md`:
- Line 431: Update the relay serialization description near the “msgspec”
transport claim to remove the assertion that request-body processing is
eliminated entirely. State only that the internal path eliminates JSON encoding
and parsing, while preserving the surrounding contrast with the external OpenAI
API.

In `@tensorrt_llm/serve/openai_server.py`:
- Line 125: Add explicit return annotations to the new async json method and the
adjacent request/route hook functions around it, using their actual concrete
return types and annotating non-returning hooks with None; do not use Any.

In `@tests/unittest/disaggregated/test_disagg_openai_client.py`:
- Around line 149-176: The existing test_request_body_is_msgpack coverage only
validates the outgoing client payload; add server-side tests covering
_MsgspecRequest and _MsgspecRoute. Verify requests marked with X-TRTLLM-Msgpack
are decoded from msgpack, while requests without that marker use the plain JSON
fallback, preserving the expected parsed request behavior in both paths.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 51fc0f40-730b-4841-b80a-7842848d584a

📥 Commits

Reviewing files that changed from the base of the PR and between cf44a1c and 53b5954.

📒 Files selected for processing (4)

• docs/source/blogs/tech_blog/blog26_DeepSeek_V4_on_NVIDIA_Blackwell_Model_Specific_and_Agentic_Workload_Optimizations_in_TensorRT-LLM.md
• tensorrt_llm/serve/openai_client.py
• tensorrt_llm/serve/openai_server.py
• tests/unittest/disaggregated/test_disagg_openai_client.py

[tensorrt-cicd]

PR_Github #61709 [ run ] triggered by Bot. Commit: 53b5954 Link to invocation

[tensorrt-cicd]

PR_Github #61709 [ run ] completed with state FAILURE. Commit: 53b5954
/LLM/main/L0_MergeRequest_PR pipeline #49914 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation