Skip to content

bugfix: set up kv cache compleition guard to prevent kv cache transfer error.#1925

Open
phantomlei3 wants to merge 1 commit into
xLLM-AI:mainfrom
phantomlei3:bugfix/kv-cache-transfer-error
Open

bugfix: set up kv cache compleition guard to prevent kv cache transfer error.#1925
phantomlei3 wants to merge 1 commit into
xLLM-AI:mainfrom
phantomlei3:bugfix/kv-cache-transfer-error

Conversation

@phantomlei3

@phantomlei3 phantomlei3 commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Description

In PD serving mode, Mooncake data transfer failures were not consistently propagated to the upper layers. Some worker exit paths could return
before signaling task completion, leaving the request in an incomplete state and potentially causing the scheduler to wait indefinitely.

Fix Approach

Introduce a completion guard to ensure that task completion is reported on every worker exit path, including early returns and failure branches.
Additionally, propagate Mooncake transfer errors consistently through the call chain so that failed transfers are detected and handled by the
scheduler instead of being silently ignored.

Change Type

  • Bug fix
  • New feature
  • Performance improvement
  • Refactor
  • Documentation
  • Test
  • Build or CI

Pull Request Checklist

Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.

PR Title and Commit Messages

  • The PR title and each commit message follow the xLLM commit format: <type>: <subject>.

Allowed types: feat, bugfix, docs, test, refactor, chore, style, revert, perf, model, build, release.
The subject should use clear English, start with a verb, include at least 4 words, and end with ..

Pre-commit Checks

  • I have installed pre-commit by running pip install pre-commit or an equivalent command.
  • I have installed the hooks with pre-commit install.
  • I have run pre-commit run --all-files and fixed any reported issues.

If you are unsure how to set up pre-commit, see the pre-commit documentation.

Self Review

  • I have self-reviewed the code according to .agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.
  • I have rebased this PR onto the latest main branch.

Build and Test Coverage

  • Tests have been added or updated as needed.
  • CUDA: python setup.py build test has passed on a CUDA machine.
  • NPU: python setup.py build test has passed on an NPU machine.
  • MLU: python setup.py build test has passed on an MLU machine.

Reviewer Notes

@phantomlei3 phantomlei3 force-pushed the bugfix/kv-cache-transfer-error branch from 94bb867 to 278f0a0 Compare July 11, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants