Skip to content

[Feat]: support multistream for CAMP2p - #161

Open
yujuancao07 wants to merge 1 commit into
vllm-project:mainfrom
yujuancao07:multistream
Open

[Feat]: support multistream for CAMP2p#161
yujuancao07 wants to merge 1 commit into
vllm-project:mainfrom
yujuancao07:multistream

Conversation

@yujuancao07

Copy link
Copy Markdown
Contributor

Purpose

Add configurable multi-stream execution to the Ascend CAMP2P connector
so that A2F/F2A communication can overlap with model computation.

Attention-side and FFN-side multi-stream execution can be enabled independently
through connector_extra_config.

Issue

Scope

This feature applies only to CAMP2pAFDConnector on NPU.

  • In scope:

    • Attention-side CAMP2P communication stream execution.
    • FFN-side CAMP2P communication stream execution.
    • Independent Attention and FFN configuration switches.
    • Independent Attention and FFN AIV core-count configuration.
    • Eager execution and the existing AFD NPU ubatch/DBO path.
    • Existing ACL graph paths where stream switching can be represented safely by
      the current graph runtime.
  • Out of scope:

    • CAMAsyncAFDConnector.
    • GPU P2pNcclAFDConnector.

Implementation Notes

  • The implementation applies only to CAMP2pAFDConnector.

  • Both switches are located under CAMP2P connector_extra_config:

    {
      "connector_extra_config": {
        "is_attn_multistream": true,
        "is_ffn_multistream": true,
        "attn_core_num": 8,
        "ffn_core_num": 8
      }
    }
    

Test Plan

Focused multi-stream tests:
pytest -q tests/unit -k multistream -vv

Test Result

image

@yujuancao07

Copy link
Copy Markdown
Contributor Author

multistream:
image
sigle stream:
image

@mergify

mergify Bot commented Jul 27, 2026

Copy link
Copy Markdown

⚠️ The sha of the head commit of this PR conflicts with #146. Mergify cannot evaluate rules on this PR. Once #146 is merged or closed, Mergify will resume processing this PR. ⚠️

Signed-off-by: yujuancao07 <yujuancao07@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Enable multi-stream parallelism for CAMP2P A2E/E2A on FFN side

1 participant