Skip to content

fix mtp cuda graph init.#1251

Merged
hiworldwzj merged 1 commit intomainfrom
hiworldwzj-patch-1
Apr 2, 2026
Merged

fix mtp cuda graph init.#1251
hiworldwzj merged 1 commit intomainfrom
hiworldwzj-patch-1

Conversation

@hiworldwzj
Copy link
Copy Markdown
Collaborator

No description provided.

@hiworldwzj hiworldwzj merged commit 0fce8c9 into main Apr 2, 2026
1 check passed
@hiworldwzj hiworldwzj deleted the hiworldwzj-patch-1 branch April 2, 2026 07:50
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request corrects the batch size calculation logic in cuda_graph.py by ensuring the mtp_step multiplier is applied correctly within the range. A review comment suggests using a more idiomatic range with a step to improve readability and reuse existing variables.

graph_grow_step_size = self.args.graph_grow_step_size * (self.mtp_step + 1)

batch_sizes = [i * (self.mtp_step + 1) for i in range(1, graph_split_batch_size + 1)]
batch_sizes = [i * (self.mtp_step + 1) for i in range(1, self.args.graph_split_batch_size + 1)]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While this line correctly fixes the bug, its logic can be expressed more directly and readably. You are generating a sequence of batch sizes that are multiples of (self.mtp_step + 1), up to graph_split_batch_size. This can be achieved more concisely using range() with a step. This approach also has the benefit of reusing the graph_split_batch_size variable defined on line 33, improving code clarity.

Suggested change
batch_sizes = [i * (self.mtp_step + 1) for i in range(1, self.args.graph_split_batch_size + 1)]
batch_sizes = list(range(self.mtp_step + 1, graph_split_batch_size + 1, self.mtp_step + 1))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant