Skip to content

[megatron] support megatron fp4#9330

Merged
Jintao-Huang merged 7 commits into
modelscope:mainfrom
Jintao-Huang:support_megatron_fp4
May 19, 2026
Merged

[megatron] support megatron fp4#9330
Jintao-Huang merged 7 commits into
modelscope:mainfrom
Jintao-Huang:support_megatron_fp4

Conversation

@Jintao-Huang
Copy link
Copy Markdown
Collaborator

No description provided.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces FP4 precision parameters (fp4_format, fp4_recipe, and fp4_param_gather) to the MegatronArguments class and updates the documentation. Reviewers identified typos in the documentation where 'FP8' was used instead of 'FP4' and suggested formatting improvements in the English version to ensure consistency with the FP8 section.

Comment thread docs/source/Megatron-SWIFT/Command-line-parameters.md Outdated
Comment thread docs/source_en/Megatron-SWIFT/Command-line-parameters.md Outdated
@Jintao-Huang
Copy link
Copy Markdown
Collaborator Author

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for FP4 precision by adding new command-line parameters (fp4_format, fp4_recipe, fp4_param_gather) and updating the argument parsing and utility functions. Reviewers suggested increasing the padding alignment for FP4 to 32 elements to ensure compatibility with Blackwell+ architecture requirements and recommended explicitly stating default values in the documentation for both Chinese and English versions.

Comment thread swift/megatron/utils/utils.py
Comment thread docs/source/Megatron-SWIFT/Command-line-parameters.md Outdated
Comment thread docs/source_en/Megatron-SWIFT/Command-line-parameters.md Outdated
@Jintao-Huang Jintao-Huang merged commit 3e609a5 into modelscope:main May 19, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants