Skip to content

Fix ZeRO-3 all_reduce param fetch stride for padded parameters#8158

Open
ebarkhordar wants to merge 2 commits into
deepspeedai:masterfrom
ebarkhordar:fix/zero3-allreduce-fetch-stride-aligned
Open

Fix ZeRO-3 all_reduce param fetch stride for padded parameters#8158
ebarkhordar wants to merge 2 commits into
deepspeedai:masterfrom
ebarkhordar:fix/zero3-allreduce-fetch-stride-aligned