Skip to content

Commit 0a7e103

Browse files
committed
update readme and license
1 parent 553e3f7 commit 0a7e103

2 files changed

Lines changed: 15 additions & 0 deletions

File tree

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,6 +338,7 @@ There are two extra options: `--forward-backward-disaggregating` and `--ignore-f
338338
339339
Enables merging forward ranks within the same TP group. After doing this, your number of ranks will be multiplied by $\frac{TP+1}{2TP}$. Be sure you are using the correct number of ranks.
340340
341+
Currently Context Parallel and Expert parallel are not supported. `--tranformer-impl` should be `local`.
341342
342343
# 🛠️ Security Policy
343344

megatron/virtual_tensor_parallel_communication.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,17 @@
1+
# Copyright 2025 Suanzhi Future Co., Ltd.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions
13+
# and limitations under the License.
14+
115
import torch
216
import threading
317
import torch.distributed as dist

0 commit comments

Comments
 (0)