Skip to content

Commit c3c0abf

Browse files
tohtanastas00
andauthored
Update training/bf16_master_weight/train.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
1 parent 2db167f commit c3c0abf

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

training/bf16_master_weight/train.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -292,7 +292,7 @@ def main():
292292
input_ids = torch.randint(0, actual_vocab_size, (args.batch_size, args.seq_length), device=device)
293293
labels = torch.randint(0, actual_vocab_size, (args.batch_size, args.seq_length), device=device)
294294

295-
# Forward pass with optional autocast.
295+
# Forward pass with an optional autocast.
296296
# DeepSpeed already applies torch.autocast inside engine.forward(), but
297297
# we wrap the entire forward+loss block so that loss_fn also runs under
298298
# autocast. The nested autocast on engine.forward() is harmless —

0 commit comments

Comments
 (0)