Release notes from Megatron-LM
Articles about Release notes from Megatron-LM
Release 26.09-alpha.rc2 · NVIDIA/Megatron-LM
Ongoing research training transformer models at scale - Release 26.09-alpha.rc2 · NVIDIA/Megatron-LM
Release 26.09-beta.rc1 · NVIDIA/Megatron-LM
Ongoing research training transformer models at scale - Release 26.09-beta.rc1 · NVIDIA/Megatron-LM
Release 26.09-alpha.rc1 · NVIDIA/Megatron-LM
perf(moe): make paged-stash copy/pop kernel launch configurable (#7897) The paged-stash copy/pop Triton kernels always launched with 1024 elements per program per inner-loop iteration and at most 2048 programs. For wide activations this leaves DRAM bandwidth unused: on VR200 an fp8 [32768, 7168] stash reaches ~3.8 TB/s, while a plain copy_ reaches ~9.9 TB/s. Add TransformerConfig options moe_paged_stash_copy_block_size (power of two, default 1024) and moe_paged_stash_copy_max_blocks (default 2048),
NVIDIA Megatron Core 0.19.2
TransformerEngine upgraded from 2.17 to 2.18 Added support for FP4 in 1F1B A2A overlap (#6135) Changelog Details ci: Revert version update (0.19.1) by @balasaajay :: PR: #6681 beep boop 🤖: Bumping Megatron Core to v0.19.1 by @nvidia-megatron-lm-release-bot[bot] :: PR: #6691 cp: treat svcnemo-autobot as internal by