Fix expert-parallel training: NaN gradients and missing gradient contributions - #48205

Fix expert-parallel training: NaN gradients and missing gradient contributions#48205

qgallouedec merged 5 commits into

mainhuggingface/transformers:mainfrom

fix-ep-training-gradientshuggingface/transformers:fix-ep-training-gradientsCopy head branch name to clipboard