Repository navigation

[`distributed`] Default MoE `ep_plans` to token dispatch - #49160

Merged

3outeille merged 119 commits into

mainfrom

ep-dispatch-default

Oct 6, 2026

Merged

[`distributed`] Default MoE `ep_plans` to token dispatch#49160
3outeille merged 119 commits into
mainfrom
ep-dispatch-default

Commits on Oct 1, 2026
- Add dense and expert device mesh views with a MeshManager
  3outeille
  committed
  eecb6a8
- cleaning
  3outeille
  committed
  29f5126
- clean
  3outeille
  committed
  4074526
- update
  3outeille
  committed
  578af0e
- clean doc
  3outeille
  committed
  083e253
- Use ep_size in the Kimi K2.5 expert parallel example
  3outeille
  committed
  2e68296
- Enable expert parallelism in the Mega MoE example
  3outeille
  committed
  29eb5a3
- warn once
  3outeille
  committed
  d6090df
- Add dense and expert device mesh views with a MeshManager
  3outeille
  committed
  76ff6f0
- cleaning
  3outeille
  committed
  9a1b525
- update
  3outeille
  committed
  ce8657f
- clean doc
  3outeille
  committed
  ea3aacb
- Update src/transformers/distributed/mixin.py
  
  3outeille
  and
  ArthurZucker
  committed
  d498e80
- style: ruff import block formatting in distributed config
  3outeille
  committed
  ce3f576
- add more comments
  3outeille
  committed
  f8415e7
- Decouple tp_plan and ep_plan for expert parallelism.
  3outeille
  committed
  a115da8
- Match expert paths with a regex and keep only expert rules under token dispatch
  3outeille
  committed
  271de16
- revert change on warning
  3outeille
  committed
  cc5200f
- Apply suggestion from @ArthurZucker
  
  3outeille
  and
  ArthurZucker
  committed
  17f7b65
- cleaning
  3outeille
  committed
  ef83726
- Fix tied embeddings for models with an EP-only base plan
  3outeille
  committed
  e7bfac9
- [distributed] Add expert-parallel token dispatch, default for Qwen3 MoE
  3outeille
  committed
  4cc84e4
- clean
  3outeille
  committed
  b756980
- fix
  3outeille
  committed
  4d5ef4c
- make check repo
  3outeille
  committed
  556f4b9
- Drop the load-time ep_size == tp_size guard that rejects token dispatch
  3outeille
  committed
  59578e0
- remove all to all warning by using functional version
  3outeille
  committed
  16fbcca
- use ep plan instead of group_gemm is_expert
  3outeille
  committed
  60fd082
- use dtensor instead of manuall all_reduce
  3outeille
  committed
  fc42fb9
- better comment
  3outeille
  committed
  1478d28
- [`distributed`]: Fix DeepSeek-V4 tied embeddings with combined TP and EP (#48824)
  3outeille
  committed
  51472f9
- linting
  3outeille
  committed
  ea814eb
Commits on Oct 5, 2026
- Add dense and expert device mesh views with a MeshManager
  3outeille
  authored
  925415d
- cleaning
  3outeille
  authored
  0c227f1
- clean
  3outeille
  authored
  a5958d4
- update
  3outeille
  authored
  6571916
- clean doc
  3outeille
  authored
  90fbd09
- Use ep_size in the Kimi K2.5 expert parallel example
  3outeille
  authored
  e4561f0
- Enable expert parallelism in the Mega MoE example
  3outeille
  authored
  7f4951f
- warn once
  3outeille
  authored
  e65e3ea
- Add dense and expert device mesh views with a MeshManager
  3outeille
  authored
  03af9e8
- cleaning
  3outeille
  authored
  69a145b
- update
  3outeille
  authored
  caa7640
- clean doc
  3outeille
  authored
  a3de4dc
- Update src/transformers/distributed/mixin.py
  
  3outeille
  and
  ArthurZucker
  authored
  84cb4be
- style: ruff import block formatting in distributed config
  3outeille
  authored
  b552195
- add more comments
  3outeille
  authored
  d1cf207
- Decouple tp_plan and ep_plan for expert parallelism.
  3outeille
  authored
  4b655fb
- Match expert paths with a regex and keep only expert rules under token dispatch
  3outeille
  authored
  a4f11e3
- revert change on warning
  3outeille
  authored
  a5cd182
- Apply suggestion from @ArthurZucker
  
  3outeille
  and
  ArthurZucker
  authored
  3d45692
- cleaning
  3outeille
  authored
  04752eb
- Fix tied embeddings for models with an EP-only base plan
  3outeille
  authored
  c41435b
- [distributed] Add expert-parallel token dispatch, default for Qwen3 MoE
  3outeille
  authored
  8dc8333
- clean
  3outeille
  authored
  865d5d9
- fix
  3outeille
  authored
  cac1b2b
- make check repo
  3outeille
  authored
  c59446d
- Drop the load-time ep_size == tp_size guard that rejects token dispatch
  3outeille
  authored
  abf336c
- remove all to all warning by using functional version
  3outeille
  authored
  6244a1c
- use ep plan instead of group_gemm is_expert
  3outeille
  authored
  c72bbf7
- use dtensor instead of manuall all_reduce
  3outeille
  authored
  c31a1b7
- better comment
  3outeille
  authored
  00ee316
- linting
  3outeille
  authored
  ec8278e
- Train models sharded at load time with the Trainer, token dispatch included
  3outeille
  authored
  39c7e41
- Use ep-dispatch version of expert_parallelism.md
  3outeille
  authored
  dfec8d1
- Sync replicated trainable parameters when training a model sharded at load time
  3outeille
  authored
  628da80
- Fix utf-8 encoding in expert parallel trainer tests
  3outeille
  authored
  e72d719
- [distributed] Default MoE expert-parallel plans to token dispatch
  3outeille
  authored
  e9bcdc5
- remove
  3outeille
  authored
  4497d74
- fix ruff
  3outeille
  authored
  9b41975
- use attribute instead
  3outeille
  authored
  69ecbf1
- Fix EP plans for youtu (dense, drop inherited plan) and zaya (token dispatch)
  3outeille
  authored
  0cb580c
- fix
  3outeille
  committed
  bcbb851
- Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/transformers into ep-mesh-infrastructure
  3outeille
  committed
  d51aca8
- Merge branch 'ep-plans' of github.com:huggingface/transformers into ep-plans
  3outeille
  committed
  448315e
- revert
  3outeille
  committed
  877afee
- Merge branch 'ep-dispatch' of github.com:huggingface/transformers into ep-dispatch
  3outeille
  committed
  bec5a12
- Merge branch 'main' into ep-mesh-infrastructure
  3outeille
  authored
  0ec824f
- Merge branch 'ep-plans' into ep-dispatch
  3outeille
  committed
  b20486c
- Merge branch 'ep-mesh-infrastructure' into ep-plans
  3outeille
  authored
  c85c8bc
- revert
  3outeille
  committed
  0d8d4f5
- Merge branch 'ep-dispatch' into ep-trainer
  3outeille
  authored
  a7a37be
- Merge branch 'ep-plans' into ep-dispatch
  3outeille
  authored
  c9d6429
- typo
  3outeille
  committed
  cedde47
- Merge branch 'main' into ep-mesh-infrastructure
  3outeille
  authored
  b62bbbc
- clearer
  3outeille
  committed
  6a23a30
- Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/transformers into ep-mesh-infrastructure
  3outeille
  committed
  daac87a
- Merge branch 'main' into ep-mesh-infrastructure
  3outeille
  authored
  6c73628
Commits on Oct 6, 2026
- clean
  3outeille
  committed
  b4d81b5
- Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/transformers into ep-mesh-infrastructure
  3outeille
  committed
  e3cc578
- Merge branch 'ep-mesh-infrastructure' into ep-plans
  3outeille
  authored
  9bb2518
- Merge branch 'ep-plans' into ep-dispatch
  3outeille
  authored
  38704ae
- cleaning
  3outeille
  committed
  7b43249
- cleaning
  3outeille
  committed
  e0f7012
- Merge branch 'ep-plans' of github.com:huggingface/transformers into ep-plans
  3outeille
  committed
  926ee70
- Merge branch 'main' into ep-mesh-infrastructure
  3outeille
  authored
  746b2c5
- Merge branch 'ep-mesh-infrastructure' into ep-plans
  3outeille
  authored
  0eac0f5
- Merge branch 'ep-plans' into ep-dispatch
  3outeille
  authored
  030900b
- dont torch cat for empty tokens
  3outeille
  committed
  af79397
- Merge branch 'ep-dispatch' of github.com:huggingface/transformers into ep-dispatch
  3outeille
  committed
  43a8c2d
- Merge branch 'ep-dispatch' into ep-trainer
  3outeille
  authored
  37089c1
- Merge branch 'ep-trainer' into ep-dispatch-default
  3outeille
  authored
  40c20a4
- check code quality
  3outeille
  committed
  865aa96
- Update src/transformers/distributed/utils.py
  
  3outeille
  and
  ArthurZucker
  authored
  7b57dd2
- Merge branch 'main' into ep-mesh-infrastructure
  3outeille
  authored
  ec77a59
- Update src/transformers/distributed/configuration_utils.py
  
  3outeille
  and
  ArthurZucker
  authored
  7af3cbc
- linting
  3outeille
  committed
  f37bc0f
- Merge branch 'ep-mesh-infrastructure' of github.com:huggingface/transformers into ep-mesh-infrastructure
  3outeille
  committed
  11f0a3d
- fix comment
  3outeille
  committed
  a4d0693
- Merge branch 'ep-mesh-infrastructure' into ep-plans
  3outeille
  authored
  6a504c1
- Merge branch 'main' into ep-mesh-infrastructure
  3outeille
  authored
  812c060
- Merge branch 'ep-plans' into ep-dispatch
  3outeille
  authored
  13d2fc7
- Merge branch 'ep-mesh-infrastructure' into ep-plans
  3outeille
  authored
  430bb2e
- Merge branch 'ep-dispatch' into ep-trainer
  3outeille
  authored
  e431d5c
- Merge branch 'ep-trainer' into ep-dispatch-default
  3outeille
  authored
  50d2f6b
- Merge branch 'ep-plans' into ep-dispatch
  3outeille
  authored
  42d0607
- Merge branch 'ep-dispatch' into ep-trainer
  3outeille
  authored
  b0fd271
- Merge branch 'ep-trainer' into ep-dispatch-default
  3outeille
  authored
  3186819
- Merge branch 'main' into ep-dispatch-default
  3outeille
  authored
  114cc5c

Commits

Commits on Oct 1, 2026

Commits on Oct 5, 2026

Commits on Oct 6, 2026