Repository navigation

[Umbrella] FSDP2 + expert-parallel token dispatch, with sharded-loading prefetch - #48204

Closed

qgallouedec wants to merge 316 commits into

mainfrom

ep-fsdp-2d-mesh

Closed

[Umbrella] FSDP2 + expert-parallel token dispatch, with sharded-loading prefetch#48204
qgallouedec wants to merge 316 commits into
mainfrom
ep-fsdp-2d-mesh

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 3, 2026
Commits on Sep 4, 2026
Commits on Sep 5, 2026
- Merge branch 'main' into moe-skip-ep-sentinels-v2
  qgallouedec
  authored
  9987b90
- Merge branch 'main' into load-shard-prefetch
  qgallouedec
  authored
  1bd524a
- Merge branch 'main' into fix-empty-fsdp-shards
  qgallouedec
  authored
  8aa5433
Commits on Sep 7, 2026
Commits on Sep 8, 2026
Commits on Sep 9, 2026
- Keep replicated_with_grad_allreduce under expert-parallel dispatch
  qgallouedec
  committed
  0817429
- Compare gradients, not only the loss, in the EP backward test
  qgallouedec
  committed
  34e63c9
- Merge branch 'pr48518' into umb48204
  qgallouedec
  committed
  caac0ad
- Merge branch 'main' into ep-fsdp-2d-mesh
  qgallouedec
  authored
  fccd0e7
Commits on Sep 10, 2026
- Merge branch 'main' into load-convert-before-shard
  qgallouedec
  authored
  5892c33
- Merge branch 'main' into fix-empty-fsdp-shards
  qgallouedec
  authored
  7b09389
- Warm the page cache before sharded loading (opt-in via HF_SHARD_PREFETCH)
  qgallouedec
  authored
  153a399
- style
  qgallouedec
  authored
  cf674aa
- Move the shard prefetch into distributed/utils.py
  qgallouedec
  authored
  77ad02d
- Apply ruff format to the prefetch helper
  qgallouedec
  authored
  70c6ac4
- Bind the prefetch barrier to the accelerator
  qgallouedec
  authored
  f70364f
- Merge branch 'main' into moe-skip-ep-sentinels-v2
  qgallouedec
  authored
  e3e2d58
- Update docs/source/en/expert_parallelism.md
  
  qgallouedec
  and
  stevhliu
  authored
  75fff71
- Update docs/source/en/expert_parallelism.md
  
  qgallouedec
  and
  stevhliu
  authored
  acba6bc
- Update docs/source/en/expert_parallelism.md
  
  qgallouedec
  and
  stevhliu
  authored
  69668c3
- Update docs/source/en/expert_parallelism.md
  
  qgallouedec
  and
  stevhliu
  authored
  9cfb2d3
- Merge branch 'main' into fsdp2-ep-2d-mesh
  qgallouedec
  authored
  2884e2f
- Merge branch 'main' into qwen3_vl_moe_tp_plan
  qgallouedec
  authored
  86fc394
- Merge branch 'fsdp2-ep-2d-mesh' into ep-token-dispatch
  qgallouedec
  authored
  4ce4700
- Merge remote-tracking branch 'upstream/load-shard-prefetch' into prefetch-rank-spans
  qgallouedec
  committed
  a687e4d
- Flag the empty shard in set_param_for_module
  qgallouedec
  committed
  fee6845
- Only build the sharding op when it is used
  qgallouedec
  committed
  cb61b1a
- Declare dispatch_experts_forward next to its only caller
  qgallouedec
  committed
  28cb482
- Merge branch 'main' into fix-empty-fsdp-shards
  qgallouedec
  authored
  f22f18a
- Let each op say whether it breaks shard-on-read
  qgallouedec
  committed
  071db77
- Merge branch 'main' into load-convert-before-shard
  qgallouedec
  authored
  5fa9a03
- Warm the page cache before sharded loading (opt-in via HF_SHARD_PREFETCH)
  qgallouedec
  authored
  11bca8c
- style
  qgallouedec
  authored
  04bedbe
- Move the shard prefetch into distributed/utils.py
  qgallouedec
  authored
  8c873dc
- Apply ruff format to the prefetch helper
  qgallouedec
  authored
  2a42ab8
- Bind the prefetch barrier to the accelerator
  qgallouedec
  authored
  31a256b
- Name the dispatch strategy instead of switching it with a bool
  qgallouedec
  committed
  a6e36c0
- Merge branch 'main' into moe-skip-ep-sentinels-v2
  qgallouedec
  authored
  70bd69e
- Merge remote-tracking branch 'upstream/load-shard-prefetch' into umb48204
  qgallouedec
  committed
  ad3b02f
- Merge remote-tracking branch 'upstream/prefetch-rank-spans' into umb48204
  qgallouedec
  committed
  9505dee
- Merge remote-tracking branch 'upstream/moe-skip-ep-sentinels-v2' into umb48204
  qgallouedec
  committed
  bbf7397
- Merge remote-tracking branch 'upstream/fix-empty-fsdp-shards' into umb48204
  qgallouedec
  committed
  18ef7f7
- Merge remote-tracking branch 'upstream/fsdp2-ep-2d-mesh' into umb48204
  qgallouedec
  committed
  6a20752
- Merge remote-tracking branch 'upstream/ep-token-dispatch' into umb48204
  qgallouedec
  committed
  7683ddd
- Merge remote-tracking branch 'upstream/load-convert-before-shard' into umb48204
  qgallouedec
  committed
  b669fc2
- Warm the page cache before sharded loading (opt-in via HF_SHARD_PREFETCH)
  qgallouedec
  authored
  6e40b0b
- style
  qgallouedec
  authored
  9a61e7b
- Move the shard prefetch into distributed/utils.py
  qgallouedec
  authored
  e8b13c2
- Apply ruff format to the prefetch helper
  qgallouedec
  authored
  eed24d1
- Bind the prefetch barrier to the accelerator
  qgallouedec
  authored
  a98cde2
- Merge branch 'main' into fix-empty-fsdp-shards
  qgallouedec
  authored
  758ed92
- Merge branch 'main' into fsdp2-ep-2d-mesh
  qgallouedec
  authored
  1be3a7f
- Merge branch 'main' into load-convert-before-shard
  qgallouedec
  authored
  10f6088
- Merge remote-tracking branch 'upstream/load-shard-prefetch' into prefetch-rank-spans
  qgallouedec
  committed
  62e85ea
- Warm the page cache before sharded loading (opt-in via HF_SHARD_PREFETCH)
  qgallouedec
  authored
  72548d0
- style
  qgallouedec
  authored
  dd8804b
- Move the shard prefetch into distributed/utils.py
  qgallouedec
  authored
  acae96c
- Apply ruff format to the prefetch helper
  qgallouedec
  authored
  7db2467
- Bind the prefetch barrier to the accelerator
  qgallouedec
  authored
  68bc14b
- Merge remote-tracking branch 'upstream/load-shard-prefetch' into prefetch-rank-spans
  qgallouedec
  committed
  e514c7f
- Merge branch 'main' into load-convert-before-shard
  qgallouedec
  authored
  14fe033
- Merge remote-tracking branch 'upstream/fsdp2-ep-2d-mesh' into ep-token-dispatch
  qgallouedec
  committed
  db78860
- Merge branch 'main' into fsdp2-ep-2d-mesh
  qgallouedec
  authored
  a6f5824
- Warm the page cache before sharded loading (opt-in via HF_SHARD_PREFETCH)
  qgallouedec
  authored
  5489cde
- style
  qgallouedec
  authored
  0b5d440
- Move the shard prefetch into distributed/utils.py
  qgallouedec
  authored
  170161c
- Apply ruff format to the prefetch helper
  qgallouedec
  authored
  2f54f17
- Bind the prefetch barrier to the accelerator
  qgallouedec
  authored
  a1846bc
- Merge branch 'main' into fix-empty-fsdp-shards
  qgallouedec
  authored
  d918357
- Merge branch 'fsdp2-ep-2d-mesh' into ep-token-dispatch
  qgallouedec
  authored
  b56e1d6
- Merge branch 'main' into load-convert-before-shard
  qgallouedec
  authored
  65b6e40
- Merge branch 'main' into qwen3_vl_moe_tp_plan
  qgallouedec
  authored
  6fbcb30
- Merge remote-tracking branch 'upstream/main' into umb48204
  qgallouedec
  committed
  dc7143f
- Merge remote-tracking branch 'upstream/load-shard-prefetch' into umb48204
  qgallouedec
  committed
  a58566d
- Merge remote-tracking branch 'upstream/prefetch-rank-spans' into umb48204
  qgallouedec
  committed
  56699c8
- Merge remote-tracking branch 'upstream/fsdp2-ep-2d-mesh' into umb48204
  qgallouedec
  committed
  fef0435
- Merge remote-tracking branch 'upstream/ep-token-dispatch' into umb48204
  qgallouedec
  committed
  21e1276
- Merge remote-tracking branch 'upstream/fix-empty-fsdp-shards' into umb48204
  qgallouedec
  committed
  cfc7f17
- Merge branch 'pr48373-head' into umb48204
  qgallouedec
  committed
  d12ff1f
- Merge remote-tracking branch 'upstream/load-convert-before-shard' into umb48204
  qgallouedec
  committed
  2a7a88d
Commits on Sep 11, 2026
Commits on Sep 12, 2026
- Merge remote-tracking branch 'upstream/main' into fsdp2-ep-2d-mesh
  qgallouedec
  committed
  79988c3
- Drop the comment on the parallel sizes
  qgallouedec
  committed
  5120748
- Trim the parallelism config comment
  qgallouedec
  committed
  0147840
- Warm the page cache before sharded loading (opt-in via HF_SHARD_PREFETCH)
  qgallouedec
  authored
  b174499
- style
  qgallouedec
  authored
  48ce8e2
- Move the shard prefetch into distributed/utils.py
  qgallouedec
  authored
  3e51ffb
- Apply ruff format to the prefetch helper
  qgallouedec
  authored
  ef758c4
- Bind the prefetch barrier to the accelerator
  qgallouedec
  authored
  cfdc8ba
- Merge remote-tracking branch 'upstream/fsdp2-ep-2d-mesh' into ep-token-dispatch
  qgallouedec
  committed
  d604a2a
- Merge remote-tracking branch 'upstream/main' into ep-token-dispatch
  qgallouedec
  committed
  b169a75
Commits on Sep 13, 2026
Commits on Sep 14, 2026
- Separate expert dispatch mesh from tensor parallel size
  3outeille
  committed
  df8fa77
- Support trunk tensor parallelism with folded expert dispatch
  3outeille
  committed
  60655f8
- rm config test
  3outeille
  committed
  d775340
- Simplify expert dispatch and rename edp to efsdp
  3outeille
  committed
  a49aa6d
- Merge branch 'ep-token-dispatch' into ep-mesh
  3outeille
  authored
  b1fdbdf
- Merge origin/ep-mesh and restore expert-parallel mesh support
  3outeille
  committed
  6d8f54d
- Unify dense and expert mesh construction
  3outeille
  committed
  ea65cfb
- remove
  3outeille
  committed
  b33e45e
- Encapsulate distributed mesh views behind named accessors
  3outeille
  committed
  0eb9ffa
- Simplify MeshManager access and keep mesh views private
  3outeille
  committed
  4d63a32
- Simplify expert parallel configuration and explicit plan selection
  3outeille
  committed
  d4da6e0
- Resolve TP and EP plans before sharding and derive expert dispatch
  3outeille
  committed
  6a1059e
- Merge branch 'main' into ep-token-dispatch
  3outeille
  authored
  56047f4
- Merge branch 'ep-token-dispatch' into ep-mesh
  3outeille
  authored
  0071737
- Inline legacy expert parallel compatibility handling
  3outeille
  committed
  c8a05bc
- Simplify parallel plan resolution and clarify TP and EP meshes
  3outeille
  committed
  cc2237c
- remove
  3outeille
  committed
  fc37645
- Remove auto EP plans and clarify parallel plan tests
  3outeille
  committed
  251b2b9
- Clarify expert parallel docs and explicit sharding mesh selection
  3outeille
  committed
  8c2588d
- Always wrap dispatched experts on the expert FSDP mesh
  3outeille
  committed
  1911858
- Clarify TP and EP layouts in FSDP comparison test
  3outeille
  committed
  49f432c
- Clarify all-reduce and dispatch layouts in EP tests
  3outeille
  committed
  6d23125
- Take TP and EP meshes in apply_tensor_parallelism
  
  3outeille
  and
  claude
  committed
  e1d33d2
Commits on Sep 15, 2026
Commits on Sep 19, 2026

Commits

Commits on Sep 3, 2026

Commits on Sep 4, 2026

Commits on Sep 5, 2026

Commits on Sep 7, 2026

Commits on Sep 8, 2026

Commits on Sep 9, 2026

Commits on Sep 10, 2026