All pull requests

Pull requests

• feat(sticky_least_loaded): opt-in bounded overflow for pinned sessions

  #540·raviguptaamd opened Nov 8, 2026·3 of 4 tasks·

  ·
  4

• [Feature] Opt-in per-worker in-flight admission capacity (#435)

  #539·mohamad-alakkad opened Nov 7, 2026·3 of 5 tasks·

  ·

• fix(cache_aware): preserve agent prefix affinity when HTTP render is unavailable

  #538·mohamad-alakkad opened Nov 7, 2026·3 tasks done·

  ·
  10

• ci: skip thin LTO in premerge release builds

  #537·amitcoder79 opened Nov 7, 2026·

  ·
  8

• [Test] Cover scheduling policies under the PD merge deployment

  #536·yenuo26 opened Nov 6, 2026Contributor·2 tasks done·

  ·
  9

• fix(pd): select the Program prefill target once, with request inputs

  #535·LiangTengwen opened Nov 6, 2026Contributor·

  ·
  1

  9

• fix(kv-aware): break equal-score ties in P/D and colocated routing

  enhancementready
  #534·KaisennHu opened Nov 6, 2026Collaborator·3 tasks done·

  ·
  1

  2

• feat: observe KV prediction accuracy

  #533·Kkhhh-star opened Nov 6, 2026·3 tasks done·

  ·
  3

• feat(kv-index): add source-scoped recovery state

  #531·Kkhhh-star opened Nov 6, 2026·

  ·
  2

• fix: Return 413 for Oversized Streamed Bodies on the Fallback

  #529·eysk-m opened Nov 5, 2026Contributor·3 tasks done·

  ·
  1

• [WIP][Conductor] Introduct Mooncake Conductor as an external KV-cache index provider for vllm-router

  #528·Bourn3z opened Nov 5, 2026·3 tasks·

  ·
  26

• feat(logging): add optional user prompt logging

  #527·sunl19ht opened Nov 5, 2026·4 tasks done·

  ·
  5

• [Bugfix][PD] Remove the Mooncake metadata handoff mutex

  #526·Li-brua opened Nov 5, 2026·3 tasks done·

  ·
  4

• [Bench] Evaluate KV retention on OffloadingConnector, hint on and off

  #525·StevenWang-CY opened Nov 5, 2026·3 tasks done·

  ·
  18

• feat(program-scheduling): score next-turn KV prefetch in shadow

  #524·StevenWang-CY opened Nov 5, 2026·3 tasks done·

  ·
  8

• feat(kv-index): discover worker cache metadata from /metrics

  #520·KaisennHu opened Nov 5, 2026Collaborator·

  ·
  1

  9

• feat(kv-aware): rank P/D decode by logical context tokens

  #519·KaisennHu opened Nov 5, 2026Collaborator·

  ·
  1

  11

• docs(program-scheduling): end-to-end design for next-turn KV prefetch

  #518·StevenWang-CY opened Nov 5, 2026·3 tasks done·

  ·
  12

• feat(kv-index)[6/6]: support DeepSeek V4/V4.1 sliding and replay windows

  #517·KaisennHu opened Nov 5, 2026Collaborator·2 tasks done·

  ·
  12

• feat(kv-index)[5/6]: support Qwen4Exp compressed attention and scratch groups

  #516·KaisennHu opened Nov 5, 2026Collaborator·2 tasks done·

  ·
  7

• feat(pd)[4/6]: add hybrid attention prefix routing for P/D

  #515·KaisennHu opened Nov 5, 2026Collaborator·2 tasks done·

  ·
  6

• feat(kv-aware)[3/6]: route hybrid attention models by prefix in colocated mode

  #514·KaisennHu opened Nov 5, 2026Collaborator·2 tasks done·

  ·
  9

• feat(kv-index)[2/6]: add hybrid KV event stream and cache layout declaration

  #513·KaisennHu opened Nov 5, 2026Collaborator·2 tasks done·

  ·
  11

• feat(kv-index)[1/6]: add hybrid attention prefix index

  #512·KaisennHu opened Nov 5, 2026Collaborator·2 tasks done·

  ·
  5

• fix(pd): keep explicit adapters out of plain decode affinity

  #510·xuxiongchen opened Nov 5, 2026Contributor·3 tasks done·

  ·
  3