Repository navigation
Worker LLM runtime envs — hot stream timeouts + startup LLM tuning #1375
Description
Worker LLM runtime envs — hot stream timeouts + startup LLM tuning
Problem
QwenPaw workers need tunable LLM streaming timeouts (first-content / idle) and
startup LLM settings, but AgentTeams has no first-class way to set them. Today
the worker entrypoint hard-codes QWENPAW_LLM_STREAM_* to 300s, and changing
them requires a container rebuild/restart — painful when a local SGLang gateway
starves a stream under concurrent load (8/26 QwenPaw001 false-kill precedent).
Proposed
Two paths, matching the QwenPaw env registry's mutability model:
- Hot stream timeouts —
WorkerSpec.llmStreamFirstContentTimeout/
llmStreamIdleTimeout(+ team-wideTeamSpecdefaults) applied to the
running worker process without a container restart via the worker
console's/api/envs(PATCH to set, POST.../resetto restore the
inherited value). Applied state is tracked by the
agentteams.io/llm-stream-timeouts-appliedannotation (first:idle,"-"
per side), which supports partial clearing: clearing one side while the
other stays configured resets the cleared key so a stale override is never
left in the process. - Startup LLM tuning — startup-only LLM settings injected at worker start
(env / runtime block), taking effect on the next (re)start.
Resolution: an explicit worker value always wins over the team default; absent
both means "inherit the QwenPaw entrypoint default" (nothing applied, nothing
to clear).
Intended scope
- Runtime:
qwenpawonly — the/api/envshot registry is qwenpaw
specific. Other runtimes are a no-op (the hot-apply path returns early). - Deployment: embedded (in-container worker console reachable on
127.0.0.1:<console-port>) and k8s. The console dial is a
trusted-network call (same trust model as the channel/skills/approval
proxies), no auth. - Failure semantics: a failed dial logs and leaves the annotation
unchanged, so the change (and any reset) retries on the next reconcile; the
worker never loses ownership of an override it still holds.
Companion PR: #1363.
Worker LLM runtime envs — hot stream timeouts + startup LLM tuning
Problem
QwenPaw workers need tunable LLM streaming timeouts (first-content / idle) and
startup LLM settings, but AgentTeams has no first-class way to set them. Today
the worker entrypoint hard-codes QWENPAW_LLM_STREAM_* to 300s, and changing
them requires a container rebuild/restart — painful when a local SGLang gateway
starves a stream under concurrent load (8/26 QwenPaw001 false-kill precedent).
Proposed
Two paths, matching the QwenPaw env registry's mutability model:
- Hot stream timeouts —
WorkerSpec.llmStreamFirstContentTimeout/
llmStreamIdleTimeout(+ team-wideTeamSpecdefaults) applied to the
running worker process without a container restart via the worker
console's/api/envs(PATCH to set, POST.../resetto restore the
inherited value). Applied state is tracked by the
agentteams.io/llm-stream-timeouts-appliedannotation (first:idle,"-"
per side), which supports partial clearing: clearing one side while the
other stays configured resets the cleared key so a stale override is never
left in the process. - Startup LLM tuning — startup-only LLM settings injected at worker start
(env / runtime block), taking effect on the next (re)start.
Resolution: an explicit worker value always wins over the team default; absent
both means "inherit the QwenPaw entrypoint default" (nothing applied, nothing
to clear).
Intended scope
- Runtime:
qwenpawonly — the/api/envshot registry is qwenpaw
specific. Other runtimes are a no-op (the hot-apply path returns early). - Deployment: embedded (in-container worker console reachable on
127.0.0.1:<console-port>) and k8s. The console dial is a
trusted-network call (same trust model as the channel/skills/approval
proxies), no auth. - Failure semantics: a failed dial logs and leaves the annotation
unchanged, so the change (and any reset) retries on the next reconcile; the
worker never loses ownership of an override it still holds.
Companion PR: #1363.
Activity
shiyiyue1102 commentedon Oct 16, 2026
对应 PR #1363 已合入 main,感谢贡献!部分清空/reset、启动参数变更触发容器重建,以及 Team 默认值传递的问题均已修正并补充回归覆盖,Worker 显式配置优先于 Team 默认值。补充部署范围:运行中热更新目前仅支持 embedded/Docker 下的 QwenPaw Worker,要求 QwenPaw ≥ 2.2.1;K8s 下的运行中热更新不包含在本次实现范围内。
Corresponding to PR #1363, it has been merged into main. Thanks for your contribution! Issues with partial clear/reset, startup parameter changes triggering container rebuilds, and Team default value passing have been fixed and regression coverage added, with Worker explicit configuration taking precedence over Team default values. Supplementary deployment scope: Running hot update currently only supports QwenPaw Worker under embedded/Docker, requiring QwenPaw ≥ 2.2.1; running hot update under K8s is not included in the scope of this implementation.