Skip to content

Run the DeepSWE eval controller as a dist_rl launch process main - #3419

Draft
susanbao wants to merge 4 commits into
mainfrom
sanbao/deepswe-eval-controller-main
Draft

susanbao wants to merge 4 commits into
mainfrom
sanbao/deepswe-eval-controller-main

Conversation

@susanbao

@susanbao susanbao commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

#3376 has merged; this PR is now based on main.

Summary

This adds eval_deepswe.controller_main(argv, context), so the DeepSWE evaluation controller can run under the shared dist_rl runtime (--process_main=tunix.dist_rl.examples.deepswe_dist.eval_deepswe.controller_main) next to stock rollout workers.

The controller:

  • Resolves the same config as the other nodes and logs its fingerprint (node_utils.log_resolved_config). It requires weight_sync_config.mode: none, since no trainer runs to give the workers weights.
  • Waits until cluster_config.rollout_replicas rollout workers register through discovery, within cluster_config.init_timeout_s. A worker whose config fingerprint differs is rejected, and so is the wait.
  • Brings the workers up the same way the training orchestrator does (ClusterOrchestrator.bring_up_workers: initialize, verify, compile, start). It then dispatches every attempt as a generate request (RolloutRequest(**request_fields(...))), with at most agentic_grpo_config.max_concurrency in flight on each worker, the same per-worker limit the rollout nodes use in training.
  • Compacts each RolloutResponse with compact_result, and records it through the existing write_json / write_results / summarize (Pass@1, plus Pass@k with several attempts). config.json records the resolved config and its fingerprint.
  • Runs inside managed_cluster_teardown, like the training orchestrator. On exit, every registered worker is stopped (honoring cluster_config.stop_workers_on_exit) and its handle closed. This includes a failed registration wait, any other error, and SIGTERM. On failure the stop is sent with abort=True, and the original error is re-raised.
  • When env_kwargs.use_agent_sandbox is set, imports examples.deepswe.sandbox_utils lazily, initializes and warms the tasks' sandbox fleet up front, and tears it down in finally, before the workers are stopped. When sandboxes are off, the module is never imported.

Shared worker registration

The orchestrator's registration wait moves to node_utils.wait_for_workers(registry, discovery, min_workers, ...), with its config check as node_utils.check_worker_config. The code is unchanged apart from taking the registry and the required workers as arguments. orchestrator.wait_for_workers keeps its signature and calls the shared function with required_workers(hp), so the orchestrator and the eval controller enforce the fingerprint check (and the rejection and deadline handling) the same way.

Other changes in eval_deepswe.py

  • collect_worker(..., rpc=...) dispatches either evaluate (this script's --role worker; the default) or generate (a stock rollout worker).
  • run_controller's result handling moves into _evaluate, which both controllers share. The one behavior change: if workers fail to become ready, run_controller now raises without writing anything. That means no config.json, and no summary recording the fatal_error.
  • request_fields no longer sets metadata["agent_config"], because a request's agent_config replaces a stock worker's configured agent_kwargs. The script's own worker passes the same {"scaffold": "r2egym"} as its agent_config instead, so --role local|controller|worker behave exactly as before.

Where eval settings come from

I added no new config section. config_args maps the resolved config onto the existing evaluation flags:

Eval setting Config source
Attempts per task agentic_grpo_config.num_generations
Turns, episode timeout max_turns, episode_timeout
In-flight attempts per worker max_concurrency (the controller's total is this times cluster_config.rollout_replicas)
Cumulative response budget per trajectory rollout_config.total_generation_steps, as in training
Sampling (applied by the workers' RolloutConfig; recorded in config.json) rollout_config.{temperature,top_p,top_k,total_generation_steps}
Model length vllm_config.max_model_len, or prompt + generation length
Tasks data_module.create_dataset(**data_config), capped by data_config.limit
Output <cluster_config.artifact_root>/eval

The rollout workers read the same config, so the config fingerprint stays identical across processes. The controller also follows training's per-request settings. The script's own default budget for a trajectory's cumulative response is max_model_len - 256 (agentic eval's convention: 32512 with deepswe.yaml). Training uses total_generation_steps (28672). So controller_main sets the budget to total_generation_steps, and an evaluated model gets the same budget it was trained with. --role local|controller|worker keep the old default. Eval-only knobs, such as a separate output directory, can be added with the launcher subcommand when they are needed.

Limitations

  • The eval config must set weight_sync_config.mode: none (enforced), and the rollout workers must load real weights themselves. deepswe.yaml uses raiden, so it cannot be reused as is.
  • The dataset, split and task cap come from data_config (e.g. deepswe.yaml's R2E-Gym-Subset train split), so an eval config must set them. With data_config.shuffle (on by default), create_dataset shuffles before applying data_config.limit, so a limit picks a seeded random subset, not the first N tasks.
  • controller_main has only been tested on CPU, with fake workers and a patched config. It has not yet run against real rollout workers on a cluster; that validation comes with the launcher subcommand PR.
  • The eval settings come from the shared config (see the table above). resolve_config requires agentic_grpo_config.num_generations > 1 in every process, and several attempts require rollout_config.temperature > 0. So controller_main runs at least 2 sampled attempts per task, and reports Pass@1 averaged over them, plus Pass@k. A single greedy attempt per task, the legacy default, still needs --role local|controller|worker.

Origin

This ports the multi-worker controller of #2389 onto the dist_rl runtime, replacing the per-replica Pathways deployment it used.

Test plan

All tests are CPU-only, with no network, sandbox, or TPU:

  • python -m pytest tests/dist_rl/examples/deepswe_dist/ tests/dist_rl/launch/
  • New tests cover:
    • The config-to-flags mapping.
    • generate dispatch: out-of-order responses are compacted, and each attempt is sent as a RolloutRequest.
    • controller_main end to end with a patched config, a stand-in cluster and a fake worker: bring-up, generate dispatch, the summary, config.json holding the resolved config and fingerprint, a clean shutdown(), and sandbox fleet init, warm and teardown.
    • With one matching and one mismatched worker, controller_main raises the rejection. The already-registered worker is covered by shutdown(abort=True), nothing is dispatched, and the sandbox fleet is still torn down.
    • controller_main rejects any weight_sync_config.mode other than none before creating anything.
    • The sandbox module is never imported when sandboxes are off.
    • The orchestrator's wait_for_workers uses the shared node_utils.check_worker_config.
  • python3 scripts/check_cl_complexity.py --strict --rev origin/main: PASS.

@susanbao
susanbao force-pushed the sanbao/deepswe-eval-controller-main branch from 8807378 to 0282fb2 Compare November 21, 2026 02:20
@susanbao
susanbao changed the base branch from sanbao/deepswe-eval-passk to main November 21, 2026 02:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants