Repository navigation
Conversation
susanbao
force-pushed
the
sanbao/deepswe-eval-controller-main
branch
from
November 21, 2026 02:20
8807378 to
0282fb2
Compare
…esolved config, and require weight_sync_config.mode none
…s response budget, validate the task, and tidy config_entries
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#3376 has merged; this PR is now based on main.
Summary
This adds
eval_deepswe.controller_main(argv, context), so the DeepSWE evaluation controller can run under the shared dist_rl runtime (--process_main=tunix.dist_rl.examples.deepswe_dist.eval_deepswe.controller_main) next to stock rollout workers.The controller:
node_utils.log_resolved_config). It requiresweight_sync_config.mode: none, since no trainer runs to give the workers weights.cluster_config.rollout_replicasrollout workers register through discovery, withincluster_config.init_timeout_s. A worker whose config fingerprint differs is rejected, and so is the wait.ClusterOrchestrator.bring_up_workers: initialize, verify, compile, start). It then dispatches every attempt as ageneraterequest (RolloutRequest(**request_fields(...))), with at mostagentic_grpo_config.max_concurrencyin flight on each worker, the same per-worker limit the rollout nodes use in training.RolloutResponsewithcompact_result, and records it through the existingwrite_json/write_results/summarize(Pass@1, plus Pass@k with several attempts).config.jsonrecords the resolved config and its fingerprint.managed_cluster_teardown, like the training orchestrator. On exit, every registered worker is stopped (honoringcluster_config.stop_workers_on_exit) and its handle closed. This includes a failed registration wait, any other error, and SIGTERM. On failure the stop is sent withabort=True, and the original error is re-raised.env_kwargs.use_agent_sandboxis set, importsexamples.deepswe.sandbox_utilslazily, initializes and warms the tasks' sandbox fleet up front, and tears it down infinally, before the workers are stopped. When sandboxes are off, the module is never imported.Shared worker registration
The orchestrator's registration wait moves to
node_utils.wait_for_workers(registry, discovery, min_workers, ...), with its config check asnode_utils.check_worker_config. The code is unchanged apart from taking the registry and the required workers as arguments.orchestrator.wait_for_workerskeeps its signature and calls the shared function withrequired_workers(hp), so the orchestrator and the eval controller enforce the fingerprint check (and the rejection and deadline handling) the same way.Other changes in
eval_deepswe.pycollect_worker(..., rpc=...)dispatches eitherevaluate(this script's--role worker; the default) orgenerate(a stock rollout worker).run_controller's result handling moves into_evaluate, which both controllers share. The one behavior change: if workers fail to become ready,run_controllernow raises without writing anything. That means noconfig.json, and no summary recording thefatal_error.request_fieldsno longer setsmetadata["agent_config"], because a request'sagent_configreplaces a stock worker's configuredagent_kwargs. The script's own worker passes the same{"scaffold": "r2egym"}as itsagent_configinstead, so--role local|controller|workerbehave exactly as before.Where eval settings come from
I added no new config section.
config_argsmaps the resolved config onto the existing evaluation flags:agentic_grpo_config.num_generationsmax_turns,episode_timeoutmax_concurrency(the controller's total is this timescluster_config.rollout_replicas)rollout_config.total_generation_steps, as in trainingRolloutConfig; recorded inconfig.json)rollout_config.{temperature,top_p,top_k,total_generation_steps}vllm_config.max_model_len, or prompt + generation lengthdata_module.create_dataset(**data_config), capped bydata_config.limit<cluster_config.artifact_root>/evalThe rollout workers read the same config, so the config fingerprint stays identical across processes. The controller also follows training's per-request settings. The script's own default budget for a trajectory's cumulative response is
max_model_len - 256(agentic eval's convention: 32512 with deepswe.yaml). Training usestotal_generation_steps(28672). Socontroller_mainsets the budget tototal_generation_steps, and an evaluated model gets the same budget it was trained with.--role local|controller|workerkeep the old default. Eval-only knobs, such as a separate output directory, can be added with the launcher subcommand when they are needed.Limitations
weight_sync_config.mode: none(enforced), and the rollout workers must load real weights themselves.deepswe.yamlusesraiden, so it cannot be reused as is.data_config(e.g. deepswe.yaml's R2E-Gym-Subsettrainsplit), so an eval config must set them. Withdata_config.shuffle(on by default),create_datasetshuffles before applyingdata_config.limit, so a limit picks a seeded random subset, not the first N tasks.controller_mainhas only been tested on CPU, with fake workers and a patched config. It has not yet run against real rollout workers on a cluster; that validation comes with the launcher subcommand PR.resolve_configrequiresagentic_grpo_config.num_generations > 1in every process, and several attempts requirerollout_config.temperature > 0. Socontroller_mainruns at least 2 sampled attempts per task, and reports Pass@1 averaged over them, plus Pass@k. A single greedy attempt per task, the legacy default, still needs--role local|controller|worker.Origin
This ports the multi-worker controller of #2389 onto the dist_rl runtime, replacing the per-replica Pathways deployment it used.
Test plan
All tests are CPU-only, with no network, sandbox, or TPU:
python -m pytest tests/dist_rl/examples/deepswe_dist/ tests/dist_rl/launch/generatedispatch: out-of-order responses are compacted, and each attempt is sent as aRolloutRequest.controller_mainend to end with a patched config, a stand-in cluster and a fake worker: bring-up,generatedispatch, the summary,config.jsonholding the resolved config and fingerprint, a cleanshutdown(), and sandbox fleet init, warm and teardown.controller_mainraises the rejection. The already-registered worker is covered byshutdown(abort=True), nothing is dispatched, and the sandbox fleet is still torn down.controller_mainrejects anyweight_sync_config.modeother thannonebefore creating anything.wait_for_workersuses the sharednode_utils.check_worker_config.python3 scripts/check_cl_complexity.py --strict --rev origin/main: PASS.