Repository navigation
Conversation
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
liangqianxing
left a comment
There was a problem hiding this comment.
Small, non-blocking follow-up at 6035c1ecfaa1c9e82f2793098fdcc7dd494fab9e: the runner guard correctly rejects PP2, but public prediction can still estimate capacity before reaching it, contrary to the documented ordering.
With workers.aggregated.parallelism.pipeline: 2, fixed or polynomial timing, default KV capacity, and no state cache, run_prediction first calls prediction_to_replay_spec. During compilation, _worker_engine_args calls materialize_aic_num_gpu_blocks (compiler.py:694–696), reaching estimate_num_gpu_blocks(pp_size=2) before this guard.
I verified this ordering on the exact Python head with a recording estimator. An injected estimator failure escapes before the PP diagnostic; returning capacity reaches the expected PP rejection. Two unmocked offline checks using the local tiny-model fixture, with default and explicit capacity, also produced the intended rejection; the real estimator succeeded in the default-capacity case. PP2 never reached the native runtime in these checks. This demonstrates ordering and injected failure propagation; no real metadata failure or incorrect replay result was observed.
A shared engine-stack preflight before compiler capacity estimation, plus a public run_prediction regression for these configurations, would close the gap. The current direct-materializer tests cover the later boundary.
Why and what changed
Engine replay accepts pipeline parallelism even though it has no pipeline-stage scheduler or transfer model. A requested PP value can affect performance-model inputs and capacity while replay still runs a single-stage worker.
Reject pipeline sizes other than one at the shared runner materialization boundary, before KV capacity estimation. Cover deployment parallelism, flat and nested legacy rank arguments, and canonical timing configuration aliases for aggregated, prefill, and decode roles. Update the existing capacity test to use supported PP1.
Review map
_materialize_engine_roleinpython/aisimulate/src/aisimulate/runner.py.Evidence
Modeling or data provenance
N/A. This enforces the documented engine-replay support boundary and adds no pipeline model.