AISim predicts LLM serving behavior and searches deployment configurations offline. Start with the quickstart to predict a deployment, search under a GPU budget, and evaluate the selected configuration.
flowchart LR
P["Performance model<br/>forward latency and memory"] --> R["Replay<br/>request and worker simulation"]
R --> S["Sweeper<br/>candidate evaluation and search"]
A["Framework adapters<br/>configuration, execution, outputs"] -.-> R
A -.-> S
The arrows show how capabilities compose. In a typical prediction-driven search, Sweeper invokes a runner, Replay schedules work in virtual time, and the engine queries the performance model for forward timing. Some analytical evaluation paths do not execute the native Replay loop; their scope is stated in the component documentation.
| Layer | Input | Output and responsibility |
|---|---|---|
| Performance model | Model, hardware, backend identity, parallelism, batch features, performance data | Forward-pass latency and memory estimates, with explicit estimator mode and coverage |
| Replay | Workload, deployment topology, scheduling and cache policies, performance model | Request timelines, throughput, latency, cache behavior and completion evidence |
| Sweeper | Search domains, workload, optimization goal and constraints | Evaluated candidates, ranking/Pareto results and concrete recommended configurations |
Adapters are integration boundaries, not a fourth simulation layer. A serving
framework can supply configuration, execution and output contracts; native
composition can customize placement and scaling without owning the event loop.
The Rust crate's engine, replay, and perfmodel namespaces describe its
implementation, while the product layers above describe user capabilities.
| Task | Read |
|---|---|
| Install and run a first prediction/search | Installation, quickstart, understand results |
| Choose or query an estimator | Model configuration, memory, Python API, Rust API |
| Collect FPM data for a model and target | FPM self-service, implementation/reference, examples |
| Replay traffic or AgentX sessions | Workloads, AgentX quickstart, engine, Dynamo integration |
| Search and materialize a deployment | Search space, goals, results, deployment generation |
| Implement an integration boundary | Adapter ABI reference |
| Extend a component | Performance modeling, Collector, Replay, Sweeper SDK |
| Look up shared syntax | CLI, top-level configuration, local resources |
| Maintain checks and releases | CI, release, accuracy publication, upstream sync |
| Use or migrate AIC workflows | AIC backward compatibility |
Performance-model coverage and Replay features answer different questions. The Feature Support Matrix summarizes Replay features per engine. The public FPE matrix probes a specific strict-native op-level query surface. It does not certify all FPM methods, orchestration paths, or end-to-end accuracy.
Use the FPM accuracy and E2E accuracy dashboards for measurement-backed results within their recorded scope. A configuration that runs successfully is not necessarily an accurate prediction. Replay and Sweeper remain experimental; validate shortlisted configurations on the target serving system before capacity or SLA decisions.
Examples on main can require capabilities newer than published wheels.
Follow the installation guide's version and optional Dynamo pairing rules.