Skip to content

Memory-level parallelism (MLP), phases 1 and 2 - #436

Merged
tinebp merged 431 commits into
masterfrom
feature/mlp-phase2
Nov 9, 2026
Merged

tinebp merged 431 commits into
masterfrom
feature/mlp-phase2

Conversation

@tinebp

@tinebp tinebp commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR adds memory-level parallelism (MLP) to the memory system, in two phases:

  • Phase 1: more requests in flight at every level of the memory system.
  • Phase 2: cheaper fences, request merging, conflict-free cache indexing, wider loads and stores, and closer agreement between SimX and the RTL.

It also brings the tooling built to land that work safely: the Beacon logic-depth gate, one synthesis front end for every flow, Foldsim, calibrated hardware profiles, and a change-impact review policy.

Every MLP feature, with its knob, default, measured gain and area, is listed in docs/designs/mlp_optimizations.md.

Perf vs master (rtlsim perf_gate, XLEN 32, master's committed baselines b447578):

  • −13.7% cycles, geomean over the 44 cases other than raycast.
  • The figure includes the compiler update this PR brings: TOOLCHAIN_REV moves to v3.0.2 (LLVM f44a6bb2af4c). Instruction counts therefore differ from master in 44 of 49 cases. The share that is hardware alone:
    • up to the Oct 26 head, on master's own LLVM, with identical instruction counts in 47 cases: −3.7%;
    • the final commits (set hash, read-merge realignment), with identical instruction counts in all 49: −1.15%;
    • the commits in between (Oct 26 to Nov 7, −9.3%) mix the LLVM update with the Phase 2 hardware that landed then; the baselines alone cannot split them.
  • core:raycast is excluded because 60e7a2b renders it at 64×64 instead of 16×12 (21× the work). Per pixel it is 34.5% faster at 4 threads and 7.4% faster at 16.

What changes

Phase 1: requests in flight

  • Hashed bank select, so power-of-two strides spread across banks.
  • Back-to-back fills into the bank pipeline.
  • Write-allocate without fetching the line at a write-back LLC (lazy fetch), on by default.
  • Flush:
    • one broadcast write, with per-port flush tokens;
    • only the L1s flushed between launches when the LLC is the coherence point.
  • Fences stop at the point of coherence; IO fences skip the cache walk.
  • Each LSU block has its own memory path; each TCU block its own metadata port and TCU_LD AGU.
  • AFU: one AXI port per GPU memory port, with many transactions in flight.
  • The command processor's DMA goes through the shared LLC, on by default.
  • Fetch width is decoupled from issue width (VX_CFG_FETCH_WIDTH).
  • Core, socket, cluster and per-slice warp counts no longer have to be powers of two.
  • Redesigned DRAM model (independent channels, refresh, data-sheet timing).

Phase 2: fences, merging, indexing, wide accesses

  • Memory fences split into acquire and release; .aq/.rl AMOs are ordered.
  • Dcache read merge, port 0 included. A merging pair that drifts a cycle apart realigns, so uniform loads keep merging.
  • Same-line icache fetches from a core's fetch slices are merged.
  • Hashed dcache set index (VX_CFG_DCACHE_SET_HASH, on): sgemv 512–2048 −20% to −24%.
  • Warp-wide local-memory unit with two pass engines (LMEM deepest path 18 → 9 levels).
  • Lane-gather writeback merge: sgemm −9.7% at 16 threads, −20.6% at 32.
  • Wide loads/stores (vx_flw.d/.q, vx_fsw.d/.q) and matrix fragment loads/stores (vx_ldm, vx_stm), as extensions that are off by default (VX_CFG_EXT_LDST_WIDE_ENABLE, VX_CFG_EXT_LDST_MATRIX_ENABLE).
  • Lazy fetch: a fully written sector skips its refetch; write-through chains stay ordered.
  • Kernel arguments are staged in clean slots, without a cache flush.
  • MSHRs and the LSU pending pool sized from the platform's bandwidth × latency.
  • Non-power-of-two way counts; victim advance only on a new line; stack interleave by whole lines.
  • Zero-latency request skids at the L2 and MMU inputs (tensor DUT 240.6 → 251.6 MHz).
  • Commit grants the FPU first; FPU cores end in a zero-latency skid.
  • New knobs: VX_CFG_L1_LINK_LATENCY, VX_CFG_L2_LINK_LATENCY, VX_CFG_LMEM_SIZE, VX_CFG_TCU_PE_RATIO, VX_CFG_FMA64_PE_RATIO; 64-thread warps build.
  • VM: socket, TEX, raster and OM traffic translate through unit MMUs.
  • TCU output is fp32 only, with bit-exact TFR numerics in SimX.

SimX: the RTL's timing model

  • Models the RTL's buffers, arbiters and registered ports at each boundary: the miss path, LSU ingress and result path, icache fill path, issue arbiters, lmem crossbar, dispatch iterator, TCU and WGMMA timing, flush walk and fences.
  • model_parity covers core, tensor, DXA and VM cases within the 5% tolerance.

Hardware profiles

  • Profiles for the GPUs that Accel-Sim, MGPUSim and NaviSim evaluate (Fermi through Blackwell, plus AMD), kept only where published latency data calibrates them. Each sets cores, warps, threads, exact cache capacities, lanes and DRAM timing.
  • ci/profile_check.py compares a profile with its reference part.

Tooling

  • Beacon / logic_gate: a per-unit logic-depth and area gate, minutes per DUT, calibrated against Vivado. It runs in ci.yml together with perf_gate (every pull request, every push that touches the RTL or the config), against baselines in ci/baselines/synthesis/beacon/. A unit that gets deeper, or a design that grows, fails it.
  • Synthesis front end: every synthesis tool (Vivado, Quartus, Design Compiler, Yosys, FireSim's image) reads the same sources: preprocessed by CIRCT (circt-verilog -E) and patched so that asynchronous-read RAMs map to block RAM (the RAM fold, pyslang). No tool reads an elaborated netlist. Yosys elaborates them with its slang frontend (read_slang). sv2v and Verilator's preprocessor are gone. CIRCT elaborates the design only for Beacon and Foldsim.
  • Foldsim: threaded FPGA simulation of Vortex on the Alveo V80, compiled by CIRCT, with a DRAM bank memory model. FireSim also runs on the V80.
  • Review policy: every change is classified R0–R3 (ci/change_impact.py, docs/review/code-review-policy.md). Changes to tests, golden values and gate thresholds are R3.
  • Timing closure: core, TCU FEDP, CTA dispatch, cache and RTU paths shortened for 300 MHz on the U55C.

Since merging master 50cb174

  • MUFU, Zfhmin and RTU perf counters from master, with the merge fixes they needed.
  • SimX RVC fix for an instruction straddling two fetch words.
  • CI fixes from the last run: the pipe buffer's UNOPTFLAT loop at -O0; the fcvt NaN-boxing check stops at FLEN on RV64; the TCU B buffer latches its descriptor only when a fetch is allocated.
  • CI: model_parity and perf_gate are split into shards of about an hour.
  • CI: logic_gate runs in ci.yml with perf_gate, against baselines recorded at this head.
  • CI: small suites share cells (isa, sw, integration, tensor_sp), with every case unchanged: a pull request plans 82 cells instead of 108.
  • TCU_LD's response assertion accepts an earlier pass's response while a later pass's request is held (NT 8 with 4 LSU lanes).

Since 7214b3b

  • Yosys reads the folded sources with read_slang instead of a CIRCT-elaborated netlist, the last flow that still read one. Eight RTL files are rewritten for slang, which is stricter than Vivado, Verilator and CIRCT (use before declaration, a negative-width cast, $bits of an interface member). None of the rewrites changes logic: Beacon reports identical depths on core, tcu, rtu and top.
  • FireSim and Altera OPAE read the folded per-file source lists. OPAE is untested: there is no OPAE SDK on the test machine.
  • Every ASIC gate build is constrained to 800 MHz; eight were at 400 or 500.
  • FPGA and ASIC synthesis baselines re-recorded; catalog budgets unchanged.
  • CI caches the ASAP7 downloads and retries them longer, after GitHub's HTTP 504.
  • A hardware profile folds into a CONFIGS that a Makefile already overrode.
  • Master merged (48fc006: RHEL 9 toolchain, softfloat options), without sv2v.

Validation

Hardware and SimX at 0f76899, plus one assertion-only RTL fix (4241405):

  • perf_gate: 49/49, baselines re-recorded; instruction counts unchanged by the final commits.
  • model_parity (core, tensor_wg, dxa, vm): 59/59.
  • cache: 70/70.
  • regression: 201/201 across XLEN 32 and 64, simx and rtlsim.
  • lint: XLEN 32 and 64, clean.
  • Beacon, run locally on each final RTL change against its parent: no unit deeper than the parent's deepest path.
  • logic_gate: 14/14 builds at +0% against the baselines, recorded with the CI toolchain (CIRCT 892a65a01, foldsim d8350bc0f).
  • CI run 42542278556 on 0f76899: 149 jobs passed. Two failed:
    • model_parity 3/5, the TCU_LD assertion, fixed in 4241405;
    • synthesis, a GitHub HTTP 504 while downloading the ASAP7 libraries.
  • CI run 42576705978 on 7214b3b: passed.
  • FPGA gate (Vivado 2024.2), run locally in update mode on 589aaf2: baselines recorded. Against the catalog: core Fmax 286 MHz (needs 294), and tex (+27.6%), gfx (+12.1%) and vm (+5.2%) over their LUT budgets.
  • ASIC gate (Yosys 0.69 with read_slang, OpenSTA 3.1.0, ASAP7 RVT TT, 800 MHz), run locally in update mode, 14/14 builds recorded. Against the catalog: core 655 MHz and top 654 MHz miss the 784 MHz floor; cache (+22.3%) and lmem (+8.0%) are over their area budgets. Every other build meets 800 MHz and its budget.
  • Since 7214b3b: testcase lint (932 cases), and configure at XLEN 32 and 64 on the merge. The merge touches no sw/ or tests/ file.

Known items

  • Synthesis gates red: the FPGA and ASIC gates above fail against the catalog budgets, which are unchanged. They run on master only, so this PR does not run them. Fixing them comes right after this merge, starting with the core and top critical paths at 800 MHz.

  • Slower than master: dxa copy-mcast is +1.66% (inside the 2% band). raycast is slower only because of its larger workload (see above).

  • Set-hash cost: two TCU tile layouts lose a little to the hashed set index: fp16 on 2 cores at 16 threads +1.8%, sgemm2 fp16 +1.1% (averaged over L1 latency +0..+3). The cost belongs to set hashing for those layouts; a cheaper fold is worse.

  • mxfp8 parity point: tensor_mx:model_parity-mxfp8-lsu-lanes passes at a 4.70% gap (5% tolerance), up from 1.24%. This kernel's gap is chaotic in size: over K = 32..192 the mean |gap| is 1.80% with or without the read-merge realignment, and K=192 improved from -5.54% to -0.22%.

  • SimX vs RTL, open: sgemv at 8 warps runs 3.5–4.6% slower in SimX than rtlsim (6.0% at m256 n500), with identical instruction counts and dcache misses within 3%. It is a steady bias outside any gate; the CI case parity-sgemv passes.

🤖 Generated with Claude Code

blaisetine and others added 30 commits October 29, 2026 00:26
A build targeted at 250 MHz (4 cores, NW=NT=4, IW=4, sockets plus six
threaded unit classes) routed with WNS -2.974 ns at 4.000 ns: a 6.974 ns
critical path, 143.4 MHz. The default is now that measured figure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 219e82e)
Every class of identical units is time-multiplexed through one physical
copy, cycle-exact against the RTL, with no Chisel, Golden Gate, Yosys or
sv2v anywhere in the flow:

- hw/syn/maxsim: circt-verilog imports Vortex; maxsim-extract-models cuts
  each level into its own logic and the class instances; maxsim-compose
  threads the classes and joins every model through one token fabric;
  firtool writes it out; the V80's native IP packaging and slashkit link
  build the image. lift_boundaries.py gives interface-port units plain
  ports and turns instance IDs into ports on the flow's copy of the
  sources, so Vortex's RTL is not changed; the copy's async-read RAM patch
  (placeholders for a Vivado hook) is replaced by the RAM it stands for.
  The flow builds the STD FPU and the RTU without vendor cores.
- hw/rtl/afu/maxsim: the kernel. maxsim_mem times the AXI4 port as
  AxiLatencyPipe does, sending requests to HBM at once; maxsim_ctrl is the
  AXI4-Lite register block and the target's control endpoint (command
  queue for DCR writes and reads, start, reset; cycle, memory-wait and
  per-class counters). Registers avoid 0x40-0xBF, which the V80 shell's
  MSI-X table answers.
- sw/runtime/maxsim (--driver=maxsim): the software command processor over
  VRT registers and QDMA on the card, or over the same kernel in Verilator
  (sim/maxsim, TARGET=avedsim).

V80, s2w4i2 (2 cores, NW=NT=4, IW=2): demo, sgemm and sgemm n=128 PASS with
cycle counts identical to Verilator; 138k routed LUTs (FireSim image 219k),
149.8 MHz; 8 host cycles per target cycle; sgemm n=128 in 196 ms, 50x SimX.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 60b965f)

Integration (phase 2): the linker-config emitter reads the descriptor's single
memory tag (phase 2's descriptor schema); CHANGELOG keeps both entries.
maxsim replaces FireSim's FAME-5 lane: the FireSim backend goes back to
running Vortex as an unthreaded BlackBox, and the lane's conversion flow
(hw/syn/firesim/fame5: sv2v, Yosys, FIRRTL regeneration), its target config
and its FAME5 make variables are gone.

The lane's VX_CFG_FAME5 hooks in the RTL are removed with it, restoring
every file to its upstream state: socket and core indices as ports, the
VX_socket_top wrapper, and the async-BRAM patch switch were simulator-only
changes to the hardware. maxsim does the same on its own copy of the
sources (lift_boundaries.py port=SOCKET_ID / port=CORE_ID, and the RAM the
patch stands for), so Vortex's RTL serves every flow unchanged.

docs/designs/firesim_integration.md now points to docs/designs/maxsim.md
for model multithreading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8036973)

Integration (phase 2): the Vortex RTL is left as it is. The FAME-5 hooks this
commit removes were not replayed (35d3e8a, 41b59d5), so the RTL already
is in the upstream state it restores.
TYPE_DEFINES comes from a configuration resolved without SYNTHESIS, which
selects the simulation FPU (STD) over the FPGA one (DSP). The image flow
used it for synthesis too, so a FireSim image carried a different FPU from
the one every native FPGA flow of the same configuration builds. Synthesis
now resolves its unit types with the synthesis defines; the metasimulator
keeps the simulation ones. The DSP FPU's Xilinx floating-point cores are
already generated and packaged by the image flow (xilinx-ips).

Not yet exercised by a FireSim image build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit e3e94aa)
The endpoint offered its command tokens (reset, DCR valid/rw/addr/data,
start) as functions of the live queue head. The simulator may take a
cycle's token before the endpoint finishes the cycle; a command the host
queued in between changed the bits of a token already taken, and was then
popped as applied though the target never saw it. On tinygpu (1 core, 2
warps x 2 threads, no L1 caches or local memory) 16 of the 20 launch DCR
writes were lost this way and the kernel never ran; on larger
configurations the timing happened to hide it.

The command applied in a target cycle is now taken from the queue when the
previous cycle finishes, and the tokens are functions of it alone. A reset
command of count 0 lasts one cycle instead of wrapping to 2^32.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit ac9a12e)
The in-process kernel's memory answered a read with the data memory held
when the response went out, so a write issued after the read but before
its latency had passed showed up in it. It now captures the data when it
takes the request, as a memory does.

Debug aids for the kernel: MAXSIM_EMUL_AXI_TRACE (the memory port's
requests and responses), MAXSIM_EMUL_REG_TRACE (the host's register
accesses), and, in a VCD=1 build, MAXSIM_EMUL_VCD with
MAXSIM_EMUL_VCD_FROM and MAXSIM_EMUL_VCD_DEPTH.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 1fbb27a)
Every synthesis flow now takes its SystemVerilog through CIRCT:

- gen_sources.sh -P preprocesses with circt-verilog -E instead of
  verilator -E. Over the whole RTL tree (full-GPU defines) the two produce
  the same code; Verilator only kept lint-pragma comments. The tinygpu
  maxsim simulator generated from it matches the old reference in token
  co-simulation (0 errors).
- The Yosys flow elaborates with CIRCT into one Verilog-2005 file
  (hw/scripts/sv_to_v.sh, replacing sv2v.sh): packed arrays and structs
  become vectors, and the SRAM macros (BB_MODULES) stay instances with
  their parameter values; yosys reads them beside their stubs in
  SystemVerilog mode.
- CIRCT installs under $TOOLDIR/circt (toolchain_install.sh --circt, in
  place of --sv2v); hw/syn/common.mk exports CIRCT_VERILOG, CIRCT_OPT and
  FIRTOOL, and maxsim takes its CIRCT from there. The asic_gate fingerprints
  its baselines on the CIRCT version.

Cache DUT through Yosys/ABC on ASAP7: SRAM area identical (16,307 um^2),
logic area 56,934 vs 53,936 um^2 (+5.6%): CIRCT folds synchronous resets
into the next value and lowers array reads to mux trees.

Needs the CIRCT prebuilt (toolchain_prebuilt.sh --circt) before CI can
install it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 5b4020c)
MAXSIM_CLASSES takes a class A+B: two modules of the same parent whose
instances CIRCT pairs and simulates as one model (maxsim-fuse-models). The
LSU slice and its scheduler exchange bits combinationally both ways (the
slice's request, the scheduler's ready on it, the slice's ready on that),
which took each thread two visits per target cycle; fused, the path is
inside one model. The documented class set fuses them.

tinygpu (V80): demo and sgemm -n16 pass with the same instructions and
cycles; 4.00 -> 3.00 host cycles per target cycle; Fmax 162.3 -> 161.2 MHz.
Area (Vivado OOC, opt_design): tinygpu 31,926 -> 31,592 LUTs, s2w4i2
166,519 -> 165,385 LUTs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 6424c4c)
- Poll backoff: the card runs the target on its own, so after ~1 ms of busy
  polls the status poll sleeps, doubling up to 100 us. The in-process
  kernel advances only when it is accessed and keeps polling back to back.
- DMA: QDMA needs only the staging-buffer offset page aligned, not the
  device address or the length, so a transfer is now exactly its bytes; a
  partial-page write no longer reads its first and last pages first.
- MAXSIM_STATUS_MEM_ERR (a burst, which the memory model does not take)
  is reported and ends the wait instead of polling a stopped target.

V80, tinygpu image, sgemm -n128: 2,785,811 -> 14,760 register reads and
2.10 -> 0.11 s of host CPU (63% -> 2%), kernel time unchanged (2,057 vs
2,059 ms); demo and sgemm -n16 cycle-identical. Writes and reads at
unaligned offsets and lengths (1 B to 3 pages + 5 B, across page edges)
match a host copy of the buffer exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 867a011)
The flow now reads the extensions' packages and include paths from
hw/syn/extensions.mk, as every synthesis flow does, so a configuration with
TCU, DXA, raster, texture, OM or RTU elaborates.

The RTU no longer forces VX_CFG_RTU_USE_DSP=0. It kept the RTU's F32 FMAs
and dividers off Xilinx's floating-point cores, whose netlists the flow can
neither read nor thread, but it also took away their DSP multiplies. Instead,
the staged copies of VX_fma_unit, VX_fdiv_unit and VX_fsqrt_unit no longer
select their vendor core (VENDOR_OK = 0), so RTU_USE_DSP keeps Vortex's FPGA
default and the RTU's soft FMAs multiply on DSPs. The build stops if a unit
ever selects its vendor core some other way. -DVIVADO stays, so the rest of
Vortex elaborates as the native Vivado flows see it. The RTU divider keeps
the soft divider's latency.

Validated: 4-core full GPU (L2, A, TCU, DXA, TEX, RASTER, OM, RTU) demo in
Verilator emulation PASSED, instrs=39104 cycles=72519 (unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 56b5fee)
The flow's VX_async_ram_patch, which VX_dp_ram and VX_sp_ram instantiate
only for a RAM that must be block RAM, now says so with the synthesis
attribute (* ram_style = "block" *). Once the target is flattened (but for
its classes), CIRCT's seq-retime-firmem-reads makes each such read
synchronous: it reads at the address the address registers are about to
take, so the RAM can be a block RAM. Unhinted RAMs are left as they are. The
pass writes gen/memories.rpt, one line per asynchronous-read RAM with its
size, hint and decision.

On the 4-core full GPU, one RAM asks for block RAM, the IPDOM stack store
(112x39, four threads), and it is converted. The other 114 asynchronous RAMs
are below Vortex's block RAM threshold.

Validated: tinygpu token co-simulation against the unconverted reference,
2000 cycles, 0 errors (host/target 3.00); full GPU demo in Verilator
emulation PASSED, instrs=39104 cycles=72519 (unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 285672b)
A RAM that must be block RAM (FORCE_BRAM) but reads asynchronously
(OUT_REG=0, read-first) is now a plain array tagged with the standard
synthesis attribute, `(* ram_style = "block" *)` (USE_BLOCK_BRAM), read
asynchronously. CIRCT's seq-retime-firmem-reads makes the read
synchronous, reading at the address the address registers are about to
take, after inlining the modules between the RAM and those registers. A
block RAM request it cannot honor stops the build with the reason.

Every synthesis flow now reads one netlist CIRCT elaborates:
hw/scripts/circt_elaborate.sh (was sv_to_v.sh) writes SystemVerilog for
Vivado (-f sv) or Verilog-2005 for Yosys (-f v2005), expands conditional
array writes into per-element selects, merges duplicate logic, and writes
<netlist>.memories.rpt, one line per asynchronous-read RAM.
gen_sources.sh -E runs it on the staged tree and makes the filelist name
the netlist (xrt, aved, dut, sandbox); FireSim and maxsim run it in their
own recipes. The Xilinx floating-point cores stay instances, declared to
the elaboration in hw/scripts/xilinx_ip_stubs.sv.

Removed: ASYNC_BRAM_PATCH, VX_async_ram_patch (both copies),
VX_placeholder, xilinx_async_bram_patch.tcl and the hooks that sourced
it (the sandbox and FireSim pre-opt hooks entirely), and RADDR_RESET,
which only the patch read. RADDR_REG stays: the ASIC branch uses it.

xilinx_slr_pblocks.tcl accepts the netlist's generate-block names,
g_x_0_inst, as well as g_x[0].inst.

Validated:
- xilinx/dut core, 16 warps, 8 threads, V80 part, 300 MHz, implemented
  both ways: 90,481 logic LUTs against 92,608 today, the same BRAM
  (65+9), LUTRAM (2,998) and DSPs (40), timing met both ways. The FF count
  differs only by registers Vivado retimed (`_bret`): the RTL registers
  are the same. The IPDOM stack store (112x39) is converted.
- Yosys: the core (16 warps, 16 threads) exports to Verilog-2005.
- maxsim: tinygpu token co-simulation, 0 errors; 4-core full GPU demo in
  Verilator emulation PASSED, instrs=39104 cycles=72519 (unchanged).

Needs the CIRCT of maxsim/phase1 (circt-maxsim) in the toolchain; the
synthesis gates' baselines move with the netlist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit c87d951)

Integration (phase 2): the Xilinx flows' source rule keeps phase 2's
dependency on the AFU memory-port stamp; the library catalog keeps phase 2's
VX_rom and drops the two removed modules.
Each model now gets hw-expand-array-writes and a topological sort before
CSE, and maxsim-extract-models moves logic of up to 64 ops that drives a
model's input from other models into the model.

s2w4i2 on the V80 part, out-of-context synthesis after opt_design: logic
LUTs 97,398 -> 93,263 (-4.2%), total LUTs 163,458 -> 159,455 (-2.4%),
LUTRAM +132, FFs +64, BRAM and DSP unchanged. Token co-simulation: s2w4i2
and tinygpu 0 errors at unchanged host cycles per target cycle; the 4-core
full GPU (Verilator, avedsim) runs demo with 39,104 instructions in 72,519
cycles, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 7098d6b)
circt_elaborate.sh and maxsim.py pass firtool's
--add-vivado-ram-address-conflict-synthesis-bug-workaround: a RAM with a
registered read gets rw_addr_collision = "yes" unless it states its own,
and an asynchronous-read RAM without a style gets ram_style =
"distributed". Without it Vivado leaves a read colliding with a write
undefined in a write-first block RAM (Synth 8-6430 on the IPDOM store
the read retiming converts). Other tools ignore the attributes.

With CIRCT cc3053a6a (memory ports sharing a clock use one clock net) this
fixes the single-bit errors of sgemm -n1024 on the full-GPU V80 image:
Vivado had mapped the issue slices' register file and the L2 data store
to block RAM in WRITE_FIRST mode with independent clocks.

s2w4i2, V80 out-of-context synthesis: area unchanged (93,192 logic LUTs,
66,061 LUTRAM, 23,943 FFs, 66 BRAM). Token co-simulation 0 errors on
s2w4i2 and tinygpu; the 4-core full GPU in Verilator runs demo with
39,104 instructions in 72,519 cycles, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 81c97e1)
A forced-block-RAM RAM read asynchronously (FORCE_BRAM_ASYNC) said
rw_addr_collision = "no". The elaboration makes its read synchronous at the
address its registers are about to take, which returns a write of the same
edge, as the asynchronous read did: write-first. With "no", Vivado built
the IPDOM stack store as a NO_CHANGE block RAM and warned (Synth 8-6430)
that a read colliding with a write is undefined. The branch now uses
RW_RAM_CHECK like every other write-first block RAM. ASIC is unaffected
(both macros are empty there).

xilinx/dut core, 16 warps, 8 threads, V80 part, synthesis: every block RAM
is READ_FIRST with CLOCK_DOMAINS = COMMON, no 8-6430 warning; LUTs 62,725 ->
62,429, block RAM unchanged (69.5).

hardware_library.md says how the elaborated netlist states read-under-write
behavior, and to check a netlist on the primitives' WRITE_MODE and
CLOCK_DOMAINS rather than the synthesis report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 25ae11e)
maxsim becomes foldsim, and its compiler moves out of the CIRCT fork into
its own project that builds on CIRCT, as POCL builds on LLVM: the flow
calls the installed foldsim driver (hw/syn/common.mk's FOLDSIM), and the
toolchain installs and packages circt and foldsim as components with
their own producer scripts (ci/circt_install.sh.in,
ci/foldsim_install.sh.in) and --circt/--foldsim flags.

foldsim now owns its target, foldsim_target (hw/rtl/afu/foldsim/target):
Vortex_axi with its ports flattened, the boundary foldsim channelizes. It
no longer elaborates the firesim backend's wrapper, and the flow, sources
and docs describe the transformations in their own terms.

Validated in-process (avedsim, Verilator) with --driver=foldsim: tinygpu
demo PASSED (1222 instrs, 24402 cycles) and the 4-core full GPU demo
PASSED (115459 target cycles), both unchanged from before the rename.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 68e2f0f)
The kernel's ID register reads "FOLD" (0x464F4C44), the runtime checks
it, and ci/foldsim_install.sh takes vortexgpgpu/foldsim at vortex_3.x by
default, the baseline building_toolchain.md names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit e077da5)
foldsim serves only the foldsim flow, so the default install (no flags)
no longer fetches it; --foldsim and --all still do, as the docs say of an
optional component. circt stays in the default install: every synthesis
flow elaborates with circt-verilog.

circt() fetched parts until any wget failed, so a network error read as
the end of the archive, and parts left by an interrupted run were
concatenated into the next one. It now removes old parts first and
fetches an explicit count, as verilator() does; the count moves with the
published parts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit a2a4895)
A re-run fetched origin and then checked out the local branch, which
stays where the first clone left it, so the build never moved to the
branch's new tip. The source is now checked out detached at
origin/<REV> when REV is a branch, or at REV itself when it names a tag
or a commit, with tags fetched (and moved tags updated) first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 888ace0)
FOLDSIM_CMD_RESET drove only the target's reset token. The memory
endpoint kept its pipe and its sticky error until the kernel's own
reset, so after a reset the fresh target was handed responses to
requests its previous run made, and a memory error could never be
cleared.

The control endpoint now gives the memory endpoint the target's reset as
a token of the same target cycle. A target cycle in reset takes no
request and finishes only once HBM has answered everything it has from
before -- reads returned, writes acknowledged, nothing left offered --
then drops the pipe (requests not answered, data not delivered, the lone
half of a write) and the error. HBM-side AXI stays in protocol, and no
response from before a reset reaches the target after it.

A burst set the error but still went out: its non-last W beats were
pushed to HBM without an AW and were not limited by w_ready, so the
16-entry W queue could overflow and the HBM port went out of protocol.
Now a burst only sets the error and is dropped; from the next cycle the
endpoint is ready for nothing. Writes go to HBM only once both halves
are in, so a lone half can be dropped without withdrawing a valid, and
`idle` covers transactions HBM still owes. The max-in-flight compares no
longer truncate the 8-bit limits to the counters' width.

hw/unittest/foldsim_mem drives the endpoint with an AXI4 master as the
target against an HBM model that checks the device-side protocol:
responses, ids and latencies, resets at random points with HBM busy,
and each kind of burst. The previous endpoint fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 9011816)
The command processor model's hooks return no status, and the backend
dropped every failure: vortex_busy read a memory error or a failed
status read as idle, so the launch retired as a success; DCR writes and
starts ignored whether the command was queued; a DCR read polled for its
response with no memory-error check and no limit.

A device error -- a memory error, a failed register or memory access, a
wait that timed out -- is now latched: it is reported once, the hooks
stop touching the target, and every register access of the runtime core
fails from then on, so the launch, DCR access or copy that hit it fails
instead of completing. Waits for queue room, a DCR response or the idle
target are bounded (60 s, or VORTEX_FOLDSIM_TIMEOUT_S), and a launch is
bounded by VORTEX_FOLDSIM_TIMEOUT_S when it is set, after the firesim
backend's VORTEX_FIRESIM_TIMEOUT_S. Open checks the memory error only
after its reset, which now clears one an earlier process left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 01cbe30)
…target source

gen_sources.sh wrote the source list before the VENDOR_OK patch, the
shell-module removal and the VX_MEM_PORTS check, so a failing check left
a list newer than its sources and the next make went on with it. The
list is now built in a temporary and moved into place last.

Every file under hw/rtl was a dependency of the list, so editing the
kernel (foldsim_token/mem/ctrl/afu.sv, foldsim_regs.vh) regenerated the
Vortex sources and reran the compiler. Those files are only read by the
kernel's packaging and emulation, which depend on them themselves; the
target's own wrapper under target/ stays a dependency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 29ab074)
…itional ones

rewrite_site kept only connections with a name: `.*` and positional
connections were dropped silently, and an implicit `.clk` became
`.clk ()`, leaving the port unconnected. `.p` is now `.p (p)`, `.*`
connects every port not named otherwise to the signal of its name, and
a positional connection -- at a lifted site, or at a site a promoted
parameter becomes a port of -- stops the lift with a message, since the
rewrite reorders ports. srcs.f is written only after every edit applied,
so a failed lift no longer leaves a list that looks finished.

The lift of the tinygpu unit set is unchanged, byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b8cab53)
Only tests/regression had run-foldsim; the OpenCL and HIP makefiles get
the same target, with their run-firesim's environment: TARGET=hw runs
FPGA_BIN_DIR's foldsim.vbin on the V80, anything else the in-process
simulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 033c522)
…x targets

simulation.md said avedsim has nothing to build; the run compiles the
simulator with foldsim and builds it with Verilator, for the app's
CONFIGS. building_toolchain.md's CIRCT configure line now matches
ci/circt_install.sh (-DCIRCT_SLANG_BUILD_FROM_SOURCE=ON), and
blackbox.sh --help names foldsim's targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 55515a2)
A command, DCR read or reset now fails only once the target has advanced
no cycle for the control timeout. In-process, a large target can take
minutes of wall time to answer a DCR read that flushes its caches; a
wall-clock bound failed such runs while the simulator was making
progress.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit fc479c4)
-f v2005 is read by tools with a Verilog-2005 frontend, which reject
SystemVerilog N'(...) casts; firtool's disallowSizeCasts keeps signed
multiplies and bitcasts free of them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 1bc0962)
On aved the simulator's clock gates become clock enables in synthesis,
which cannot enter an out-of-context vendor core. VX_fpu_dsp's FMA,
divider and square root select Xilinx cores on Vivado, so a V80 image
failed the platform's check for logic-driven clocks. The aved build now
takes the STD FPU and its units' soft paths in the flow's own copy of the
sources, as the foldsim flow does; the metasimulator and the image run
the same FPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 80f322a)
The generated simulator depended on the foldsim driver script alone, so
a rebuilt foldsim-opt or CIRCT tool left a stale simulator in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 5238cd5)
…cover the V80

An installed LLVM has no llvm-lit, so the producer could not run
check-foldsim against an installed CIRCT; it now takes pip's lit (or LIT).
building_toolchain.md's FireSim section described the U55C alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit c808baa)
tinebp and others added 14 commits November 7, 2026 20:30
Between the coalescer merge and commit, the RTL LSU has three 2-entry
skid buffers, each with a registered ready: rsp_buf and no_rsp_buf
(VX_lsu_block), the response-first rsp_arb output buffer, then the lane
gather (VX_lsu_unit). A stall at commit reaches the LSU's response input
one stage per cycle. SimX pooled the first two stages into one 4-entry
output that freed its slots at once, so it kept accepting responses after
the RTL had stopped (two FPU commits at 770-771 on parity-sgemv make the
RTL refuse responses at 773-774), and two ports' responses that the RTL
realigns into one fragment stayed two.

SimX now models the stages separately: response and no-response buffers
(2 entries each), a response-first arbiter into a 2-entry output, and a
registered-ready LSU commit queue. Total capacity and latency are
unchanged; stores, direct commits and fence releases now pass through the
no-response buffer, as skipped requests do in the RTL.

Core model_parity at the arbiter change: 40/40; without it, parity-sgemv
0.19%, sgemm-mc64 2.24%, mixfu-simd 1.62%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The FPU has two tags, and a tag is freed only when its result commits.
Commit was fixed priority ALU > LSU > SFU > FPU > TCU, so the scarcest
resource waited behind the most plentiful results: on sgemm-mc64 an FPU
result waited 3.3-3.7 cycles per op at commit and the FPU issue was
tag-bound for 87.7k of 131k cycles. Convoys of L2-hit fills with the
command processor's DMA through the LLC (60d7947) made the LSU bursts
longer, which is why that change cost sgemm-mc-nt4 +1.05%.

Commit order is now FPU > ALU > LSU > SFU > TCU, in RTL (a permutation of
VX_commit_slice's existing fixed-priority arbiter inputs, no new state)
and in SimX (kCommitPrio). The two tags bound how many beats the FPU can
take, so the others lose little. Round-robin was rejected (fp16-mc-nt16
+14%), as were FPU-first-when-tags-full (+4.7%) and ALU > FPU > LSU.

Each FPU core's 2-entry registered output buffer (FPU_CORE_OUT_BUF) is
replaced by a zero-latency skid register (VX_bypass_buffer; new OUT_SKID
parameter of VX_pe_serializer, default off, so other users are
unchanged). Its ready is still registered, so the core's clock enable
still starts at a register; the result no longer pays a cycle. DPI, DSP
and STD change together and SimX drops the cycle.

rtlsim, 22 cases vs the parent: geomean -3.89%; sgemm-mc-nt16 -25.1%,
sgemm-mc-nt4 -23.6%. Four single points are slower (perf_gate-fp16
+2.98%, mxfp8 +3.24%, sgemv-nt16 +2.49%, fp16-isw4 +1.80%); over four L1
link latencies their means are within 0.4-0.9% of the parent's, inside
the parent's own 3-9% spread, so they are phase effects.
Beacon core and top_vortex: no register deeper; core LUT -1.4%, FF -1.8%.
Core model_parity 40/40. XLEN 64 build clean; zfhmin std/dsp, isa-32f-dsp
and isa-64d-dsp-fma64 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recorded at 727273b with the v3.0.1 toolchain refresh (LLVM
f44a6bb2af4c), all 49 rtlsim XLEN 32 cases, three shards:
  pytest ci -m "perf_gate and rtlsim" --update-baselines

Every case is attributed per docs/bug_fixes.md section 6 against the
previous baselines (48f4e42) and the 4ecd8a0 recording.
42 cases are faster, among them sgemm-mc-nt16 -24.3%, sgemm-mc-nt4
-21.9%, the DXA sgemm2 cases -29% to -42%, raster -10%, om -6% to -9%.
Slower than the previous baselines:
- sgemv-nt4 +7.87%: the compiler drops a stack spill (384 fewer
  instructions), which moves the loop 8 bytes; the 512x512 case is
  bistable (rows alias onto 2 dcache sets). Over L1 link latency +0..+3
  it runs 934290, 848936, 781082, 1009304; the old baseline is inside.
- mxfp8 +4.56%: bistable; over L1 link latency +0..+3 it runs 80153,
  79640, 81188, 76072, mean 0.4% below the parent's; old baseline inside.
- copy-mcast +1.42%, fp16 +0.73%, wgmma-dxa-mcast-nt4/nt16 +0.71%/+0.45%,
  softmax-nt16 +0.13% (lazy fetch): within the gate, attributed in the
  sign-off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VX_pipe_buffer held valid_in in valid[0] and the stage registers in
valid[1..DEPTH], one vector (same for data). The logic is acyclic, but the
debug CI category builds xrtsim and opaesim with Verilator -O0, which skips
the pass that splits vectors, so any outer path from valid_out back to
valid_in reads as a loop through that one variable. The cache's flush token
makes such a path: mem_flush_valid waits for the memory-request crossbar to
be empty, which is its output buffers' registered valid_out, and the token
enters the same crossbar (f93df32). Every debug xrtsim/opaesim job has
failed on UNOPTFLAT since.

The stage registers now live in their own vectors and each stage takes its
input from valid_in or the previous register. Proven equivalent to the old
module with yosys equiv_induct for DEPTH 1-3, with and without reset (a
mutant is rejected). debug demo -O0 now builds and passes on xrtsim and
opaesim at XLEN 32 and 64 with no UNOPTFLAT.

The Verilog guideline's UNOPTFLAT section said packed vectors are always
auto-split; it now says the split does not happen at -O0 and forbids mixing
a combinational input with registers in one vector.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 2's perf_gate baselines (f30b606) were recorded with LLVM
f44a6bb2af4c, which the prebuilt ships from 063b1ca. That commit had been
published by moving the rolling v3.0.1 tag, which master also pins; master's
baselines predate the new LLVM, and perf_gate checks retired instructions
exactly, so the move would fail master's perf_gate (sgemv drops 384
instructions). v3.0.1 is back at 8a50bce, the toolchain master was last
validated with, and 063b1ca is tagged v3.0.2 for this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RISC-V reads a narrower value as valid only if every bit from its top up to
FLEN-1 is one; bits above FLEN are not part of the FP register. VX_fcvt_unit
tested the half operand's boxing over dataa[XLEN-1:16]. A unit built with
FLEN 32 on XLEN 64 therefore read every half boxed to 32 bits as the
canonical NaN. The single-precision check in the same unit, and
VX_fncp_unit's, already stop at FLEN.

The hw-fpu unittest builds the unit that way and failed at XLEN 64 (fcvt_unit
h2f errors=92353) ever since the half check came in with b447578. The core
cannot see the difference: every FP register write fills ones up to XLEN.

hw-fpu unittest: all five units errors=0 at XLEN 64 and 32; reverting the
fix reproduces h2f=92353. zfhmin category 3/3 (XLEN 64) and 6/6 (XLEN 32)
on simx and rtlsim. Verilator lint of VX_fcvt_unit is clean at XLEN64/FLEN64
with and without half, XLEN64/FLEN32 and XLEN32/FLEN32.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VX_tcu_bbuf re-latched the slot's descriptor fields (B row base, K-major
stride, row-major mode) on every first compute uop, even while a fetch was
in flight. The buffer's request comes from the lowest-indexed active TCU
block, so another CTA's first uop arriving on a lower block mid-fetch
overwrote the slot base for a cycle. The K-major fetch reads that base live
per row, so one B row came from the other CTA's shared memory.

sgemm_tcu_wg_sp-6 at XLEN 64 hit this window: the trace shows CTA 2's fetch
reading row 0x26 (CTA 3's buffer). It got 64 wrong C elements, columns 32-33
over two warps' rows. The defect dates from the descriptor latch in
e2b9745 and is on master too; FETCH_WIDTH=2 (e587204) lines up the two
CTAs' first uops within one fetch window.

The descriptor fields are now latched at allocation with the step and CTA
fields, and the fetch address, mode and stride at allocation come from the
request's own descriptor, which every compute uop carries. SimX already keys
and addresses the B buffer from the requesting warp's descriptor at
allocation, so it is unchanged and no cycle count moves (sp-6 XLEN 64 stays
at 12826).

Beacon tcu vs 3b2b192: deepest path 25 -> 25, mean of the 50 deepest
17.4 -> 17.4, registers at 8+ levels 156 -> 156, LUT -9. slot_row_major_r
goes from 4 to 10 levels. It now shares the allocation enable of the other
slot registers, which are at 10, so the cost belongs to latching at
allocation; it stays below 12.

rtlsim: sp-6 passes at XLEN 64 and 32; tensor_wg 10/10 (XLEN 64) and 17/17
(XLEN 32, with its model_parity and perf_gate cases); tensor 31/31 (XLEN 32,
6 model_parity, 6 perf_gate).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dcache took its set index straight from the address bits above the bank
field, so a power-of-two stride of NUM_BANKS x LINE x 2^k or more kept the low
set bits fixed and piled into NUM_SETS / 2^k sets. The default dcache (16 KB,
4 ways, 2 banks of 32 sets) mapped the 16 rows sgemv 512x512 keeps live, 2 KB
apart, into 2 sets; the run flipped between a thrashing and an escaping state on
any one-cycle timing change.

The set index is now the raw set bits XORed with an XOR fold of the tag's low
32 bits, in chunks of the set-index width (cache_set_hash in VX_gpu_pkg). The
tag is stored unchanged, so the same XOR maps a bank's line address back:
VX_cache applies it ahead of the request crossbar and undoes it on each bank's
memory request before the raw bank bits are formed, and CS_BANK_TO_FULL_ADDR
undoes it so trace and assert addresses stay real. Tags, MSHR, sectors,
replacement and the flush walk are untouched. SimX's params_t::set_xor is the
same function, in addr_set_id and mem_addr.

VX_CFG_DCACHE_SET_HASH is on by default. The L2 and L3 get no set hash: in
every measured case their working sets fit without conflicts and the knob
changed nothing, so it is not carried there.

Gains (rtlsim):
- sgemv 512x512: -20.1% at 4 threads, -23.9% at 16; read hit rate 29% -> 85%;
  spread over L1 latency +0..+3 is 2.1% (29% without the hash)
- sgemv 512x1024 and 512x2048: -23.2% and -23.8%
- mxfp8 tensor -4.8%, fp16 tensor -3.7%, fp16 at issue width 4 -3.5%
- perf_gate geomean over 49 cases: -1.12%

Cost, inherent to set hashing for two TCU tile layouts (averaged over L1
latency +0..+3): fp16 on 2 cores at 16 threads +1.8% (misses +39%), sgemm2
fp16 +1.1% (misses +21%). The low-tag-only XOR variant is worse (+3.4% on the
first), so the cost belongs to moving which lines conflict, not to the fold.
Area: +8 LUT per dcache; Beacon shows no deeper register.

Test change, approved by the user: cache_ways placed its "same set" lines a
raw set stride apart, which the hash spreads over several sets. It now picks,
with the hash on, a start and a step (in whole set strides) that keep every
line's tag fold equal, so the lines still share one set; with the hash off the
placement is unchanged. cache_ways-1/2/3 pass on simx and rtlsim.

Verified on the final Phase 2 tree (this change + the read-merge realignment):
perf_gate 49/49 (re-recorded in a following commit), cache 70/70, model_parity
(core, tensor_wg, dxa, vm) 59/59, regression 201/201 at XLEN 32 and 64 on simx
and rtlsim, lint at XLEN 32 and 64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dcache read merge serves a uniform load's two channels with one bank access
only when both reach the port arbiter in the same cycle. After the LSU each
channel has its own FIFO path, and a contiguous load or store sends its two
channels to the same bank with different words. The bank crossbar serves them
a cycle apart, the first-served channel's FIFO moves on, and from then on its
requests reach the arbiter one cycle ahead of its partner's. Every following
uniform read of the pair is then unmerged. Nothing realigned the pair except a
busy-bank stall. Both models lose merges this way, by different chaotic amounts:
vm:parity-vm-vecadd-wide (IW 4, 64 warps, 32 threads, 4 LSU blocks x 8 lanes)
swung from -0.9% to +7.2% between nearby sizes and failed the 5% gate in CI.

Each same-word pair of ports (READ_MERGE group of 2, no line merge) now
realigns. A port that moves past a head it shared with its partner (same tag,
same line, a regular request) while the partner still holds that part waits:
it is not valid to the crossbar or as a merge leader, and not ready, until the
partner's head moves on. If both ports of a pair waited in the last cycle, both
go (registered mutual release), so tag aliases cannot deadlock. The pair's next
requests, including the next uniform read, then arrive together. RTL:
VX_cache g_read_merge.g_realign; SimX: CachePortArb::update_realign.

The rule's scope comes from measurement:
- pairs only: with 4-port groups the first port waits up to 3 cycles and gives
  up the overlap of the next request on other banks (sgemm-mc-nt16 +4.7%,
  sgemv-nt16 +3.5% at that step); 4-port groups are unchanged;
- same line only: a tag-only rule held ports on TCU strided stores whose parts
  sit in different banks (wgmma-dxa-mcast gap 2.68% -> 4.41%); that case is
  now unchanged in both models;
- same-word groups only: the icache's line-merge slices are independent clients
  and livelocked when realigned;
- off the grant path: the wait reads registers and one tag/line compare against
  the partner's current head, and the mutual guard is registered.

vecadd-wide over 7 sizes (12288-32768), SimX vs rtlsim gap: mean 3.90% (range
-0.88..7.19) before, 1.81% (1.03..2.92) after; n=16384 rtlsim 16410 -> 14563
cycles (-11.3%).

Beacon (cache, top_vortex) against the parent: deepest path 23 -> 23 levels;
registers at or above 8 levels 675 -> 675; 18 registers on the dMMU input and
the dcache request crossbar 1-2 levels deeper, all below the deepest path;
top_vortex +136 LUT. The cache DUT (4-port groups) is unchanged.

perf_gate on the final tree, this change alone: sgemm-mc-nt4 -1.11%,
sgemm2-fp16 +0.02%; the other 47 cases unchanged.

Verified on the final Phase 2 tree: perf_gate 49/49, cache 70/70, model_parity
(core, tensor_wg, dxa, vm) 59/59, regression 201/201 at XLEN 32 and 64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Regenerated with pytest ci -m perf_gate --update-baselines on the final Phase 2
tree (the two preceding commits), as approved by the user: the dcache set-index
hash on by default, plus the read-merge pair realignment. Retired-instruction
counts are unchanged in all 49 cases; only cycles move. Geomean -1.15%
(set hash alone -1.12%).

The realignment moves only sgemm-mc-nt4 (-1.11% of the -2.38% below) and
sgemm2-fp16 (+0.02%); every other movement is the set hash. The two TCU
regressions are its inherent cost (more conflict misses for those tile
layouts; see the set-hash commit), and sgemmx-nt16 moves with the changed miss
timing.

Every case that moved by 0.5% or more (rtlsim, XLEN 32, cycles):
 -23.87%  core:sgemv-nt16                      375500 -> 285869
 -20.12%  core:sgemv-nt4                       934290 -> 746326
  -4.81%  tensor_mx:perf_gate-mxfp8             80153 ->  76299
  -3.73%  tensor:perf_gate-fp16                254637 -> 245149
  -3.48%  tensor:perf_gate-fp16-isw4           164504 -> 158780
  -2.38%  core:sgemm-mc-nt4                   1720808 -> 1679928
  -1.61%  dxa:perf_gate-wgmma-dxa-mcast-nt4    360918 -> 355108
  -0.78%  core:sgemm-nt4                      5915949 -> 5869813
  -0.74%  tensor:perf_gate-fp16-mc-nt4         221154 -> 219517
  +0.90%  core:sgemm-nt16                     1542695 -> 1556536
  +0.92%  dxa:perf_gate-wgmma-dxa-mcast-nt16    78705 ->  79428
  +2.07%  core:sgemmx-nt16                    1002839 -> 1023552
  +3.74%  tensor:perf_gate-sgemm2-fp16         527038 -> 546754
  +4.97%  tensor:perf_gate-fp16-mc-nt16         35429 ->  37191
The other 35 cases move by less than 0.5%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the LSU is narrower than the warp, a TCU_LD goes out in passes, and the
LSU can hold a later pass's request while an earlier pass's response returns.
The AGU handles that order (responses are counted by eop and written by their
pass tag), but its assertion expected a response only after the last pass had
been requested, so it fired on correct traffic.

tensor_mx:model_parity-mxfp8-lsu-lanes (NT 8, 4 LSU lanes, 2 passes) stopped
in rtlsim at cycle 6889: pass 0's response arrived while pass 1's request was
held. The dcache timing of the last push brought that order out; the case
passed before it.

The logic is unchanged. Verified: both tensor_mx lsu-lanes cases pass on
rtlsim (the 2-pass and the 4-pass shape), and both build clean under Verilator
-Wall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
logic_gate guards what perf_gate guards from the other side: perf_gate fails a
change that costs cycles, logic_gate one that makes a unit deeper or the design
bigger. It ran only from its own workflow, so a pull request could land a
deeper unit with every ci.yml cell green.

Its gate case is now an rtlsim-driver case at tier smoke, so the planner plans
its cell exactly when it plans perf_gate's rtlsim cells: every pull request,
and every push that forces rtlsim in (the RTL or the config). The cell installs
foldsim through `needs`. The whole-GPU group stays opt-in (tier timing_gpu,
~95 GB per build), and logic_gate.yml keeps only manual dispatch, for a subset
of the catalog or the gpu group.

The baselines (ci/baselines/synthesis/beacon/) are recorded with
`ci/logic_gate.py --update-baseline` at 0f76899 with the CI toolchain
(v3.0.2: CIRCT 892a65a01, foldsim d8350bc0f), for all 14 builds of the
catalog. The cell run against them passes all 14 at +0%. The RTL commits
since then change only a simulation assertion.

AGENTS.md states the rule; CHANGELOG and the CI design doc describe the cell.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A test category can now be a directory of files, ci/testcases/<category>/*.yaml,
each file with its own defaults, that runs as one cell per driver and XLEN.
Eighteen of the smallest suites ran their own cells of 2-6 minutes, most of it
the cell's checkout, toolchain restore and build; they now share four:

  isa          mufu, zfhmin, rvc, riscv, amo
  sw           opencl, hip, kernel, runtime
  integration  gem5, sst, dtm, cupbop, mpi
  tensor_sp    tensor_sp, tensor_sp_mx

A pull request plans 82 cells instead of 108, a push 35 instead of 49, the
nightly 120 instead of 151.

A cell runs every case of its category on its driver whatever the case's
tier, so each directory holds suites of one tier only: debug (smoke) and
scope (nightly) stay apart, and mpi joins the nightly integration suites.

Every case keeps its command, configs, drivers, XLENs and tier: all 932 cases
resolve to the same specs as before apart from their category. Each file is
also a marker named after it, so `-m mufu` and `regression.sh --test gem5`
still select one suite. Case ids are unique within a category, so 29
placeholder ids drafted from regression.sh (isa-N, legacy, ...) take their
suite's name (rvc-N, hip-N, gem5, ...); ci/runtimes.json follows the new ids.

The mpi and gem5 installs key on the cell's `needs` instead of the category
name, and mpi declares `needs: [mpi]`. configure copies category directories
into the build tree, the lint rejects a category or suite named after a
check, driver or tier, and the change-impact detectors cover the nested
files.

Verified: lint; every merged cell collects in pytest from a freshly configured
build; ci/test_change_impact.py 134/134.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rrode

The Xilinx and Altera DUT flows' build.mk append the DUT's flags with
`override CONFIGS += ...` before they include config.mk. config.mk then folded
the profile in with a plain `CONFIGS := $(PROFILE_CONFIGS)`, because CONFIGS
did not come from the command line, and make ignores a plain assignment to an
overridden variable. Every DUT-flow build that names a profile therefore ran
without it, with no error: logic_gate's gpu_rtx5080 and gpu_h100 builds Beacon'd
a default core. They are opt-in and have no committed baselines. The xrt, aved
and yosys flows include config.mk before their own overrides and were not
affected.

The fold now takes the override for a CONFIGS from the command line or one an
including Makefile already overrode.

Verified in a DUT work dir with PROFILE=rtx5080: the build's flags now carry
the profile (48 warps x 32 threads, 128 KB dcache, 64 MB L2, RTU, TLAS, ...); an
explicit -DVX_CFG_L2_SIZE in CONFIGS still wins over the profile; a build with
no profile is untouched, since the block is skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The synthesis cell of run 42542278556 failed before Yosys ran: GitHub
answered HTTP 504 to all four attempts at an ASAP7 Liberty file, and
curl's --retry 3 backs off only 1, 2 and 4 s. The 64-bit cell of the same
run, 45 min earlier, passed.

The synthesis cases now declare `needs: asap7`, and ci.yml caches the
downloads directory of such a cell, keyed on the manifest's hash. The
installer checksum-verifies each file and skips any already present, so
a cache hit fetches nothing. On a miss it now retries six times 20 s
apart (wget: 7 tries, retrying 429 and 5xx).

Verified: testcase lint (932 cases); both synthesis cells plan with
needs [asap7]; a fresh install with the new fetch, and an install from
restored downloads that fetches nothing and gives identical libs; the
XLEN 32 synthesis case passes locally at 589aaf2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tinebp and others added 6 commits November 9, 2026 08:13
FPGA (Vivado 2024.2) baselines are recorded on 589aaf2, the Phase 2
tree whose synthesis flows read the preprocessed, RAM-folded sources.
ASIC (Yosys 0.69 + OpenSTA 3.1.0, ASAP7 RVT TT) baselines are recorded
on the same sources read through Yosys's slang frontend, all 14 builds,
at the catalog clocks of this commit.

Catalog clocks and budgets are unchanged. Against them the ASIC gate
reports cache (+22.3%) and lmem (+8.0%) over their area budgets, and
the FPGA gate reports core Fmax 286 MHz (needs 294) and tex, gfx and vm
over their LUT budgets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
core, lmem, top, tensor, rtu, gfx and vm were constrained to 400 MHz and
tcu to 500, which let ABC map them for a clock no Vortex ASIC targets and
hid how far they are from 800. Every build now carries the same 800 MHz
commitment. Area budgets are unchanged. The comments name Yosys's slang
frontend in place of sv2v, which the flow no longer uses.

The baselines of the eight retargeted builds are re-recorded at 800 MHz
in a follow-up commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…C flow

No synthesis tool reads an elaborated netlist: Vivado, Quartus, Design
Compiler, FireSim's image and now Yosys all read the CIRCT-preprocessed,
RAM-folded sources (gen_sources.sh -P -F) and elaborate them themselves.
A CIRCT-elaborated netlist cost Fmax in Vivado (core 295 -> 286 MHz,
cache 300 -> 276) and the Yosys flow was the last one still fed one.

Yosys reads the sources with its built-in read_slang (Yosys >= 0.67),
with the SRAM macros black-boxed through --blackboxed-module. OpenSTA's
Verilog reader rejects parameter overrides, so run_synth.sh strips them
from the black-box instances of the mapped netlist; the SRAM estimate
reads port widths from the JSON netlist written before the strip. sv2v
stays out of the toolchain. circt_elaborate.sh now writes MLIR only, for
Beacon, and gen_sources.sh loses -E/-X. FireSim and Altera OPAE pass the folded
per-file lists; OPAE generates afu_json_info.vh and copies the platform
headers before preprocessing, since the preprocessor inlines them
(untested: no OPAE SDK on this machine). xilinx_slr_pblocks.tcl drops
the CIRCT netlist's g_x_i_ generate-scope names.

slang rejects RTL that Vivado, Verilator and CIRCT accept: wires used
before their declaration (VX_socket, VX_decompressor, VX_cp_dma,
VX_tcu_core), a localparam in a port width before its declaration
(VX_cp_unpack), an interface member expanded through SEXT (VX_alu_int),
a negative-width cast checked uninstantiated at UUID_WIDTH=1
(VX_uuid_gen) and $bits of an interface member as a cast width
(VX_rtu_core). All are rewritten with
no logic change; Beacon reports identical register depths on core, tcu,
rtu and top.

The FPGA and ASIC baselines are recorded on this flow (b16dd30).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in RHEL 9 toolchain support (48fc006) and the softfloat fix.
Conflicts: CHANGELOG.md (both entries kept, RHEL 9 entry without sv2v)
and ci/toolchain_install.sh.in (ours: no sv2v); the rhel/9 note in
docs/building_toolchain.md drops sv2v the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
core, lmem, top, tensor, tcu, rtu, gfx and vm, now constrained to
800 MHz (88d621f). Cell area, SRAM area and cell count are identical
to the 400/500 MHz recording, so ABC maps these designs the same way
whatever the target; Fmax moves 1-5% and power doubles with the clock.
core (655 MHz) and top (654 MHz) miss the 784 MHz floor; lmem stays
8.0% and cache 22.3% over their area budgets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A hand-kept count goes stale as modules are added or removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tinebp
tinebp merged commit 127178d into master Nov 9, 2026
93 checks passed
@tinebp
tinebp deleted the feature/mlp-phase2 branch November 9, 2026 17:54
tinebp pushed a commit that referenced this pull request Nov 11, 2026
… + feature/beacon)

Upstream master since 50cb174: 48fc006 RHEL 9 toolchain, 127178d MLP
phases 1+2 (#436, a squash of origin/feature/mlp-phase2 and its later
commits), 2565c92 docs, ca18837 synthesis-gate fixes after the MLP merge
(#461).

This branch already holds mlp-phase2 948db57 (a100/integ-1106's 11-06
merge into a100/cov b83da51, 117 conflicts resolved there), so the merge
is three-way against 948db57, not against 50cb174 (a plain merge of
the squash reported 207 conflicts, most of them the same commits rebased
on both sides). 12 conflicts:

- VX_cache (RED WARNING): upstream's read-merge pair realignment is written
  for its flat front (core_req_line_addr / core_req_bid / core_req_live);
  our front is the VX_cache_req_port structure (a100/l2-front 1d93c36).
  Not ported: no realignment in the RTL or in SimX (its SimX twin is
  removed too, so the two stay in step). SET_HASH (set-index hashing, on by
  default for the dcache) is ported: VX_cache_req_port hashes the line it
  packs for the banks, as upstream's front does, since the memory side's
  unhash (per_bank_mem_req_pdata, CS_BANK_TO_FULL_ADDR) merged cleanly.
- sim/simx/mem/cache.cpp: our request stage / arbiter-tree model kept;
  upstream's realign model dropped (above); set hashing merged.
- VX_socket: our registered MMU drain status kept; upstream only moved
  dmmu_lane_idle's declaration up (merged).
- VX_alu_int: upstream's imm20 wire with our is_lui_op.
- VX_cp_dma, VX_cp_unpack, VX_tcu_core: ours (the same declare-before-use
  fixes, already made here); VX_tcu_agu: ours (the same TCU_LD response
  assertion fix, on our batch counter).
- VX_fpu_pkg: upstream's (FPU_CORE_OUT_BUF goes: the FPU cores' registered
  output buffer is upstream's zero-latency skid + per-unit registered clock
  enables, VX_pe_serializer PE_ENABLES, which merged cleanly, with SimX's
  FPU latency one cycle shorter in step).
- hw/scripts/circt_elaborate.sh: upstream's (MLIR output only), with -f mlir
  still accepted for older callers.
- CHANGELOG, docs/designs/floating_point_unit.md: both sides; the realign
  entry is left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants