Repository navigation
Expand file tree
/
Copy pathsynth_gate.py
More file actions
executable file
·2471 lines (2163 loc) · 111 KB
/
Copy pathsynth_gate.py
File metadata and controls
executable file
·2471 lines (2163 loc) · 111 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
#!/usr/bin/env python3
"""synth_gate — synthesis-regression gate over a checked-in golden baseline.
Same shape as the perf_gate (ci/perf_baseline.py): a golden baseline per build,
a hard threshold around it, and baselines that only a human updates. The
assertion differs: instead of rtlsim cycles it compares THIS commit's
post-synthesis timing and area against the recorded numbers, so an RTL change
that quietly costs timing closure or area cannot land green.
Four backends, one gate. Everything that is not "how do I drive the tool and
what does it call its numbers" is shared:
--tool=xilinx Vivado synthesis + place-and-route -> ci/fpga_gate.py
--tool=yosys Yosys + OpenSTA on an ASIC PDK -> ci/asic_gate.py
--tool=beacon logic levels per RTL unit, no synthesis -> ci/logic_gate.py
--tool=beacon_asic Beacon's cell stages on asic_gate's builds (ASAP7)
Spec : ci/testcases/<gate>.yaml (what to build)
Baselines : ci/baselines/synthesis/<tool>/<group>.json (last accepted metrics)
Metrics : <build>/synth_summary.csv (beacon_summary.csv for Beacon),
written by the flow itself
Same division as every other check in the catalog: the yaml is hand-authored,
commented and reviewed; the baseline is machine-written and never hand-edited.
A `config_hash` (dut/clock/configs/tool env/xlen) ties the two together, so
metrics recorded under one config can never be compared against another.
See docs/designs/continuous_integration.md 3.5.
"""
import argparse
import concurrent.futures
import csv
import fcntl
import glob
import hashlib
import json
import os
import re
import shlex
import shutil
import signal
import subprocess
import sys
import time
import yaml
# Fmax is held to the build's target clock and area to its fixed budget, both
# from the catalog and both one-sided: Fmax may not fall more than 2% below the
# target, area may not grow more than 5% over the budget, and doing better never
# fails. Neither is measured against the last recorded baseline, so re-recording
# cannot ratchet the limit; raising a budget is a reviewed catalog edit. Any
# other metric gated with --gate is compared with its baseline both ways: the
# improvement side is the ratchet, so a win must be recorded to lock it in.
#
# These are the GLOBAL defaults. They are overridden per-metric on the command
# line (--metric-threshold lut=0.10) and per-build in the catalog ("thresholds":
# {"fmax_mhz": 0.03}) -- most specific wins. Same shape as model_parity's
# `tolerance` (per-case entry > category defaults > DEFAULT_PARITY_TOLERANCE).
TOLERANCE = 0.05
METRIC_TOLERANCE = {"fmax_mhz": 0.02}
# Higher-is-better metrics. Everything else is lower-is-better.
HIGHER_IS_BETTER = ("fmax_mhz",)
# Verdicts that do not fail the run. KNOWN-ISSUE is a tracked expected failure
# (see gate()); XPASS is one that stopped reproducing -- reported loudly, but a
# gate that hard-failed on an unexpected PASS would be the wrong incentive.
# Against a reference (--against-rev), ACCEPTED is a regression the baseline's
# re-record accepted, and NO-REFERENCE a build the reference revision does not
# have at all (a DUT the change adds), judged against its baseline alone. REF-FAIL, a reference that failed to measure the build, fails.
PASSING = ("PASS", "RECORDED", "KNOWN-ISSUE", "XPASS", "ACCEPTED",
"NO-REFERENCE")
POLL_INTERVAL = 5
HEARTBEAT = 300 # seconds between "still alive, here is where it is" lines
# Tool jobs per build, clamped so N parallel builds do not oversubscribe.
MIN_TOOL_JOBS = 2
MAX_TOOL_JOBS = 8
class BuildError(Exception):
pass
# ---------------------------------------------------------------------------
# tools
#
# A backend supplies four things and nothing else: what it measures, how it is
# invoked, how its log reads, and what makes two runs comparable. The gating
# core below never names a tool.
# ---------------------------------------------------------------------------
class Tool:
"""One synthesis backend."""
name = "" # --tool value, and ci/baselines/synthesis/<name>/
gate = "" # ci/testcases/<gate>.yaml, and the build-dir/prefix/stamp name
spec = "" # another gate's ci/testcases/<spec>.yaml, if not the gate's own
flow = "" # DUT entry point, relative to a configured build tree
metrics = () # recorded, in report order; build_time_s is bookkeeping
gated = () # asserted by default (--gate overrides)
area = "" # the gated area metric, held to the catalog's budget
budget_key = "" # the catalog field holding that budget
# The metric whose presence in a baseline entry means "recorded".
key_metric = "fmax_mhz"
# The flow's one-row metric file, in the build's work directory.
summary = "synth_summary.csv"
# Default gate tolerance, overridden by --threshold and the catalog.
tolerance = TOLERANCE
# Per-metric defaults over `tolerance`, overridden by --metric-threshold
# and the catalog.
metric_tolerance = {}
integral = () # recorded as ints; every other metric is a rounded float
columns = () # (header, metric, format, width) for the report table
# Whether the build's clock_mhz is part of config_hash: it is an input of
# a synthesis run, but only a gate threshold for an estimate that does not
# read it, where raising the target must not invalidate the measurements.
clock_in_hash = True
# What a register's depth is counted in, for the reasons --against gives.
depth_unit = "levels"
# The dut/catalog.mk variable naming a DUT's top module (<dut><top_var>).
top_var = "_PRJ"
# Scheduling weights (seconds) for a build with no recorded build_time_s yet.
# Only ever used to ORDER the queue, so a rough magnitude is enough.
weights = {}
fallback_weight = 1800
# Peak memory (GB) of a build, for admission: builds start together only if
# their peaks fit in the machine. A DUT not listed is not limited.
mem_gb = {}
job_label = "tool" # what --tool-jobs buys, in the schedule banner
# Log lines that mark a phase transition, in order. Progress is reported
# against these, so a long build says where it actually is.
phases = ()
# The point past which a build can no longer die of a config/RTL typo (see
# watch()), and -- where the tool prints one -- the line that says it did.
elab_done = ""
elab_fail = ""
elab_label = "elaboration"
error_re = re.compile(r"^ERROR:")
def check_env(self):
"""Fail fast if the tool is not on this machine."""
def env(self, args):
"""The tool environment recorded alongside every baseline."""
raise NotImplementedError
def hash_key(self, env):
"""The env fields that make two runs comparable, for config_hash."""
raise NotImplementedError
def env_warning(self, build, env):
"""Optional warning when a baseline was recorded under a different tool."""
return None
def dut_root(self, build_dir):
return os.path.join(build_dir, *self.flow.split("/"))
def work_dir(self, dut_root, prefix, build):
raise NotImplementedError
def prepare_flow(self, dut_root, env):
"""Once-per-session setup, before any build starts."""
def clean(self, dut_root, work, build, benv):
raise NotImplementedError
def build_command(self, dut_root, work, build, env, args):
"""(argv, extra environment) that runs this build to completion."""
raise NotImplementedError
def extras(self, work):
"""Diagnostics recorded next to the metrics, never gated."""
return {}
def compare(self, metrics, base):
"""Reasons beyond the per-metric tolerance check, as gate() words them:
a reason containing "improvement" asks for a re-record, any other
fails."""
return []
def evidence(self, work):
"""Files of a finished build that explain its reasons."""
return []
def notes(self, build, metrics, args):
"""Advisory lines printed under the report table; never a verdict."""
return []
def top_module(self, dut_root, dut):
"""The DUT's top module, from dut/catalog.mk ("" if not declared)."""
return _make_var(_catalog(dut_root), dut + self.top_var)
def diff_levels(self, build, metrics):
"""(watch, fix) thresholds for `beacon.py diff` against a reference."""
return metrics.get("watch_levels") or 8, metrics.get("fix_levels") or 12
def incomplete(self, metrics):
"""Whether a cached result predates something this tool now records.
Reusing it would silently keep the gap forever, so --resume re-runs from
the checkpoint instead -- minutes of reporting, not hours of synthesis.
"""
return False
def describe(self, metrics):
"""One-line result summary for the progress log."""
return " ".join("%s=%s" % (c[0], metrics.get(c[1])) for c in self.columns)
def describe_baseline(self, base):
"""One-line baseline summary for --list."""
return " ".join("%s=%s" % (c[0], base.get(c[1])) for c in self.columns)
# --- catalog.mk parsing, shared by both DUT trees ---------------------------
def _catalog(dut_root):
"""The DUT catalog text (dut/catalog.mk is the single DUT list per flow)."""
try:
with open(os.path.join(dut_root, "catalog.mk")) as fh:
return fh.read()
except OSError:
return ""
def _make_var(text, name):
"""One `NAME := value` from a catalog, with line continuations folded in."""
m = re.search(r"^%s\s*:?=\s*((?:.*\\\n)*.*)$" % re.escape(name), text, re.M)
return m.group(1).replace("\\\n", " ").strip() if m else ""
def dut_names(dut_root):
"""DUT ids declared by dut/catalog.mk."""
return set(_make_var(_catalog(dut_root), "DUTS").split())
# --- Beacon, both targets ----------------------------------------------------
def harness_ties(work):
"""The instance-input bits the DUT top ties to constants, from the Beacon
run's netlist (ci/beacon.py ties): the count and the ports that carry
the most. A finished run always has its netlist, so a missing or
unreadable one is a build error, never a count of zero."""
run = os.path.join(work, "beacon")
netlist = os.path.join(run, "netlist.mlir")
if not os.path.exists(netlist):
raise BuildError("no %s to count the harness's constant ties in" % netlist)
import beacon
try:
rows = beacon.const_ties(netlist, beacon.netlist_top(run))
except ValueError as e:
raise BuildError("counting the harness's constant ties: %s" % e)
return {"tied_bits": sum(r[0] for r in rows),
"tied_ports": [{"port": p, "bits": n, "width": w}
for n, w, p in rows[:10]]}
def ties_reasons(metrics, base):
"""A harness whose constant ties changed: the synthesis gates fold the
logic behind a tied input and Beacon does not, so they now measure a
different design than the one their budgets were set on. Either way it
needs a reviewed re-record; a baseline without the count is not gated."""
now, was = metrics.get("tied_bits"), base.get("tied_bits")
if now is None or was is None or now == was:
return []
return ["harness: %d -> %d instance-input bits tied to constants -- "
"CHANGED: the synthesis gates now fold %s of the logic Beacon "
"counts; review the fpga_gate/asic_gate budget, then re-record"
% (was, now, "more" if now > was else "less")]
class Xilinx(Tool):
"""Vivado synthesis + place-and-route (hw/syn/xilinx/dut)."""
name = "xilinx"
gate = "fpga_gate"
flow = "hw/syn/xilinx/dut"
metrics = ("fmax_mhz", "wns_ns", "lut", "lutram", "ff", "bram", "uram",
"dsp", "build_time_s")
# Fmax is held to the build's clock target (gate()'s BELOW TARGET check).
gated = ("lut",)
area = "lut"
budget_key = "lut_budget"
integral = ("lut", "lutram", "ff", "bram", "uram", "dsp", "build_time_s")
columns = (("fmax", "fmax_mhz", "%.0f", 9), ("lut", "lut", "%d", 9),
("ff", "ff", "%d", 9), ("lutram", "lutram", "%d", 9),
("bram", "bram", "%d", 8), ("uram", "uram", "%d", 6),
("dsp", "dsp", "%d", 6))
weights = {"top": 14400, "core": 7200, "tcu": 3600, "rtu": 3600,
"vortex": 3600, "cache": 1800, "raster": 1800, "dxa": 1800,
"tex": 1200, "om": 1200}
job_label = "Vivado"
# Vivado prints the ELAB_DONE timing line on the way out of elaboration
# EITHER WAY, so a clean pass is elab_done with no elab_fail behind it.
elab_done = "Finished RTL Elaboration"
elab_fail = "RTL Elaboration failed"
elab_label = "RTL elaboration"
phases = (
("create_project", "setup"),
("Starting synth_design", "elaboration"),
(elab_done, "synthesis"),
("Synth Design complete", "opt"),
("Starting Logic Optimization Task", "opt"),
("Starting Placer Task", "placement"),
("Starting Routing Task", "routing"),
("report_timing", "reporting"),
)
# Top critical paths, read from the timing.rpt project.tcl already writes.
# Recorded whether or not timing closed, and never gated: they are the
# diagnostic that says WHERE the design is tight, so a Fmax regression comes
# with its location instead of just its magnitude.
TIMING_RPT = "timing.rpt"
NUM_PATHS = 10
# Top high-fanout nets (high_fanout_nets.rpt). Recorded, never gated -- the
# "why" behind a route-bound critical path.
FANOUT_RPT = "high_fanout_nets.rpt"
NUM_FANOUT = 10
def check_env(self):
if not os.environ.get("XILINX_VIVADO"):
sys.exit("ERROR: XILINX_VIVADO is not set -- source the Xilinx env "
"first (source ~/dev/xilinx_setup.sh)")
def env(self, args):
return {"device": os.environ.get("DEVICE", "xcu55c-fsvh2892-2L-e"),
"vivado": os.path.basename(os.environ["XILINX_VIVADO"].rstrip("/")),
"opt_level": args.opt_level, "xlen": args.xlen}
def hash_key(self, env):
return [env["device"], str(env["opt_level"]), str(env["xlen"])]
def env_warning(self, build, env):
ref = build.get("baseline_env", {}).get("vivado")
if ref and ref != env["vivado"]:
return ("baseline was recorded on Vivado %s, running %s -- results "
"across tool versions are not comparable" % (ref, env["vivado"]))
return None
def work_dir(self, dut_root, prefix, build):
return os.path.join(dut_root, "%s_%s" % (prefix, build["dut"]))
def clean(self, dut_root, work, build, benv):
subprocess.run(["make", "-C", work, "clean"], env=benv,
stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT,
check=True)
def build_command(self, dut_root, work, build, env, args):
# build.mk plays the role the former dut/<name>/Makefile did: it is
# copied in as the work tree's Makefile and run there with DUT=<name>.
os.makedirs(work, exist_ok=True)
shutil.copy2(os.path.join(dut_root, "build.mk"),
os.path.join(work, "Makefile"))
benv = dict(DUT=build["dut"],
CONFIGS=build["configs"],
CLK_FREQ_MHZ=str(build["clock_mhz"]),
OPT_LEVEL=str(env["opt_level"]),
MAX_JOBS=str(args.tool_jobs))
if build.get("impl_strategy"):
benv["IMPL_STRATEGY"] = build["impl_strategy"]
return ["make", "-C", work], benv
def extras(self, work):
return {"critical_paths": read_paths(os.path.join(work, self.TIMING_RPT),
self.NUM_PATHS),
"high_fanout_nets": read_fanout(os.path.join(work, self.FANOUT_RPT),
self.NUM_FANOUT)}
def incomplete(self, metrics):
return not (metrics.get("critical_paths")
and metrics.get("high_fanout_nets") is not None)
def describe(self, m):
return "Fmax=%.1f MHz LUT=%d" % (m["fmax_mhz"], m["lut"])
def describe_baseline(self, base):
return "Fmax=%.0f MHz LUT=%d" % (base["fmax_mhz"], base["lut"])
class Yosys(Tool):
"""Yosys + OpenSTA on an ASIC PDK (hw/syn/yosys/dut).
No licence and no dedicated machine, so unlike the Vivado gate this one is
cheap enough to run on a hosted runner. The metrics change with the target:
LUT/BRAM/DSP have no ASIC meaning, cell area and SRAM area do.
"""
name = "yosys"
gate = "asic_gate"
flow = "hw/syn/yosys/dut"
top_var = "_TOP"
metrics = ("fmax_mhz", "wns_ns", "tns_ns", "cell_area_um2", "seq_area_um2",
"sram_area_um2", "cell_count", "power_mw", "build_time_s")
# Fmax is held to the build's clock target (gate()'s BELOW TARGET check),
# not to its baseline. ABC maps against that target (-D) and spends any
# slack beyond it recovering area, so above the target Fmax moves with
# whatever else the design contains: #424 left the RTU's worst path in an
# unchanged FMA stage yet moved it 957 -> 903 MHz.
gated = ("cell_area_um2",)
area = "cell_area_um2"
budget_key = "area_budget_um2"
integral = ("cell_count", "build_time_s")
columns = (("fmax", "fmax_mhz", "%.0f", 9),
("area_um2", "cell_area_um2", "%.0f", 11),
("sram_um2", "sram_area_um2", "%.0f", 10),
("cells", "cell_count", "%d", 9),
("power_mw", "power_mw", "%.1f", 9))
weights = {"vortex": 7200, "rtu": 11000, "gfx": 5400, "core": 3600,
"tcu": 3600, "tensor": 3600, "vm": 1800, "cache": 900,
"raster": 300, "dxa": 300, "tex": 300, "om": 120}
fallback_weight = 900
job_label = "make"
# gen_sources.sh fails on a bad define or a missing include while it writes
# the sources; run_synth.sh stamps `gen-ys` once they are written. The design
# is elaborated later, by read_slang inside Yosys, so a hierarchy error still
# surfaces during the synthesis phase.
elab_done = "TIME gen-ys"
elab_label = "source generation"
phases = (
("gen_sources.sh", "sources"),
(elab_done, "synthesis"),
("TIME yosys", "sram"),
("run_sta.tcl", "timing"),
("DONE. Top:", "reporting"),
)
error_re = re.compile(r"^(?:ERROR|FATAL|Error)\b|^\S+: \*\*\* ")
def env(self, args):
return {"pdk": os.environ.get("PDK", "asap7"),
"vt": os.environ.get("ASAP7_VT", "rvt"),
"corner": os.environ.get("CORNER", "tt"),
"yosys": tool_version("yosys", "-V"),
"sta": tool_version("sta", "-version"),
"xlen": args.xlen}
def hash_key(self, env):
# ASAP7 RVT-TT and LVT-SS are not comparable, so the PDK corner is part
# of the fingerprint the way the FPGA part number is for Vivado.
#
# The tool versions are in the fingerprint too, which they are NOT on the
# Vivado side: area and Fmax move with the Yosys/ABC release far more
# than Vivado's do, and often by more than the 5% gate tolerance. A
# toolchain bump must therefore read as STALE ("re-record"), not as a
# regression -- the alternative is a red gate that blames the RTL for a
# tool change. Both ship in one tarball pinned by TOOLCHAIN_REV, so
# this makes that pin part of the baseline contract.
return [env["pdk"], env["vt"], env["corner"], str(env["xlen"]),
env["yosys"], env["sta"]]
def env_warning(self, build, env):
# Area and Fmax move with the Yosys/ABC version far more than Vivado's
# do, so a version change is worth saying out loud on every build.
for key, label in (("yosys", "Yosys"), ("sta", "OpenSTA")):
ref = build.get("baseline_env", {}).get(key)
if ref and env.get(key) and ref != env[key]:
# The version is in config_hash, so this build is already going
# to read STALE. Say WHICH tool moved -- STALE alone cannot.
return ("baseline was recorded on %s %s, running %s -- results "
"across tool versions are not comparable, so this reads "
"as STALE until re-recorded" % (label, ref, env[key]))
return None
def tops(self, dut_root):
"""DUT -> top module, from dut/catalog.mk. The flow builds into
<PREFIX>_<top>, so the work directory is not knowable without it."""
text = _catalog(dut_root)
return {d: _make_var(text, d + "_TOP") for d in _make_var(text, "DUTS").split()}
def work_dir(self, dut_root, prefix, build):
top = self.tops(dut_root).get(build["dut"])
if not top:
raise BuildError("no <dut>_TOP for '%s' in %s/catalog.mk"
% (build["dut"], dut_root))
# hw/syn/yosys/Makefile builds in $(PREFIX)_$(TOP_LEVEL_ENTITY), one
# level above the DUT dispatcher.
return os.path.join(os.path.dirname(dut_root), "%s_%s" % (prefix, top))
def prepare_flow(self, dut_root, env):
"""Install the PDK once, before the pool starts.
The flow's `asap7` target is a prerequisite of every build and is phony,
so N parallel builds would otherwise race the same download into the
same directory. Installed once up front, they all hit its marker fast
path instead.
"""
subprocess.run(["make", "-C", os.path.dirname(dut_root), "asap7"],
stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT,
check=True)
def _make(self, dut_root, build, target):
return ["make", "-C", dut_root, build["dut"], "TARGET=" + target,
"PREFIX=%s_%s" % (self.gate, build["id"]),
"CLOCK_FREQ=%d" % build["clock_mhz"],
"CONFIGS=" + build["configs"]]
def clean(self, dut_root, work, build, benv):
subprocess.run(self._make(dut_root, build, "clean"), env=benv,
stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT,
check=True)
os.makedirs(work, exist_ok=True)
def build_command(self, dut_root, work, build, env, args):
os.makedirs(work, exist_ok=True)
return self._make(dut_root, build, "timing"), {}
def describe(self, m):
return "Fmax=%.1f MHz area=%.0f um^2" % (m.get("fmax_mhz") or 0,
m.get("cell_area_um2") or 0)
def describe_baseline(self, base):
return "Fmax=%.0f MHz area=%.0f um^2" % (base["fmax_mhz"],
base.get("cell_area_um2") or 0)
class Beacon(Tool):
"""Beacon: logic levels per RTL unit and the design's area
(hw/syn/xilinx/dut, target `beacon`).
foldsim-estimate-timing ranks every register-to-register path of the CIRCT
netlist the Xilinx flow elaborates by its logic levels, counted as Vivado
counts them on DEVICE's family; foldsim-estimate-area counts its LUTs,
flip-flops, distributed and block RAM, UltraRAM and DSPs, placing each
memory as Vivado infers it. It needs no Vivado and no licence and takes
seconds per DUT, so it can run on every change where the Fmax gate runs
nightly.
Depth is deterministic -- the same RTL and tools give the same levels -- so
the gate has no tolerance: a unit that gets deeper fails, by name, until it
is fixed or a human re-records it. Area is as deterministic, but a feature
adds some by design, so a resource of the design fails past its baseline
by more than fpga_gate's LUT tolerance (metric_tolerance), and a memory
that moves between distributed and block RAM shows as both columns
moving. What it gates is depth and cells, not Fmax or placement: Beacon
does not see wires (docs/designs/continuous_integration.md 3.5).
"""
name = "beacon"
gate = "logic_gate"
flow = "hw/syn/xilinx/dut"
area_metrics = ("lut", "ff", "lutram", "bram", "uram", "dsp")
metrics = ("max_levels", "deep_endpoints", "watch_endpoints",
"endpoints") + area_metrics + ("pred_fmax_mhz", "risk_endpoints",
"build_time_s")
gated = ("max_levels",) + area_metrics
integral = ("max_levels", "deep_endpoints", "watch_endpoints",
"endpoints", "lut", "ff", "lutram", "uram", "dsp",
"risk_endpoints", "build_time_s")
key_metric = "max_levels"
summary = "beacon_summary.csv"
tolerance = 0.0
metric_tolerance = dict.fromkeys(area_metrics, 0.05)
columns = (("max_lv", "max_levels", "%d", 7),
("deep", "deep_endpoints", "%d", 9),
("watch", "watch_endpoints", "%d", 9),
("ends", "endpoints", "%d", 9),
("lut", "lut", "%d", 9), ("ff", "ff", "%d", 9),
("lutram", "lutram", "%d", 9), ("bram", "bram", "%g", 8),
("uram", "uram", "%d", 6), ("dsp", "dsp", "%d", 6),
("p_fmax", "pred_fmax_mhz", "%.1f", 8),
("risk", "risk_endpoints", "%d", 6),
("tied", "tied_bits", "%d", 6))
# Elaboration and mapping of the widest DUTs; the level count itself is
# seconds.
weights = {"top": 600, "vortex": 600, "gfx": 600, "core": 300,
"tensor": 300, "tcu": 300}
fallback_weight = 120
# Measured peaks: together they exceed a 16 GB hosted runner.
mem_gb = {"gfx": 9.5, "gfxvm": 10.0}
job_label = "make"
# The netlist exists once foldsim-opt starts mapping it; a typo in a define
# or an include dies in gen_sources.sh or circt_elaborate.sh before that.
elab_done = "foldsim-beacon-map"
elab_label = "elaboration"
phases = (
("gen_sources.sh", "sources"),
("circt_elaborate.sh", "elaboration"),
(elab_done, "mapping"),
("foldsim-estimate-timing", "levels"),
("foldsim-estimate-area", "area"),
("beacon_summary.py", "reporting"),
)
error_re = re.compile(r"^(?:ERROR|FATAL|Error)\b|: error:|^\S+: \*\*\* ")
UNITS = "beacon_units.json"
PATHS = "beacon_paths.txt"
AREA = "beacon_area.txt"
RISK = "beacon_risk.txt"
# The learned calibration per device family (docs/designs/beacon.md 7.1).
CALIBRATIONS = {"xcu55c": "beacon_calibration_u55c.json"}
RISK_ROWS = 40
def env(self, args):
return {"circt": tool_version("circt-verilog", "--version",
package="circt", line_prefix="CIRCT"),
"foldsim": tool_version("foldsim-opt", "--version",
package="foldsim",
line_prefix="foldsim"),
"device": os.environ.get("DEVICE", "xcu55c-fsvh2892-2L-e"),
"xlen": args.xlen}
def hash_key(self, env):
# The levels are the model's: a new foldsim or CIRCT can map the same
# RTL to other levels, and another device's family counts them
# differently; either must read as STALE, not as a regression.
return [env["circt"], env["foldsim"], env["device"], str(env["xlen"])]
def env_warning(self, build, env):
for key, label in (("foldsim", "foldsim"), ("circt", "CIRCT")):
ref = build.get("baseline_env", {}).get(key)
if ref and env.get(key) and ref != env[key]:
return ("baseline was recorded on %s %s, running %s -- this "
"reads as STALE until re-recorded" % (label, ref, env[key]))
return None
def work_dir(self, dut_root, prefix, build):
return os.path.join(dut_root, "%s_%s" % (prefix, build["dut"]))
def clean(self, dut_root, work, build, benv):
subprocess.run(["make", "-C", work, "clean"], env=benv,
stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT,
check=True)
def build_command(self, dut_root, work, build, env, args):
os.makedirs(work, exist_ok=True)
shutil.copy2(os.path.join(dut_root, "build.mk"),
os.path.join(work, "Makefile"))
return ["make", "-C", work, "beacon"], dict(
DUT=build["dut"], CONFIGS=build["configs"],
BEACON_INTERFACES=build.get("interfaces", ""))
def extras(self, work):
try:
with open(os.path.join(work, self.UNITS)) as fh:
doc = json.load(fh)
except (OSError, ValueError):
return {}
out = {"units": doc.get("units", {}),
"interfaces": doc.get("interfaces", {}),
"deepest_paths": doc.get("deepest_paths", []),
"fix_levels": doc.get("fix_levels"),
"watch_levels": doc.get("watch_levels")}
out.update(self.risk(work))
out.update(harness_ties(work))
return out
def risk(self, work):
"""The learned calibration's Fmax for the design, the slowest register's,
and its registers likely to fail, written to beacon_risk.txt. Levels
miss what wires decide (a wide enable driven across instances); the
calibration reads width, fan-out and span as well, and ranks a design's
registers far better than levels do. Advisory: its Fmax is not gated."""
device = os.environ.get("DEVICE", "xcu55c-fsvh2892-2L-e")
cal = next((f for d, f in self.CALIBRATIONS.items()
if device.startswith(d)), None)
run = os.path.join(work, "beacon")
if cal is None or not os.path.isdir(run):
return {}
import beacon
with open(os.path.join(source_root(), "hw", "scripts", cal)) as fh:
model = json.load(fh)
rows, pathless = beacon.score_run(run, model)
if not rows:
return {}
period = model.get("period_ns", beacon.CAL_PERIOD)
rows.sort(key=lambda r: r[1])
fmax = 1000.0 / (period - rows[0][1])
risky = sum(1 for r in rows if r[2] >= 0.5)
with open(os.path.join(work, self.RISK), "w") as fh:
fh.write("predicted Fmax %.1f MHz (slowest register, advisory); %d of %d "
"registers P(fail) >= 0.5 at %.3f ns; model %s\n"
% (fmax, risky, len(rows), period, cal))
if pathless:
fh.write("warning: registers at %d levels or more lack a path; "
"the scores of those are partial\n"
% model["path_levels"])
fh.write("\n%8s %7s %7s %4s register\n"
% ("fmax", "slack", "P(fail)", "lv"))
for p_near, slack, p_fail, lv, reg in rows[:self.RISK_ROWS]:
fh.write("%8.1f %+7.3f %7.2f %4d %s\n"
% (1000.0 / (period - slack), slack, p_fail, lv, reg))
return {"pred_fmax_mhz": round(fmax, 1), "risk_endpoints": risky}
def incomplete(self, metrics):
return metrics.get("units") is None or "tied_bits" not in metrics
def notes(self, build, metrics, args):
# The calibrated Fmax against the clock is a pointer, not a verdict:
# held to it, it failed known-good builds as often as bad ones.
pfmax, clock = metrics.get("pred_fmax_mhz"), build.get("clock_mhz")
if pfmax is None or not clock or pfmax >= clock:
return []
return ["p_fmax %.1f MHz is %.1f%% below the %g MHz clock; slowest "
"registers in %s (advisory, not gated)"
% (pfmax, 100.0 * (clock - pfmax) / clock, clock, self.RISK)]
def compare(self, metrics, base):
"""Per unit: deeper than its baseline fails; a new unit at the fix
threshold fails; shallower, or gone from the watch list, is an
improvement to lock in. Per interface output: more bits reached
combinationally from outside its instance fails, fewer is an
improvement. A harness whose constant ties changed fails."""
reasons = ties_reasons(metrics, base)
cur, ref = metrics.get("units"), base.get("units")
if cur is None or ref is None:
return reasons
fix = metrics.get("fix_levels") or 0
for unit in sorted(set(cur) | set(ref)):
now, was = cur.get(unit), ref.get(unit)
if was is None:
if now >= fix:
reasons.append("unit %s: new at %d levels (fix threshold "
"%d) -- REGRESSION" % (unit, now, fix))
elif now is None:
reasons.append("unit %s: %d levels -> below the watch threshold "
"-- improvement, update the baseline to lock it "
"in" % (unit, was))
elif now != was:
reasons.append("unit %s: %d -> %d levels -- %s"
% (unit, was, now,
"REGRESSION" if now > was else
"improvement, update the baseline to lock "
"it in"))
cur, ref = metrics.get("interfaces") or {}, base.get("interfaces") or {}
for port in sorted(set(cur) | set(ref)):
now, was = cur.get(port, 0), ref.get(port, 0)
if now != was:
reasons.append("interface %s: %d -> %d bits reached "
"combinationally from outside -- %s"
% (port, was, now,
"REGRESSION, register it" if now > was else
"improvement, update the baseline to lock "
"it in"))
return reasons
def evidence(self, work):
# Every unit at or above the watch threshold, its paths op by op: a
# failing unit is a heading there; the area by unit and each memory's
# placement.
return [p for p in (os.path.join(work, self.PATHS),
os.path.join(work, self.AREA),
os.path.join(work, self.RISK))
if os.path.exists(p)]
def describe(self, m):
area = (", %d LUTs, %g BRAM, %d DSP" % (m["lut"], m["bram"], m["dsp"])
if m.get("lut") is not None else "")
return "max %d levels, %d end points >= %s%s" % (
m["max_levels"], m["deep_endpoints"], m.get("fix_levels"), area)
def describe_baseline(self, base):
return "max %d levels" % base["max_levels"]
class BeaconAsic(Tool):
"""Beacon on the ASIC flow (hw/syn/yosys/dut, target `beacon`,
target=$(PDK)): asic_gate's builds, DUT for DUT and config for config, in
minutes instead of hours.
foldsim-estimate-timing counts cell stages (docs/designs/beacon.md 4.3)
and converts the deepest path to a period with the library's delays;
foldsim-estimate-area counts cells, flip-flops and SRAM macros, a memory
without a macro as flip-flops. Flip-flops and cells are held to the
baseline (5 %), so a memory that leaves its SRAM macro, or one Yosys must
build from flip-flops, fails here before the nightly synthesis. The
predicted Fmax is held to the build's clock_mhz with a wide band (see
FMAX_BAND): a backstop for a path far too deep to close, not a timing
verdict. The sensitive timing check is register by register against a
reference (--against-rev), where a path that got deeper fails by name.
"""
name = "beacon_asic"
gate = "logic_gate_asic"
spec = "asic_gate"
flow = "hw/syn/yosys/dut"
top_var = "_TOP"
metrics = ("fmax_mhz", "period_ps", "max_stages", "cells", "ff", "macros",
"endpoints", "stage_ps", "overhead_ps", "build_time_s")
gated = ("ff", "cells")
integral = ("period_ps", "max_stages", "cells", "ff", "macros",
"endpoints", "stage_ps", "overhead_ps", "build_time_s")
key_metric = "period_ps"
summary = "beacon_asic_summary.csv"
# The predicted Fmax fails only more than FMAX_BAND below clock_mhz.
# Beacon's period runs 0.59-1.05x OpenSTA's on the 14 DUTs (it prices
# vm's L2 TLB MRU update at 46 stages where STA closes it at 1266 MHz),
# so a tight band fails designs that close. Over every validated (design,
# target) pair, the lowest prediction of a design STA closes is 0.749 of
# its target (vm and the fixed top, 749 MHz at 1 GHz), and the highest of
# one that misses badly is 0.615 (core and top before their FPU fixes,
# 492 MHz at 800). 0.32 splits that gap evenly in ratio (about 10 % margin
# each side) and holds at both the 800 MHz and the 1 GHz target.
FMAX_BAND = 0.32
metric_tolerance = {"fmax_mhz": FMAX_BAND}
# Beacon reads no clock: the target only sets the band and the stage
# thresholds, so a new target must not invalidate the measurements.
clock_in_hash = False
depth_unit = "stages"
columns = (("fmax", "fmax_mhz", "%.0f", 7),
("stages", "max_stages", "%d", 7),
("cells", "cells", "%d", 9),
("ff", "ff", "%d", 8),
("macros", "macros", "%d", 7),
("tied", "tied_bits", "%d", 6))
weights = {"top": 600, "gfx": 600, "core": 300, "tensor": 300, "vm": 300}
fallback_weight = 120
job_label = "make"
elab_done = "foldsim-beacon-map"
phases = (
("gen_sources.sh", "sources"),
("circt_elaborate.sh", "elaboration"),
(elab_done, "mapping"),
("foldsim-estimate-timing", "stages"),
("foldsim-estimate-area", "area"),
("asic-summary", "reporting"),
)
error_re = re.compile(r"^(?:ERROR|FATAL|Error)\b|: error:|^\S+: \*\*\* ")
REPORT = os.path.join("beacon", "report.txt")
AREA = os.path.join("beacon", "area.txt")
def env(self, args):
return {"circt": tool_version("circt-verilog", "--version",
package="circt", line_prefix="CIRCT"),
"foldsim": tool_version("foldsim-opt", "--version",
package="foldsim",
line_prefix="foldsim"),
"pdk": os.environ.get("PDK", "asap7"),
"xlen": args.xlen}
def hash_key(self, env):
return [env["circt"], env["foldsim"], env["pdk"], str(env["xlen"])]
def env_warning(self, build, env):
return Beacon.env_warning(self, build, env)
tops = Yosys.tops
work_dir = Yosys.work_dir
def diff_levels(self, build, metrics):
"""(watch, fix) stages for `beacon.py diff`: the most that fit 90 %
and 100 % of the target period, plus one, with the run's own delays
(period = overhead + stages x stage)."""
period = 1e6 / float(build["clock_mhz"])
stage = metrics.get("stage_ps") or 24
over = metrics.get("overhead_ps") or 231
return (int((0.9 * period - over) // stage) + 1,
int((period - over) // stage) + 1)
def _make(self, dut_root, build, target):
return ["make", "-C", dut_root, build["dut"], "TARGET=" + target,
"PREFIX=%s_%s" % (self.gate, build["id"]),
"CONFIGS=" + build["configs"]]
def clean(self, dut_root, work, build, benv):
shutil.rmtree(work, ignore_errors=True)
os.makedirs(work, exist_ok=True)
def build_command(self, dut_root, work, build, env, args):
os.makedirs(work, exist_ok=True)
script = os.path.join(os.path.dirname(os.path.abspath(__file__)),
"beacon.py")
make = " ".join(shlex.quote(a) for a in self._make(dut_root, build, "beacon"))
return ["sh", "-c", "%s && echo asic-summary && %s %s asic-summary %s %s"
% (make, shlex.quote(sys.executable), shlex.quote(script),
shlex.quote(os.path.join(work, "beacon")),
shlex.quote(os.path.join(work, self.summary)))], {}
def extras(self, work):
return harness_ties(work)
def incomplete(self, metrics):
return "tied_bits" not in metrics
def compare(self, metrics, base):
return ties_reasons(metrics, base)
def notes(self, build, metrics, args):
# Below the target but inside the band: worth a look, not a verdict.
# Beyond the band, gate() already failed it.
fmax, clock = metrics.get("fmax_mhz"), build.get("clock_mhz")
if fmax is None or not clock or fmax >= clock:
return []
band = threshold(build, "fmax_mhz", args)
shortfall = (clock - fmax) / float(clock)
if shortfall > band:
return []
return ["predicted Fmax %.0f MHz is %.1f%% below the %g MHz target, "
"inside the %.0f%% band; deepest path in %s (advisory)"
% (fmax, 100 * shortfall, clock, 100 * band, self.REPORT)]
def evidence(self, work):
return [p for p in (os.path.join(work, self.REPORT),
os.path.join(work, self.AREA))
if os.path.exists(p)]
def describe(self, m):
return "Fmax=%.0f MHz (%d stages) %d cells, %d FFs" % (
m.get("fmax_mhz") or 0, m.get("max_stages") or 0,
m.get("cells") or 0, m.get("ff") or 0)
def describe_baseline(self, base):
return "Fmax=%.0f MHz %d FFs" % (base.get("fmax_mhz") or 0, base.get("ff") or 0)
TOOLS = {t.name: t for t in (Xilinx(), Yosys(), Beacon(), BeaconAsic())}
def tool_version(name, flag, package=None, line_prefix=None):
"""First line of `<tool> <flag>`, or "" if it cannot be asked.
Resolved out of $TOOLDIR the way hw/syn/common.mk does, with a PATH fallback:
the prebuilt toolchain is not on PATH, and a version recorded from some other
copy of yosys would make the baseline's tool env a lie.
"""
tooldir = os.environ.get("TOOLDIR") or os.path.expanduser("~/tools")
package = package or name
# <PACKAGE>_PATH overrides the toolchain copy, as in hw/syn/common.mk.
root = os.environ.get(package.upper() + "_PATH") or \
os.path.join(tooldir, package)
binary = os.path.join(root, "bin", name)
if not os.path.exists(binary):
binary = name
try:
out = subprocess.run([binary, flag], stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, timeout=30)
except (OSError, subprocess.SubprocessError):
return ""
text = out.stdout.decode(errors="replace").strip()
if not text:
return ""
# A tool that prints several components' versions names the one that
# matters on a line of its own (circt-verilog: "CIRCT <sha>").
if line_prefix:
lines = [l.strip() for l in text.splitlines()
if l.strip().startswith(line_prefix)]
return lines[0] if lines else ""
# Up to the build details: yosys reports its git sha AND the g++ that built
# it ("Yosys 0.57+157 (git sha1 ..., g++ 11.4 ...)"). Those identify the
# binary, not the tool, and this string is part of the baseline fingerprint
# -- rebuilding the same yosys must not invalidate every recorded number.
return text.splitlines()[0].split("(")[0].strip()
# ---------------------------------------------------------------------------
# catalog
# ---------------------------------------------------------------------------
def source_root():
"""Source checkout root (has configure + VX_config.toml), not a build tree."""
d = os.path.dirname(os.path.abspath(__file__))
while d != "/":
if os.path.exists(os.path.join(d, "configure")) and \
os.path.exists(os.path.join(d, "VX_config.toml")):
return d
d = os.path.dirname(d)
sys.exit("ERROR: not inside a Vortex source checkout")
def spec_path(root, gate):
return os.path.join(root, "ci", "testcases", gate + ".yaml")
def baseline_dir(root, tool, override=None):
"""ci/baselines/synthesis/<tool>, or <override>/<tool> to read and write
proposed baselines outside the tree."""
return os.path.join(override or os.path.join(root, "ci", "baselines",
"synthesis"), tool)
def baseline_path(root, tool, group, override=None):
return os.path.join(baseline_dir(root, tool, override), group + ".json")
def load_catalog(root, tool, xlen=32, baselines_from=None):
"""Builds keyed by id: the SPEC from ci/testcases/<gate>.yaml, joined to its
recorded metrics from ci/baselines/synthesis/<tool>/<group>.json.
Same division as every other check in the catalog: the yaml says what to run
(and is hand-authored, commented, reviewed), the baseline says what the last
accepted answer was (and is machine-written). Neither file can be mistaken
for the other's job.
A tool that measures another gate's builds (`spec`) takes their designs
and targets, not their verdict policy: that gate's `known_issue` and