Skip to content

Commit daf186b

Browse files
committed
docs(ci): keep benchmark summaries without raw exports
1 parent 5358300 commit daf186b

5 files changed

Lines changed: 44 additions & 857 deletions

File tree

‎docs/adr/0002-lightweight-ci-preparation.md‎

Lines changed: 44 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -64,16 +64,26 @@ confirmed preparation/build/lint/test failure, missing-directory and workspace
6464
boundary rejection, disabled checks and cancellation. Expected failures are
6565
successful negative tests, not successful workflow conclusions.
6666

67-
The [measurement record](evidence/0002-ci-preparation.json) retains source
68-
revisions, image digest, probe outcomes and each run's metrics. The
69-
[job timestamps](evidence/0002-ci-job-timings.csv) permit independent calculation.
70-
The first six probes used `40b3b81`; boundary/cancellation probes and all measured
71-
runs used `5bccd49`, which additionally preserves whitespace in directory names.
67+
The initial baseline was `aaab44d37f5f675dc6789ea377ffc5b63cc3fe0f` and the
68+
candidate was `5bccd49fb33a6c04d5332cb32754b28261b12103`. After
69+
[priming](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42283024208),
70+
the five measured pairs were
71+
[1](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42284009431),
72+
[2](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42284313836),
73+
[3](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42284695321),
74+
[4](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42285014018) and
75+
[5](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42285335029).
7276

7377
The benchmark uses the existing preinstalled npm fixture with one test, an empty
74-
build and an immutable image digest. Baseline and candidate run together in each
75-
dispatch. One priming pair precedes five consecutive measured pairs; all measured
76-
pairs are retained. Measurement starts after the full implementation CI finishes.
78+
build and this immutable image in both series:
79+
80+
```text
81+
ghcr.io/hoverkraft-tech/ci-github-nodejs/ci-npm@sha256:324a3de0633bb5f7f0d41394ccf902195665fdaf2c23798f6145922161511574
82+
```
83+
84+
Baseline and candidate run together in each dispatch. One priming pair precedes
85+
five consecutive measured pairs; all measured pairs are retained. Measurement
86+
starts after the full implementation CI finishes.
7787

7888
All five measured pairs passed. All 48 sampled job logs (including priming) used
7989
runner `2.338.2`, Ubuntu image `ubuntu-26.04` version `20261101.186.1`; container
@@ -113,23 +123,34 @@ is recorded separately; do not treat that series as measurements of a later SHA.
113123
The [integrated CI](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42288071590)
114124
passed at `2f87ce61c0641934d24c39162732660b90451004`, including requested-build
115125
artifact transfer and source packaging without compilation. Its workflow/action
116-
sources equal the measured candidate `540c5f4`. All eight failure, disabled-check
117-
and cancellation probes also matched their expected outcomes on that candidate.
126+
sources equal the measured candidate `540c5f4ea785d853affae704ee5c4c270d96abcd`.
127+
Eight probes matched their expected outcomes:
128+
[preparation failure](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287006075),
129+
[disabled checks](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287010624),
130+
[build failure](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287014811),
131+
[test failure](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287019554),
132+
[lint failure](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287024618),
133+
[missing directory](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287029142),
134+
[workspace boundary](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287033354) and
135+
[cancellation](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42287037745).
118136

119137
The preceding [integration run](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42286746512)
120-
exposed a fixture collision: upstream package-result collection groups reports by
121-
project directory. Two test calls packaged `tests/npm`, yielding duplicate results
122-
for one project. The no-build test now uses `tests/source-package`, whose build
123-
command deliberately fails if invoked. This isolates the two cases; it does not
124-
change the upstream collector's grouping contract.
125-
126-
Five consecutive updated pairs after priming all passed. The
127-
[integrated measurement record](evidence/0002-ci-integration.json) and
128-
[job timestamps](evidence/0002-ci-integration-job-timings.csv) retain the same
129-
metric definitions as the initial series, run links, source revisions and all
130-
probe outcomes. All 48 sampled job logs again used runner `2.338.2` and Ubuntu
131-
image `20261101.186.1`; both variants used the recorded immutable fixture digest
132-
and passed the same one-suite, one-test inventory.
138+
exposed duplicate reports when two cases packaged the same project. The no-build
139+
case uses `tests/source-package`, whose build command deliberately fails if invoked,
140+
to isolate it without changing the collector's project-based grouping.
141+
142+
The same method compared baseline `55afe8ef3c9b59932f87a99cb2620226f18b2062`
143+
with candidate `540c5f4ea785d853affae704ee5c4c270d96abcd`. After
144+
[priming](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42286929531),
145+
all five measured pairs passed:
146+
[1](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42289175642),
147+
[2](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42289482879),
148+
[3](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42289790837),
149+
[4](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42290030964) and
150+
[5](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42290263969).
151+
The runs retain job timestamps and outcomes. All 48 sampled job logs again used
152+
runner `2.338.2` and Ubuntu image `20261101.186.1`; both variants used the image
153+
above and passed the same one-suite, one-test inventory.
133154

134155
| Metric | Baseline median (range) | Candidate median (range) | Median change |
135156
| -------------------------------------- | ----------------------- | ------------------------ | ------------- |

‎docs/adr/evidence/0002-ci-integration-job-timings.csv‎

Lines changed: 0 additions & 49 deletions
This file was deleted.

0 commit comments

Comments
 (0)