Skip to content

Commit c19c5d8

Browse files
committed
docs(ci): record measurements against workspace-aware upstream
1 parent c2ee50c commit c19c5d8

3 files changed

Lines changed: 487 additions & 2 deletions

File tree

‎docs/adr/0002-lightweight-ci-preparation.md‎

Lines changed: 54 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
## Context
88

9-
The CI workflow starts an application container only to resolve paths and build
9+
For application callers, CI starts a container only to resolve paths and build
1010
options. An empty build then starts another container before tests can run. These
1111
jobs perform no application validation and delay callers with prebuilt images.
1212

@@ -30,6 +30,12 @@ Use an explicit cancellation-aware condition so an intentional skip does not
3030
propagate to these jobs. A failed, cancelled or unexpectedly skipped requested
3131
build must block them. Lint remains independent of build and test results.
3232

33+
Public-package discovery needs the checked-out source. Run it in a separate host
34+
job only when packaging is enabled, in parallel with compilation. Package matrix
35+
validation requires successful discovery and the build condition above. Collect
36+
results only after successful package validation. Preserve workspace selection,
37+
consumer tests and artifact outputs without adding discovery to application CI.
38+
3339
Do not change application commands, report handling, build-secret delivery or
3440
CodeQL/dependency-review policy. Preserve package validation when no compilation
3541
is needed. No additional runtime dependency or public workflow input is needed.
@@ -47,7 +53,7 @@ empty build jobs changes emitted check names/statuses: callers requiring those
4753
individual jobs must update their required checks when adopting the release.
4854
Rollback restores the previous workflow pin and corresponding required checks.
4955

50-
## Implementation evidence: 2026-11-06
56+
## Initial implementation evidence: 2026-11-06
5157

5258
The [implementation CI](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42282825359)
5359
passed on `5bccd49fb33a6c04d5332cb32754b28261b12103`. The 56 contract tests
@@ -90,3 +96,49 @@ and empty build; both versions still execute lint and the same test.
9096
These measurements isolate reusable-workflow overhead. They do not establish a
9197
Pleey end-to-end improvement or a cold-cache guarantee. A reviewed release and
9298
consumer CI remain required before adopting its immutable SHA in Pleey.
99+
100+
## Upstream integration: 2026-11-06
101+
102+
Main advanced to `55afe8ef3c9b59932f87a99cb2620226f18b2062` during review
103+
preparation, adding workspace packaging and updating shared action pins. Retain
104+
that package API and move its source-dependent discovery out of the removed
105+
setup job. The adapted suite has 60 contract tests, including discovery failure
106+
and validation-result collection when compilation is omitted. The existing
107+
source-package integration case also checks the resulting artifact output.
108+
109+
The initial measurements above retain their exact revisions and are historical
110+
evidence for the pre-integration implementation. Validation against updated main
111+
is recorded separately; do not treat that series as measurements of a later SHA.
112+
113+
The [integrated CI](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42288071590)
114+
passed at `2f87ce61c0641934d24c39162732660b90451004`, including requested-build
115+
artifact transfer and source packaging without compilation. Its workflow/action
116+
sources equal the measured candidate `540c5f4`. All eight failure, disabled-check
117+
and cancellation probes also matched their expected outcomes on that candidate.
118+
119+
The preceding [integration run](https://github.com/hoverkraft-tech/ci-github-nodejs/actions/runs/42286746512)
120+
exposed a fixture collision: upstream package-result collection groups reports by
121+
project directory. Two test calls packaged `tests/npm`, yielding duplicate results
122+
for one project. The no-build test now uses `tests/source-package`, whose build
123+
command deliberately fails if invoked. This isolates the two cases; it does not
124+
change the upstream collector's grouping contract.
125+
126+
Five consecutive updated pairs after priming all passed. The
127+
[integrated measurement record](evidence/0002-ci-integration.json) and
128+
[job timestamps](evidence/0002-ci-integration-job-timings.csv) retain the same
129+
metric definitions as the initial series, run links, source revisions and all
130+
probe outcomes. All 48 sampled job logs again used runner `2.338.2` and Ubuntu
131+
image `20261101.186.1`; both variants used the recorded immutable fixture digest
132+
and passed the same one-suite, one-test inventory.
133+
134+
| Metric | Baseline median (range) | Candidate median (range) | Median change |
135+
| -------------------------------------- | ----------------------- | ------------------------ | ------------- |
136+
| Elapsed from preparation | 101s (89–107s) | 52s (50–58s) | −48.51% |
137+
| Preparation to first validation action | 61s (58–68s) | 31s (29–36s) | −49.18% |
138+
| Summed job time | 135s (121–149s) | 90s (77–91s) | −33.33% |
139+
| Initial queue | 4s (4–4s) | 4s (3–4s) | 0.00% |
140+
141+
This confirms the reduction for the controlled application fixture against the
142+
workspace-aware upstream implementation. Package-discovery jobs remain skipped
143+
for this caller. The same scope limits apply: Pleey adoption and end-to-end
144+
measurement require a reviewed release.
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
sample,run_id,variant,job_id,job,started_at,completed_at,validation_action_started_at
2+
0,42286929531,baseline,128130051617,📦 Prepare configuration,2026-11-06T20:16:38Z,2026-11-06T20:16:43Z,
3+
0,42286929531,baseline,128130207747,⚙️ Setup,2026-11-06T20:16:59Z,2026-11-06T20:17:29Z,
4+
0,42286929531,baseline,128130574024,👕 Lint,2026-11-06T20:18:45Z,2026-11-06T20:19:30Z,2026-11-06T20:19:14Z
5+
0,42286929531,baseline,128130574270,🏗️ Build,2026-11-06T20:19:23Z,2026-11-06T20:19:51Z,
6+
0,42286929531,baseline,128131737182,🧪 Test,2026-11-06T20:19:57Z,2026-11-06T20:20:26Z,2026-11-06T20:20:22Z
7+
0,42286929531,candidate,128130051239,📦 Prepare configuration,2026-11-06T20:16:32Z,2026-11-06T20:16:37Z,
8+
0,42286929531,candidate,128130144528,👕 Lint,2026-11-06T20:16:43Z,2026-11-06T20:17:22Z,2026-11-06T20:17:09Z
9+
0,42286929531,candidate,128130148172,🧪 Test,2026-11-06T20:16:45Z,2026-11-06T20:17:22Z,2026-11-06T20:17:17Z
10+
1,42289175642,baseline,128137790804,📦 Prepare configuration,2026-11-06T20:32:15Z,2026-11-06T20:32:18Z,
11+
1,42289175642,baseline,128137837328,⚙️ Setup,2026-11-06T20:32:20Z,2026-11-06T20:32:51Z,
12+
1,42289175642,baseline,128138098907,👕 Lint,2026-11-06T20:32:53Z,2026-11-06T20:33:46Z,2026-11-06T20:33:23Z
13+
1,42289175642,baseline,128138098910,🏗️ Build,2026-11-06T20:32:53Z,2026-11-06T20:33:18Z,
14+
1,42289175642,baseline,128138317698,🧪 Test,2026-11-06T20:33:20Z,2026-11-06T20:33:57Z,2026-11-06T20:33:53Z
15+
1,42289175642,candidate,128137791333,📦 Prepare configuration,2026-11-06T20:32:15Z,2026-11-06T20:32:18Z,
16+
1,42289175642,candidate,128137838965,👕 Lint,2026-11-06T20:32:20Z,2026-11-06T20:33:07Z,2026-11-06T20:32:46Z
17+
1,42289175642,candidate,128137839731,🧪 Test,2026-11-06T20:32:21Z,2026-11-06T20:33:01Z,2026-11-06T20:32:57Z
18+
2,42289482879,baseline,128138843980,📦 Prepare configuration,2026-11-06T20:34:25Z,2026-11-06T20:34:29Z,
19+
2,42289482879,baseline,128138898487,⚙️ Setup,2026-11-06T20:34:31Z,2026-11-06T20:34:57Z,
20+
2,42289482879,baseline,128139126552,🏗️ Build,2026-11-06T20:34:59Z,2026-11-06T20:35:31Z,
21+
2,42289482879,baseline,128139126589,👕 Lint,2026-11-06T20:34:59Z,2026-11-06T20:35:39Z,2026-11-06T20:35:26Z
22+
2,42289482879,baseline,128139413068,🧪 Test,2026-11-06T20:35:33Z,2026-11-06T20:36:12Z,2026-11-06T20:36:08Z
23+
2,42289482879,candidate,128138844346,📦 Prepare configuration,2026-11-06T20:34:25Z,2026-11-06T20:34:29Z,
24+
2,42289482879,candidate,128138900590,👕 Lint,2026-11-06T20:34:31Z,2026-11-06T20:35:20Z,2026-11-06T20:34:58Z
25+
2,42289482879,candidate,128138901343,🧪 Test,2026-11-06T20:34:32Z,2026-11-06T20:35:10Z,2026-11-06T20:35:06Z
26+
3,42289790837,baseline,128139904753,📦 Prepare configuration,2026-11-06T20:36:35Z,2026-11-06T20:36:39Z,
27+
3,42289790837,baseline,128139958822,⚙️ Setup,2026-11-06T20:36:41Z,2026-11-06T20:37:06Z,
28+
3,42289790837,baseline,128140172838,🏗️ Build,2026-11-06T20:37:08Z,2026-11-06T20:37:32Z,
29+
3,42289790837,baseline,128140173015,👕 Lint,2026-11-06T20:37:08Z,2026-11-06T20:37:46Z,2026-11-06T20:37:34Z
30+
3,42289790837,baseline,128140378795,🧪 Test,2026-11-06T20:37:34Z,2026-11-06T20:38:04Z,2026-11-06T20:38:01Z
31+
3,42289790837,candidate,128139904223,📦 Prepare configuration,2026-11-06T20:36:34Z,2026-11-06T20:36:39Z,
32+
3,42289790837,candidate,128139955265,👕 Lint,2026-11-06T20:36:41Z,2026-11-06T20:37:26Z,2026-11-06T20:37:05Z
33+
3,42289790837,candidate,128139956080,🧪 Test,2026-11-06T20:36:41Z,2026-11-06T20:37:09Z,2026-11-06T20:37:06Z
34+
4,42290030964,baseline,128140741084,📦 Prepare configuration,2026-11-06T20:38:20Z,2026-11-06T20:38:24Z,
35+
4,42290030964,baseline,128140786845,⚙️ Setup,2026-11-06T20:38:25Z,2026-11-06T20:38:50Z,
36+
4,42290030964,baseline,128140989709,👕 Lint,2026-11-06T20:38:51Z,2026-11-06T20:39:34Z,2026-11-06T20:39:18Z
37+
4,42290030964,baseline,128140989740,🏗️ Build,2026-11-06T20:38:52Z,2026-11-06T20:39:20Z,
38+
4,42290030964,baseline,128141226579,🧪 Test,2026-11-06T20:39:21Z,2026-11-06T20:39:51Z,2026-11-06T20:39:48Z
39+
4,42290030964,candidate,128140741253,📦 Prepare configuration,2026-11-06T20:38:19Z,2026-11-06T20:38:24Z,
40+
4,42290030964,candidate,128140787208,👕 Lint,2026-11-06T20:38:26Z,2026-11-06T20:39:17Z,2026-11-06T20:38:55Z
41+
4,42290030964,candidate,128140788114,🧪 Test,2026-11-06T20:38:26Z,2026-11-06T20:39:01Z,2026-11-06T20:38:57Z
42+
5,42290263969,baseline,128141559120,📦 Prepare configuration,2026-11-06T20:40:04Z,2026-11-06T20:40:09Z,
43+
5,42290263969,baseline,128141616194,⚙️ Setup,2026-11-06T20:40:11Z,2026-11-06T20:40:41Z,
44+
5,42290263969,baseline,128141878503,👕 Lint,2026-11-06T20:40:43Z,2026-11-06T20:41:24Z,2026-11-06T20:41:11Z
45+
5,42290263969,baseline,128141878643,🏗️ Build,2026-11-06T20:40:44Z,2026-11-06T20:41:14Z,
46+
5,42290263969,baseline,128142143383,🧪 Test,2026-11-06T20:41:16Z,2026-11-06T20:41:45Z,2026-11-06T20:41:41Z
47+
5,42290263969,candidate,128141559445,📦 Prepare configuration,2026-11-06T20:40:04Z,2026-11-06T20:40:07Z,
48+
5,42290263969,candidate,128141605586,👕 Lint,2026-11-06T20:40:09Z,2026-11-06T20:40:54Z,2026-11-06T20:40:33Z
49+
5,42290263969,candidate,128141606855,🧪 Test,2026-11-06T20:40:09Z,2026-11-06T20:40:38Z,2026-11-06T20:40:34Z

0 commit comments

Comments
 (0)