Skip to content

Commit c9e0277

Browse files
committed
tools,doc: check and backfill notarization of darwin tarballs
Add tools/macos-tarball-notarization.sh. Given release versions, or --all-supported for the lines currently in their support window, it downloads the darwin tarballs, verifies the bin/node signature and asks Apple's Gatekeeper ticket service, by code directory hash, whether a notarization ticket exists. That is the lookup macOS itself performs; `codesign --verify -R=notarized` only reads the local ticket cache and reports a notarized binary as unticketed on a machine that never fetched its ticket. A failed lookup is reported as such, never as a missing ticket. With --submit it zips and submits the binaries that lack one through the NODE_RELEASE_PROFILE keychain profile, requires Apple's Accepted verdict, and waits for the ticket. --latest N limits the run to the N most recent releases per line, since every check downloads a tarball. Notarizing a published binary does not modify it, so checksums stay valid; this is how already shipped tarballs get backfilled. Document in release-overview.md how macOS signing and notarization are wired, that tarball binaries usually receive a ticket only days after a release when third parties notarize apps that bundle them, that the NOTARIZATION_* variables in iojs+release must be exported on the tarball axes too, and how to check and backfill a release. Describe in jenkins-jobs.md the scheduled ci-release job that runs the script nightly, which both backfills and detects a tarball axis that stopped notarizing. Note in the orka README that pkg and tarball nodes share the release image. Refs: nodejs/node#67140 Refs: nodejs/node#67143 Refs: #3538 Refs: #4545 Assisted-by: Claude Assisted-by: Codex Signed-off-by: Ulises Gascon <ulisesgascongonzalez@gmail.com>
1 parent 8954a60 commit c9e0277

4 files changed

Lines changed: 471 additions & 0 deletions

File tree

‎doc/jenkins-jobs.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,46 @@ https://github.com/nodejs/email/blob/HEAD/iojs.org/aliases.json.
1515

1616
This job needs to be updated each time a we roll over to a new Current
1717
release.
18+
19+
## notarize-darwin-tarballs (ci-release, to be created)
20+
21+
Nightly job on ci-release that checks the darwin tarballs of the supported
22+
release lines for an Apple notarization ticket and submits the ones that lack
23+
it, see [nodejs/node#67140](https://github.com/nodejs/node/issues/67140). It
24+
has to run on a macOS release node (full Xcode and the `NODE_RELEASE_PROFILE`
25+
keychain profile), so it cannot live on the test CI.
26+
27+
Freestyle job restricted to a `macos*-release-*` label, checking out
28+
`nodejs/build`, with the `SSH_RELEASE_*` credential binding from
29+
`iojs+release` and `NOTARIZATION_TEAM_ID` set to the Node.js team id,
30+
running the same keychain unlock `iojs+release` uses (the notary credentials
31+
are read from the keychain profile, so `NOTARIZATION_PASSWORD` is not
32+
needed):
33+
34+
```shell
35+
security unlock-keychain -p "$SSH_RELEASE_PASSWORD"
36+
tools/macos-tarball-notarization.sh --submit --all-supported --latest 3
37+
```
38+
39+
Each check downloads a tarball, so only the three most recent releases per
40+
line are covered nightly; older ones are a manual run with a larger
41+
`--latest`. Supported lines come from the Release schedule, so the job needs
42+
no editing on roll-over. Submissions are serial: each waits for Apple's
43+
verdict (bounded to 30 minutes by `notarytool --timeout`) and then up to
44+
eight minutes for the ticket to become visible, so 18 binaries can take over
45+
11 hours if Apple is slow on every one. Disable concurrent builds and give the
46+
first run a 16 hour timeout, or run it by hand in two or three batches with
47+
explicit versions. A rerun skips every binary that already has a ticket and
48+
resubmits the rest, so a run that hits the timeout is simply rerun. One hour
49+
is enough once everything is ticketed. A rejected submission prints
50+
Apple's notary log in the build output.
51+
52+
The ticket check queries Apple's ticket service by code directory hash, the
53+
same lookup macOS performs; a failed lookup is reported as such and never
54+
counted as a missing ticket. Exit 1 means a completed check found a published tarball whose
55+
`bin/node` has no ticket, or an invalid signature: in the nightly `--submit` run that
56+
should not happen and points at a tarball axis of `iojs+release` that
57+
stopped notarizing. Exit 3 means a check or a submission could not complete:
58+
download, extraction, a rejected or timed-out notarytool submission, or a
59+
binary not signed by the expected team; read the build output for which.
60+
Notify `release-validation-alert` on both, as for `validate-downloads`.

‎doc/release-overview.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,5 +159,40 @@ flowchart TD
159159
r2Staging ~~~ website
160160
```
161161

162+
## macOS code signing and notarization
163+
164+
The macOS release VMs are built from
165+
[`orka/templates/macos-release.pkr.hcl`](../orka/templates/macos-release.pkr.hcl),
166+
which installs the Developer ID certificate and a `notarytool` keychain
167+
profile (`NODE_RELEASE_PROFILE`) on every release image, pkg and tarball nodes
168+
alike. In `nodejs/node`, `make pkg` and `make binary` codesign `bin/node` and
169+
call `tools/osx-notarize.sh` on their output; the pkg gets its ticket stapled,
170+
the tarball binary cannot and relies on Gatekeeper fetching the ticket by
171+
code directory hash, so its contents and checksums are unaffected.
172+
173+
The script only runs when `NOTARIZATION_ID`, `NOTARIZATION_PASSWORD` and
174+
`NOTARIZATION_TEAM_ID` are set. `iojs+release` injects them from a
175+
conditional step per node type; the tarball steps need them as well as the
176+
pkg step, otherwise tarballs ship signed but not notarized
177+
([nodejs/node#67140](https://github.com/nodejs/node/issues/67140)). A
178+
successful tarball axis logs `Verification was successful.`; `Skipping
179+
notarization` means the variables were not injected.
180+
181+
Tarball binaries often receive a ticket anyway, days after the release,
182+
when a third party notarizes an app that bundles them; the window right after
183+
a release and any version nobody bundles stay uncovered, which is what the
184+
build step closes. To check a published release from any Mac (the script asks
185+
Apple's ticket service by code directory hash; `codesign -R=notarized` alone
186+
only reads the local cache), or to backfill tickets for already published
187+
tarballs from a machine with the keychain profile:
188+
189+
```shell
190+
tools/macos-tarball-notarization.sh v26.13.0
191+
tools/macos-tarball-notarization.sh --submit --all-supported --latest 10
192+
```
193+
194+
The scheduled `notarize-darwin-tarballs` job in [jenkins-jobs.md](./jenkins-jobs.md)
195+
runs the second form nightly.
196+
162197
[nodejs.org]: https://nodejs.org
163198
[release process]: https://github.com/nodejs/node/blob/main/doc/contributing/releases.md

‎orka/templates/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ We need the private key for node-www for the release images, as well as the appl
5757

5858
Xcode Command-line tools are not enough to perform a full notarization cycle, full Xcode must be fully installed on the release images.
5959

60+
The release image is used by both the `.pkg` and the tarball nodes of `iojs+release`, and both notarize their output, so every release image needs full Xcode and the `NODE_RELEASE_PROFILE` notarytool profile that this template stores in the System keychain.
61+
6062
* Login to https://developer.apple.com using the apple-operations@openjsf.org account
6163
* Download Xcode: https://developer.apple.com/download/more/ - find non-beta version, open Developer Tools in browser, Networking tab, start download (then cancel), in Networking tab "Copy as cURL" (available in Chrome & FF)
6264
* On OSX 15 we currently install 16.4

0 commit comments

Comments
 (0)