Skip to content

fix(support): no gate rebuilds the guest, and the reproducible build works on amd64 only #1811

Description

@Toby1009

Updated 2026-08-20 after #1821. That PR fixed the guest lockfile, brought RISC0_VERSION and
RISC0_TOOLCHAIN into stage scope, made build.rs derive the guest-builder tag from the
Dockerfile, and removed the tracked Elf.sol. All three original blockers are gone: I ran the
reproducible build and it now reaches the guest source. The title no longer fits and the first
section below replaces items 1, 2 and 3a.

Re-checked 2026-08-21 against b5811892. The reproducible build was run again and behaves
exactly as described below, down to the same builder digest. Line references into ci.yml,
releases.yml and bump-versions.ts moved with #1846 and #1848 and are corrected here.

The protocol image ID identifies the guest code that the protocol ciphertext verifier accepts.
Acceptance also requires the protocol-bound journal and the E3 program's separate verification, so
the image ID is not the whole gate. It is the part that identifies code. A reproducible build is
what would connect it back to reviewed source. Without one, a receipt supports "a binary matching
this image ID ran correctly" rather than "the source at commit X ran correctly", and the tracked
ImageID.sol carries the value alone, with no revision or build metadata
(crates/support/contracts/ImageID.sol:17-22).

#1821 added docs/pages/verifying-the-compute-provider.mdx and pnpm provenance:manifest, which
record that connection. The page states the same gap in its own words: "Nothing in the repository
reproduces the image ID from the guest source any more, so a match here shows the deployment agrees
with what was committed, not that what was committed is what the source builds."

The reproducible build now runs, and stops only on an arm64 host

Run against 3c7172c8 from a detached worktree, RISC0_USE_DOCKER=1 cargo build in
crates/support/methods. The first two blockers are gone:

#2 [internal] load metadata for docker.io/risczero/risc0-guest-builder:r0.1.91.1
#3 [build 1/5] FROM docker.io/risczero/risc0-guest-builder:r0.1.91.1@sha256:fafb377a44e1cfca415577c48d2f7012bda99ed36f2fae27f9a663b9fe6048f0
#7 [build 4/5] RUN cargo +risc0 fetch --locked --target riscv32im-risc0-zkvm-elf --manifest-path methods/guest/Cargo.toml
#8 [build 5/5] RUN cargo +risc0 build --release --locked ...

cargo fetch --locked succeeds, so the guest lockfile resolves. guest_builder_tag
(crates/support/methods/build.rs:56-80) reads ARG RISC0_TOOLCHAIN=1.91.1 and hands
r0.1.91.1 to docker_container_tag, and that image carries rustc 1.91.1, which satisfies the
MSRV of the fhe fork and tfhe-ntt. Over two hundred crates compile, risc0-zkp v3.0.3 among
them.

It then fails on this host:

#8 60.30 error: failed to run custom build command for `fhe v0.2.1 (https://github.com/gnosisguild/fhe.rs?tag=v0.2.2#f2c1d225)`
#8 60.30   process didn't exit successfully: .../build/fhe-56712c74a11d6369/build-script-build (exit status: 1)
#8 60.30   --- stderr
#8 60.30   Error: Custom { kind: Other, error: "protoc failed: " }

This is specific to arm64. risczero/risc0-guest-builder:r0.1.91.1 publishes a single
manifest, application/vnd.docker.distribution.manifest.v2+json with no platform list, so there is
no arm64 image and buildkit emulates. It also ships no protoc:

$ docker run --rm --entrypoint sh risczero/risc0-guest-builder:r0.1.91.1 -c 'command -v protoc || echo absent; rustc --version'
absent
rustc 1.91.1 (ed61e7d7e 2025-11-07)

fhe's build script guards for exactly that, with
Command::new("protoc").arg("--version").output().is_ok(), and falls back to skipping proto
compilation. Under emulation the guard inverts. The same six-line probe, compiled and run in each
environment:

Image output().is_ok() Result
rust:1.91-slim, native linux/arm64 false err=No such file or directory (os error 2)
rust:1.91-slim, emulated linux/amd64 true status=ExitStatus(unix_wait_status(32512)), empty stdout and stderr
risc0-guest-builder:r0.1.91.1, emulated true same, 32512

32512 is 127 << 8, the shell's "command not found". Emulated, the missing binary reports through
the child's exit status instead of an error in the parent, so is_ok() is true, the guard passes,
and prost-build reports protoc failed: with nothing after the colon.

The build may well complete on a native amd64 host. I have not verified that it does, having no
amd64 machine. What the run does establish is that nothing in the guest tree blocks it any more, and
that an arm64 reader following docs/pages/verifying-the-compute-provider.mdx gets a failure that
names a protobuf compiler and says nothing about architecture.

Two things the same run settles for the record:

  • The builder resolves to a digest today, sha256:fafb377a44e1cfca415577c48d2f7012bda99ed36f2fae27f9a663b9fe6048f0.
    The tag is mutable, so the digest is what makes a build record a record.
  • pnpm provenance:manifest names a different builder from the one the build uses.
    defaultBuilderTag (scripts/generate-provenance-manifest.ts:116-130) reads risc0-build's
    compiled-in DEFAULT_DOCKER_TAG, which is r0.1.88.0, while build.rs overrides that with
    r0.1.91.1. Running the generator on this tree prints
    "tag": "risczero/risc0-guest-builder:r0.1.88.0" beside "risc0GuestToolchain": "1.91.1". The
    function's own comment says it exists so the manifest does not "record a builder nobody used".
    fix(support): record the builder image the guest build actually pulls #1854 removes defaultBuilderTag and derives the tag from ARG RISC0_TOOLCHAIN, the source
    build.rs reads.

SKIP_SOLIDITY is still out of stage scope

crates/support/Dockerfile:1-4 declares RUST_VERSION, RISC0_VERSION, RISC0_TOOLCHAIN, and
SKIP_SOLIDITY before the first FROM on line 5. #1821 re-declared RISC0_VERSION and
RISC0_TOOLCHAIN inside the stage (:42-43) and added a fail-closed guard (:47-49).
SKIP_SOLIDITY was not among them, so --build-arg SKIP_SOLIDITY=1 still has no effect, while
crates/support/methods/build.rs:104 genuinely reads it.

An ARG declared before the first FROM is out of scope inside a stage unless re-declared, which a
minimal check confirms:

ARG MYVAR=pinned-value
FROM alpine:3
RUN echo "[${MYVAR}]"      ->  []
ARG MYVAR
RUN echo "[${MYVAR}]"      ->  [pinned-value]

docker build --check reports no warnings for the current file, because the name is declared
somewhere in it. The linter is not evidence either way here.

No gate compiles the guest or compares the image ID

.github/workflows/ci.yml:319-320 says the build_e3_support_risc0 job "Guards the RISC Zero guest
artifact. The on-chain imageId is immutable, so a guest change that leaves ImageID.sol untouched
ships a verifier that no longer matches this tree."

The job's steps are checkout, tag generation, buildx setup, registry login, and
docker/build-push-action on crates/support (:321-364). The image's last instruction is
CMD ["/bin/bash"] (crates/support/Dockerfile:69). Nothing in the job compiles the guest, and
nothing compares the result against crates/support/contracts/ImageID.sol. releases.yml:147-179 builds
the same image.

crates/support is also excluded from the root workspace (Cargo.toml:49-56), and the root scripts
operate on that workspace, so no root build, test, or lint reaches it either.

Dangling references introduced alongside the fix

crates/support/Cargo.toml cites ImageID.stamp.json (:50) and pnpm check:image-id three times
(:50, :62, :68), describing what each would assert or warn about. Neither exists on main:
crates/support/contracts/ holds only ImageID.sol, and package.json has no check:image-id
script. As written the comments describe a gate that is not there.

Three ImageID.sol values, one with no consumer

File Value
crates/support/contracts/ImageID.sol:22 0x7ac60209…44e105
examples/CRISP/.interfold/generated/contracts/ImageID.sol:22 0x0ad904cf…7368d6
templates/default/.interfold/generated/contracts/ImageID.sol:22 0xaf928ebf…1a76ef

A repository-wide search finds no consumer for the crates/support copy; only
methods/build.rs:31 writes it. CRISP reads its generated copy with readFileSync
(examples/CRISP/packages/crisp-contracts/deploy/crisp.ts:16-18). The template compiles its copy through Hardhat
(templates/default/hardhat.config.ts:79) and deploys and reads it (templates/default/deploy/default.ts:44-56).

On a normal checkout, predev:all does not rebuild the guest, because its guard
[ ! -f './.interfold/generated/contracts/ImageID.sol' ] sees the tracked file
(templates/default/package.json:20). An explicit interfold program compile still rebuilds it.

Smaller items in the same area

  • crates/support/Cargo.toml:3 sets msrv under [workspace]. Cargo has no such key. Builds in that
    workspace print warning: crates/support/Cargo.toml: unused manifest key: workspace.msrv;
    cargo metadata does not. Closed audit item I46 (Synchronize independent Rust consumer lockfiles after Alloy upgrade #1692) removed exactly this key from the root manifest,
    for exactly this reason, and that remediation did not reach the support workspace.
  • crates/support/README.md:293 points at .github/workflows/support-docker.yml, which does not
    exist. The jobs are ci.yml:321 and releases.yml:147. fix(docs): correct stale API references in support and building guides #1813 corrects this line.
  • crates/support is its own workspace at 0.1.0 (crates/support/Cargo.toml:8) while the root is
    0.10.0 (Cargo.toml:73). scripts/bump-versions.ts:383-394 passes only the root manifest to its
    Rust updater, so this crate does not take part in pnpm bump:versions.
  • agent/CONTEXT.md:16 still states that the unified version is 0.4.0.

What would close this

A gate that rebuilds the guest with a digest-pinned builder and compares the result against the
committed value, run in the existing build_e3_support_risc0 job, which is already filtered on
crates/support/**.

Once it works, the same commands are what an outside reader would run, so the external verification
procedure in docs/pages/verifying-the-compute-provider.mdx becomes reproducible rather than a
comparison against a committed constant. Two details decide whether such a procedure is real: the
builder has to be identified by digest rather than by a mutable tag, and a plain SHA-256 of the ELF
is not the image ID. RISC Zero loads the program into a memory image and takes that image's Merkle
root (risc0-binfmt-3.0.2/src/image.rs:253).

Happy to open a PR for any of this if it is useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions