Skip to content

Streamed file input, stencil/depth bias, blend constant, occlusion queries, and an interpreter stack fix for Halo - #71

Draft
replghost wants to merge 12 commits into
mainfrom
work/halo-streamed-file-input
Draft

replghost wants to merge 12 commits into
mainfrom
work/halo-streamed-file-input

Conversation

@replghost

@replghost replghost commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Draft: runtime support needed to run the Halo CE engine (polkavm-app-kit apps/halo) on any Host. Everything is additive within application runtime ABI 1.

Changes

Runtime file handlers and streamed file input (ADR 0003; docs/runtime/polkavm-app-abi-v1.md)

  • Guests register file handlers at runtime, and a selected file can be delivered as a stream. Guests read it in bounded ranges (host_file_read) and keep a private per-selection cache (host_file_cache_*).
  • Implemented in the native and browser backends. Large inputs are not copied into guest memory or packaged as assets.

Stencil and depth bias, gated by RasterFeatures bit 1 (GPU_RASTER_FEATURE_STENCIL_DEPTH_BIAS)

  • Texture format 8 (Depth24PlusStencil8).
  • Render-pass stencil load, store and clear (pass flags 16, 32 and 64; flag 64 adds a 4-byte stencil clear value).
  • A pipeline trailer (flag 2) with stencil state and depth bias.
  • Opcode 30, SetStencilReference.
  • The wire decoder now rejects unknown pass and pipeline flag bits, because payload length depends on them.

Blend constant, gated by RasterFeatures bit 2 (value 4)

  • Opcode 31, SetBlendConstant: 16 bytes, RGBA f32 little-endian, every value finite. Valid only inside a render pass; it applies to the draws recorded after it.
  • Every pass starts with the constant 0,0,0,0, as in WebGPU. Before this change, native wgpu panicked at draw time when a pipeline used constant blend factors without a constant.

Occlusion queries, gated by RasterFeatures bit 3 (value 8)

  • BeginRenderPass flag 128 declares a pass's query count and a guest token. Passes in one batch declare at most 4,096 queries in total.
  • BeginOcclusionQuery (32) and EndOcclusionQuery (33) follow WebGPU's nesting rules on both Hosts.
  • Each declaring pass is resolved and read back without delaying the batch. Its results arrive as GPU event 9 (token, count, u64 sample counts), in submission order.
  • Native Hosts receive outstanding results from NativeGpuRenderer::poll_events.
  • Native and browser Hosts now advertise RasterFeatures 15.

Interpreter stack-residency fix

  • PolkaVM 0.37 (c160c13) treated a grown stack buffer's whole power-of-two capacity as resident guest stack. For a non-power-of-two declared stack, that exposed the lower guard page and corrupted Host reads of stack-backed memory.
  • The engine crate is vendored under rust/vendor/polkavm with residency bounded to the declared size, and with its provenance and licences recorded. Upstream PolkaVM is unchanged.

Merge of main (#57, mediated input runtime API)

  • Conflicts are resolved in the merge commit.
  • A file delivered while the runtime is paused is held until resume. This is covered by a new test.

Verification

  • cargo +nightly fmt --check, clippy with -D warnings, and cargo test --workspace --all-features: 193 passed.
  • npm test: 216 passed. Browser assets were regenerated, and verify-browser-assets passes.
  • polkavm-app-kit's Halo guests on this runtime:
    • The real engine starts on an original, freely licensed test room, on native compiler and interpreter.
    • A pixel-exact D3D frame check passes on native compiler and interpreter: 85 rectangles, 0 mismatching pixels. It covers stencil, z-bias, constant blending and occlusion counts (0 behind an occluder, exact counts in front).
    • The play check passes on native and in headless Chromium, both backends: real input moves, turns and collides with the walls.

Notes for reviewers

  • The vendored engine is a local patch, not an upstream PolkaVM release. It should be dropped once an upstream PolkaVM revision includes the fix.
  • No retail game data is involved in any test.

replghost and others added 12 commits September 29, 2026 12:19
Add ADR 0003 and the ABI v1 §File input contract. File input moves from the
App v2 capabilities.fileInput manifest block to a runtime host_file_register
call on the mediated-input lifecycle, with inline (8 MiB) and relaunch
(128 MiB) delivery and host_file_info for the selected file's name, MIME type
and size. The manifest keeps only an advisory top-level fileTypes hint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Implement ABI v1 §File input on the mediated-input lifecycle.
host_file_register validates the JSON descriptor (4 KiB, strict fields,
integer maxBytes, id/label/extension/MIME/mount-path rules), shares the
handle space and the eight-registration quota with host_input_register,
and reports -2/-4 from the deliveries the Host declares.
host_file_info describes a ready file.

Hosts see registrations with file_registrations, receive guest-triggered
picker requests as MediatedInputCommand::FileRequest, and deliver a
selection with send_file_input, which rejects empty or over-bound files
with status 6. A relaunch delivery stops the execution and returns a
FileRelaunch that set_file_relaunch mounts over the asset of a fresh
execution, where the re-registered handler reports status 3.
host_input_cancel discards a ready result and every successful cancel
leaves the registration idle. AppDescriptor parses the advisory fileTypes
hint, and the wasm bridge exposes the same calls to the browser core.

The file-input fixture is assembled from source in tests/file_input.rs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The translated backend parses descriptors with the native strictness and
both backends exchange the same Host messages: file-registrations,
file-input-request, file-input, and file-input-delivery, with the
fileInput and fileRelaunch start options. A Host still receives the
registrations of an execution that fails before it becomes ready.
Shared descriptor vectors and the file-input fixture run against both
backends. Browser assets are regenerated from the build.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The PolkaVM 0.37 interpreter (c160c13) grew its stack buffer to a power-of-two
capacity and treated the whole capacity as resident guest stack. For a
non-power-of-two declared stack (e.g. 384 KiB grown to 512 KiB) this exposed
the lower guard page and corrupted Host reads of stack-backed guest memory.

Vendor the pinned engine crate under rust/vendor/polkavm with residency
bounded to the declared stack size, record upstream provenance and licenses,
and ship the PolkaVM license texts with the browser package.
File input: a selected file can be delivered as a stream that guests read in
bounded ranges (host_file_read) and cache privately (host_file_cache_*), in
both the native and browser backends, so large inputs need not be copied into
guest memory or packaged as assets.

GPU wire, additive within application runtime ABI 1 and gated by
RasterFeatures bit 1 (GPU_RASTER_FEATURE_STENCIL_DEPTH_BIAS): texture format
Depth24PlusStencil8, render-pass stencil load/store/clear, a pipeline trailer
with stencil state and depth bias, and opcode 30 SetStencilReference. Unknown
pass and pipeline flag bits are now rejected because payload length depends on
them. Implemented in the native renderer and the browser GPU worker, with wire,
native and browser tests and the ABI document updated.
Rebuild the browser runtime (npm run build) with the vendored PolkaVM, file
streaming, and stencil/depth-bias support, and record the new asset digests.
… input

Resolve the merge of main's pause/background/random runtime work (#57) with
runtime file handlers, streamed file input, stencil/depth-bias GPU wire and
the vendored PolkaVM interpreter stack-residency fix.

- Cargo.toml: keep the vendored PolkaVM path dependency; take main's 0.3.1
  sibling crate versions.
- lib.rs: keep both test sets (resident stack growth and native pause).
- runtime-core.js: keep both state sets (file streams/caches and
  pause/background/held inputs); main's inactive frame/audio guards precede
  the file-input-request tracking in postRuntimeOutput; start sets both
  fileCacheEnabled and starting; start and pause/background failures use the
  asynchronous terminate() so cache cleanup completes before "terminated",
  while a disposed endpoint stays silent; the disposed-start teardown tracks
  translated cleanup.
- wasm-translated.js: initialize() marks initialization and takes main's u64
  gas budget.
- Docs: file pickers count as mediated-input prompts that inactive Hosts
  cancel; file status changes while paused or backgrounded wait for the next
  executed update.
- Tests: the malformed-background test awaits the asynchronous termination;
  add a regression that a file delivered while paused waits for resume in both
  backends.
- Regenerate browser assets (npm run build), SHA256SUMS and embedded digests.
Blend factors 12 (constant) and 13 (one minus constant) were accepted but
had no way to set the constant. SetBlendConstant carries four finite f32
values (RGBA, 16 bytes), is valid only inside a render pass, and applies to
later draws in that pass. Every pass starts with constant 0,0,0,0.

- gpu-wire: opcode 31 with an exact 16-byte payload and
  GPU_RASTER_FEATURE_BLEND_CONSTANT = 4.
- native: records the constant in pass order and calls
  set_blend_constant. Every pass now starts with an explicit zero constant,
  because wgpu otherwise rejects constant-factor draws with a validation
  panic.
- browser: parses, validates (in pass, finite), and calls
  setBlendConstant.
- Both hosts advertise RasterFeatures 7.
- ABI doc, wire and browser tests, and regenerated browser assets.
BeginRenderPass flag 128 declares a pass's query count and a guest token
(at most 4,096 per batch). BeginOcclusionQuery (32) and EndOcclusionQuery
(33) follow WebGPU's nesting rules on both Hosts. After the batch completes,
each declaring pass is resolved and read back asynchronously and delivered as
GPU event 9 (token, count, u64 sample counts) in submission order. Native
Hosts receive outstanding results from NativeGpuRenderer::poll_events.
Implemented in the wire crate, the native renderer and the browser GPU
worker, with tests, the ABI document and rebuilt browser assets.
@replghost replghost changed the title Streamed file input, stencil/depth bias, and an interpreter stack fix for Halo Streamed file input, stencil/depth bias, blend constant, occlusion queries, and an interpreter stack fix for Halo Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant