fix(vmm): enable KVM_CAP_ARM_WRITABLE_IMP_ID_REGS on aarch64 by rogersnm · Pull Request #6116 · firecracker-microvm/firecracker

rogersnm · 2026-08-15T15:13:18Z

Changes

On aarch64, enable KVM_CAP_ARM_WRITABLE_IMP_ID_REGS on the VM fd before vCPUs are created, whenever the host kernel offers the capability.

Reason

Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) writable from userspace (KVM: arm64: writable MIDR/REVIDR), gated behind KVM_CAP_ARM_WRITABLE_IMP_ID_REGS, which the VMM must enable per VM before any vCPU is created. Firecracker never enables it, so a custom CPU template with a modifier for one of these registers fails at boot with EINVAL even on kernels that support the write:

Start microvm error: System configuration error: Error configuring the vcpu:
Error applying template: Failed to set register 0x603000000013c000 to value
0x00000000410fd0c0: Invalid argument (os error 22)

Being able to rewrite these registers matters for the same reason the capability was added to KVM: the guest sees the host CPU identity while KVM masks features like SVE from the ID feature registers, and guest software that dispatches on the identity rather than on the feature registers or hwcaps then executes instructions the VM traps, dying with SIGILL. We hit this in production with WebKit on Graviton4 (both Skia and Mesa llvmpipe's LLVM JIT key on the CPU part number and emit SVE). A CPU template that rewrites MIDR_EL1 to a generic part resolves the whole class, but only once this capability is enabled.

Enabling the capability on its own does not change guest visible state: the registers keep their host values unless a template writes them, and same-value writes (for example on snapshot restore) were already accepted before the capability existed.

VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls (up to and including current main), so the ioctl is issued directly with the same definition kvm-ioctls uses on other architectures. The kvm-ioctls side is now up as rust-vmm/kvm#382; once a release containing it exists, the direct ioctl here can be replaced with a plain enable_cap call, and I am happy to follow up with that change or to rebase this PR onto it now if waiting for the dependency is preferred.

Verified on an EC2 c7g.metal running Ubuntu 26.04 (kernel 7.0):

Without this patch, the boot fails with the EINVAL above.
With this patch, a microVM booted with a custom template containing a MIDR_EL1 reg modifier comes up and the guest reads the templated value: /proc/cpuinfo reports CPU part: 0xd0c on a host whose part is 0xd40.
The new unit test passes on the same host (it returns early on kernels without the capability, verified against a 6.1 host where KVM_CHECK_EXTENSION reports 0).

License Acceptance

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check CONTRIBUTING.md.

PR Checklist

This functionality cannot be added in rust-vmm. (The decision to enable the capability belongs in the VMM. The missing piece in rust-vmm is only the aarch64 enable_cap binding, which this PR works around with a direct ioctl; see the note above about lifting that cfg gate upstream.)

rogersnm · 2026-08-15T15:20:53Z

The kvm-ioctls side of this is now up as rust-vmm/kvm#382 (exposing VmFd::enable_cap on all architectures). Once that lands in a release, the direct ioctl in this PR can be replaced with a plain enable_cap call. Happy to follow up with that change, or to rebase this PR onto it now if you would rather wait for the dependency.

The KVM_ENABLE_CAP vm ioctl is architecture independent, but both the ioctl definition and VmFd::enable_cap were compiled only for x86_64, s390x and powerpc. aarch64 now needs them: Linux 6.15 gates writes to the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) behind KVM_CAP_ARM_WRITABLE_IMP_ID_REGS, which a VMM must enable on the VM before creating vCPUs. Without enable_cap, VMMs fall back to issuing the raw ioctl themselves (see firecracker-microvm/firecracker#6116). Remove the cfg gates and guard the x86 specific portion of the doc example so it builds and runs on every architecture. Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>

ShadowCurse · 2026-08-24T13:01:50Z

+- [#6116](https://github.com/firecracker-microvm/firecracker/pull/6116): On
+  aarch64, enable `KVM_CAP_ARM_WRITABLE_IMP_ID_REGS` before vCPU creation when
+  the host kernel offers it (Linux 6.15 and later). Custom CPU templates that
+  modify the implementation ID registers (`MIDR_EL1`, `REVIDR_EL1`, `AIDR_EL1`)
+  previously failed at boot with `Failed to set register ... Invalid argument`
+  because KVM rejects such writes unless the capability is enabled on the VM.


This should be in the Added section.

Done, moved to the Added section and reworded accordingly.

ShadowCurse · 2026-08-24T13:08:50Z


    /// Pre-vCPU creation setup.
    pub fn arch_pre_create_vcpus(&mut self, _: u8) -> Result<(), KvmVmError> {
+        // KVM gates writes to the implementation ID registers (MIDR_EL1,


This logic should be moved inside the new function

Done, the capability is now enabled in KvmVm::new, right after the VM fd is created. I think this is also strictly safer than the previous placement: KVM returns EINVAL for this KVM_ENABLE_CAP once any vCPU exists, so enabling at VM creation removes any possibility of hitting that if create_vcpus were ever called more than once.

ShadowCurse · 2026-08-24T13:47:52Z

+            if ret != 0 {
+                return Err(KvmVmError::EnableWritableImpIdRegs(errno::Error::last()));
+            }


I don't think this should be an error (since it will abort the whole VM boot process). The warning message should be enough.

Agreed. I downgraded it to a warning and removed the error variant. A template that does write these registers still fails loudly at template application, so nothing is silently lost.

ShadowCurse · 2026-08-24T13:51:01Z

+            // the other architectures.
+            // SAFETY: The ioctl is safe because we allocated the struct and
+            // the kernel will only read the size of the struct.
+            let ret = unsafe { ioctl_with_ref(self.fd(), ioctls::KVM_ENABLE_CAP(), &cap) };


Since there is already a PR open to enable this call for vmfd, I would add a TODO comment referencing it and saying that this call can be replaced by one from the crate when PR is merged.

Done, added a TODO on the ioctl module referencing rust-vmm/kvm#382. (That PR has since been approved, so the follow-up should be possible once it is in a release.)

The KVM_ENABLE_CAP vm ioctl is architecture independent, but both the ioctl definition and VmFd::enable_cap were compiled only for x86_64, s390x and powerpc. aarch64 now needs them: Linux 6.15 gates writes to the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) behind KVM_CAP_ARM_WRITABLE_IMP_ID_REGS, which a VMM must enable on the VM before creating vCPUs. Without enable_cap, VMMs fall back to issuing the raw ioctl themselves (see firecracker-microvm/firecracker#6116). Remove the cfg gates and guard the x86 specific portion of the doc example so it builds and runs on every architecture. Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>

rogersnm · 2026-08-24T15:13:11Z

Thanks for the review @ShadowCurse. All four points are addressed, force-pushed as an amended commit since the PR will squash anyway:

CHANGELOG entry moved to the Added section.
The capability is now enabled in KvmVm::new rather than in arch_pre_create_vcpus, which is back to its empty body. The unit test simplifies with it, since setup_vm_with_memory already goes through KvmVm::new.
A failure to enable the capability is now a warning instead of aborting the boot, and the error variant is gone.
Added a TODO on the direct ioctl referencing Expose VmFd::enable_cap on all architectures rust-vmm/kvm#382.

I also added a sentence to the existing ARM note in docs/cpu_templates/cpu-templates.md documenting that the implementation ID registers need a 6.15+ host kernel, and updated the commit message to match the new shape.

Re-verified the full path on a 7.0 kernel (Ubuntu 26.04, aarch64): the new unit test enables the capability via KvmVm::new, writes a fake MIDR_EL1, and reads the templated value back.

Linux 6.15 made the implementation ID registers (MIDR_EL1, REVIDR_EL1, AIDR_EL1) writable from userspace, gated behind a capability that the VMM must enable on the VM before any vCPU is created. Firecracker never enables it, so a custom CPU template with a modifier for one of these registers fails at boot with EINVAL even on kernels that support the write: Failed to set register 0x603000000013c000 to value 0x410fd0c0: Invalid argument (os error 22) Enable the capability when the VM is created, whenever the host kernel offers it. This has no effect on guest visible state by itself: the registers keep their host values unless a template writes them, and writes of unchanged values (snapshot restore) were already accepted before the capability existed. A failure to enable it is logged rather than fatal: a VM whose template does not touch these registers is unaffected, and one that does still fails loudly when the template is applied. VmFd::enable_cap is not exposed for aarch64 by kvm-ioctls, so the ioctl is issued directly, with the same definition kvm-ioctls uses on other architectures (to be replaced with enable_cap once rust-vmm/kvm#382 is released). Signed-off-by: Nick Rogers <1903140+rogersnm@users.noreply.github.com>

rogersnm force-pushed the aarch64-writable-imp-id-regs branch 2 times, most recently from 3f7b27e to e6f77eb Compare August 15, 2026 15:16

rogersnm mentioned this pull request Aug 15, 2026

Expose VmFd::enable_cap on all architectures rust-vmm/kvm#382

Merged

4 tasks

ShadowCurse reviewed Aug 24, 2026

View reviewed changes

rogersnm force-pushed the aarch64-writable-imp-id-regs branch from e6f77eb to 409364d Compare August 24, 2026 15:11

rogersnm force-pushed the aarch64-writable-imp-id-regs branch from 409364d to c5e1036 Compare August 24, 2026 15:16

Navigation Menu

Uh oh!

fix(vmm): enable KVM_CAP_ARM_WRITABLE_IMP_ID_REGS on aarch64 - #6116

fix(vmm): enable KVM_CAP_ARM_WRITABLE_IMP_ID_REGS on aarch64#6116
rogersnm wants to merge 1 commit into
firecracker-microvm:mainfrom
rogersnm:aarch64-writable-imp-id-regs

rogersnm commented Aug 15, 2026 •

edited

Loading

Uh oh!

rogersnm commented Aug 15, 2026

Uh oh!

ShadowCurse Aug 24, 2026

Uh oh!

rogersnm Aug 24, 2026

Uh oh!

ShadowCurse Aug 24, 2026

Uh oh!

rogersnm Aug 24, 2026

Uh oh!

ShadowCurse Aug 24, 2026

Uh oh!

rogersnm Aug 24, 2026

Uh oh!

ShadowCurse Aug 24, 2026

Uh oh!

rogersnm Aug 24, 2026

Uh oh!

rogersnm commented Aug 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

rogersnm commented Aug 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Reason

License Acceptance

PR Checklist

Uh oh!

rogersnm commented Aug 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rogersnm commented Aug 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rogersnm commented Aug 15, 2026 •

edited

Loading