Skip to content

VM migration between hosts, plus detection of migrations done outside the API - #351

Merged
v0l merged 1 commit into
masterfrom
vm-migration
Aug 2, 2026
Merged

VM migration between hosts, plus detection of migrations done outside the API#351
v0l merged 1 commit into
masterfrom
vm-migration

Conversation

@v0l

@v0l v0l commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Fixes #66

vm.host_id is what every lifecycle operation aims at — start, stop, reinstall, firewall sync, state polling — so a VM that moves without the database being told breaks all of them at once, with no signal that anything is wrong. A VM migrated by hand in the Proxmox UI is exactly that case, and one such VM is out of sync in production today.

Detection and repair

WorkJob::ReconcileVmHosts runs every 10 minutes: it lists the VMs on every reachable host and re-points vm.host_id/vm.disk_id (matching the storage pool it was found on) at whichever host actually has each VM. Each correction is recorded in VM history as the new migrated action with detected: true, and raises an admin notification — nobody asked this API for the move, so it is worth surfacing.

Two rules keep it from doing harm:

  • a host that could not be polled is omitted entirely rather than reported as empty, so an unreachable host does not make its VMs look like they moved away;
  • a VM seen on two hosts — the signature of a leftover copy on the source node after a migration — is reported as ambiguous and never reconciled, so the record cannot flap between them on every pass.

Migration

POST /api/admin/v1/vms/{id}/migrate with { target_host_id, live?, reason? } (permission virtual_machines::update) queues a MigrateVm job and returns a job_id. Pre-flight refuses:

  • a target in a different region — IP assignments are not rewritten, so the VM would come up unreachable;
  • a different hypervisor kind or CPU architecture;
  • a disabled host, or one without free CPU, memory or disk for the VM.

The destination pool is the one whose name matches the source where such a pool exists — on shared or mirrored storage that means nothing is copied — otherwise the emptiest pool with room, copied across with with-local-disks/targetstorage. live: false stops a running VM first and starts it again on the destination. host_id/disk_id are updated only after the hypervisor confirms the move, so a failed migration leaves the VM on, and pointed at, the source host; the firewall ruleset is re-applied on arrival.

Placement moves through a new LNVpsDb::update_vm_host rather than update_vm, which has never written host_id: where a VM lives is owned by the hypervisor and must not change as a side effect of saving an unrelated edit to a Vm.

Implemented for Proxmox. Other host kinds return a clear "not supported" error instead of half-moving a VM.

Tests

14 new tests, all green with cargo test --workspace --exclude lnvps_e2e -- --test-threads=1:

  • drift detection: VM moved to another host, VM on two hosts (ambiguous), unreachable host, host VMs outside the managed id range;
  • migration planning: same-name pool preferred without copying, fallback to the emptiest pool, and each refusal (same host, disabled, cross-region, cross-arch, no CPU/memory/disk);
  • end-to-end migrate_vm and reconcile_vm_hosts over MockDb and the dummy hypervisor, asserting the database follows the host and the history entry is written;
  • two wiremock tests pinning the Proxmox request body (notably: no with-local-disks when the disk does not have to move);
  • an e2e test for the endpoint's 409/404 refusals.

ADMIN_API_ENDPOINTS.md and API_CHANGELOG.md are updated.

… has

`vm.host_id` is aimed at by every lifecycle operation there is — start,
stop, reinstall, firewall sync, state polling — so a VM that moves
without the database being told breaks all of them at once, silently. A
VM migrated by hand in the Proxmox UI is exactly that case, and one such
VM is out of sync in production today.

Both halves work off the same rule: the host is the source of truth for
where a VM lives. `ReconcileVmHosts` runs every 10 minutes, lists the VMs
on every reachable host, and re-points host_id/disk_id at whichever host
actually has each VM, recording a `migrated` history entry with
`detected: true` and notifying admins. A host that could not be polled is
left out rather than treated as empty, and a VM seen on two hosts (the
signature of a leftover copy on the source node) is reported but never
reconciled, so the record cannot flap between them.

`POST /api/admin/v1/vms/{id}/migrate` performs the move. Pre-flight
refuses a target in another region — IP assignments are not rewritten, so
the VM would come up unreachable — as well as a different hypervisor kind
or CPU architecture, a disabled host, or one without free CPU, memory or
disk. The destination pool is the one whose name matches the source where
it exists, which on shared or mirrored storage means nothing is copied;
otherwise the emptiest pool with room, copied across. host_id/disk_id
change only after the hypervisor confirms the move, and the firewall
ruleset is re-applied on the destination.

Placement moves through a new `update_vm_host`, not `update_vm`, which
has never written host_id: where a VM lives is owned by the hypervisor,
so it must not change as a side effect of saving an unrelated edit.

Migration is implemented for Proxmox; other host kinds return a clear
"not supported" rather than half-moving a VM.

Fixes #66
@v0l
v0l merged commit 162eb47 into master Aug 2, 2026
6 checks passed
@v0l
v0l deleted the vm-migration branch August 2, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VM Migration

1 participant