Skip to content

Add metrics, drift repair, completion taints and AWS IPAM - #179

Merged
mthorson merged 16 commits into
masterfrom
prep/gcp-lab-issues
Oct 14, 2026
Merged

mthorson merged 16 commits into
masterfrom
prep/gcp-lab-issues

Conversation

@mthorson

@mthorson mthorson commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Adds optional assignment labels, metrics, GCP drift repair, completion taints and AWS IPAM allocation. New behavior defaults off. This draft includes the assignment-label change from #178 and should merge after it.

Issue Change
#126 Legacy assignment label with changes limited to its own Node key.
#147 Prometheus assignment metrics, health/readiness endpoints and GCP/AWS reservation inventory.
#150 GCP drift repair restores the persisted reservation and refuses an address assigned elsewhere.
#154 V1 migration and node-label troubleshooting guidance; no reproduced v2 defect is claimed.
#156 Separate completion taint with durable success/exhaustion state and protection against repeating uncertain cloud mutations.
#166 AWS public IPv4 IPAM allocation with ownership tags, durable intent, restart recovery and owned-allocation release.

Node state writes verify UID and resource version. Persistent assignment modes require release disabled and Node patch permission. Completion cannot be combined with drift repair or IPAM. AllocateAddress disables SDK retries because it has no idempotency token; uncertain outcomes recover by tags or require explicit reconciliation. Association failures retain the reservation.

GCP operation-status reads retry transient transport errors without repeating mutations. AWS release retries transient InvalidIPAddress.InUse after disassociation, and IPAM recovery supports a primary interface without an existing public IP. Deployment guidance covers required IAM actions and independent API connectivity. Shutdown grace is 330 seconds to cover the five-minute release timeout.

Validation: full race-enabled Go tests, lint and CI passed for the runtime changes. Live GCP and AWS checks covered assignment, restart recovery, ownership refusal, permission failures, release, metrics, drift repair and completion-state behavior. The injected lost-allocation-response case used real AWS allocation and verified no duplicate allocation. No merge or release has been performed.

@mthorson
mthorson changed the base branch from fix/assignment-label to master October 14, 2026 18:41
@mthorson mthorson changed the title Add AWS IPAM allocation and restart recovery for lab validation Prepare remaining issue fixes for GCP and AWS lab validation Oct 14, 2026
@mthorson mthorson changed the title Prepare remaining issue fixes for GCP and AWS lab validation Add metrics, drift repair, completion taints and AWS IPAM Oct 14, 2026
@mthorson
mthorson marked this pull request as ready for review October 14, 2026 21:29
@mthorson
mthorson merged commit 3328e2e into master Oct 14, 2026
3 checks passed
@mthorson
mthorson deleted the prep/gcp-lab-issues branch October 14, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant