Skip to content

fix(monitors): reject stale overlapping observe commits - #118

Open
kvnloo wants to merge 3 commits into
CopilotKit:mainfrom
kvnloo:fix/monitor-observe-version-guard
Open

kvnloo wants to merge 3 commits into
CopilotKit:mainfrom
kvnloo:fix/monitor-observe-version-guard

Conversation

@kvnloo

@kvnloo kvnloo commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Problem

Two monitor observations can overlap if a worker lease expires while the first observation is still in flight and another worker claims the task.

Both runs can read the same monitor snapshot and later pass the current CAS predicate of only { status: "active" }. The second commit can then overwrite the first run's newer baseline and lose a checks increment.

Change

Include the monitor's current checks value in the publication CAS predicate.

checks already increments on every successful observation, so it acts as a lightweight commit version without adding schema.

A stale overlapping run now loses the CAS and follows the existing LostLeaseError retry path instead of clobbering the newer observation.

Regression

The test freezes two observe calls on the same monitor snapshot:

  • first observe commits successfully;
  • second observe rejects with LostLeaseError;
  • exactly one additional checks increment lands.

No monitor matching, scheduling, or notification semantics change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant