Skip to content

fix(resilience): ignore outcomes from older circuit recovery cycles - #425

Open
er-s-an wants to merge 1 commit into
HKUDS:mainfrom
er-s-an:fix/circuit-stale-outcomes-20261022
Open

er-s-an wants to merge 1 commit into
HKUDS:mainfrom
er-s-an:fix/circuit-stale-outcomes-20261022

Conversation

@er-s-an

@er-s-an er-s-an commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Description

A request admitted before a circuit opens can finish after the breaker starts recovery. Its late success currently closes the breaker during cooldown or releases a newer half-open probe; its late failure can interrupt that probe or reopen a recovered circuit.

Track the recovery generation when admitting each decorated call and apply its outcome only while that generation is current. Keep the generation check and state update atomic under the shared lock. The original return value or exception still reaches the caller. Public manual recording and cancellation ownership behavior are preserved.

Related Issues

Found through concurrent reproduction. Circuit issue/PR searches found no equivalent fix. #377 covers cancelled-probe cleanup and explicitly leaves success/failure accounting unchanged; this addresses late completion accounting.

Validation

  • Twelve event-synchronized cases exercise real threads and async tasks, completing an older successful/failing call while the breaker is open, probing, or recovered. Before the fix: 8 failed, 31 passed with the existing resilience suite. After: all 39 passed.
  • Fresh full suite: 910 passed, Linux/Python 3.12.14. ReportLab and OpenAI were added to the local test environment after the first run exposed missing optional dependencies; project dependencies are unchanged.
  • Repository pre-commit hooks and diff checks passed. Final diff reviewed separately for atomic admission/completion checks, public behavior and probe ownership.

No hosted model, production load test, or other OS/Python matrix was exercised. This uses the existing clock and timeout semantics.

Checklist

  • Changes tested locally
  • Code reviewed
  • Unit tests added
  • Documentation reviewed; no public API change

Developed with OpenAI Codex assistance; no independent human review is claimed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant