Repository navigation
test: stop the quick suite sitting out real sleeps - #5673
Merged
Merged
Conversation
A census of every real time.sleep / asyncio.sleep during ./run_all --quick found several tests waiting on wall-clock back-offs and poll ticks that their assertions do not depend on. - hanchu_api: the transport-retry test slept the real 2s + 4s back-off; the sleep is now recorded and the 2s, 4s sequence asserted (6.0s -> 0.0s) - component_base: the first-flag test waited a fixed 200 simulated seconds; it now polls for the steady-state run and stops when it happens - annual_job: the elapsed-freeze check moves annual_job's clock five seconds on instead of time.sleep(1.1), and the stub poll is 20ms not 100ms (still caught: breaking the freeze reports elapsed 0 then 5) - solis: the transient-retry case shortens SOLIS_INITIAL_RETRY_DELAY - gateway: the non-blocking commit test polls the ack at 10ms, not 0.5s - chat: the confirmation-timeout test uses a 0.1s timeout with the existing fast confirm poll Deliberate timing proofs (chat heartbeat and hand-off, Solis lock-blocking, agent_tools) are left alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
springfall2008
commented
Oct 21, 2026
springfall2008
left a comment
Owner
Author
There was a problem hiding this comment.
Automated comment from the triage bot.
Reviewed head 92e178a (test: stop the quick suite sitting out real sleeps): test-only changes to six files, aimed at sleeps the tests do not assert on. No issues found (0 inline, 0 in body).
What I checked, working from the PR head against the production code each mock touches:
- annual_job —
annual_job.status()readstime.time()only whenfinished_atis unset (apps/predbat/annual_job.py:60), sopatch("annual_job.time", SimpleNamespace(...))rebinds just that module's global: when the freeze works, the patched clock is never reached and both reads returnint(finished_at - started_at); if the freeze were removed, the 5 s jump is caught deterministically (int truncation cannot collapse a ≥5 s gap). This is strictly more deterministic than thetime.sleep(1.1)it replaces, which its own old comment admitted was probabilistic. - chat —
chat.pyreads bothCONFIRM_TIMEOUT_SECONDSandCONFIRM_POLL_SECONDSat call time (chat.py:2291,chat.py:2309), so timeout 0.1 s with the existing_fast_confirm_poll()(poll 0.002 s, restored in its ownfinally) keeps ~50 waiter iterations and the same decline-on-timeout assertions; the module attribute is restored in the test'sfinally. - component_base — the poll breaks on
any(flag is False for flag in component.first_flags), andfirst_flagsis the test component's own list of bools, so the break condition is exactly the assertion it feeds; worst case remains the old 200 simulated seconds (fast_sleep scales sleeps by 1/100), typical case ends early. Assertions are unchanged. - gateway —
gateway._commit_pause()reads the module global_COMMIT_POLL_SECONDSat call time (gateway.py:109), so the 0.01 s patch takes effect, andpoll_patch.stop()runs in the test'sfinally. Later gateway tests that exercise the ack wait patch_commit_pauseor_monotonicthemselves, so even the short unprotected window betweenpoll_patch.start()and thetry(loop/thread setup) cannot meaningfully leak into them. - hanchu — with
HANCHU_RETRIES = 3andawait asyncio.sleep(2 * attempt)starting fromattempt = 1on the second try (apps/predbat/hanchu.py:348-362), a transport failure produces exactly 3 posts and sleeps of 2 s then 4 s, so the newawait_args_list == [2, 4]assertion is correct and actually tightens the test (it now pins the back-off shape, not just the attempt count).AsyncMock/patchare imported, and the two-patchwithis scoped to the one call. - solis —
_with_retryre-readsSOLIS_INITIAL_RETRY_DELAYat call entry (apps/predbat/solis.py:811), so the 0.01 s patch shortens the real back-off whilemax_retry_time=5and the retry semantics are untouched; the retry/no-retry assertions are unchanged.
Also checked: all six runners stay registered in TEST_REGISTRY, nothing assigns to my_predbat.* without restore, every patch is restored on all paths, no production code (and therefore no kernel-parity, docs, CONFIG_ITEMS or Storage implications), and the deliberately-left-alone sleeps listed in the PR description are indeed ones whose tests assert on the sleep itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Several quick-suite tests were waiting out real back-offs and poll intervals that their checks don't depend on. To find them, I wrapped every real
time.sleep/asyncio.sleepduring./run_all --quickand charged the time to whichever test was running. This PR fixes the sleeps that are not themselves what the test checks. All changes are in test files.On the same
origin/mainbase the quick suite goes from 84.5s to 71.4s locally. CI runners are about 2.5× slower, so that should save roughly 30s there.hanchu_apihanchu.asyncio.sleepis now anAsyncMock, and the test also asserts the delays are 2s then 4s.annual_jobannual_job's clock 5s forward (only that module'stimereference) instead oftime.sleep(1.1). Stub polling is 20ms instead of 100ms.component_basefirst=False) run happens, instead of waiting a fixed 200 simulated seconds.solisSOLIS_INITIAL_RETRY_DELAY.chattest_write_confirmation_timeoutuses a 0.1s timeout with the existing_fast_confirm_poll().gatewaytest_write_switch_handler_does_not_blockpolls the ack at 10ms instead of 0.5s. The patch is stopped in the test's ownfinally, becauserun_gateway_testsnever callstearDown.Still testing the same thing
status()reading as iffinished_atwere unset) and the test still fails:elapsed should freeze once a run is complete, got 0 then 5.first=Falserun happens and that the startup run happens exactly once. The poll only ends the wait sooner._commit_pause,_monotonicand_COMMIT_POLL_SECONDSafter a fullrun_gateway_tests. All three are restored, both before and after this change.Left alone on purpose
agent_tools.web.pyserver loop's 1s sleep: it runs in the background and doesn't hold up any test.db_manager's ~14 × 0.1s waits for its worker thread (~1.4s): these could become condition waits in a follow-up.Test plan
./run_all --quickpasses (71.4s, against 84.5s fororigin/main's versions of the same six files)component_base,annual_job,chat,solis,gateway,hanchu_api🤖 Generated with Claude Code