Skip to content

test: stop the quick suite sitting out real sleeps - #5673

Merged
springfall2008 merged 1 commit into
mainfrom
test/quick-suite-real-sleeps
Oct 22, 2026
Merged

springfall2008 merged 1 commit into
mainfrom
test/quick-suite-real-sleeps

Conversation

@springfall2008

Copy link
Copy Markdown
Owner

Summary

Several quick-suite tests were waiting out real back-offs and poll intervals that their checks don't depend on. To find them, I wrapped every real time.sleep / asyncio.sleep during ./run_all --quick and charged the time to whichever test was running. This PR fixes the sleeps that are not themselves what the test checks. All changes are in test files.

On the same origin/main base the quick suite goes from 84.5s to 71.4s locally. CI runners are about 2.5× slower, so that should save roughly 30s there.

Test Before After Change
hanchu_api 6.03s 0.02s The transport-retry test sat through the real 2s + 4s back-off. hanchu.asyncio.sleep is now an AsyncMock, and the test also asserts the delays are 2s then 4s.
annual_job 2.83s 1.10s The elapsed-freeze check moves annual_job's clock 5s forward (only that module's time reference) instead of time.sleep(1.1). Stub polling is 20ms instead of 100ms.
component_base 2.41s 1.03s The first-flag test polls until a steady-state (first=False) run happens, instead of waiting a fixed 200 simulated seconds.
solis 1.46s 0.48s The transient-retry case shortens SOLIS_INITIAL_RETRY_DELAY.
chat 2.72s 2.04s test_write_confirmation_timeout uses a 0.1s timeout with the existing _fast_confirm_poll().
gateway 0.75s 0.25s test_write_switch_handler_does_not_block polls the ack at 10ms instead of 0.5s. The patch is stopped in the test's own finally, because run_gateway_tests never calls tearDown.

Still testing the same thing

  • annual_job: I broke the elapsed freeze in memory (status() reading as if finished_at were unset) and the test still fails: elapsed should freeze once a run is complete, got 0 then 5.
  • component_base: it still asserts that a first=False run happens and that the startup run happens exactly once. The poll only ends the wait sooner.
  • gateway: I checked _commit_pause, _monotonic and _COMMIT_POLL_SECONDS after a full run_gateway_tests. All three are restored, both before and after this change.

Left alone on purpose

  • Sleeps that are the point of their test: the chat heartbeat and turn hand-off checks, the Solis lock-blocking waits and agent_tools.
  • The web.py server loop's 1s sleep: it runs in the background and doesn't hold up any test.
  • db_manager's ~14 × 0.1s waits for its worker thread (~1.4s): these could become condition waits in a follow-up.

Test plan

  • ./run_all --quick passes (71.4s, against 84.5s for origin/main's versions of the same six files)
  • Each changed test run on its own: component_base, annual_job, chat, solis, gateway, hanchu_api
  • pre-commit clean on the changed files

🤖 Generated with Claude Code

A census of every real time.sleep / asyncio.sleep during ./run_all --quick
found several tests waiting on wall-clock back-offs and poll ticks that
their assertions do not depend on.

- hanchu_api: the transport-retry test slept the real 2s + 4s back-off;
  the sleep is now recorded and the 2s, 4s sequence asserted (6.0s -> 0.0s)
- component_base: the first-flag test waited a fixed 200 simulated seconds;
  it now polls for the steady-state run and stops when it happens
- annual_job: the elapsed-freeze check moves annual_job's clock five
  seconds on instead of time.sleep(1.1), and the stub poll is 20ms not 100ms
  (still caught: breaking the freeze reports elapsed 0 then 5)
- solis: the transient-retry case shortens SOLIS_INITIAL_RETRY_DELAY
- gateway: the non-blocking commit test polls the ack at 10ms, not 0.5s
- chat: the confirmation-timeout test uses a 0.1s timeout with the existing
  fast confirm poll

Deliberate timing proofs (chat heartbeat and hand-off, Solis lock-blocking,
agent_tools) are left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the BOT_REVIEW Trigger an autotriage label Oct 21, 2026

@springfall2008 springfall2008 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated comment from the triage bot.

Reviewed head 92e178a (test: stop the quick suite sitting out real sleeps): test-only changes to six files, aimed at sleeps the tests do not assert on. No issues found (0 inline, 0 in body).

What I checked, working from the PR head against the production code each mock touches:

  • annual_job — annual_job.status() reads time.time() only when finished_at is unset (apps/predbat/annual_job.py:60), so patch("annual_job.time", SimpleNamespace(...)) rebinds just that module's global: when the freeze works, the patched clock is never reached and both reads return int(finished_at - started_at); if the freeze were removed, the 5 s jump is caught deterministically (int truncation cannot collapse a ≥5 s gap). This is strictly more deterministic than the time.sleep(1.1) it replaces, which its own old comment admitted was probabilistic.
  • chat — chat.py reads both CONFIRM_TIMEOUT_SECONDS and CONFIRM_POLL_SECONDS at call time (chat.py:2291, chat.py:2309), so timeout 0.1 s with the existing _fast_confirm_poll() (poll 0.002 s, restored in its own finally) keeps ~50 waiter iterations and the same decline-on-timeout assertions; the module attribute is restored in the test's finally.
  • component_base — the poll breaks on any(flag is False for flag in component.first_flags), and first_flags is the test component's own list of bools, so the break condition is exactly the assertion it feeds; worst case remains the old 200 simulated seconds (fast_sleep scales sleeps by 1/100), typical case ends early. Assertions are unchanged.
  • gateway — gateway._commit_pause() reads the module global _COMMIT_POLL_SECONDS at call time (gateway.py:109), so the 0.01 s patch takes effect, and poll_patch.stop() runs in the test's finally. Later gateway tests that exercise the ack wait patch _commit_pause or _monotonic themselves, so even the short unprotected window between poll_patch.start() and the try (loop/thread setup) cannot meaningfully leak into them.
  • hanchu — with HANCHU_RETRIES = 3 and await asyncio.sleep(2 * attempt) starting from attempt = 1 on the second try (apps/predbat/hanchu.py:348-362), a transport failure produces exactly 3 posts and sleeps of 2 s then 4 s, so the new await_args_list == [2, 4] assertion is correct and actually tightens the test (it now pins the back-off shape, not just the attempt count). AsyncMock/patch are imported, and the two-patch with is scoped to the one call.
  • solis — _with_retry re-reads SOLIS_INITIAL_RETRY_DELAY at call entry (apps/predbat/solis.py:811), so the 0.01 s patch shortens the real back-off while max_retry_time=5 and the retry semantics are untouched; the retry/no-retry assertions are unchanged.

Also checked: all six runners stay registered in TEST_REGISTRY, nothing assigns to my_predbat.* without restore, every patch is restored on all paths, no production code (and therefore no kernel-parity, docs, CONFIG_ITEMS or Storage implications), and the deliberately-left-alone sleeps listed in the PR description are indeed ones whose tests assert on the sleep itself.

@springfall2008 springfall2008 removed the BOT_REVIEW Trigger an autotriage label Oct 21, 2026
@springfall2008
springfall2008 merged commit 3db0f2b into main Oct 22, 2026
3 checks passed
@springfall2008
springfall2008 deleted the test/quick-suite-real-sleeps branch October 22, 2026 07:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant