Fix output loss in quiet mode when a task fails - #22
Merged
Conversation
The quiet buffer was flushed as soon as the child exited, but the stdout and stderr handlers that fill it were only awaited afterwards. A child exiting does not mean its pipes have been drained, so on a loaded machine the flush emitted nothing and the real output was appended to a buffer nobody read again. Drain the handlers first. This is what made the quiet-mode-failure golden test flaky - it dropped both expected lines and left only "-- output:" and "-- exit code: 1". It blocked the v0.18.0.13 release: the arm64 job failed on it and matrix fail-fast cancelled x86_64, so no release was published. Also make the append atomic. Both handlers write to the buffer concurrently, and plain modifyIORef can lose one of two racing updates. The new test fails 8/8 without the fix and passes 8/8 with it, where the existing single-line test only failed about half the time locally. Detection needs enough lines that draining them takes measurable time after exit, which in turn exposed the deadlock already flagged by a FIXME in the harness: hGetContents is lazy, so nothing was read until after waitForProcess and a task writing more than the pipe buffer holds (~64 KiB) blocked forever. Forcing the read first is enough, since UseHandle closes the parent's copy of the write end and the read ends at EOF.
zyla
approved these changes
Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
quiet-mode-failuregolden test has been flaky, and it is a real bug rather thantest noise. It blocked the
v0.18.0.13release: the arm64 job failed on it, matrixfail-fast cancelled x86_64, and no release was published.
The bug
App.hsflushed the quiet buffer as soon as the child exited, but the stdout/stderrhandlers — which are what fill that buffer (
Utils.hs:46) — were only awaitedafterwards:
A child exiting does not mean its pipes are drained. Under load the flush emitted
nothing and the real output was appended to a buffer nobody read again — hence the
observed symptom of both lines vanishing, leaving only
-- output:and-- exit code: 1. Fix: drain the handlers first.Separately, both handlers append concurrently via plain
modifyIORef, which can loseone of two racing updates. Now
atomicModifyIORef'.Test
quiet-mode-failure-large-outputfails 8/8 without the fix and passes 8/8 with it.The existing single-line test only failed about half the time locally, which is why this
survived so long.
Detection needs enough output that draining it takes measurable time after exit — and
that exposed the deadlock already flagged by a
FIXMEin the harness.LBS.hGetContentsis lazy, so nothing was read until after
waitForProcess, and any task writing more thanthe pipe buffer holds (~64 KiB) blocked forever. Forcing the read before reaping is
sufficient:
std_out/std_errare passed asUseHandle, so theprocesslibrary closesthe parent's copy of the write end and the read terminates at EOF.
Full suite passes.
Note
The release workflow's matrix
fail-fastis worth turning off regardless — one flakyarm64 job should not cancel the x86_64 build and leave a tag with no release asset.
🤖 Generated with Claude Code