server: release the request body of a streaming call once its handler is gone - #313
Merged
Merged
Conversation
… is gone The reader behind a client- or bidi-streaming call kept the partial message it had buffered, its task and the request body until the client ended the stream, even after the handler had returned, an interceptor had rejected the call, or the request timeout had fired. A client that stalled part-way through a message could hold them for as long as it kept the connection open. The reader now races a body with nothing ready against the handler's end of the request channel, so a gone handler is noticed while the client is stalled. It frees the partial message and discards the rest of the body for at most 1 MiB and 5 seconds (bytes only on wasm32, which has no clock), then drops the body: an HTTP/2 stream is reset with NO_ERROR, an HTTP/1.x connection is closed. The drain itself stays, on both versions: HTTP/1.x needs it for keep-alive, and on HTTP/2 dropping the body at once makes h2 charge frames still in flight from the client to connection-wide budgets and answer with GOAWAY(ENHANCE_YOUR_CALM). The end-to-end tests pin both. The handler's request stream now ends when the decoder finishes (the END_STREAM envelope or a decode error) rather than when the body does. Signed-off-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>
…alls Feeds envelope-framed messages to ConnectRpcService from an in-memory body, so the reader task, the channel to the handler and the call around them are timed without a client or a server sharing the machine. Cases: a light handler with the body ready, in 16 KiB chunks and pending before every frame, and the BenchService client-stream handler. Each case is checked once, untimed, so a failing call cannot pass for a fast one. Signed-off-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>
rpb-ant
approved these changes
Sep 21, 2026
iainmcgin
added a commit
that referenced
this pull request
Sep 21, 2026
Patch release for 0.9.0 carrying the fix from #313. A client- or bidi-streaming call can end while the client is still sending its request: the handler returns, an interceptor rejects the call, or the request timeout fires. The server kept reading the request body in a background task after that, so if a client stalled mid-message, that task, the partial message it had buffered and the HTTP/2 stream were held until the client ended the stream or closed the connection. The reader now frees the partial message when the handler is gone and discards the rest of the body for at most 1 MiB and 5 seconds before dropping it. A client still uploading when the limit is reached loses its HTTP/2 stream or its HTTP/1.x connection. It is cut from the v0.9.0 tag on a `release-0.9` branch because main has unreleased API additions that do not belong in a patch. Unlike #313, the handler's request stream still ends when the drain does, not when the decoder finishes. The companion crates stay at 0.9.0. After merge, a signed `v0.9.1` tag runs `release.yml` and `publish-crates.yml`. --------- Signed-off-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>
iainmcgin
added a commit
that referenced
this pull request
Sep 21, 2026
Patch release for 0.8.1 carrying the fix from #313. A client- or bidi-streaming call can end while the client is still sending its request: the handler returns, an interceptor rejects the call, or the request timeout fires. The server kept reading the request body in a background task after that, so if a client stalled mid-message, that task, the partial message it had buffered and the HTTP/2 stream were held until the client ended the stream or closed the connection. The reader now frees the partial message when the handler is gone and discards the rest of the body for at most 1 MiB and 5 seconds before dropping it. A client still uploading when the limit is reached loses its HTTP/2 stream or its HTTP/1.x connection. It is cut from the v0.8.1 tag on a `release-0.8` branch because main has unreleased API additions that do not belong in a patch. It enables tokio's `macros` feature in `connectrpc`, which 0.8.1 lacks and the reader's `select!` needs on wasm32; 0.9.0 already does. Unlike #313, the handler's request stream still ends when the drain does, not when the decoder finishes. The companion crates stay at 0.8.0. After merge, a signed `v0.8.2` tag runs `release.yml` and `publish-crates.yml`. --------- Signed-off-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>
This was referenced Sep 21, 2026
iainmcgin
added a commit
that referenced
this pull request
Sep 23, 2026
A client-streaming or bidi handler received its request messages from a reader task spawned for each call, which decoded envelopes from the HTTP body and passed each message through an `mpsc::channel(1)`. Every message therefore crossed one more task boundary on its way from hyper to the handler. The request stream now owns the body and the `EnvelopeDecoder` and decodes in `poll_next`, in whichever task polls it. The detached reader task, its channel and the `StreamingResponseBody` field that held its join handle are gone. The drain from #313 is kept, with the same bounds: when the stream ends (END_STREAM, a decode error) or is dropped before the body has ended, it drops the decoder with any partial message and hands the rest of the body to a detached task that reads and discards frames until the body ends, 1 MiB has been discarded (counting what the decoder left of the last frame) or `DRAIN_TIMEOUT` has passed. Reading rather than dropping still matters for h2's small-frame budget and for HTTP/1.1 connection reuse. A body that already reports its end needs no task. The drain now starts when the handler drops the stream rather than when the handler's end of a channel is seen to close, so there is nothing to race while a client is stalled. The body is now read only while the handler polls its request stream; the old reader read up to two messages ahead. The drain task is spawned on the runtime the stream was created on, normally the server's, so a handler that drops its request stream on another thread or runtime still gets the drain. `spawn_detached`, the fallback for a stream created outside a runtime, no longer returns a handle and does nothing outside a Tokio runtime instead of panicking. The reader tests now drive `decode_request_body` directly and wait for the body to be dropped instead of joining the reader task; a per-thread panic count stands in for the join handle's panic check. New tests cover frames that split or pack envelopes, trailers and empty frames, a body already at its end, a drop on another thread or outside a runtime, trailing data in the END_STREAM frame that alone exceeds the drain limit, and a drain that stops at a body error. `http1_connection_survives_unread_streaming_request_body` (tests/streaming) checks over a raw socket that a connection whose streaming request was abandoned after a decode error serves the next request; it fails with the drain disabled, as do the four `*_early_return_*` server tests. Stale mentions of a body reader in the `intercept_head` docs and the guide are updated. Fixes #298. Signed-off-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A client- or bidi-streaming call can end while the client is still sending its request: the handler returns, an interceptor rejects the call, or the request timeout fires. The server kept reading the request body in a background task after that. If the client stalled mid-message, that task, the partial message it had buffered (up to
max_message_size) and the HTTP/2 stream were held until the client ended the stream or closed the connection, and hyper allows 200 streams per connection by default.The reader now notices that the handler is gone while the client is stalled and frees the partial message. It discards the rest of the body for at most 1 MiB and 5 seconds (
DRAIN_TIMEOUT; bytes only on wasm32, which has no clock), then drops it, if the client has not finished by then: an HTTP/2 stream is reset withNO_ERROR, an HTTP/1.x connection is closed. The handler's request stream now ends when the decoder finishes (END_STREAM or a decode error), not when the body does.The drain stays on HTTP/2 too. Dropping the body at once makes h2 count the client's in-flight small DATA frames against connection-wide budgets and send
GOAWAY(ENHANCE_YOUR_CALM). That still happens for a bidi call whose handler stops reading requests but keeps its response open for longer than the drain, while the client keeps sending small frames.reader_benchon bare metal, three runs against main: +2% per message with the body ready, +5% when the body is pending before every frame, unchanged with a real handler.The 5 s bound is fixed, not a setting. A slow uploader still sending 5 s after the handler left loses its stream or connection; before, only exceeding 1 MiB did.