In the cross-implementation benchmark (benches/rpc, cross/client_stream_grpc: one gRPC client stream of 10 small messages, one request in flight, the same connectrpc-rs client driving every server), the connect-rust server is the slowest of the three Rust stacks: 186 µs per stream against 170 µs for tonic and 161 µs for tonic with the protobuf codec (bare-metal c7i.metal-24xl, turbo off, 2026-09-08). Unary and server-streaming calls on the same servers are level, so the gap is specific to the request-streaming path.
The cause is structural rather than decode cost. For client-streaming and bidi requests, handle_client_streaming_request / handle_bidi_streaming_request call spawn_body_reader, which spawns a detached task that polls the request body, decodes envelopes, and forwards each message to the handler over a tokio::sync::mpsc::channel(1). Every message therefore passes through an extra task and a bounded channel between hyper's connection task and the handler's task, and with a capacity of 1 the reader also waits for the handler to take each message before it reads the next. On a multi-threaded runtime those hand-offs mostly land on different workers: under strace -c the connect-rust server makes about 20 syscalls per 10-message stream (9.8 futex, 4.1 epoll_wait, 2.0 eventfd write, plus the 2 recvfrom / 2 writev that tonic also makes), where tonic makes 6. Pinning the server runtime to one worker thread (TOKIO_WORKER_THREADS=1) removes most of the gap on its own (174 µs to 160 µs per stream on the metal box, tonic 155 µs), which is what identifies the cost as wakeups rather than work.
The reader task exists for HTTP/1.1: when a handler returns without consuming the whole request stream, something must keep reading the body (bounded by MAX_DRAIN_BYTES) or hyper cannot reuse the connection, and aborting a reader part-way through the body had been found to race with hyper's end-of-body detection. That property does not require a task in the common case. A request stream that owns the body and the EnvelopeDecoder and decodes in poll_next, in the handler's own task, can hand the unread remainder of the body to a detached drain task only when it finishes decoding (END_STREAM, a decode error) or is dropped before the body has ended; a handler that reads its stream to the end then involves no extra task at all, and the early-return path drains exactly as today.
Measured that way on the same box (three alternating 10 s rounds per server, 8 server cores):
| server |
µs per 10-message stream, 1 in flight |
connect-rust main |
174.3 |
| connect-rust, inline decoding |
152.1 |
| tonic |
157.7 |
| tonic + protobuf codec |
149.5 |
The syscall profile drops to tonic's 6 per stream, and server instructions per stream fall about 7%. With 8 streams in flight all four servers are within 4% of each other before and after, so the change is latency at low concurrency and CPU per message rather than peak throughput.
In the cross-implementation benchmark (
benches/rpc,cross/client_stream_grpc: one gRPC client stream of 10 small messages, one request in flight, the same connectrpc-rs client driving every server), the connect-rust server is the slowest of the three Rust stacks: 186 µs per stream against 170 µs for tonic and 161 µs for tonic with the protobuf codec (bare-metal c7i.metal-24xl, turbo off, 2026-09-08). Unary and server-streaming calls on the same servers are level, so the gap is specific to the request-streaming path.The cause is structural rather than decode cost. For client-streaming and bidi requests,
handle_client_streaming_request/handle_bidi_streaming_requestcallspawn_body_reader, which spawns a detached task that polls the request body, decodes envelopes, and forwards each message to the handler over atokio::sync::mpsc::channel(1). Every message therefore passes through an extra task and a bounded channel between hyper's connection task and the handler's task, and with a capacity of 1 the reader also waits for the handler to take each message before it reads the next. On a multi-threaded runtime those hand-offs mostly land on different workers: understrace -cthe connect-rust server makes about 20 syscalls per 10-message stream (9.8futex, 4.1epoll_wait, 2.0 eventfdwrite, plus the 2recvfrom/ 2writevthat tonic also makes), where tonic makes 6. Pinning the server runtime to one worker thread (TOKIO_WORKER_THREADS=1) removes most of the gap on its own (174 µs to 160 µs per stream on the metal box, tonic 155 µs), which is what identifies the cost as wakeups rather than work.The reader task exists for HTTP/1.1: when a handler returns without consuming the whole request stream, something must keep reading the body (bounded by
MAX_DRAIN_BYTES) or hyper cannot reuse the connection, and aborting a reader part-way through the body had been found to race with hyper's end-of-body detection. That property does not require a task in the common case. A request stream that owns the body and theEnvelopeDecoderand decodes inpoll_next, in the handler's own task, can hand the unread remainder of the body to a detached drain task only when it finishes decoding (END_STREAM, a decode error) or is dropped before the body has ended; a handler that reads its stream to the end then involves no extra task at all, and the early-return path drains exactly as today.Measured that way on the same box (three alternating 10 s rounds per server, 8 server cores):
mainThe syscall profile drops to tonic's 6 per stream, and server instructions per stream fall about 7%. With 8 streams in flight all four servers are within 4% of each other before and after, so the change is latency at low concurrency and CPU per message rather than peak throughput.