Skip to content

Release native socket listener slot before closing the connection - #21442

Open
David Fowler (davidfowl) wants to merge 1 commit into
mainfrom
agent/fix-socket-listener-slot-release
Open

David Fowler (davidfowl) wants to merge 1 commit into
mainfrom
agent/fix-socket-listener-slot-release

Conversation

@davidfowl

Copy link
Copy Markdown
Collaborator

Description

Fixes an intermittent connection-slot race in SocketConnectionListener. It caused NativeCliProtocolTests.SilentAndPartialFramesExpireWithoutBlockingOtherCliClients(frame: "") to fail on Ubuntu.

Failure: main run 40802338563, job 123215117869 (2026-10-29, 84881cc54). The test threw OperationCanceledException after 20 s at NativeCliProtocolTests.cs:120. That line is the replacement client's WaitForConnectionAsync, so the reconnecting client had been rejected.

Window / denominator: main CI attempt-1 runs, 2026-10-16 to 2026-10-30. Hosting.Native.Core on Ubuntu failed 1 of 64 jobs that ran; on Windows it failed 0 of 64. The test and listener source were unchanged since f4ae2cc42 (2026-10-21), and all 64 jobs ran on that source, so the passes are source-comparable.

Root cause

The listener enforced MaximumConnections by counting the not-yet-completed ServeAsync tasks. However, ServeAsync disposes the socket (using var owned) before its task completes. A client can observe the close and reconnect inside that window. The accept loop then still counts the old task as active and disposes the new connection as over capacity. With MaximumConnections = 2, the test's replacement client sometimes hit this window.

Fix

  • Track capacity with an explicit activeConnections counter. The counter is incremented before the connection's task is added.
  • ServeAsync releases its slot in finally before disposing the socket. A peer can therefore only observe the close after the slot is free.

Assertions, timeouts and connection limits are unchanged. The fix adds no sleeps or retries.

Regression test

New test: SocketConnectionListenerTests.ClientThatObservedCloseCanReconnectAtCapacity.

  • Uses maximumConnections: 1 and runs 2000 sequential connections.
  • The handler writes a byte and then reads one, so it completes on an I/O continuation.
  • The client waits until it observes EOF and then asserts that the next connection is served.

Native before/after qualification

Both runs used Hosting.Native.Core on ubuntu-latest, filtered to SocketConnectionListenerTests and NativeCliProtocolTests.

Branch Run Result
main + new test only (5b6a341c5) 41033521471 ❌ 22/23. Connection 279 was rejected after the previous connection's close was observed.
fix + new test (f8237a496) 41033525414 ✅ 23/23

I also ran a standalone harness that compiles the real listener source:

  • Unmodified listener: rejections in 5 of 5 runs of 2000 iterations.
  • Fixed listener: 0 rejections in 5×2000 iterations plus a 50,000-iteration run.

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
    • No

🤖 Posted by AspireAgent (AI Agent)

A client that observes the server closing its connection could reconnect before
the serving task completed, so the accept loop still counted the old connection
against MaximumConnections and rejected the reconnect. Track active connections
explicitly and release the slot before the socket is closed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 21442

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 21442"

@aspire-repo-bot
aspire-repo-bot Bot requested a balanced review from Copilot October 30, 2026 09:37
@github-actions github-actions Bot added the area-integrations Issues pertaining to Aspire Integrations packages label Oct 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Tests selector

2 / 100 PR test projects · 5 PR jobs, from 2 changed files.

Selected PR test projects (2 / 100)

Aspire.Cli.EndToEnd.Tests, Aspire.Hosting.Native.Core.Tests

Selected PR jobs (5)

cli-starter-validation, extension-e2e, homebrew-installer, typescript-api-compat, winget-installer


How these were chosen — grouped by what changed

📦 affected project Aspire.Hosting.Native.Server
→ 1 test: Aspire.Cli.EndToEnd.Tests

🧪 tests/Aspire.Hosting.Native.Core.Tests/SocketConnectionListenerTests.cs (changed test)
→ 1 directly: Aspire.Hosting.Native.Core.Tests

Job reasons

Job Triggered by
cli-starter-validation affected project Aspire.Hosting.Native.Server
extension-e2e affected project Aspire.Hosting.Native.Server
homebrew-installer affected project Aspire.Hosting.Native.Server
typescript-api-compat affected project Aspire.Hosting.Native.Server
winget-installer affected project Aspire.Hosting.Native.Server

Selection computed for commit add76fe.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the socket listener’s connection-capacity race fix.

No open issues 🎉

What changed in this PR

Fixes a connection-capacity race in Aspire’s native socket listener that could reject a replacement client after the previous connection had closed.

Changes:

  • Tracks active connections explicitly and releases capacity before disposing the socket.
  • Adds a 2,000-connection regression test verifying that clients can reconnect after observing closure at capacity.
File Description
tests/​Aspire.Hosting.Native.Core.Tests/​SocketConnectionListenerTests.cs Adds repeated reconnect coverage with a one-connection limit.
src/​Aspire.Hosting.Native.Server/​Transport/​SocketConnectionListener.cs Releases the active-connection slot before closing the socket.

Review effort: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Copy link
Copy Markdown
Collaborator Author

CI status for exact head add76fe9:

  • Every test job passed on attempt 1, including both Hosting.Native.Core jobs and the Native hosting validation. The PR stays in draft. Only Prepare Homebrew cask failed, and the Final Results aggregators failed because of it.
  • The cause is Refusing to load cask local/aspire/aspire from untrusted tap local/aspire. The failing jobs ran on macOS image 20261026, which ships Homebrew 7.0.11. Automatic retry attempt 2 hit the same failure on the same image.
  • Trust local tap cask before brew audit in Homebrew validation #21429 fixes this, and this PR doesn't touch installer scripts. I'll mark this PR ready only after exact-head CI is green.

🤖 Posted by AspireAgent (AI Agent)

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Copy link
Copy Markdown
Collaborator Author

Another occurrence of the same signature, on an unrelated PR branch (#21626, Native hosting validation run 41277664307, job 124763008835, 2026-11-01). SilentAndPartialFramesExpireWithoutBlockingOtherCliClients(frame: "") threw OperationCanceledException after 20s at NativeCliProtocolTests.cs:120, the replacement client's WaitForConnectionAsync. This matches the listener slot race this PR fixes, so I'm not opening a separate issue or PR.

🤖 Posted by AspireAgent (AI Agent)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-integrations Issues pertaining to Aspire Integrations packages flaky-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants