Skip to content

Isolate CONNECT headers from origin HTTPS responses - #18

Open
cursor[bot] wants to merge 3 commits into
mainfrom
fix/connect-header-security
Open

cursor[bot] wants to merge 3 commits into
mainfrom
fix/connect-header-security

Conversation

@cursor

@cursor cursor Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Fixes the CONNECT header issues from the vulnerability scan:

  • Reject CR/LF/NUL in CONNECT request header names and values so proxy_headers cannot smuggle extra headers.
  • Parse CONNECT responses only through the header terminator.
  • Do not copy origin-semantic CONNECT fields (Set-Cookie, Content-Type, Location, Referrer-Policy, and similar) onto the HTTPS response, and do not overwrite headers the origin already sent.
  • Keep documented proxy headers such as X-ProxyMesh-IP on response.headers.
  • Store the full CONNECT header set on request.meta["proxy_response_headers"].

Adds test_header_safety.py for the CRLF and merge behavior. The Scrapy 2.17.0 floor for CVE-2026-84366 remains in #17.

Open in Web View Automation 

The existing proxy integration workflow runs test_header_safety.py in a unit job on ubuntu-24.04 (no PROXY_URL). The new checkout and setup-python actions in that job are pinned by commit SHA. The integration job is unchanged.

README, docs, and the agent skill describe which CONNECT headers are copied onto the response.

Open in Web Open in Cursor 

Cursor and others added 2 commits November 1, 2026 19:05
Reject CR/LF/NUL in CONNECT request lines so proxy_headers cannot
smuggle extra headers. Parse CONNECT responses only through the header
terminator, skip origin-semantic fields such as Set-Cookie and
Content-Type, and never overwrite existing origin headers. Keep
X-ProxyMesh-* visible on response.headers and store the full CONNECT
set on request.meta['proxy_response_headers'].

Co-authored-by: ProxyMesh AI <proxymeshai@users.noreply.github.com>
Add a unit job to the existing proxy integration workflow so
test_header_safety.py runs on ubuntu-24.04 without PROXY_URL.
Pin the new checkout and setup-python actions by commit SHA.

Co-authored-by: ProxyMesh AI <proxymeshai@users.noreply.github.com>

Copy link
Copy Markdown
Collaborator

bugbot review

@cursor

cursor Bot commented Nov 1, 2026

Copy link
Copy Markdown
Author

Skipping Bugbot: Bugbot is disabled for this repository. Visit the Bugbot dashboard to update your settings.

processProxyResponse parsed only the latest TCP chunk, so a split
CONNECT response dropped earlier header lines and could treat bytes
after a split header terminator as headers.

Co-authored-by: ProxyMesh AI <proxymeshai@users.noreply.github.com>
@cursor

cursor Bot commented Nov 1, 2026

Copy link
Copy Markdown
Author

Review found one bug, fixed in 4eaeec6.

processProxyResponse parsed only the latest TCP chunk. Scrapy buffers the CONNECT response until \r\n\r\n, so a split response dropped header lines from earlier chunks (including X-ProxyMesh-IP) and could treat bytes after a split terminator as headers. It now parses the reassembled _connectBuffer once the terminator is present. test_header_safety.py covers that split.

No other correctness or security issues in the diff. The new unit job keeps contents: read and pins checkout and setup-python by commit SHA. Local test_header_safety.py passed. CI: unit pass, integration pass.

@proxymeshai
proxymeshai marked this pull request as ready for review November 1, 2026 21:08

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant