Skip to content

Conversation

@drshvik
Copy link

@drshvik drshvik commented Jan 31, 2026

Summary

Fixes #1023.

When using an HTTP proxy with tunneling (HTTPS requests), if the CONNECT handshake fails (e.g. RemoteProtocolError, timeouts, or TLS errors during the tunnel setup), the underlying network stream was not explicitly closed.

This caused the connection to remain in the pool as "active" but dead. If max_connections is set, this eventually leads to pool starvation where no new requests can be made.

The Fix

  • Wrapped the connection setup logic in AsyncTunnelHTTPConnection.handle_async_request with a try...except block.
  • Explicitly calls await self._connection.aclose() if any exception occurs during the handshake or TLS upgrade steps.

Verification

I verified this with a local reproduction script that simulates a proxy server accepting a connection and then immediately closing it (crashing the handshake).

Before the fix:

  • The pool count remained at 1 (stuck connection).
  • Subsequent requests hung indefinitely waiting for a pool slot (if max_connections=1).

After the fix:

  • The pool count correctly returns to 0.
  • Subsequent requests fail immediately (correct behavior) instead of hanging.

@drshvik
Copy link
Author

drshvik commented Jan 31, 2026

This relates to #1049 but provides a broader fix by wrapping the entire handshake process, ensuring cleanup even if the failure occurs before the TLS upgrade step.

@drshvik
Copy link
Author

drshvik commented Feb 2, 2026

@simonw Please review this PR and let me know your opinion.

@baizhufbb
Copy link

Thanks for looking into this. I believe the broader fix overlaps with existing exception handling:
Current state

  • AsyncHTTP11Connection.handle_async_request has try...except BaseException (http11.py#L132-136), calling _response_closed() on any exception
  • _response_closed() calls aclose() when request is incomplete (L250), transitioning state to CLOSED
  • Non-2xx CONNECT responses already call aclose() explicitly (http_proxy.py#L296)

The only gap is TLS handshake After CONNECT 200 returns, start_tls() (http_proxy.py#L317) sits outside the exception handler above. When TLS fails, the underlying socket is closed but _state remains ACTIVE → zombie connection.

My PR #1049 adds minimal try-except for this specific gap. Wrapping the entire handshake duplicates HTTP/1.1 layer's existing cleanup logic.

Detailed analysis: https://www.reddit.com/r/Python/comments/1qs7crk/bug_fix_connection_pool_exhaustion_in_httpcore/
,or https://juejin.cn/post/7601020578491744265 (for Chinese readers)

Is there a specific edge case I'm missing that isn't covered by the existing handle_async_request try-except?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exceptions not handled properly in the proxy's AsyncTunnelHTTPConnection leading to the leaks and pool starvation

2 participants