gh-149746: Handle CIDRs in NO_PROXY env var

[turettn]

The change

This PR allows IPv4 & IPv6 CIDRs to be specified in NO_PROXY. This is a fairly common convention, as described in #149746.

With this change, if the following env var is set:

NO_PROXY=192.168.0.0/16

the proxy will not be used for any IP address in that range (e.g. 192.168.4.4).

Caching

Since parsing NO_PROXY involves a non-trivial amount of string manipulation, object allocation, and attempts to parse, it turned out significantly more expensive for long NO_PROXY strings than I expected. Since I would expect the environment to change very infrequently, I added a lru_cache around the repeated parsing of the same NO_PROXY string. In the overwhelming majority of production instances, I would expect the size of the cache to sit at 1, and the number of cache hits to be high (e.g. one per request).

Tightly coupled bug

Along the way, I discovered a bug in urllib.parse.splitport, which is deprecated but still used heavily internally:

>>> urllib.parse.splitport('::1')
<python-input-4>:1: DeprecationWarning: urllib.parse.splitport() is deprecated as of 3.8, use urllib.parse.urlparse() instead
(':', '1')

I tried to work around this issue throughout my PR, but it kept getting uglier and I decided to bundle the fix with this change together. If you'd like a different approach (e.g. 2 stacked PRs) just let me know and I'm happy to refactor.

Existing Issue & PR

I foolishly wrote this PR before looking for existing issues (it started as a monkey-patch to fix an urgent issue in prod). When I found the issue, I also found an existing PR #149744.

The big difference in our approach seems to be that I focused solely on the environment variable, and not on environment-specific settings. It seems like Windows doesn't support CIDRs in the registry value, and I couldn't find a clear indicator that MacOS supports it either.

I went back and forth about posting this, but figured if its not helpful the close button is right there :).

Final notes

This is my first attempt to upstream code to Python. If I missed anything important or violated convention somewhere along the way, please let me know - I'm happy to amend my PR as needed.

• Issue: CIDR support for proxy bypass list #149746

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34215229 | 📁 Comparing 46818b2 against main (ee521e8)

  🔍 Preview build  

2 files changed

± library/urllib.request.html
± whatsnew/changelog.html