Requests ProxyError on a Private API? Check the Environment First

VPN is up, the internal dashboard opens in a browser, and the Python script still raises a proxy error. Nothing is wrong with the tunnel in that picture: the interpreter is reading a proxy address from its own environment and sending the request there. This runbook reads the environment the process actually sees, applies the narrowest exclusion, and only then considers an explicit session policy. Every step ends with a command whose output decides whether you continue.

Read the environment the process actually sees

Desktop proxy settings, shell exports and a systemd unit's Environment= lines are three different sources, and Requests only reads the last one it inherits. Run the diagnostic below in the same shell, container or unit that runs the failing script, not in your interactive terminal. A variable exported in your login shell is invisible to a container process, and that gap explains many reports of a proxy that appears from nowhere. The two commands print the same data at different levels of detail.

env | grep -i -E 'proxy|requests_ca_bundle|curl_ca_bundle'
python3 -c "import os;[print(k,'=',v) for k,v in sorted(os.environ.items()) if 'proxy' in k.lower()]"

Both spellings of each variable are read, and duplicate entries are applied in the order the process received them, so a stale HTTP_PROXY can override the value you just exported. The CA-bundle variables matter here too, because they change what verify resolves to without appearing anywhere in the code. If the output is empty, the environment is not your problem and you can stop reading this section. If it lists a proxy you do not recognise, find the file or unit that sets it before you change anything.

Ask Requests what it would do with a specific URL. This call shows the merged proxy, verify and cert values for that request without sending a packet.

python3 -c "import requests;s=requests.Session();print(s.merge_environment_settings('https://api.corp.example/status',{},None,None,None))"

A non-empty proxies entry for the private host is your answer, and the printed verify value tells you which CA bundle the request would trust. If both look correct, the failure is later in the stack and you should skip to the verification section. Do not guess from the code: the merge happens per request, which is why one session can behave differently for two hosts.

Exclude the private host before you disable anything

When the proxy is genuinely required for other destinations, the narrow fix is an exclusion for this host. Append to the existing list instead of replacing it, because wiping NO_PROXY breaks every other internal destination that relied on it. The commands below append the entry and then ask Requests whether the bypass applies.

export NO_PROXY="${NO_PROXY:+$NO_PROXY,}api.corp.example,.corp.example"
python3 -c "import requests;print(requests.utils.should_bypass_proxies('https://api.corp.example/status',None))"

Matching is by host suffix, so .corp.example covers the whole domain while api.corp.example covers one name. A bare asterisk disables proxying for every host and is almost never what a production job wants. Whether a CIDR entry is honoured depends on the Requests and urllib3 versions in the image, so test the exact entry instead of assuming it works. Check the result of should_bypass_proxies for the real URL, not for a hostname you typed from memory.

Keep the two layers separate in your head. Bypassing the HTTP proxy does not bypass the VPN, because the proxy decision happens in the library and the tunnel decision happens in the kernel routing table.

Disable trust_env only as a deliberate session policy

If the whole job should ignore ambient configuration, build the session explicitly and record the decision in code. That is a policy, not a workaround, and it belongs in a diff a reviewer can see. The example below keeps a timeout, an explicit CA path and a status check.

import requests

with requests.Session() as session:
    session.trust_env = False
    response = session.get(
        "https://api.corp.example/status",
        timeout=(5, 20),
        verify="/etc/ssl/certs/corp-ca.pem",
    )
    response.raise_for_status()

Switching trust_env off changes three behaviours at once, and only one of them is about proxies. Requests stops reading .netrc credentials, and the REQUESTS_CA_BUNDLE and CURL_CA_BUNDLE variables stop affecting verify, which then falls back to the bundled certificate store. If your organisation terminates TLS with its own CA, that fallback raises a verification error that looks unrelated to the change you just made. Passing verify explicitly is what keeps the two effects separate.

What you should not do is set verify=False to silence the resulting error. That trades a proxy misconfiguration for an unverified connection, and it hides the certificate problem the exclusion was meant to expose.

Verify the failing layer and keep a rollback

Three failures look similar in a log and have nothing in common: a proxy that cannot reach the destination raises ProxyError or returns 502, a trust problem raises SSLError, and an application rejection returns 401 or 403. Reproduce the request outside Requests to see which one you have. The commands below remove the proxy, test the certificate chain, and then repeat the call through the library.

curl -sv --noproxy '*' https://api.corp.example/status -o /dev/null
openssl s_client -connect api.corp.example:443 -servername api.corp.example </dev/null
python3 -c "import requests;r=requests.get('https://api.corp.example/status',timeout=10);print(r.status_code)"

If the curl call succeeds and the Python call fails, the environment is still the difference. If both fail at the TLS stage, the proxy was never the problem and the certificate chain is. If both return 401, the request reached the application and the remaining work is authentication. Recording which of the three you observed saves the next person an hour.

Rollback is one line in either direction: unset the exclusion, or flip trust_env back. Save the previous NO_PROXY value before you append to it, and prefer a wrapper script or a unit-level Environment= entry over an export that lives only in your terminal history. Both changes are visible in the process environment, so a restart usually returns you to the old behaviour.

Finish by confirming four things: the merged settings for the private URL show no proxy, the certificate validates against the intended CA, the request returns the status you expect, and the change is committed somewhere reproducible. A green script with an unexplained environment is a fix that breaks again on the next machine. The commands in this runbook are the ones to paste into the ticket.