requests: how to disable / bypass proxy

I am getting an url with:

r = requests.get("http://myserver.com")

As I can see in the ‘access.log’ of “myserver.com”, the client’s system proxy is used. But I want to disable using proxies at all with requests.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

The only way I’m currently aware of for disabling proxies entirely is the following:

• Create a session
• Set session.trust_env to False
• Create your request using that session

import requests

session = requests.Session()
session.trust_env = False

response = session.get('http://www.stackoverflow.com')

This is based on this comment by Lukasa and the (limited) documentation for requests.Session.trust_env.

Note: Setting trust_env to False also ignores the following:

• Authentication information from .netrc (code)
• CA bundles defined in REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE (code)

If however you only want to disable proxies for a particular domain (like localhost), you can use the NO_PROXY environment variable:

import os
import requests

os.environ['NO_PROXY'] = 'stackoverflow.com'

response = requests.get('http://www.stackoverflow.com')

Method 2

You can choose proxies for each request. From the docs:

import requests

proxies = {
  "http": "http://10.10.1.10:3128",
  "https": "http://10.10.1.10:1080",
}

requests.get("http://example.org", proxies=proxies)

So to disable the proxy, just set each one to None:

import requests

proxies = {
  "http": None,
  "https": None,
}

requests.get("http://example.org", proxies=proxies)

Method 3

The way to stop requests/urllib from proxying any requests is to set the the no_proxy (or NO_PROXY) environment variable to * e.g. in bash:

export no_proxy='*'

Or from Python:

import os
os.environ['no_proxy'] = '*'

To understand why this works is because the urllib.request.getproxies function first checks for any proxies set in the environment variables (e.g. http_proxy, HTTP_PROXY, https_proxy, HTTPS_PROXY, etc) or if none are set then it will check for system configured proxies using platform specific calls (e.g. On MacOS it will check using the system scutil/configd interfaces, and on Windows it will check the Registry). As mentioned in the comments if any proxy variables are set you can reset them as @udani suggested, or unset them like this from Python:

del os.environ['HTTP_PROXY']

Then when urllib attempts to use any proxies the proxyHandler function it will check for the presence and setting of the no_proxy environment variable – which can either be set to specific hostnames as mentioned above or it can be set the special * value whereby all hosts bypass the proxy.

Method 4

requests library respects environment variables.
http://docs.python-requests.org/en/latest/user/advanced/#proxies

So try deleting environment variables HTTP_PROXY and HTTPS_PROXY.

import os
for k in list(os.environ.keys()):
    if k.lower().endswith('_proxy'):
        del os.environ[k]

Method 5

With Python3, jtpereyda’s solution didn’t work, but the following did:

proxies = {
    "http": "",
    "https": "",
}

Method 6

 r = requests.post('https://localhost:44336/api/',data='',verify=False)

I faced the same issue when connecting with localhost to access my .net backend from a Python script with the request module.

I set verify to False, which cancels the default SSL verification.

P.s – above code will throw a warning that can be neglected by below one

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
r=requests.post('https://localhost:44336/api/',data='',verify=False)

Method 7

I implemented @jtpereyda‘s solution in our production codebase which worked fine on normal successful HTTP requests (200 OK), but this code ended up not working when receiving an HTTP redirect (301 Moved Permamently). Instead use:

requests.get("https://pypi.org/pypi/pillow/9.0.0/json", proxies={"http": "", "https": ""})

For comparison, this line causes a requests.exception.SSLError (pypi.org tries to redirect us to Pillow with an uppercase P):

requests.get("https://pypi.org/pypi/pillow/9.0.0/json", proxies={"http": None, "https": None})

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0