Simple URL GET/POST function in Python

I can’t seem to Google it, but I want a function that does this:

Accept 3 arguments (or more, whatever):

• URL
• a dictionary of params
• POST or GET

Return me the results, and the response code.

Is there a snippet that does this?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

requests

https://github.com/kennethreitz/requests/

Here’s a few common ways to use it:

import requests
url = 'https://...'
payload = {'key1': 'value1', 'key2': 'value2'}

# GET
r = requests.get(url)

# GET with params in URL
r = requests.get(url, params=payload)

# POST with form-encoded data
r = requests.post(url, data=payload)

# POST with JSON 
import json
r = requests.post(url, data=json.dumps(payload))

# Response, status etc
r.text
r.status_code

httplib2

https://github.com/jcgregorio/httplib2

>>> from httplib2 import Http
>>> from urllib import urlencode
>>> h = Http()
>>> data = dict(name="Joe", comment="A test comment")
>>> resp, content = h.request("http://bitworking.org/news/223/Meet-Ares", "POST", urlencode(data))
>>> resp
{'status': '200', 'transfer-encoding': 'chunked', 'vary': 'Accept-Encoding,User-Agent',
 'server': 'Apache', 'connection': 'close', 'date': 'Tue, 31 Jul 2007 15:29:52 GMT', 
 'content-type': 'text/html'}

Method 2

Even easier: via the requests module.

import requests
get_response = requests.get(url='http://google.com')
post_data = {'username':'joeb', 'password':'foobar'}
# POST some form-encoded data:
post_response = requests.post(url='http://httpbin.org/post', data=post_data)

To send data that is not form-encoded, send it serialised as a string (example taken from the documentation):

import json
post_response = requests.post(url='http://httpbin.org/post', data=json.dumps(post_data))
# If using requests v2.4.2 or later, pass the dict via the json parameter and it will be encoded directly:
post_response = requests.post(url='http://httpbin.org/post', json=post_data)

Method 3

You could use this to wrap urllib2:

def URLRequest(url, params, method="GET"):
    if method == "POST":
        return urllib2.Request(url, data=urllib.urlencode(params))
    else:
        return urllib2.Request(url + "?" + urllib.urlencode(params))

That will return a Request object that has result data and response codes.

Method 4

import urllib

def fetch_thing(url, params, method):
    params = urllib.urlencode(params)
    if method=='POST':
        f = urllib.urlopen(url, params)
    else:
        f = urllib.urlopen(url+'?'+params)
    return (f.read(), f.code)


content, response_code = fetch_thing(
                              'http://google.com/', 
                              {'spam': 1, 'eggs': 2, 'bacon': 0}, 
                              'GET'
                         )

[Update]
Some of these answers are old. Today I would use the requests module like the answer by robaple.

Method 5

I know you asked for GET and POST but I will provide CRUD since others may need this just in case: (this was tested in Python 3.7)

#!/usr/bin/env python3
import http.client
import json

print("n GET example")
conn = http.client.HTTPSConnection("httpbin.org")
conn.request("GET", "/get")
response = conn.getresponse()
data = response.read().decode('utf-8')
print(response.status, response.reason)
print(data)


print("n POST example")
conn = http.client.HTTPSConnection('httpbin.org')
headers = {'Content-type': 'application/json'}
post_body = {'text': 'testing post'}
json_data = json.dumps(post_body)
conn.request('POST', '/post', json_data, headers)
response = conn.getresponse()
print(response.read().decode())
print(response.status, response.reason)


print("n PUT example ")
conn = http.client.HTTPSConnection('httpbin.org')
headers = {'Content-type': 'application/json'}
post_body ={'text': 'testing put'}
json_data = json.dumps(post_body)
conn.request('PUT', '/put', json_data, headers)
response = conn.getresponse()
print(response.read().decode(), response.reason)
print(response.status, response.reason)


print("n delete example")
conn = http.client.HTTPSConnection('httpbin.org')
headers = {'Content-type': 'application/json'}
post_body ={'text': 'testing delete'}
json_data = json.dumps(post_body)
conn.request('DELETE', '/delete', json_data, headers)
response = conn.getresponse()
print(response.read().decode(), response.reason)
print(response.status, response.reason)

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0