I am trying to log in to an ASP.NET website using the requests module in Python.
While logging in manually in the website I can see the following headers as well as cookies.
Request Headers:
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Encoding:gzip,deflate Accept-Language:en-US,en;q=0.8 Cache-Control:max-age=0 Connection:keep-alive Content-Length:810 Content-Type:application/x-www-form-urlencoded Cookie:ASP.NET_SessionId=sfiziz55undlnz452gfc2d55; __utma=120481550.280814175.1411461613.1411461613.1411479534.2; __utmb=120481550.1.10.1411479534; __utmc=120481550; __utmz=120481550.1411461613.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none) Host:www11.davidsonsinc.com Origin:http://www11.davidsonsinc.com Referer:http://www11.davidsonsinc.com/Login/Login.aspx?ReturnUrl=%2fdefault.aspx User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36
Form Data:
__EVENTTARGET: __EVENTARGUMENT: __LASTFOCUS: __VIEWSTATE:/wEPDwUKMTY3MDM5MDAxNQ9kFgJmD2QWAgIDD2QWAgIDD2QWAgIBD2QWBAIBD2QWAmYPZBYCAg0PEA8WAh4HQ2hlY2tlZGdkZGRkAgMPDxYCHgdWaXNpYmxlaGRkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYBBUBjdGwwMCRDb250ZW50UGxhY2VIb2xkZXJOYXZQYW5lJExlZnRTZWN0aW9uJFVzZXJMb2dpbiRSZW1lbWJlck1lsSFPYUYvIbQNBPs/54aHYcx6GyU= __VIEWSTATEGENERATOR:1806D926 __EVENTVALIDATION:/wEWBQLy8oGOCwKanaixDwKPr7TsAQKu3uTtBgKs+sa/CQVDEisOu4Iw1m9stXWgAAz9TWQn ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName:Username ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password:password ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$RememberMe:on ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton:Log In
Request Cookies
ASP.NET_SessionId: nz452gfc2d55
Response Cookies
.ASPXAUTH: 1F5A05237A1AA18795ECA108CE6E70D48FE5CBB5B38D061E0770618F6C069ABA03604335B6209CF8198AD3E98AE934F14056F5C887A92BB099BF38D639A22BC12972DEEE91BCE0BF36239BD1728E228E0E9CA1E5146A6C69E906E177CC8FB27395CE2F56B4013535C62E821384231EF0AD632474D6EBCFCD859882DBE9D420B6A8816BE6
Following is the script I use to log in in to websites using Python/Django.
import requests
with requests.Session() as c:
url = 'http://www.noobmovies.com/accounts/login/?next=/'
USERNAME = 'user name'
PASSWORD = 'password'
c.get(url)
csrftoken = c.cookies['csrftoken']
login_data = dict(csrfmiddlewaretoken=csrftoken, username=USERNAME, password=PASSWORD, next='/')
c.post(url, data=login_data, headers={"Referer":"http://www.noobmoviews.com/"})
page = c.get('http://www.noobmovies.com/user/profile/0/')
print page.status_code
But I don’t know how to log in into an ASP.NET website. How do I post the data on the ASP.NET website?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
import requests
from bs4 import BeautifulSoup
URL="http://www11.davidsonsinc.com/Login/Login.aspx"
headers={"User-Agent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36"}
username="username"
password="password"
s=requests.Session()
s.headers.update(headers)
r=s.get(URL)
soup=BeautifulSoup(r.content)
VIEWSTATE=soup.find(id="__VIEWSTATE")['value']
VIEWSTATEGENERATOR=soup.find(id="__VIEWSTATEGENERATOR")['value']
EVENTVALIDATION=soup.find(id="__EVENTVALIDATION")['value']
login_data={"__VIEWSTATE":VIEWSTATE,
"__VIEWSTATEGENERATOR":VIEWSTATEGENERATOR,
"__EVENTVALIDATION":EVENTVALIDATION,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName":username,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password":password,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton":"Log In"}
r=s.post(URL, data=login_data)
print r.url
Method 2
I was initially using requests+bs4 as well however I was running into similar issues with the ASPX site I’m scrapping. I found another library called robobrowser that wraps requests+bs4. With this you no longer have to manually set items such as “__VIEWSTATE” and friends when interacting with ASPX sites.
from robobrowser import RoboBrowser url = ' http://www11.davidsonsinc.com' login_url = url + '/Login/Login.aspx' username = "username" password = "password" browser = RoboBrowser(history=True) # This retrieves __VIEWSTATE and friends browser.open(login_url) signin = browser.get_form(id='aspnetForm') signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName"].value = username signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password"].value = password signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton"].value = "Log In" browser.submit_form(signin) print browser.url
Method 3
I think this is cleaner and more generic.
import requests
from bs4 import BeautifulSoup
url="http://www11.davidsonsinc.com/Login/Login.aspx"
username="username"
password="password"
session = requests.Session()
# Dont botter with headers at first
# s.headers.update(headers)
response = session.get(url)
soup = BeautifulSoup(response.content)
login_data = {}
# get the aspnet state form data needed with bsoup
aspnetstates = ['__VIEWSTATE', '__VIEWSTATEGENERATOR', '__EVENTVALIDATION', '__EVENTTARGET',
'__EVENTARGUMENT', '__VIEWSTATEENCRYPTED' ];
for aspnetstate in aspnetstates: # search for existing aspnet states and get its values
result = soup.find('input', {'name': aspnetstate})
if not (result is None): # when existent (some may not be needed!)
login_data.update({aspnetstate : result['value']})
login_data.update(
{"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName" : username,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password" : password,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton" : "Log In"})
response = session.post(url, data=login_data)
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0