Unable to log in to ASP.NET website with requests module of Python

I am trying to log in to an ASP.NET website using the requests module in Python.

While logging in manually in the website I can see the following headers as well as cookies.

Request Headers:

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:810
Content-Type:application/x-www-form-urlencoded
Cookie:ASP.NET_SessionId=sfiziz55undlnz452gfc2d55; __utma=120481550.280814175.1411461613.1411461613.1411479534.2; __utmb=120481550.1.10.1411479534; __utmc=120481550; __utmz=120481550.1411461613.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
Host:www11.davidsonsinc.com
Origin:http://www11.davidsonsinc.com
Referer:http://www11.davidsonsinc.com/Login/Login.aspx?ReturnUrl=%2fdefault.aspx
User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36

Form Data:
__EVENTTARGET:
__EVENTARGUMENT:
__LASTFOCUS:
__VIEWSTATE:/wEPDwUKMTY3MDM5MDAxNQ9kFgJmD2QWAgIDD2QWAgIDD2QWAgIBD2QWBAIBD2QWAmYPZBYCAg0PEA8WAh4HQ2hlY2tlZGdkZGRkAgMPDxYCHgdWaXNpYmxlaGRkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYBBUBjdGwwMCRDb250ZW50UGxhY2VIb2xkZXJOYXZQYW5lJExlZnRTZWN0aW9uJFVzZXJMb2dpbiRSZW1lbWJlck1lsSFPYUYvIbQNBPs/54aHYcx6GyU=
__VIEWSTATEGENERATOR:1806D926
__EVENTVALIDATION:/wEWBQLy8oGOCwKanaixDwKPr7TsAQKu3uTtBgKs+sa/CQVDEisOu4Iw1m9stXWgAAz9TWQn
ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName:Username
ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password:password
ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$RememberMe:on
ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton:Log In

Request Cookies
ASP.NET_SessionId: nz452gfc2d55

Response Cookies
.ASPXAUTH: 1F5A05237A1AA18795ECA108CE6E70D48FE5CBB5B38D061E0770618F6C069ABA03604335B6209CF8198AD3E98AE934F14056F5C887A92BB099BF38D639A22BC12972DEEE91BCE0BF36239BD1728E228E0E9CA1E5146A6C69E906E177CC8FB27395CE2F56B4013535C62E821384231EF0AD632474D6EBCFCD859882DBE9D420B6A8816BE6

Following is the script I use to log in in to websites using Python/Django.
import requests
with requests.Session() as c:
    url = 'http://www.noobmovies.com/accounts/login/?next=/'
    USERNAME = 'user name'
    PASSWORD = 'password'
    c.get(url)
    csrftoken = c.cookies['csrftoken']
    login_data = dict(csrfmiddlewaretoken=csrftoken, username=USERNAME, password=PASSWORD, next='/')
    c.post(url, data=login_data, headers={"Referer":"http://www.noobmoviews.com/"})
    page = c.get('http://www.noobmovies.com/user/profile/0/')

    print page.status_code

But I don’t know how to log in into an ASP.NET website. How do I post the data on the ASP.NET website?

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

import requests
from bs4 import BeautifulSoup

URL="http://www11.davidsonsinc.com/Login/Login.aspx"
headers={"User-Agent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36"}

username="username"
password="password"

s=requests.Session()
s.headers.update(headers)
r=s.get(URL)
soup=BeautifulSoup(r.content)

VIEWSTATE=soup.find(id="__VIEWSTATE")['value']
VIEWSTATEGENERATOR=soup.find(id="__VIEWSTATEGENERATOR")['value']
EVENTVALIDATION=soup.find(id="__EVENTVALIDATION")['value']

login_data={"__VIEWSTATE":VIEWSTATE,
"__VIEWSTATEGENERATOR":VIEWSTATEGENERATOR,
"__EVENTVALIDATION":EVENTVALIDATION,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName":username,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password":password,
"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton":"Log In"}

r=s.post(URL, data=login_data)
print r.url

Method 2

I was initially using requests+bs4 as well however I was running into similar issues with the ASPX site I’m scrapping. I found another library called robobrowser that wraps requests+bs4. With this you no longer have to manually set items such as “__VIEWSTATE” and friends when interacting with ASPX sites.

from robobrowser import RoboBrowser

url = ' http://www11.davidsonsinc.com'
login_url = url + '/Login/Login.aspx'

username = "username"
password = "password"

browser = RoboBrowser(history=True)
# This retrieves __VIEWSTATE and friends
browser.open(login_url)

signin = browser.get_form(id='aspnetForm')
signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName"].value = username
signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password"].value = password
signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton"].value = "Log In"
browser.submit_form(signin)
print browser.url

Method 3

I think this is cleaner and more generic.

import requests
from bs4 import BeautifulSoup

url="http://www11.davidsonsinc.com/Login/Login.aspx"

username="username"
password="password"

session = requests.Session()
# Dont botter with headers at first
# s.headers.update(headers)
response = session.get(url)
soup = BeautifulSoup(response.content)
login_data = {}
# get the aspnet state form data needed with bsoup
aspnetstates = ['__VIEWSTATE', '__VIEWSTATEGENERATOR', '__EVENTVALIDATION', '__EVENTTARGET', 
                '__EVENTARGUMENT', '__VIEWSTATEENCRYPTED' ];    
for aspnetstate in aspnetstates: # search for existing aspnet states and get its values 
    result = soup.find('input', {'name': aspnetstate})
    if not (result is None): # when existent (some may not be needed!)
        login_data.update({aspnetstate : result['value']})      

login_data.update(
    {"ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName" : username,
    "ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password" : password,
    "ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton" : "Log In"})

response = session.post(url, data=login_data)


All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x