Based on the posts here and here I am trying to use a chrome webdriver in selenium to be able to download a file. Here is the code so far
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
chrome_options.add_experimental_option("profile.default_content_settings.popups", 0)
chrome_options.add_experimental_option("download.prompt_for_download", "false")
chrome_options.add_experimental_option("download.default_directory", "/tmp")
driver = webdriver.Chrome(chrome_options=chrome_options)
But this alone results in the following error:
WebDriverException: Message: unknown error: cannot parse capability: chromeOptions from unknown error: unrecognized chrome option: download.default_directory (Driver info: chromedriver=2.24.417424 (c5c5ea873213ee72e3d0929b47482681555340c3),platform=Linux 4.10.0-37-generic x86_64)
So how to fix this? Do I have to use this ‘capability’ thing? If so, how exactly?
Answers:
Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.
Method 1
Try this. Executed on windows
(How to control the download of files with Selenium Python bindings in Chrome)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_experimental_option("prefs", {
"download.default_directory": r"C:UsersxxxdownloadsTest",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing.enabled": True
})
Method 2
I think that the easiest way to save arbitrary file (i.e. image) using WebDriver is to execute JavaScript which will save file. No configuration required at all!
I use this library FileSaver.js to save a file with desired name with ease.
from selenium import webdriver
import requests
FILE_SAVER_MIN_JS_URL = "https://raw.githubusercontent.com/eligrey/FileSaver.js/master/dist/FileSaver.min.js"
file_saver_min_js = requests.get(FILE_SAVER_MIN_JS_URL).content
chrome_options = webdriver.ChromeOptions()
driver = webdriver.Chrome('/usr/local/bin/chromedriver', options=chrome_options)
# Execute FileSaver.js in page's context
driver.execute_script(file_saver_min_js)
# Now you can use saveAs() function
download_script = f'''
return fetch('https://cdn.sstatic.net/Sites/stackoverflow/company/img/logos/so/so-logo.svg?v=a010291124bf',
{{
"credentials": "same-origin",
"headers": {{"accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-language":"en-US,en;q=0.9"}},
"referrerPolicy": "no-referrer-when-downgrade",
"body": null,
"method": "GET",
"mode": "cors"
}}
).then(resp => {{
return resp.blob();
}}).then(blob => {{
saveAs(blob, 'stackoverflow_logo.svg');
}});
'''
driver.execute_script(download_script)
# Done! Your browser has saved an SVG image!
Method 3
Some tips:
-
chromium and chromedriver should have same version.
Typically chromium package should have chromedriver inside, you can find it in the install dir. If you are using ubuntu/debian, execute
dpkg -L chromium-chromedriver. -
Have a correct Chrome preference config.
as Satish said, use
options.add_experimental_option("prefs", ...)to config selenium+chrome. But sometimes the config may change by time. The beset way to get newest and workable prefs is to check it in the chromium config dir.
For example,- Launch a chromium in Xorg desktop
- Change settings in menu
- Quit chromium
- Find out the real settings in
~/.config/chromium/Default/Preferences - Read it, pick out the exact options you need.
In my case, the code is:
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
options = webdriver.ChromeOptions()
options.gpu = False
options.headless = True
options.add_experimental_option("prefs", {
"download.default_directory" : "/data/books/chrome/",
'profile.default_content_setting_values.automatic_downloads': 2,
})
desired = options.to_capabilities()
desired['loggingPrefs'] = { 'performance': 'ALL'}
driver = webdriver.Chrome(desired_capabilities=desired)
Method 4
for chrome in mac os, the download.defaultdirectory did not work for me and fortunately savefile.default_directory works.
prefs = {
"printing.print_preview_sticky_settings.appState": json.dumps(settings),
"savefile.default_directory": "/Users/creative/python-apps",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"download.safebrowsing.enabled": True
}
Method 5
One of the reasons you can’t set “download.default_directory” may be that you have a system variable XDG_DOWNLOAD_DIR in file ~/.config/user-dirs.dirs
You can remove variable form that file or, you can set it to whatever you like before running your program.
I was looking for a solution for two days…
My SW set:
- Ubuntu bionic, 18.04.5 LTS
- chromedriver.86.0.4240.22.lin64
- Python 3.9
- selenium 3.141.0
- splinter 0.14.0
Method 6
From your exception, you are using chromedriver=2.24.417424.
What are the versions of Selenium and Chrome browser that are you using?
I tried the following code with:
- Selenium 3.6.0
- chromedriver 2.33
- Google Chrome 62.0.3202.62 (Official Build) (64-bit)
And it works:
from selenium import webdriver
download_dir = "/pathToDownloadDir"
chrome_options = webdriver.ChromeOptions()
preferences = {"download.default_directory": download_dir ,
"directory_upgrade": True,
"safebrowsing.enabled": True }
chrome_options.add_experimental_option("prefs", preferences)
driver = webdriver.Chrome(chrome_options=chrome_options,executable_path=r'/pathTo/chromedriver')
driver.get("urlFileToDownload");
Make sure you are using a browser that is supported by your chromedriver (from here, should be Chrome v52-54).
Method 7
Can’t you use requests lib?
If so, here’s an example:
import re
import requests
urls = [ '...' ]
for url in urls:
# verify = False ==> for HTTPS requests without SSL certificates
r = requests.get( url, allow_redirects = True, verify = False )
cd = r.headers.get( 'content-disposition' )
fa = re.findall( 'filename=(.+)', cd )
if len( fa ) == 0:
print( f'Error message: {link}' )
continue
filename = fa[ 0 ]
f = open( os.path.join( 'desired_path', filename ), 'wb' )
f.write( r.content )
f.close()
All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0