82

Basically I want to at least check that a download-able file exists / download link works and preferably get stuff like the file size too.

Here's an example:

link = self.browser.find_element_by_link_text('link text')
href = link.get_attribute('href')
download = self.browser.get(href)
print download

That fourth line prints "None", presumably because I haven't manually clicked the Save button, and even if I had, I doubt WebDriver would be able to "see" the file.

Any ideas? I'm using Firefox as my browser-under-test, and I understand that the file handling for downloads is somewhat browser and/or OS-specific.

3
  • Maybe it's better to see [this][1]. [1]: stackoverflow.com/questions/18439851/… Commented Oct 30, 2013 at 8:17
  • this works: downloadBytes($(selector)[0].@href) using geb.Page Commented Aug 9, 2017 at 10:51
  • 1
    One potential solution is to obtain the URL for the file via Selenium, create a (non-Selenium) connection, copy Selenium's cookies to the connection (if necessary), and download the file. Since this method utilizes non-Selenium APIs to download the file, it will work with (or without) any browser. For more info, see my answer here: stackoverflow.com/questions/16746707/… Commented Oct 16, 2017 at 16:25

14 Answers 14

26

As far as I know there is no easy way to make Selenium download files because browsers use native dialogs for it which cannot be controlled by JavaScript, so you need some "hack". Check this, hope it helps.

3
  • Thanks. I might try to do it with the Python requests module.
    – Aaron
    Commented Dec 2, 2011 at 22:05
  • 5
    +1 but "Provide context for links Links to external resources are encouraged, but please add context around the link so your fellow users will have some idea what it is and why it’s there. Always quote the most relevant part of an important link, in case the target site is unreachable or goes permanently offline." as stated here: sqa.stackexchange.com/help/how-to-answer
    – dzieciou
    Commented Jun 28, 2014 at 17:53
  • @Mark Mayo 's answer worked for me (sqa.stackexchange.com/a/6317/49307)
    – Moltres
    Commented Apr 24, 2021 at 15:01
46

Here's a solution. Set Firefox's preferences to save automatically, and not have the downloads window popup. Then you just grab the file, and it'll download.

So, something like this:

FirefoxProfile fxProfile = new FirefoxProfile();

fxProfile.SetPreference("browser.download.folderList",2);
fxProfile.SetPreference("browser.download.manager.showWhenStarting",false);
fxProfile.SetPreference("browser.download.dir","c:\\mydownloads");
fxProfile.SetPreference("browser.helperApps.neverAsk.saveToDisk","text/csv");

        FirefoxOptions option = new FirefoxOptions();
        option.Profile = fxProfile;
        driver = new FirefoxDriver(option);
driver.navigate().to("http://www.foo.com/bah.csv");

and given you now have the download directory, never ask to save, and no download manager appearing, automation from this point should be straightforward.

8
  • 1
    Worked like a charm, but I used my user profile. stackoverflow.com/questions/15954682/…
    – jmunsch
    Commented Apr 27, 2014 at 23:13
  • Should this answer still be working with Firefox 58? I can't manage to make it work.
    – meles
    Commented Mar 1, 2018 at 10:12
  • We have been using this for a long time with Firefox pre-version 47 and it worked great. Now we've upgraded to 58 and it no longer works.
    – k-den
    Commented Mar 30, 2018 at 20:15
  • 1
    Here it still / again works with the following setting: profile.setPreference("browser.download.dir", "path\\..."); profile.setPreference("browser.download.folderList", 2); profile.setPreference("browser.helperApps.neverAsk.saveToDisk", "text/plain");
    – Clerenz
    Commented May 6, 2019 at 15:31
  • 2
    It seems that you have to use now FirefoxOptions option=new FirefoxOptions(); option.setProfile(fxProfile); driver = new FirefoxDriver(option);
    – ka3ak
    Commented Nov 26, 2020 at 8:42
30

You can check the header response to check that you get a 200 OK (or maybe a redirect, depends on your expected outcome) and it will tell you that a file exists.

Here is my implementation.

This finds the link on the page and extracts the URL being linked to. It then uses apache commons to replicate the browser session used by selenium and then download the file. There are some instances where it won't work (where the link found on the page does not actually link to the download file but a layer to prevent automated file download).

Generally, it works well and is cross-platform/cross-browser compliant.

The code is:

 /*
  * Copyright (c) 2010-2011 Ardesco Solutions - http://www.ardescosolutions.com
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
  *
  * http://www.apache.org/licenses/LICENSE-2.0
  *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */

package com.lazerycode.ebselen.customhandlers;

import com.google.common.annotations.Beta;
import com.lazerycode.ebselen.EbselenCore;
import com.lazerycode.ebselen.handlers.FileHandler;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.cookie.CookiePolicy;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.GetMethod;

import java.io.*;
import java.net.URL;
import java.util.Set;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Beta
public class FileDownloader {

    private static final Logger LOGGER = LoggerFactory.getLogger(EbselenCore.class);
    private WebDriver driver;
    private String downloadPath = System.getProperty("java.io.tmpdir");

    public FileDownloader(WebDriver driverObject) {
        this.driver = driverObject;
    }

    /**
     * Get the current location that files will be downloaded to.
     *
     * @return The filepath that the file will be downloaded to.
     */
    public String getDownloadPath() {
        return this.downloadPath;
    }

    /**
     * Set the path that files will be downloaded to.
     *
     * @param filePath The filepath that the file will be downloaded to.
     */
    public void setDownloadPath(String filePath) {
        this.downloadPath = filePath;
    }


    /**
     * Load in all the cookies WebDriver currently knows about so that we can mimic the browser cookie state
     *
     * @param seleniumCookieSet
     * @return
     */
    private HttpState mimicCookieState(Set<org.openqa.selenium.Cookie> seleniumCookieSet) {
        HttpState mimicWebDriverCookieState = new HttpState();
        for (org.openqa.selenium.Cookie seleniumCookie : seleniumCookieSet) {
            Cookie httpClientCookie = new Cookie(seleniumCookie.getDomain(), seleniumCookie.getName(), seleniumCookie.getValue(), seleniumCookie.getPath(), seleniumCookie.getExpiry(), seleniumCookie.isSecure());
            mimicWebDriverCookieState.addCookie(httpClientCookie);
        }
        return mimicWebDriverCookieState;
    }

    /**
     * Mimic the WebDriver host configuration
     *
     * @param hostURL
     * @return
     */
    private HostConfiguration mimicHostConfiguration(String hostURL, int hostPort) {
        HostConfiguration hostConfig = new HostConfiguration();
        hostConfig.setHost(hostURL, hostPort);
        return hostConfig;
    }

    public String fileDownloader(WebElement element) throws Exception {
        return downloader(element, "href");
    }

    public String imageDownloader(WebElement element) throws Exception {
        return downloader(element, "src");
    }

    public String downloader(WebElement element, String attribute) throws Exception {
        //Assuming that getAttribute does some magic to return a fully qualified URL
        String downloadLocation = element.getAttribute(attribute);
        if (downloadLocation.trim().equals("")) {
            throw new Exception("The element you have specified does not link to anything!");
        }
        URL downloadURL = new URL(downloadLocation);
        HttpClient client = new HttpClient();
        client.getParams().setCookiePolicy(CookiePolicy.RFC_2965);
        client.setHostConfiguration(mimicHostConfiguration(downloadURL.getHost(), downloadURL.getPort()));
        client.setState(mimicCookieState(driver.manage().getCookies()));
        HttpMethod getRequest = new GetMethod(downloadURL.getPath());
        FileHandler downloadedFile = new FileHandler(downloadPath + downloadURL.getFile().replaceFirst("/|\\\\", ""), true);
        try {
            int status = client.executeMethod(getRequest);
            LOGGER.info("HTTP Status {} when getting '{}'", status, downloadURL.toExternalForm());
            BufferedInputStream in = new BufferedInputStream(getRequest.getResponseBodyAsStream());
            int offset = 0;
            int len = 4096;
            int bytes = 0;
            byte[] block = new byte[len];
            while ((bytes = in.read(block, offset, len)) > -1) {
                downloadedFile.getWritableFileOutputStream().write(block, 0, bytes);
            }
            downloadedFile.close();
            in.close();
            LOGGER.info("File downloaded to '{}'", downloadedFile.getAbsoluteFile());
        } catch (Exception Ex) {
            LOGGER.error("Download failed: {}", Ex);
            throw new Exception("Download failed!");
        } finally {
            getRequest.releaseConnection();
        }
        return downloadedFile.getAbsoluteFile();
    }
}
6
  • Just curious, will that work for HTTPS? I guess no, as there is no support for SSL in your HttpClient configuration...
    – dzieciou
    Commented Jun 28, 2014 at 17:51
  • 1
    It should work (hc.apache.org/httpclient-3.x/sslguide.html), but the http client library is now EOL so you are better off using the HTTP Component library hc.apache.org/httpcomponents-client-ga/examples.html
    – Ardesco
    Commented Jun 30, 2014 at 6:31
  • I want to download a file to check if it is containing the right data. Commented Aug 14, 2019 at 6:13
  • This won't work with HttpOnly cookies would it? Commented Mar 1, 2020 at 17:50
  • Kind of a weird idea to respond to a question with: "you probably don't need to do this". Sometimes we need to do things. Be it because they make sense or because the boss said so. Making my boss ooo and ahh pays the bills. So, there's a reason. Imagine being stuck and someone tells you your whole premise is a waste of time... As for the solution, I also disagree that this is a main idea. Most files out there are dynamically generated or dynamically referenced, you don't have a "storage.com/myfile.txt"... It's usually a java script action or some magic (I am not a browser programmer)
    – DraxDomax
    Commented Dec 10, 2020 at 3:48
7

I made my own version of the downloader, by using an Ajax request and returning the bytes. Has the advantage that it uses the browser directly, so authentication and cookies do not need to be dealt with. It has the disadvantage that you're restricted by same-origin rule, it might need a lot of memory and maybe also fail in older browsers.

Still sometimes is very useful:

import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.concurrent.TimeUnit;

public class AjaxFileDownloader {

    private WebDriver driver;

    public AjaxFileDownloader(WebDriver driverObject) {
        this.driver = driverObject;
        driver.manage().timeouts().setScriptTimeout(15, TimeUnit.SECONDS); // maybe you need a different timeout
    }

    public InputStream download(String url) throws IOException {
        String script = "var url = arguments[0];" +
                "var callback = arguments[arguments.length - 1];" +
                "var xhr = new XMLHttpRequest();" +
                "xhr.open('GET', url, true);" +
                "xhr.responseType = \"arraybuffer\";" + //force the HTTP response, response-type header to be array buffer
                "xhr.onload = function() {" +
                "  var arrayBuffer = xhr.response;" +
                "  var byteArray = new Uint8Array(arrayBuffer);" +
                "  callback(byteArray);" +
                "};" +
                "xhr.send();";
        Object response = ((JavascriptExecutor) driver).executeAsyncScript(script, url);
        // Selenium returns an Array of Long, we need byte[]
        ArrayList<Long> byteList = (ArrayList<Long>) response;
        byte[] bytes = new byte[byteList.size()];
        for(int i = 0; i < byteList.size(); i++) {
            bytes[i] = (byte)(long)byteList.get(i);
        }
        return new ByteArrayInputStream(bytes);
    }

}
3
  • 2
    Amazing this works great, thank you very much for this up-to-date answer!
    – Bruno
    Commented May 25, 2017 at 0:11
  • I modified this a little by setting the responsetype to "text" and just doing the callback without the array stuff. It works beautifully. I needed it for csv file.
    – user890332
    Commented Feb 19, 2019 at 18:25
  • This is probably the most beautiful hack I've seen in years! Well, maybe not the most beautiful (ok, it's ugly, like all hacks), but definitely the most useful! I tried every other chrome-flavored Java solution (for two days), so I was very happy that this one finally worked. I never would have thought of it myself, so congratulations! (I'm using it to search an in-house Nexus repo, so security isn't an issue, but having zillions of files and directories are).
    – Tihamer
    Commented Jan 12, 2022 at 23:20
5

The best way I have found to do this is by accessing the page, getting the download link, and performing a HEAD request for the file with an HTTP library. The response will contain the length of the file and it's type.

A HEAD request is preferable since it will only retrieve the headers instead of pulling down the entire file.

And if the file is behind auth, you will need to pull the session cookie from Selenium's cookie store and pass it into the HTTP library when performing the request.

That, or you can configure the browser you're using to auto-download files to a specific location and then perform checks against the file on disk.

I've outlined each of these approaches in detail with working Ruby code here:

1
  • In general, this is nice, as long the page does not require a login. As soon the user has to log in, this solution will break.
    – andreas
    Commented Nov 27, 2017 at 15:57
4

How about this approach:

How to auto save files using custom Firefox profile?

2
  • Cross platform and Python. This is the best approach.
    – Cerin
    Commented Oct 29, 2012 at 12:06
  • 2
    Link only answers are discouraged. Please add the solution part from the link to your answer. Commented Apr 28, 2015 at 15:57
2

There are many ways to download file in Selenium, one of the easiest way in Firefox using Firefox Profile.

First add preferences in profiles and specify the MIME type of file and then you can open Firefox with above preferences.

I found below article interesting which cover above scenario: How to Download files in Selenium Webdriver.

2

This blog post (Archived version) describes a straight forward way of invoking another library to download the file (so not through the browser) whilst maintaining selenium's session with the site - so it works on password-protected files, etc.

Here's the code from the blog post:

using System.Net;
void downloadFile(IWebDriver driver, string url, string localPath)
{
    var client = new WebClient();
    client.Headers[HttpRequestHeader.Cookie] = cookieString(driver);
    client.DownloadFile(url, localPath);
}
string cookieString(IWebDriver driver)
{
    var cookies = driver.Manage().Cookies.AllCookies;
        return string.Join("; ", cookies.Select(c => string.Format("{0}={1}", c.Name, c.Value)));
}
4
  • I have realised this is similar to Ardesco's answer. However, I think the solution in the blog post is simpler. Also, it is for .NET rather than Java, so it may be useful to people targeting that platform. Commented Jun 11, 2012 at 17:21
  • 1
    Windows only? Blargh.
    – Cerin
    Commented Oct 29, 2012 at 12:03
  • Bad link. Please fix it.
    – Coder
    Commented Sep 17, 2022 at 23:34
1

First, think about - do you really need to download an image? Or You just need to make sure that it exists and it is able to be downloaded?

Here you may find full trusted description how to check that image is available and exists, just by following by image's URL.

Main steps are:

  • extract authorization cookies (if user session required)
  • use them for building new HTTP request
  • send such request with image's URL to check status code
  • if status code is 200 - image exists
  • To get webDriver-like cookies just use something like that:

    webDriver.manage().getCookieNamed("JSESSIONID");
    

    Note, that it is not apache-like cookies, you can not use them strictly with apache http client. But you could build one apache-like based on it.

    2
    • 1
      welcome to sqa.stackexchange.com. This is a nicely detailed answer but the OP was pretty clear about wanting to download a document vs checking an image. The answers are most effective when the address the question asked.
      – Dan Snell
      Commented Feb 20, 2013 at 23:21
    • Sorry! :))) I was so attracted by searching for approaches about how to check that image exists that focused on it too much and miss "document". Ok, anyway - download file should be quite the same, I think. Thanks for comment
      – Gadget
      Commented Feb 21, 2013 at 8:12
    0

    An approach that I took I took recently to this was to capture to response via fiddler. In my case I am making a call directly to the export handler in our application. I have wrapped the selenium bits and intercepted the traffic with fiddler core. I actually just cared about the data in the file not that the browser correctly interpreted the request. You could also accomplish this using the a click action as well

    static string Test2ExportGeneration()
    {
        IWebDriver driver = new InternetExplorerDriver();
        FiddlerApplication.Startup(8877, true, true);
    
        List<Session> oAllSessions = new List<Session>();
    
        LoginPage login = new LoginPage(driver);
        //PerformancePage perf = new PerformancePage(driver);
    
        FiddlerApplication.AfterSessionComplete += delegate(Session oS)
        {
            Monitor.Enter(oAllSessions);
            oAllSessions.Add(oS);
            Monitor.Exit(oAllSessions);
        };
    
        driver.Navigate().GoToUrl(test2BaseURL + "login.aspx");
        login.LoginToView("[email protected]", "pw");
        driver.Navigate().GoToUrl(test2BaseURL + "/handlers/export.ashx?id=74&accountId=164");
    
        FiddlerApplication.Shutdown();
        
        driver.Quit();
        string responseBody = "";
    
        foreach (Session oS in oAllSessions)
        {
            if (oS.fullUrl.Contains("handlers"))
            {
                oS.utilDecodeResponse();
                string type = oS.oResponse.MIMEType;
    
                responseBody = oS.GetResponseBodyAsString();
                
            }
        }
        return responseBody;
    }
    

    The GetResponseBodyAsString() method in FiddlerCore unpacks the response and returns a string I can then manipulate and work with in code.

    I am using C# and WebDriver for my implementation. If you are using a different language I think there are some other tools as well. I my opinion the advantage of this is that I can strip away having to try to interact with the file download mechanisms of any of the browsers. If I want to create the file later all of the information the browser would have received is in the response header.

    2
    • Great approach. However, in the browser once you started the download, you still have to handle any native download dialogs. One would have to combine your approach with somehing like automated save of downloads (sqa.stackexchange.com/a/6317/17012) Commented Mar 2, 2016 at 11:31
    • In this case I simply did not care about dealing with the alert. I just read the data into memory and worked from there and validated it. I never actually read it to disk. If I really wanted to work with the file I would have just saved it with a write.File or some other mechanism in code and by passed the browser all together. After all I am not testing the browser file handling. I simply kill the browser session when I am done.
      – Dan Snell
      Commented Apr 13, 2016 at 16:58
    0

    A simple but somewhat flimsy solution (depending on whether you're expecting a consistent screen size in your target environment); is to use the java.awt.Robot class as below.. You could also try the ChromeDriver for selenium; as I've noticed chrome doesn't have an OS dialogue for download confirmation.

    //add wait time to for file download dialogue to appear
    Sleeper.SYSTEM_SLEEPER.sleep(new Duration(5,java.util.concurrent.TimeUnit.SECONDS));
    Robot r = new Robot();
    r.mouseMove(888, 458);//coordinates of save button
    //focus dialogue window
    r.mousePress(InputEvent.BUTTON1_MASK);
    r.mouseRelease(InputEvent.BUTTON1_MASK);
    Sleeper.SYSTEM_SLEEPER.sleep(new Duration(1,java.util.concurrent.TimeUnit.SECONDS));
    //click save button
    r.mousePress(InputEvent.BUTTON1_MASK);
    r.mouseRelease(InputEvent.BUTTON1_MASK);
    //wait for file to download
    Sleeper.SYSTEM_SLEEPER.sleep(new Duration(5,java.util.concurrent.TimeUnit.SECONDS));
    
    0

    When you are using selenium web driver with for Firefox profile , the best way to deal with the modal window is by changing the Firefox profile settings to automatically downloading the file to the desired location.

    The other answers here has useful info but not a working solution based on JAVA.

    here is the code snippet that does work like a trick -

    FirefoxProfile profile = new FirefoxProfile();          
    profile.setPreference("browser.download.folderList",2);
    profile.setPreference("browser.download.manager.showWhenStarting",false);
    profile.setPreference("browser.download.dir","C\\downloads");
    profile.setPreference("browser.helperApps.neverAsk.saveToDisk", 
    "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;"
    + "application/pdf;" 
    + "application/vnd.openxmlformats-officedocument.wordprocessingml.document;" 
    + "text/plain;" 
    + "text/csv");  
    Webdriver driver = new FirefoxDriver(profile);
    
    0

    SCALA:

    val remote = new RemoteWebDriver(new URL("http://localhost:9515/"),
    DesiredCapabilities.chrome())
    val a =  remote.findElements(By.xpath("""//td[6]/a[2]""")).iterator()
    
    while(a.hasNext){   
      val xmlclick = a.next()   
      println("a href: " + xmlclick.getAttribute("href") + "; "+ xmlclick.getText  )  
      xmlclick.click()   
      Thread.sleep(1000) 
    }
    

    for full automatic work you need to read about DesiredCapabilities as this: How to handle downloading a file in selenium webdriver?

    0

    The way we have accomplished this is sending keys to the browser window. Agreed its not the best solution but its quick and works. We did this on IE9 with the selenium web driver for IE and the scenario we wanted to automate was a submit button click that would end up downloading a file on the browser.

    The way we accomplished this was clicking submit on a thread because submit blocks till the action is taken to either progress or cancel the download on the download bar. Then wait for few seconds and send Tab keys to the browser which eventually ends the focus on the save button (Sending ALT+S would work if you just want to save in the default location in IE) but in our case we wanted to save the file with a specific name in a specific location. Once on the save button send the Down arrow key which opens up the menu on the save button and then a couple of down arrow keys more followed by Enter opens up the save as dialog. Here is the code snippet (written in C#):

     Thread ts = new Thread(new ThreadStart(()=> submitButton.click();));
     ts.start();
     System.Threading.Thread.Sleep(5000);
     System.Windows.Forms.SendKeys.SendWait("{TAB}");
     System.Threading.Thread.Sleep(1000);
     System.Windows.Forms.SendKeys.SendWait("{TAB}");
     System.Threading.Thread.Sleep(1000);
     System.Windows.Forms.SendKeys.SendWait("{TAB}");
     System.Threading.Thread.Sleep(1000);
     System.Windows.Forms.SendKeys.SendWait("{DOWN}");
     System.Threading.Thread.Sleep(1000);
     System.Windows.Forms.SendKeys.SendWait("{DOWN}");
     System.Threading.Thread.Sleep(1000);
     System.Windows.Forms.SendKeys.SendWait("{ENTER}");
     System.Threading.Thread.Sleep(1000);
    

    The Save dialog opens up. Send more keys to write name of file and then send ALT+S to download it.

    Not the answer you're looking for? Browse other questions tagged or ask your own question.