-1

I have got a huge list of domains/URLs (about a 1000). The task is to test if each one of them, if accessed with http://domain.com then it redirects to https://domain.com.

All the domains that do not redirect are to be reported to devs.

One way of doing this would be,

  1. To open each URL with Selenium WebDriver's get(url); method.
  2. To get the current URL from the browser using getCurrentUrl();.
  3. Compare the to to ensure the latter begins with HTTPS.

But, I guess there should be a faster way to do this without needing to use Selenium.

Any ideas?

2
  • 1
    Literally anything that can make a request and look at the response code ("is it 3xx?") can do this, surely? You don't need a browser, or to actually follow the redirect (if you want to know where it goes, the Location header tells you).
    – jonrsharpe
    May 27 at 9:22
  • 3xx response code would ensure that it redirects but I need to check if the destination URL begins with HTTPS. May 27 at 9:35

2 Answers 2

1

You can use CURL for it. For instance, if you run

curl -Ls -o /dev/null -w %{url_effective} http://sqa.stackexchange.com/

it will show https://sqa.stackexchange.com/, which is the redirected URL.

The flags mean the following:

-L         Follow redirects
-s         Silent mode. Don't output anything
-o FILE    Write output to FILE instead of stdout
-w FORMAT  What to output after completion (url_effective is the final URL)

With some simple scripting, you can check if the returned strings form CURL start with https

-1

Searched for a bit on the internet and from few websites gather and compiled this bit of code which reads domains from csv, checks redirection and then writes result in a csv,

public static void main(String[] args) {
    try {
        FileReader filereader = new FileReader(System.getProperty("user.dir") + "//data/file.csv");
        CSVParser parser = new CSVParserBuilder().withSeparator(';').build();
        CSVReader csvReader = new CSVReaderBuilder(filereader).withCSVParser(parser).withSkipLines(1).build();
        List<String[]> allData = csvReader.readAll();
        List<String[]> data = new ArrayList<String[]>();
        URL url;
        int count = 0;
        for (String[] row : allData) {
            for (String cell : row) {
                int responseCode = 0;
                String Location = null;
                url = new URL("http://" + cell);
                try {
                    HttpURLConnection.setFollowRedirects(false);
                    HttpURLConnection huc = (HttpURLConnection) url.openConnection();
                    responseCode = huc.getResponseCode();
                    if (responseCode == HttpURLConnection.HTTP_SEE_OTHER || responseCode == HttpURLConnection.HTTP_MOVED_PERM || responseCode == HttpURLConnection.HTTP_MOVED_TEMP) {
                        Location = huc.getHeaderField("Location");
                    }
                    if (Location != null && Location.startsWith("/")) {
                        Location = url.getProtocol() + "://" + url.getHost() + Location;
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
                count++;
                if (Location != null && Location.startsWith("https")) {
                    data.add(new String[] { Integer.toString(count), String.valueOf(url), Integer.toString(responseCode), Location, "Pass" });
                } else {
                    data.add(new String[] { Integer.toString(count), String.valueOf(url), Integer.toString(responseCode), Location, "Fail" });
                }
            }
        }
        DateFormat df = new SimpleDateFormat("yyyy-MM-dd-HH-mm-ss");
        String currentDate = df.format(new Date());
        File file = new File(System.getProperty("user.dir") + "//data/report-" + currentDate + ".csv");
        FileWriter outputFile = new FileWriter(file);
        CSVWriter writer = new CSVWriter(outputFile, ',', CSVWriter.NO_QUOTE_CHARACTER, CSVWriter.DEFAULT_ESCAPE_CHARACTER, CSVWriter.DEFAULT_LINE_END);
        String[] header = { "#", "URL", "Response Code", "Redirected To", "Status" };
        writer.writeNext(header);
        writer.writeAll(data);
        writer.close();
    } catch (
    Exception e) {
        e.printStackTrace();
    }
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.