1

I want to check the status code of a URL by reading the URL from a CSV file and "hitting it" with a browser.

How can I achieve this functionality? Are there any tools for that?

  • Use broken link checker tools. – Helping Hands Feb 16 '15 at 11:12
  • Thanks for ur answer Halping Hands , here i dont want to check all the links in a page . I want to check a list of urls response code – QAMember Feb 16 '15 at 11:52
1

There are several solutions.

The simplest would probably be to list your links in HTML format, and then run that page through a link checker tool. Here's an open source example.

Another solution is to set up a testing project in Visual Studio (or a similar environment). You can provide source data in CSV format, see this topic. It's fairly easy to have Selenium (or Coded UI) open the webpage and check the contents for a 404 message.

| improve this answer | |
  • Thanks for ur answer @Frederik , I have used the selenium to check the response code of a url but I am having a 10000 url's so my system is hanging. So let me know is there any alternative or any free tool to do this . – QAMember Feb 18 '15 at 6:33
  • If you're using Selenium, try splitting your CSV into several files over several test methods? Especially if you can subdivide your links into functional/logical groups, that would be a great match. Also, don't forget to close() your browser after each url/test. – FDM Feb 18 '15 at 6:46
  • What you said is True @ Frederik, but here I have one problem for doing like this bcoz after test case execution I am inserting the results(the count of broken url's) in to DB.So that I cant subdivide test case. If I misunderstood ur comment please provide me an example. I am appreciating you for spending your valuable time on my query. Thank you @ Frederik – QAMember Feb 19 '15 at 5:15
  • @sunilkumar I see, but that shouldn't be an obstacle. You can keep a global variable to add up all the broken links returned by each method. At the end of the entire test suite, you can write one result to the DB. – FDM Feb 19 '15 at 6:16
3

A nice automation question.

This question has already been asked on Stack Overflow, in Check the server response code, then export to csv. I don't know how good your python coding skills are, but most of the code is given on that page. It seems to aim to achieve exactly what you want to do. It probably will need tidying up and debugging first though :-( However, as rSF suggests, it should only take an hour or two.

import csv

with open("urls.csv", 'r') as csvfile:
    urls = [row[0] for row in csv.reader(csvfile)]

import urllib2
for url in urls:
    try:
        connection = urllib2.urlopen(url)
        print connection.getcode()
        connection.close()
    except urllib2.HTTPError, e:
        print e.getcode()

As an aside, there is an online tool of the same functionality, HTTP Status Code Checker, although coding that up using that programatically is probably more trouble than it is worth.

| improve this answer | |
  • Thanks for your answer @ Greenonline. Actually I am using the selenium to automate the checking the response code for URL's but I have 10000 url's so my script is taking lot of time sometimes my system going to hang . So I am looking for tool like what you have shared "HTTP Status Code Checker" but here is there any feature's like importing the bulk url's through csv/excel or etc ... – QAMember Feb 18 '15 at 5:36
  • With 10k URLs, you'll probably want to use the python multiprocessing library as well to parallelize this . . . – ernie Feb 20 '15 at 1:32
0

Or just code it in your favorite language- Python, Perl, JAVA etc.

I estimate it in less than an hour of coding, verifying it and having a cup of coffee

| improve this answer | |
0

I think that you should be able to use Apache JMeter for it. JMeter acts on protocol level so in any case it will be faster than using a browser. Besides JMeter is designed for load testing so you will be able to check all the URLs in matter of seconds.

Components you'll need are:

| improve this answer | |
0

I'm pretty new to coding, but I wrote some code that I believe does what you want. The output is a .csv file with two fields.

import csv
import urllib2

file_to_open = raw_input('Please enter the name of the csv file containing the URLs: ')
file_to_write = file_to_open[0:-4] + '-review.csv'

with open(file_to_open, 'r') as old_csvfile:
    urls = [row[0] for row in csv.reader(old_csvfile)]

with open(file_to_write, 'wb') as new_csvfile:
    new_csv_writer = csv.writer(new_csvfile, delimiter=',')
    new_csv_writer.writerow(['URL'] + ['Code'])
    for url in urls:
        try:
            connection = urllib2.urlopen(url)
            new_csv_writer.writerow([url] + [connection.getcode()])
            connection.close()
        except urllib2.HTTPError, e:
            new_csv_writer.writerow([url] + [e.getcode()])
| improve this answer | |
-2

I came across similar requirement so I used core Java's Connection URL API, URLConnection. It was much faster:

String  myString = null;
URLConnection urlConn = null;
try{
    URL url = new URL("enter url here");
    urlConn = url.openConnection();
    urlConn.setUseCaches(false);
    HttpURLConnection httpConn = (HttpURLConnection)urlConn;
    InputStream is=null;
    if (httpConn.getResponseCode() >= 400) {
        if(httpConn.getErrorStream()!=null){
            is = httpConn.getErrorStream();
        }else{System.out.println("no error");}
    } else {
        is = httpConn.getInputStream();
    }
    myString = IOUtils.toString(is, "UTF-8");
    System.out.println(myString);
}catch(Exception e){
    throw new RuntimeException("Exception while calling url:",e);
}
| improve this answer | |
  • Hello Satya Ravi, can u give me the sample code – QAMember Feb 24 '15 at 5:43
  • Kudos to those who downvoted without giving any argument. – dzieciou Jun 15 '15 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.