Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I need to find a program that can take a list of url's and actually run them to see where they go. For example:

Take - www.example.com and see if it goes to www.example.com (no error/redircts) or if it goes to a redirect (i.e. sub.example.com) or if it throws a error such as 404, & 500.

The list is about 900 domains long and we used to copy and paste each one into a browser and see what the end result was, then document what it returned. This has now become a pain since the client base is increasing extremely fast. A desktop app would be ideal, we would have to consider web apps.

Thanks

share|improve this question
    
Is there some programming language/environment that you'd prefer the solution to run under? For example C#/Windows or JAVA/Linux or ??? –  dcaswell Sep 17 '13 at 23:34

3 Answers 3

Although I like Joe's answer, I've seen this fail a few times. I've seen it happen a few times where for a 'quick fix' a developer has put in javascript redirects on page load when they don't have access to the server itself. In these kind of circumstances, I've found even using something like Selenium useful.

It could be as simple as the following;

[Test]
public void FindSites()
{
  var listOfSites[] = new list[]{'site1', 'site2', 'site3'}
  var driver = new FireFoxDriver();
  foreach (var site in listOfSites)
    {
    driver.Navigate().GoToUrl(site);
    if(site != driver.Url)
      Console.WriteLine("Attempted to navigate to {0} 
        but was redirected to {1}", site, driver.Url);
    }
}

It may not be horribly efficient, but, it works.

share|improve this answer

Xenu Link Sleuth (http://home.snafu.de/tilman/xenulink.html) can do that and more.

It's easy. It's fast. Best of all, it's free! (Free as in $0, but not Open Source)

share|improve this answer
    
This looks perfect for what we need, but the company doesn't like using free open source programs. Ive sent it up the ladder anyways just in case. Thanks for the comment. –  ZacNespral21 Sep 16 '13 at 18:22
    
@Duke03 - Perfect, because Xenu isn't open source! See: home.snafu.de/tilman/xenulink.html#FAQ –  Joe Strazzere Sep 16 '13 at 18:24
    
Right I realized that, but still the company recognizes anything to be free as a potential danger to our system. –  ZacNespral21 Sep 16 '13 at 18:26
    
@Duke03 - You are talking about tools to assist testing, right? My company also dislikes open source when used in Production systems. But they let us choose whatever we wish for test tools. –  Joe Strazzere Sep 16 '13 at 18:28
    
I don't get the Free is a danger issue, but I do use Xenu to go through our site to help me find dead links on occasion so I can have them resolved through Content Editing or redirects. It's very nice, but spend time with the reporting because the default HTML one is not Business User friendly. I do my own summary of links using PowerShell to link the reports together for easy readability –  MichaelF Sep 16 '13 at 20:24

If you are comfortable writing up a script, it probably will take less than 50 lines to do what you want.

If you want to use a programming language like python or java or something, you can use one of the URL libraries and make a request to the url. If the response code is 200, that URL passes your test. If not, you can mark it.

You can also write up a quick bash script using curl

share|improve this answer
    
I could write a script, but do you know of any examples anywhere that could get me started? Also what other languages could i write this in? Not great with those two but i could give it a shot. Thanks for the comment, –  ZacNespral21 Sep 16 '13 at 18:21
    
@Duke03, For Java see Apache HTTP Client with example here. –  dzieciou Sep 16 '13 at 18:57
    
@Suchit, great answer. One thing, the OP wanted also to recognize URL redirections. As there are many different approaches for redirection on client- and server-side: meta refresh, java script redirection, etc., his script would need to parse it somehow from HTTP response. Any library that can simplify that? –  dzieciou Sep 16 '13 at 19:03
    
@Duke03 - Take a look at this answer on SE too (php) - stackoverflow.com/a/3520085/781810. The response also has a link to a pretty good article. May not be the exact response you are looking for, but some good information. –  Suchit Parikh Sep 17 '13 at 16:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.