There is a requirement from the client, they want us to put a check on set of pages, that these pages and link on these pages should never throw 404 error. And Un-fortunately if any 404 appears an email is sent automatically to Dev team.

We can easily do it using Jmeter and Selenium but we don't how to keep it running 24x7. If we use Jenkins jobs (cron jobs) we dont know would it be optimal way to keep it running? Keep hitting these pages and links on those pages, wouldn't hit be performance issue?

Is their any other open source tool to accomplish this task?

Am I missing something?

  • What are they wanting to check for? That the server is up? That the server is working properly? That the backend is working properly? Sep 11, 2017 at 20:01
  • check link is not dead., not response code other than 200
    – paul
    Sep 12, 2017 at 18:52
  • I'm not sure this is a QA/test question, then. Sep 12, 2017 at 18:53
  • @KevinMcKenzie why not? Being tester you don't want to test (validate) whether links have gone dead (404) or not?
    – paul
    Sep 12, 2017 at 19:31
  • Philosophical issue, possibly. If I understand the question you're asking, it has to do with monitoring a live system. So you're trying to find out if, say, a server has gone down, or a network fiber was broken, or a program has run out of memory, or whatever, and impacted the service in question. To me, that's not QA/test. That's availability measurement/management/monitoring, which is very distinct from QA/test. But I'll leave it up to the moderators to decide. Sep 12, 2017 at 19:39

3 Answers 3


Don't use selenium for something like this, the overhead from carrying a browser with you doesn't benefit you in anyway and just eats up ram. I'm not going into the question if something like this makes sense since it's a client requirement and it's not a QA job to argue about this. Just set up a daemon that sends out requests to the pages and check the return code something along the lines of

import urllib3 as ul
from time import sleep
Pages = ["List", "of","urls", "to", "check"]
    for Page in Pages:
        with ul.PoolManager() as http:
           Resp = http.request(url=Page, method="GET")
           RespCode = Resp.status
           if RespCode.startswith("4") or RespCode.startswith("5):#reports if status code is in error range
    sleep(60*5)#change intervals to whatever you need

This is easily maintainable, does the same as selenium would do and is softer on the machine.

  • What language is this? Can you please share its Java version, so that I can understand and test it?
    – paul
    Sep 13, 2017 at 17:33
  • The language is python. In java you can do the same using the HttpUrlConnection class but I personally wouldn't recommend java foe daemonized services.
    – Daniel
    Sep 14, 2017 at 22:09

From the testing point of view you can't keep running a test suite to verify if at anytime there is any 404 pages come then it should trigger a mail. 404 error shows when the client was able to communicate with a given server, but the server could not find what was requested.(which may occurs because huge load or missing sources)

I would suggest that ask your UI developer to include the their UI development package and they can trigger an event when 404 errors happen.


First, it's absolutely not viable to constantly call each page. You'd have performance issues, and distort any kind of data collection you're doing.

That said, you do have some options.

  • Modify the application or host error handling to email the team - this would required development assistance to code the changes, or potentially network management or devops assistance to change the host settings to email the team whenever there are 404 errors.
  • Use a cloud-based monitoring service - AWS provides a cloud based monitoring service that can send a notification if a site or URL is unreachable. I'm sure there are others.
  • Scheduled availability check - You don't want to do this real-time, but you can schedule an availability check to run daily at a low-activity time. If you can't convince your dev team or devops/network management team to set up a notification on receiving a 404, and you can't find a viable cloud-based monitoring service, this is probably your best option.

Regardless of the method you think is best, you should research your options and take them to you manager so they can work with the client. It's best if your client knows what method you're using, why you're using it, and approves of the decision.

  • This is one of the reason I posted the question, that I want to go with a presentable idea. So that client would buy it one go? I like @godisalie92's idea. It looks lightweight and easy to maintain.
    – paul
    Sep 13, 2017 at 17:35
  • It's not feasible to predict whether the client would agree or not, but giving them a number of proposals with advantages/disadvantages gives them the chance to get what they want (someone gets notified of any 404 errors) in a way that won't impact performance or statistics.
    – Kate Paulk
    Sep 13, 2017 at 19:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.