My organization would like to choose between the TestNG and Robot Framework. I thought it better to approach this from the stand point of usage statistics, since both are open source. How do I go about doing this. I did not find any head to head comparison information when I looked it up and so I thought it better to go with usage stats. Could someone point me in the right direction?

By the way we're a product company providing solutions for the Supply chain Management domain. We have both J2EE Web applications (flagship) and mobile applications (iOS and Android)

Currently we're white-boxing with Groovy/testNG but there's confusion for black boxing. We're planning to use Selenium as the automation tool with either TestNG or Robot.

  • 1
    Would you mind elaborating on what you mean by usage statistics? Are you talking about code coverage statistics, or how much market share TestNG & Robot have, or something else?
    – user246
    Commented Mar 13, 2014 at 15:39
  • Yep I meant market share
    – funkfan
    Commented Mar 14, 2014 at 14:52

1 Answer 1


I think you want to pick the framework which has the most active development and the most documentation resources on the internet.

Of-course you first need to check which framework fits your requirements, I would pilot all for a short while (starting with the most active one. If you have multiple candidates.)

Which is more popular:

  • Number of (recent) commits in the version control system
  • Number of contributors in the version control system
  • Number of tags on StackOverflow (Robot + RobotFramework:500 vs TestNG:1665)
  • Number of Google results
  • Active users on the IRC channel on irc.freenode.org

In the past we have picked a slightly "worse" framework, because it was actively developed compared with the single developer for the other framework.

Other things I would look into are:

  • Is the framework in a programming language your team has experience with
  • Does the framework have good test coverage

This will make it easier to extend it in the future. When the community does not fix a defect your are running into.

  • Thanks for your timely reply, I will take all of the above under advisement.
    – funkfan
    Commented Mar 14, 2014 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.