I found TestNG failing occasionally and when I started to investigate I found it has little test coverage. It looks like the test framework itself is poorly tested.

Which other test execution frameworks for Java are better tested with unit tests?

  • Where exactly does TestNG go wrong? Next best known alternative springing to mind is JUnit. – FDM Feb 12 '16 at 14:31
  • @FDM what's the test/code coverage for JUnit? – dzieciou Feb 12 '16 at 14:45
  • Correct me if I'm wrong, but this question is all about "eating your own dogfood" right? What test-frameworks are, themselves, actually tested and how much so, right? – corsiKa Feb 12 '16 at 21:12
  • @corsiKa. Yes, initially it was my way to relieve frustration after debugging TestNG, RestAssured and other test frameworks code. They contain bugs, often they are open source code created by passionate programmers and testers. I was trying to understand how they are tested and which one I can trust. – dzieciou Feb 12 '16 at 21:52

I have never used it but I would be surprised to find out that the popular test framework TestNG is actually the source of your intermittent results.

That being said, if you want to use a different one I would recommend JUnit. The interface is similar to TestNG (Annotation based) and I've never had a problem with it.

|improve this answer|||||
  • This does not answer the question. The question was about which test framework is better tested. – dzieciou Feb 12 '16 at 21:58
  • If if you don't believe that TestNG and test frameworks in general contain bugs, check issues reported for TestNG here: github.com/cbeust/testng/issues. – dzieciou Feb 12 '16 at 22:46

When talking about unit testing with java, then no one beats JUnit.

JUnit is the awesome unit testing framework and widely used in software companies. As compare to TestNG, JUnit is lightweight and faster. But almost same as TestNG.

TestNG itself is extension of JUnit, and it has just added new features to JUnit. But much powerful then Junit.

I will suggest to go for JUnit, but still it is as same as TestNG.

Another framework combination can be using JTest with JUnit.

Jtest complements and extends JUnit. Jtest not only runs JUnit test cases, but also automatically designs and executes additional test cases that verify the code and increase test coverage. Moreover, Jtest automatically creates JUnit test class templates into which you can easily enter test cases, and exports Jtest test cases in JUnit-compatible format. It even enforces best practices for JUnit test classes.

|improve this answer|||||
  • By "no one beats JUnit", in terms of the question, this would imply that when JUnit is built that it goes through a series of unit tests and if what you say is true that "no one beats JUnit" that those tests have 100% code coverage and that none of the tests ever fail on the production build. Is this your claim? Because that's what the question appears to be asking. – corsiKa Feb 12 '16 at 21:12
  • @corsiKa - my answer was for "Which other test execution frameworks for Java are better tested with unit tests?" – Jeevan Bhushetty Feb 12 '16 at 21:29
  • To be precisely, TestNG introduced many features before JUnit 3 had them. Then they were added in JUnit4, so functionally JUnit and TestNG are very comparable now. – dzieciou Feb 12 '16 at 21:57
  • What is test/code coverage of JUnit framework itself? – dzieciou Feb 12 '16 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.