5

I have a wishlist for code coverage tools and am wondering if people could recommend some tools that fulfill all or many of my wishes. I am starting my search for tools that work specifically with .net code, but will also need a tool for Java code as well, so recommendations for either would be appreciated.

  1. Integration with my continuous integration tool - TeamCity to run during unit test execution.
  2. Run against a deployed service or web service to track coverage during execution of automated and/or manual tests.
  3. Ability to merge results from multiple runs - I want to see combined coverage from all tests and then split it up to see results from unit tests vs functional automation vs manual regression.
  4. Ability to show coverage for specific executed tests.

I am currently evaluating NCover 4.0 for .net which is very promising, but can be expensive to license. I wanted to do a fair comparison to other tools before investing in the cost of NCover.

Improve this question

6 Answers 6

Reset to default
2

Can't help with .net as I mainly have experience at a java shop. We use a combo of sonar and Jacoco.

Check out Jacoco!

I just installed it on the machines where we execute our service tests and the reports it generates are nice and can be integrated into CI easily (using ant). It's actively worked on, and has a Maven plug-in. The coolest feature for me is using the java agent to monitor lines of code used during testing. Also has a feature to combine reports. So you could, in theory, take unit, service, and regression testing coverage reports and combine them together (pretty damn cool).

I think that covers all your requirements, but of course not .net...

Improve this answer
1
  • Awesome, I'll definitely check this out when I get to the Java side!
    – Sam Woods
    Commented May 6, 2013 at 17:18
2

Feb 10 2016 Update

For the .net world mstest continues to provide code coverage. We also worked on integrating the Microsoft build technologies (msbuild, TeamBuild and the new TFS Build) with SonarQube.

https://blogs.msdn.microsoft.com/visualstudioalm/2015/08/24/build-tasks-for-sonarqube-analysis/

Btw, TFS build also supports Java builds (via Maven, Ant and Gradle) and you can enable code coverage there. For Maven you even enable SonarQube analysis.

Original answer

Visual Studio 2012 has a pretty good code coverage tool. I have used it to measure code coverage of a web service.

It is easy to integrate into build systems, you have out of the box support for TeamBuild (the TFS build) - see a more detailed article here.

According to msdn it has result merging.

Code coverage is also integrated in Microsoft Test Manager, so you can have manual testers do a round of testing on your service and at the end see how much of the code they actually hit.

Improve this answer
1

If price is an issue, you can use ncover 1.5, which is still open source on sourceforge. Another option is coverity test advisor.

Improve this answer
0

Look at Sonar: http://www.sonarsource.org

it is pretty good for any static test methods. Perfect for Java (plenty of metrics and plugins) okay for .NET (many metrics).

Improve this answer
0

Our family of test coverage tools satisfy directly your requirements 2/3/4 and your interest in tools that cover C# as well as Java (and several other languages). The tools are scriptable, so integrating into TeamCity should be straightforward, but they don't do that out of the box.

Improve this answer
-1

Dot net's build in Unit test and NUnit can resolve all of your needs...and for Java, Junit is the best . You may choose TestNG.

Improve this answer
6
  • Can you expand your answer? How do mstest, nunit and junit resolve my needs for code coverage?
    – Sam Woods
    Commented Apr 30, 2013 at 19:33
  • Actually Unit tests can test your codes before building any solution and you can cover code coverage (along with decision coverage). I thought of code coverage as tested code.
    – Shantonu
    Commented Apr 30, 2013 at 19:50
  • Still not clear to me how your answer relates to Sam's question?
    – testerab
    Commented Apr 30, 2013 at 21:30
  • 3
    @Shantonu I think Sam was asking about tools that measure how much of the code under test is executed. The packages you described are frameworks for organizing and executing automated tests. To see the difference, try comparing ncover.com to nunit.org.
    – user246
    Commented Apr 30, 2013 at 22:08
  • I think one of the key issues here is explaining how it relates instead of just listing them. For example, there may be some JUnit or NUnit plugin that you can use to get what you want, but without that extra knowledge it isn't as useful.
    – corsiKa ♦
    Commented May 5, 2013 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.