Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Has anyone had experience getting a combined code coverage measure across both unit tests and functional/UI automation tests? I'm in a Java stack with Cobertura, Jenkins and Ant. I can successfully get the unit test coverage report in Jenkins already, and I am looking to get that build to also spin up an app server with an instrumented version of the app, and include the coverage from the UI automation along with the unit tests.

What I'm hoping to do is identify the code that is covered by the UI automation tests, so I could eliminate some unit tests that don't test any meaningful behavior on their own and are there mostly to increase coverage.

In other words, I'd like to have a way to report that I don't need a unit test for a controller class (for example) in a Spring MVC/Grails application because it is already covered by the UI automation, and because there is no real business logic in the controller by itself.

share|improve this question
    
Measuring code coverage requires code instrumentation, which may be harder in end-to-end tests, because you need an code instrumentation agent in each location where your Java files are executed. For instance, if Web app pages are served by Tomcat server, JaCoCo agent must be run on Tomcat. So it all depends. Where and how your app is run? –  dzieciou Jul 13 at 6:59
    
I wonder also about the purpose of a combined code coverage report. Usually, in end-to-end tests were more interested in test coverage (what features where coverage), while in unit tests in code coverage. I have seen you original question (stackoverflow.com/questions/24703790/…) and it explains better the purpose. Might be helpful to have similar explanation here as well. –  dzieciou Jul 13 at 7:05
    
that is precisely what I'm asking- how would I get my app instrumented when it runs on Tomcat/Glassfish, and include the coverage output added to the unit tests? –  wrschneider99 Jul 14 at 13:37

1 Answer 1

Yes, we do just that. We use Python and Coverage.

I am curious what is business reason to eliminate unit tests. Unit tests are much closer to code and if any fails, finding and fixing the bug is much easier. The only reason to eliminate unit test is if another unit test covered the same execution path.

OK, after comment from OP: yes, it is valid reason to eliminate unit test: you mocked the heck out of controller without really testing much. Especially if maintaining mocks is brittle, tedious and does not buy you much because you need to test it with real life examples anyway. If you have some specific business logic for controller, unit-test it.

But 100% code coverage is wrong goal. Better goal is to get the best value for the time of developers and testers spend on writing and maintaining both production code and test code. Code (production, unit-tests, system-tests) is code, you need resources to maintain it, modify it in response to changes in requirements. What is best business value for resources spent? What makes your customers happy? If your code is covered twice over, by both unit and system tests, maybe you need channel more resources for development to have more features to be tested :-)

share|improve this answer
1  
Yes, unit tests are still good in most cases. Still there are situations like controllers with no real logic of their own, which may be easier to cover with end-to-end system tests than mock-heavy unit tests. Ultimately I want to get 100% coverage across all forms of testing and then use the most appropriate form for the situation. –  wrschneider99 Jul 14 at 18:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.