2

Has anyone had experience getting a combined code coverage measure across both unit tests and functional/UI automation tests? I'm in a Java stack with Cobertura, Jenkins and Ant. I can successfully get the unit test coverage report in Jenkins already, and I am looking to get that build to also spin up an app server with an instrumented version of the app, and include the coverage from the UI automation along with the unit tests.

What I'm hoping to do is identify the code that is covered by the UI automation tests, so I could eliminate some unit tests that don't test any meaningful behavior on their own and are there mostly to increase coverage.

In other words, I'd like to have a way to report that I don't need a unit test for a controller class (for example) in a Spring MVC/Grails application because it is already covered by the UI automation, and because there is no real business logic in the controller by itself.

Improve this question
3
  • Measuring code coverage requires code instrumentation, which may be harder in end-to-end tests, because you need an code instrumentation agent in each location where your Java files are executed. For instance, if Web app pages are served by Tomcat server, JaCoCo agent must be run on Tomcat. So it all depends. Where and how your app is run?
    – dzieciou
    Jul 13, 2014 at 6:59
  • I wonder also about the purpose of a combined code coverage report. Usually, in end-to-end tests were more interested in test coverage (what features where coverage), while in unit tests in code coverage. I have seen you original question (stackoverflow.com/questions/24703790/…) and it explains better the purpose. Might be helpful to have similar explanation here as well.
    – dzieciou
    Jul 13, 2014 at 7:05
  • that is precisely what I'm asking- how would I get my app instrumented when it runs on Tomcat/Glassfish, and include the coverage output added to the unit tests?
    – wrschneider
    Jul 14, 2014 at 13:37

1 Answer 1

Reset to default
1

Yes, we do just that. We use Python and Coverage.

I am curious what is business reason to eliminate unit tests. Unit tests are much closer to code and if any fails, finding and fixing the bug is much easier. The only reason to eliminate unit test is if another unit test covered the same execution path.

OK, after comment from OP: yes, it is valid reason to eliminate unit test: you mocked the heck out of controller without really testing much. Especially if maintaining mocks is brittle, tedious and does not buy you much because you need to test it with real life examples anyway. If you have some specific business logic for controller, unit-test it.

But 100% code coverage is wrong goal. Better goal is to get the best value for the time of developers and testers spend on writing and maintaining both production code and test code. Code (production, unit-tests, system-tests) is code, you need resources to maintain it, modify it in response to changes in requirements. What is best business value for resources spent? What makes your customers happy? If your code is covered twice over, by both unit and system tests, maybe you need channel more resources for development to have more features to be tested :-)

Improve this answer
2
  • 1
    Yes, unit tests are still good in most cases. Still there are situations like controllers with no real logic of their own, which may be easier to cover with end-to-end system tests than mock-heavy unit tests. Ultimately I want to get 100% coverage across all forms of testing and then use the most appropriate form for the situation.
    – wrschneider
    Jul 14, 2014 at 18:31
  • 1
    If you have heavy mocking at your controller, your controller is likely doing too much and you should break out logic into more composed services. Unit tests are the foundation and I agree with the original comment that deleting these is not really the right solution.
    – Alex White
    Apr 12, 2021 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.