10

in my project I started code coverage for unit and integration test with gradle + jacoco. It's already working and generating reports.

I saw a lot of questions and tutorials about how to do code coverage for both tests, but I didn't see any question about why do code coverage for integration test. Which type of data do we expect for integration test (code coverage)? The same as unit test?

Why I'm asking this: On unit test, we explicit call methods, objects and other system under test code. In this scenario, jacoco understand this and do a metric for code coverage based on what methods/properties is being called. And this is ok, makes sense to me. But on integration test, I only use some model classes to use in test and jacoco only report this models as being code coverage but not the methods being used during the tests.

Makes sense to do a code coverage for integration test? Which type of data do we expect? Or my integration test code coverage is wrong?

Improve this question
5
  • 2
    What do you mean with an integration test, what do you test? end-2-end, multiple classes combined? API with Database? I am confused as you say I only use some model classes. Sounds like your testing models?
    – Niels van Reijmersdal
    Commented Jan 20, 2017 at 16:10
  • Are you asking how to measure much of your product code is covered by integration tests, or how to measure much of your integration code is covered?
    – user246
    Commented Jan 20, 2017 at 18:35
  • Hey Niels, by integration test I mean e2e test. I used "integration" because of maven pattern. I use SUT's model in my test (like Event.java, User.java) to create an expected object. @user246, I'm asking why do this measure in e2e test.
    – Thiago Fioravante
    Commented Jan 23, 2017 at 13:43
  • @ThiagoFioravante You did not answer my question. Please be specific. Which measurement? Do you want to measure how much of your SUT is executed, or do you want to know how much of your integration test is executed? Which one?
    – user246
    Commented Jan 23, 2017 at 13:56
  • @user246 how much of SUT is executed.
    – Thiago Fioravante
    Commented Jan 23, 2017 at 23:16

2 Answers 2

Reset to default
10

I'll assume the question is this:

Why measure how much of the SUT's code is covered by integration tests?

and not this:

Why measure how much of the integration test code is covered?

I think you should avoid measuring code coverage for integration tests, for two reasons:

  • Unit tests are a much more effective way to exercise code than integration tests. Compared to integration tests, unit tests are lighter weight, faster, and more targeted. Using an integration test to cover code is like using a sledge hammer to drive in a nail.
  • Unit tests are about exercising code, whereas integration tests are about testing assumptions. When we write unit tests, we mock/fake/stub out the interfaces that the code depends on, and in the process we encode assumptions about how those interfaces behave. Integration tests are where we find out whether those assumptions were valid.

Another way to put it: unit tests are about testing whether the code does what the developer intended. Integration tests are about determining whether components work together.

Improve this answer
1
  • Hey user246, I have the same opinion about this but I saw a lot of questions/tutorials about this and decided to give it a shot. But when I saw the report it made no sense for me and I thought that I could do something wrong. Today I found the same discussion here: sqa.stackexchange.com/questions/9519/… and I'll try @jpjwolli answer for now.
    – Thiago Fioravante
    Commented Jan 23, 2017 at 13:57
2

You typically exclude test code from code coverage reports. The entire goal is to learn which parts of production code are touched by your test code.

If you run integration tests against a deployed version of your application you should look into remote code coverage, typically exposed via JMX. Jacoco has support for this, see the agent documentation.

Improve this answer
3
  • 1
    I'm pretty sure they're asking how to measure what code their integration tests touch, not if something else is testing their tests . ..
    – ernie
    Commented Jan 20, 2017 at 19:42
  • Thanks @jpjwolli, I'll try this and return a feedback about the remote code coverage.
    – Thiago Fioravante
    Commented Jan 23, 2017 at 13:59
  • "You typically exclude test code from code coverage reports" -- I actually don't do this for one specific reason: sometimes it reveals tests that aren't actually being run, or test fixtures that aren't actually being used. Like if I see tests/foobar.py at less than 100% coverage I immediately go "hey that's weird" and look inside to see what in that file isn't being exercised as part of a test run.
    – Adam Parkin
    Commented Apr 16, 2021 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.