Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

what is really the difference between a component/unit test and an integration test?

Suppose you do very fine unit testing with stubs and drivers and test each component as far as you can with unit tests, which kind of tests would you then do in integration testing?

Things I can think of:

(looking bottom up, from unit to integration level)

  • If you test on unit level with BVA, perhaps it's sufficient to only
    test with EP on integration level?
  • You can do positive and negative testcases on unit level, while you would restrict yourself to more positive testcases on integration level?
  • On integration level you focus more on data. But how would you do this exactly?

(looking top down from system to integration level)

  • You see which use cases can be sensibly executed with the few components that are integrated already.

So if you do not want to do double work, which NEW tests would you write for integration level?

share|improve this question
add comment

4 Answers

Welcome to SQA, Chris.

First, regarding terminology, there are many terms for describing different kinds of testing. Everyone uses them differently. In some situations, those terms have specific meanings defined in contracts or regulatory documents. Usually though, the terms are just labels assigned to vague concepts that individuals (rather they realize it or not) do not necessarily agree on. When someone talks about "unit testing" or "integration testing", I try to ask what those terms mean to them so that we speak the same language. In any case, I will assume your question does not apply to a project with specific test processes defined in a contract or a regulatory document.

Yes, if you unit test with boundary value analysis (BVA), equivalence partitioning (EP) should suffice in higher-level tests.

I do not mean to be evasive, but your integration tests should be whatever they need to be. I think unit-testing is about testing individual (and perhaps indivisible) components, and system testing is about ensuring that the system works as a whole. Software is an assembly of components, but a component may itself be a sub-assembly of other components. Unit testing only verifies that a component behaves as the developer intended. It does not guarantee that a component behaves the way everyone else assumes it behaves. Integration testing is an opportunity to verify those assumptions.

It may help to think about the system-test/integration-test/unit-test hierarchy as the inverse of the diagnostic process. When the dishwasher repairman arrived to fix my dishwasher, he started diagnosing the problem by interacting with the system interface, i.e. he closed the door, pressed some buttons on the control panel, listened to the sounds coming from the dishwasher, and then opened the dishwasher back up to see whether the dishes were wet. After that, he narrowed his search to a specific sub-assembly, which has its own interface that is different from the system's. Eventually he narrowed down the problem to a specific part, which has a different interface from the system's or the sub-assembly's.

Integration testing is a means of identifying problems in a more easily-diagnosable manner than system testing. The problems are more easily diagnosable because the integration test is specific to a sub-assembly (with fewer moving parts) rather than to the system as a whole. Because it works at the sub-assembly level, the integration test may have access to addition diagnostic information that is not exposed at the system level.

Looking at this from the other direction, then, your integration test should focus on the sub-assembly's interface, not the interfaces of its component parts. If you find a large overlap between your component tests and your sub-assembly tests, it may mean that you do not really need to test the sub-assembly, or conversely that you need to define your sub-assembly in a different way.

share|improve this answer
    
Hi,good points you make. My problem is that I don't see a difference in the contents of the tests. Or only that we test a subset of what we test in unit tests (IF we test all components completely before we integrate) This is the EP/BVA thing. –  Chris Feb 7 '12 at 19:40
    
I agree with most of the above. I have always looked at integration tests as tests that validate interaction between classes, or between a class and an external component. True unit tests can always be run without needing stubs or mock objects. Integration tests may (not always) require mock objects to get thorough coverage of the possible interactions. –  Sam Woods Feb 7 '12 at 23:38
    
Chris, the best way to discover what you need in an integration test may be to pay attention to what happens when you do not use one. –  user246 Feb 7 '12 at 23:55
add comment

The objective is to resolve conflicts between components being integrated. Those kind of conflicts cannot be found during unit tests. The example of dishwasher of @user246 is very illustrative here.

Type of conflicts you may encounter:

  • A component transmits syntactically wrong or no data. The receiving component cannot operate or crashes (functional fault in a component, incompatible interface formats, protocols faults).
  • The communication works but the involved components interpret the received data in a different way (functional fault of a component, contradicting or misinterpreted specifications).
  • Data is transmitted correctly but it is transmitted at the wrong time, or is late (time problem), or intervals between the transmissions are too short (throughput, load, or capacity problem).

Based on the "Software Testing Foundations" book by Spillner et al.

Note, you may find those problems both during integration of your software and of your software with off-the-shelf components, e.g., libraries.

EDIT: I started to analyze different bugs that I and my colleagues found in integration tests. I believe it can be useful to know also examples of the conflict classes I mentioned above. Being more sensitive on what can go wrong in integration can allow us to stress those points in tests.

EDIT 2: I found a more comprehensive source on integration bugs found "Empirical Study of Software Interface Faults — An Update". It's from 1987 but does not seem much outdated.

share|improve this answer
    
I completely agree with your statements, but what is the content of these tests? I think these test will not contain anything new related to component tests. - Your first bullet can be found with a subset of the component tests of the high level component where a stub of B is replaced by the real B. - Your second bullet: exactly the same. - Third bullet: I think these may contain new testcases, mostly negative ones. –  Chris Feb 13 '12 at 8:18
    
Yes, you may reuse those test scenarios that focus on the areas where you expect most integration problems. You may follow negative cases. You may combine several small tests to simulate stateful interaction protocol between components. Also, integration itself may require some glue to put components together. For instance in frontend/backend (Flex/J2EE) integration the glue is the mapping between methods transfer objects. Components may require particular setup to work together. What if frontend session timeout is longer than backend session timout? User will be logged in for too long. –  dzieciou Feb 13 '12 at 19:42
add comment

I agree with user246 in regards to always confirming implied meaning of terminology. I was taught and have always used this reference wrt understanding the abstract "levels of testing" and wrote a bit about it here.

In my current role I manage a team of testers that do 90% integration testing. What that means is that while we rely on our developers to do unit testing of individual functions and components prior to any check-in. Oour responsibility lies primarily in testing functional "scenarios" of the 'components' below or without the user interface. This is beneficial because often time the functional capabilities or "business logic" is well tested before the user interface stabalizes enough to automate tests. (Also IMHO, it is more effective to automate "functional" tests that exercise the "business logic" below the UI, and use UI automation to mostly focus on behavioral aspects of the UI code.)

Also, as Sam indicates above unit tests may rely on stubs or mocks, integration tests should never use a mocked component.

WRT BVA and EP, I do think that boundaries should be throughly tested in unit tests. But, EP is a mechanism that can help identify sub-boundary conditions at the edges of 'ranges' of data partitions in some cases. The edge cases of EP (e.g. unique or special sets of EP data) can typically be tested more efficiently at the integration level. (Efficient in terms of time. Unit tests can't take forever, or be completely inclusive. In our case our integration tests are developed in-sync with the dev code in an effort to identify "functional" issues way up stream before they are incorporated into the main branch.

share|improve this answer
add comment

Any tests that involve some subsystem that isn't part of your code base are considered integration tests. So this includes things like databases, the file system, web services and so on.

So you should go back to your business requirements, and identify those requirements which use one of these "external" subsystems as part of their implementation. So, to start with, any requirement that involves storing and recalling information, where you've chosen a database or file system based implementation is a candidate.

Then proceed as you would for a unit test. That is, work out if the requirement has different cases depending on different inputs or different system states. Then set up tests that exercise whichever components contribute to the fulfilment of each case, both within your code base and outside of it.

Don't forget to include negative cases. Whereas you've probably already tested these at the unit test level, you probably want to make sure that a failure case or a "no data" case within one leve

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.