Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

The test pyramid is a concept developed by Mike Cohn, described in his book "Succeeding with Agile". Basically, end-to-end tests (often called UI tests) go across all the layers of the application, while unit tests cover single components.

Test pyramid

While the idea is sound and logical, I found implementing a well-balanced test pyramid hard, particularly when different teams are responsible for different levels of tests. For instance, in our teams devs focus on unit tests, while testers focus on end-to-end tests. Service-level tests are done either by testers, devs or by both, depending on the knowledge of architecture, testing skills, available time, etc.

Therefore, I'm looking for resources (books, blogs) that could help me answer:

  1. How do you make sure you do not duplicate test cases at different levels? Sure, when you verify how the system handles invalid input from a user, you may discover different problems at unit level, service level and UI level. Does it mean you want to replicate all unit tests at higher level as well? I would like to avoid that, as end-to-end tests takes longer and thus their number should be limited.

  2. Following the above question, how do you recognize redundant test cases?

  3. When a tester discover a case on end-to-end level (UI) that could be easily tested at unit level (and is missing in unit test suite), should she add it to the unit test suite?

I understand the area is wide, hence I'm more interested in hands-on experiences and lessons learned than silver bullet solutions.

share|improve this question
add comment

1 Answer 1

I've been wondering the same thing for the last couple of years. In my organization we've begun using BDD style testing to help with our pyramid problems. Right now we have about 800 automated acceptance tests, 3000 manual tests, 1500 integration/service tests, and 1000 unit tests. Obviously not ideal.

I've realized that a lot of those manual tests, automated or not, can be tested more efficiently through integration or even unit tests, but our QA analysts have no idea what those lower level tests are doing. Using BDD to describe the behavior that is being tested is helping QA analysts, SDETs, developers, etc all get on the same page as to what is being tested. In most cases it doesn't really matter to anybody what the implementation of a specific test is, just that some behavior is functioning as intended.

As for your specific questions, I think this helps us avoid #1 and #2 before it's an issue. Regarding test duplication, certain things like input validation are requirements at multiple levels (service, web) and we prefer to validate those at both levels with unit tests if possible.

For #3 when a tester discovers a defect, he can write a BDD test for it which will be implemented at the correct level as determined by a developer or SDET (see the Depth of Test link below).

Here are a couple of things I've read about this topic:

Depth of test

Automated Testing and the Test Pyramid

share|improve this answer
    
Thanks. Is it like this that (A) your QA analysts defines BDD scenarios in advance, and the team implements them at different levels, or (b) it is more about common notation, e.g., every one in the team names the test in given/when/then grammar? –  dzieciou Dec 31 '13 at 17:58
1  
It's kind of both. Ideally you'd have everybody defining the BDD scenarios. BDD is generally meant for business people and developers to be able to communicate. We use it more for QA/dev communication, so QA analysts are mostly writing the scenarios. The scenarios are then automated at the appropriate level. The common notation part doesn't really matter. What matters is that behavior is described in English instead of code so that everybody can understand it. –  thebeekeeper Dec 31 '13 at 18:35
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.