6

I've been struggling a bit when writing test cases. Our team follows a 2 week sprint, and I'm writing acceptance test cases on these user stories.

I've tried the BDD format (Given/When/Then) and it feels...short. I'm not automating everything (although what I do automate I automate after development is stable, and I use cucumber). I'm not sure if this is the right way to go (Although I love the BDD methodology and I want to try it).

Then again this is also an "outline style", a kind of a "Verify X works this way" with a sort of checklist.

I'm not really sure which way makes the most sense, and really what the advantage/disadvantage is. I'm not really looking for a "Which is better" but more of a "Which makes more sense for BDD" or "advantages/disadvantages"

4

I would recommend doing both approaches, for different purposes.

Given / When / Then is good for testing communication. This format brings assurance that what you are testing is what the business owner / requirements writer intended. The bugs mainly caught by these tests, IME, are ones where the business owner intended X and the dev did Y. Often, these tests (if the devs read them) will "catch bugs" before they are implemented, because the dev will come to tell you that the test is wrong. A conversation with the business owner usually clears up where the issue is, and often it is a communication issue. These tests will also catch basic "the feature doesn't work" issues that somehow leak through the developer's unit tests.

However, Given / When / Then tests often carry a substantial amount of overhead. First, writing the G/W/T format tends to take more time than a checklist, especially to write this format well. Second, to get the most value from G/W/T tests, you need to ensure that both the business owner and the developer read them before coding starts. Often developers, especially, horde their precious work time - the less you have for them to review, the more likely they are to be involved. Finally, IME maintenance for tests implemented with G/W/T languages like Cucumber tends to be more costly than e.g., JUnit tests, mostly because there is the additional "English" layer on top of the libraries. Moving from English to code effectively has some quirks that do create additional maintenance work. It's easy to end up with test steps that seem to do one thing in English but actually do another in code, especially if you have multiple test implementors. In general, you want a test pyramid - acceptance tests should be the smallest group of tests, then system integration tests, then component integration tests, then unit tests.

Personally, I use the Given / When / Then format for only those things that have clear requirements around them that the business owner cares about. For some code, this will include mainly happy path scenarios while the engineers will be the best ones to determine how error cases should be handled. In other cases, there are requirements around error cases that the business cares about deeply and additional Given / When / Then style outlines will be needed for those cases.

I usually then use a list or checklist to cover the other scenarios or test work that the business owner doesn't care about. This is mostly error cases, but will also include things like logging, metrics, documentation, very technical aspects of the feature like queue processing performance, and so on.

| improve this answer | |
  • 1
    Were just now really starting the "Scrum"/Agile process. So i've just been using the Given/When/Then tests for acceptance test cases based off user stories. Not sure if thats a good idea however. But since im a single QA in this company doing all different kinds of test plans seems like impossible work as a single person. – Mercfh Oct 22 '15 at 18:35
  • 1
    For me, I usually do that and then just throw an "additional test notes" section under it. It's very casual, and scales well to the free time available. I will sometimes call out certain tests as "as time allows", if they are lower priority. ETA: Also, I do this in the JIRA / ticket itself. Not even a separate test plan. And, I think using the G/W/T as you are is perfect. – Ethel Evans Oct 22 '15 at 19:52
  • 1
    Sorry one more thing: Do you find when you do G/W/T test cases off user stories they are...short? They just don't seem very long. I mean user stories in general shouldn't be superlong regardless. (im still new to the whole BDD thing haah) – Mercfh Oct 23 '15 at 15:17
  • 1
    Yes, absolutely. 3 to 7 lines long, usually. An excellent book for writing great G/W/T tests, BTW, is "The Cucumber Book" - links to both the default (in Ruby) and a newer Java version are at the bottom of this page: cucumber.io It's a very easy read, and covers both writing the G/W/T (aka Gherkins) and implementing the test steps using Cucumber. I hand it to everyone in the office who is getting started with BDD. – Ethel Evans Oct 23 '15 at 21:18
  • 1
    I actually just ordered that book but haven't gotten the chance to flip through it. Thanks! – Mercfh Oct 26 '15 at 2:32
2

The more detailed the test case, the more time you will spend on revisions as the software evolves.

The less detailed the test case, the more you are leaving it up to your testers to read between the lines.

Maybe you should try it both ways and see how it goes.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.