"Conventional" Test cases in an Agile environment?

Question

I've just started at a Web Development company that does websites/mobile apps. I've done QA for 5-6 years now and they wanted someone that had QA experience but wasn't set in their ways to be their QA lead for their new QA dept. (Which just consists of me).

In the past, the company had kinda taken an ad-hoc approach, no real standardized approach to collecting requirements etc... However, now they are doing a normal Scrum approach, tracking user stories in Jira (Which I think personally is a good start).

In my previous job, we had very strict requirements defined at the beginning of a project (Like a LOT of requirements) based off an FSD (Functional Spec Document). I would write my test cases based on these.

However, User Stories tend to be a little more generic and vague while still identifying the requirement.

What's the best process for really creating test cases/test plan based off of this. I feel like conventional test cases (IE: User clicks this button: Steps: Expected Result: etc...) seems a little...slow and cumbersome? and requires a lot of maintenance especially if things change?

Obviously automating everything is impossible...but is there like a halfway point maybe? My idea is to create acceptance tests based on the user story, I've read about cucumber and Gherkin...which seems cool because you can write a High-level Acceptance test that can be automated later. This kinda covers the Front Acceptance Test end while making automated tests later not be such a hassle.

Am I missing anything? I guess I wonder about all the "conventional" test cases...and if/when they are needed. After my experience with doing them and spending more hours than I'd like to admit writing STEPS/EXPECTED OUTCOME/Etc.... x10000 they just don't seem as needed...is this wrong? I just wanna make sure what I said doesn't sound....well stupid.

Answer 1

To start with, I'd suggest browsing the answers of the questions showing as related, particularly Test Automation in Agile? and Agile Tester vs Traditional Tester...........?

As far as test case writing, it doesn't really matter how you handle that, as long as someone who is moderately familiar with the application can follow your test cases. The main things you need are the goal of the test case (I often state this at a high level in the title of the test case, e.g. "Users with permission level admin or higher can access feature X"), the desired outcome (you can take "does not crash" as implicit in all of these if you have a stability non-functional requirement as part of the user stories), and the actual results. That's the minimum necessary, and can be stated in English or in a Cucumber-style fixture. I'll typically state the generic case, then have a list of data-specific cases I plan to look at (usually exploring around boundaries and crucial configurations).

The approach you take is going to depend on the dynamics of your team and the nature of your web applications, but I can make some suggestions here:

• Developers should be writing unit tests and these should form the bulk of the automation. Your role with these tests is to work with developers to generate good unit tests.
• Developers should be writing module/integration tests and these should be your next largest number of tests. Once again, you need to work with developers to help generate good tests.
• Testers or Developers write system-level tests. These tests have some overlap with integration tests, but largely cover system processes from end to end. If you can't do this kind of testing through your application API you have a problem.
• All the unit, integration, module and system tests should be automated and run with every build. These are all "headless" tests in that they don't interact with the GUI, so they run quickly.
• If there is any way to avoid GUI automation short of manually testing everything, use it. GUI automation is slow, expensive and even with the best design can require a large maintenance cost. You want to avoid this wherever possible.
• Tester-built automation should happen early in the next sprint. The reason for this is that automating something that hasn't stabilized is wasted time. The ROI for any automation lies in future releases, not in the current cycle.
• When starting, look for high value and easy to automate and maintain tests. This is particularly important when you're looking at something with several years of development and no automation of any kind in place.

Also - and this is particularly important - You may not be able to automate everything. Many of the most experienced folk here will tell you that you should automate 100% of the tests that should be automated - the fun is in working out what those are.

Some things that can't or shouldn't be automated are:

• Look and feel - until computers can beat the human eyeball and brain at pattern recognition and complain about awkward/unwieldy processes, this has to be done manually.
• Not all widgets (web or otherwise) can be accessed by automation tools.
• You can automated checking printouts, but it's clumsy, time-consuming, and very error-prone. The alternative: automate checking the printout text and run a printout for a manual lookover is a whole lot easier, faster (and therefore cheaper) and less likely to break.
• Unless you know how power-users actually use your application you can't automate that. When you do know how they use it, the best you're likely to be able to do is make sure all the shortcuts keep working.

Answer 2

You are on the right track. I used to do the granular test plans until relatively recently. I have gone the direction of making a Test Plan mostly consisting of User Stories. You need to remember your audience when writing anything. My Test Plans are part of the deliverable to clients so it makes the most sense that anyone can read it and use it. That means cutting out the granular, technical stuff that made something you could explain in 5 sentences into a 200 row spreadsheet.

The part about automation shouldn't be a major factor this early into the game. As J M Juran says, people want to automate to be quicker but they end up automating bad processes. I recommend you take the time to iron out your manual processes and then decide what is worth automating and how.

Answer 3

I had the exact same questions as you when I started the "testing" compartment in my current Agile project.

It is indeed not feasible to write and maintain test cases in the tradition format. User stories are delivered every few weeks and change rapidly afterwards. Test cases would become obsolete faster than you can write them.

One approach that I found useful in our small Agile team is session-based testing. You can find some explanation here and here.

Because we're using Microsoft Test Manager, I've created a custom template based on this SBT method. (No use sharing it here because it's not in English.)

The main advantages are: one test charter is linked to all related bugs and test execution history. One test charter also contains all relevant tips and tricks needed to test the user story in question. There's little administrative overhead. Worth looking into, as you can easily adapt this idea to your needs.

Answer 4

Obviously automating everything is impossible...

How do you think teams that practice continuous delivery are working?

Being Agile means that you optimize the feedback cycle on all levels, for long-running projects a rule of thumb should be to automate everything. This means the whole pipeline up-to deployment in production.

I would rather say: of-course automating everything is possible!

Just work with the development team and make it testable. Cut out any pieces of infrastructure or external libraries that are "untestable" and replace them with something else that is testable.

I love the BDD concepts, but if your team is not using it as a communication tool between with the business it might be a layer of waste since it adds overhead when creating automated tests.

you can write a High-level Acceptance test that can be automated later.

The automation should be done before releasing the functionality described in this acceptance test. If you don't you will start falling behind with test automation. How can the team release new features that build on top of these in the next iteration? Manual testing? Writing and executing manual test-cases is not sustainable in an ever fast-changing world.