Writing test cases before implementation?

Question

So currently at my company it is common practice to write test case high level steps before implementation. Myself and others find this to be a somewhat wasted effort. I often end up blowing away much of the steps when I write the final draft.

In my opinion the final draft is very quick to write once implemented.

So my question/problem to solve is, what can we better do with our time during this phase(pre-implementation)? What's a more constructive endeavor for this phase?

My current suggestion is to map the use case and data/control flow of the requirement being tested. That way writing the test case should be simple later because time was spent understanding the problem.

Answer 1

Well, first of all - congrats buddy, glad to know you work for customer / company with understanding that involving QA at early stages is definitely worth it!

I'll reveal below several related points from my own experience of managing QA team (up to 10 FTE) involved in testing of fairly big project, where reqs page count was about 1000 pages and more, and business logic complicated enough so that typical project investigation for QA team newcomer took about 2-3 weeks.

Many customers do believe that investment in any kind of test documentation is a waste of efforts in any case: indeed, during these hours we do not test and do not reveal any bugs, so what's the profit? Consider the following before making your mind! (:

1. Creating test cases (or simply any kind of test documentation) at early project stages, like prototyping, reqs refinement, early DEV has one great advantage: at the very same time we test requirements! Bugs are expected there sometimes even more often than in the application itself, but cost of their fixing at early stages is several hours of BA / analyst instead of hundreds of hours of DEV work later. In my case initial team of 3 QAs involved in test cases creation 2 months before the first build saw the sunlight was able to reveal more than 500 reqs issues of different severity: from incorrect warning messages to contradictions in business process. All of these were easily handled by team of 3 analysts - compare to 40+ DEV team at active stage of implementation and actual app testing.
2. At the same time, I must admit that many of early test cases were blown away from final suites. In fact, from about 2.5k+ of overall test cases count created before the 1st build only about 1.5k were included in running suites. Was that really worth it? Well, in addition to the above point related to reqs testing, by the time of 1st tests QA team itself was familiar enough with the functionality so that testing efforts for the same tests at the very start were only about 20-25% higher than later. Having in mind complexity of the system and unmovable project release dates (REALLY unmovable in our case!) any other approach was actually impossible, since for the team of any size project investigation would take the same 2-3 weeks. So, in our case that was the most important part of product investigation.
3. Regarding test cases format and detail: sure thing creating detailed test cases at reqs refinement stage is not at all the best choice. In the described case we finally ended up with the following approach: at first short document with the high level description of test directions only, e.g. "For the document submission form: personal info details integrity, attachments availability for download and printed form version are tested using this and that sets of test data". Such document, which we called "Test Design" (our PM just liked that (: ) was then passed to analysts, who then crossed out, corrected and adjusted priorities for the described scenarios. Only after that stage (which took about 10-15% of time comparing to detailed test cases creation) such refined test approach was transformed to the detailed test cases. No doubts this is worth it when the logic is complex and project is huge. For me that looks just about the same as "My current suggestion is to map the use case and data/control flow of the requirement being tested." from the original question!)
4. Do not neglect testing environment preparation or at least planning at this stage: this will help to avoid surprises later and save hours and even days of precious time at release week (besides, for me that was THAT project where true understanding of "hour lost at project start is that very same hour you so terribly miss at release date" came).

Perhaps the above is a bit broader than expected from the OP, but I hope some ideas will be helpful for anyone else. Sure thing actual approach at early stages with QA involvement depends on the project / team / complexity / release calendar, etc. - and therefore "test cases before the application" approach is doubtful for projects of small size / simple logic, but for big projects these QA efforts will make a definite overall profit when planned wisely.

Answer 2

You're absolutely right to feel that your time in the early stages is best spent understanding the problem.

Mapping the use cases and data/control flow of the requirement is one good way of building up that understanding, and in doing so, you are also testing those requirements as you will discover questions that you want to ask, gaps and ambiguities that need to be clarified.

I would suggest that you also step back from the requirements doc and spend time understanding what value the system under test is meant to provide. This can be surprisingly hard to uncover by simply looking at lists of requirements - you may need to follow up with business analysts, or other representatives of the business to understand the motivation behind the requirements. (You might find that the business isn't actually all that clear about what value they want, or that different stakeholders have wildly differing and contradictory views on that. Be diplomatic here, and bear in mind your company culture. In some companies you'll be thanked for uncovering this kind of problem, in others, not.)

Without understanding what value the system is meant to provide, however, you don't have any way of designing your tests so that they are most likely to reveal risks to that value. And that is what your testing is for, isn't it? Not just to unquestioningly confirm each requirement and go no further?

Something else that I seek to do in early stages is to gain an understanding of the technology, especially if it's new to me. The depth I go to depends on the project - some require that I build up a reasonably good technical understanding in order to identify risks, others don't. I'm not just looking for risk though when I'm doing this - I'm often trying to understand what tools might be useful to me for exploring the system. Sometimes I write some of these tools ahead of time. I also think about what test data might be useful, what test environments I might require, and so on.

All of these things are more valuable than spending time on documenting something that you don't yet understand well enough.

Answer 3

Sharing my thought as I almost asked the same question.

I find that keyword-driven test framework in a way helps the QA to start before the application is ready to be tested. Although in reality, your QA team probably has something else (products/releases already in production) to test while implementation of the current relese isn't ready yet.

The problem we face is that specs/designs keep getting minor updates, testcases can go out-of-date when first build is out. With keyword-driven framework, you can just change/adjust the library for your keywords then your testcases are golden again. This may take some practice to create keywords in a way that testcases don't need to be touched, but should shorten the time needed to get the testcases working again after some "minor changes" to the application under test.

A simple example, testing the login page before you even see the page.

Test case with keyword-driven framework:

    log in to site with ${valid credential}
    veryfiy login was successful
    log out 
    page should not contain $(element)

• You will probably also have 50 other similar testcases with invalid/empty/double bite credentials.

So your login page may get a complete revamp, all you need to do is to change the underline code for the keywords, then all your tescases should start to work again.

You can argue that well defined test subroutines will do the same thing, but I just find keyword-driven framework makes it easier and naturally done.

Answer 4

The top tasks are:

• Read and refine requirements
• Prepare test cases
• Map test cases to requirements
• Create test data
• Create test environment
• In fact try to understand the complete application in depth
• Tasks leftover from the previous sprint.

Visit Testers tasks before implementation for detailed description of each suggestion and see if it helps or not.