6

I've managed tests for a project that I was with since its infancy. I used Microsoft Test Manager to tie tests to requirements and organize them in folders. It went well.

However, now I'm being tasked with organizing tests for a program that has been in production for a decade and is much more complex. The regression test we have is just a list of almost 7,000 steps in an Excel sheet, not even broken into tests. I'm looking for any advice on how to manage this project. I want to organize test cases without wasting a lot of time. Thoughts?

Something else to consider is that I've also been tasked with automating the majority of these tests (using TestComplete). While organizing the existing regression tests will be faster than automating them, I'm considering that automation will naturally result in organization. So should I propose to focus on well-documented automation and not have many manual regression tests?

2
  • 4
    How do you know that the 7000 steps are valid or useful tests? Commented Feb 21, 2014 at 23:08
  • 2
    7000 steps in an excel spreadsheet? Are they atomic? Or does the order with which you execute them matter.
    – corsiKa
    Commented Mar 7, 2014 at 16:21

3 Answers 3

6

As the other answers have said, your first task is to organize the list. There are a number of reasons you want to do this:

  • The items may not be accurate - This is ridiculously common. What tends to happen is that the people who execute the test steps know enough about the system to adjust for flaws in the list and act accordingly.
  • The items may not be relevant - This is another common problem with long manual regression tests. They tend not to be updated, so they'll specify actions or features that the application no longer supports.
  • There's a lot of repetition - Chances are a document like your 7k item spreadsheet grew organically, and the lack of organization in it means it's a serious pain to try to find out if it already has a test for some condition that's needed: so the test gets added. It's also probably never been reorganized, so there's going to be a whole lot of similar actions in there that could be handled more effectively as a grouping.
  • It's the easiest way to figure out what's in it - I'm speaking from experience here: with something like this, the easiest way to work out what's there is to go through it and organize and group it.
  • The tests may not be suited for automation - Good candidates for automation and existing manual regression test cases are not the same. There may be very little overlap (and that's a separate problem).

It's going to be a major time-sink no matter what you do, but organizing the list of test cases will give you a feel for the problem areas of the application (because I guarantee you your spreadsheet will have grown by adding reported bugs to the list) and the areas where automated regression should be best targeted. It will also allow you to identify redundancies and tests that are no longer relevant or valid.

The structure you use to organize the tests can then be echoed in your automation, but you'll start that project with a much clearer idea of what's needed - which will allow you to target your automation projects much more cleanly.

TestComplete does support manual test suites if you wish to use that. If the project team has a life cycle management tool in place, I'd use that (whether it be Team Foundation Server with Microsoft Test Manager, QAComplete with Test Complete, HP ALM or a home-grown setup) unless they use spreadsheets... If they don't have one in place, you might want to consider something like TestLink as your organizing tool (to be honest, even a set of smaller spreadsheets on network shares would be better than what you have right now). Regardless of the tool you choose, you're still going to want to get that list tamed before you start automating.

4
  • Are there any tips to organize tests such that they are easier to search and find in a test management tool like (the clunky) HP ALM ?
    – JohnSink
    Commented Jul 18, 2017 at 20:13
  • 1
    Where you can group them into heirarchies, do it. If tags are available, use them. Add key information to titles, so that your titles include your likely search keyword (I use things like Application - Module - to start my titles, e.g. "Store - Cart - Remove $qty $product reduces total by $qty*$product"
    – Kate Paulk
    Commented Jul 19, 2017 at 11:43
  • What if we have api tests ? Long descriptive names are nice in ALM, but can be a problem when you want to check in automated tests into your local windows.
    – JohnSink
    Commented Jul 24, 2017 at 18:50
  • 1
    @JohnSink it helps to figure out your test case naming convention and use that: ShoppingAPI - MethodName - IsOK gives you plenty of information in a relatively short title, as does ShoppingAPI - MethodName - InsufficientQty.
    – Kate Paulk
    Commented Jul 24, 2017 at 18:59
1

If the tests are all in one big Excel file importing them into a tool is going to be basically impossible so you should expect to to it manually (copy/paste).

Most test case management tools have a hierarchy setup of some sort that will allow you to organize a huge number of tests in a way that makes them manageable. You will just need to determine what sort of structure you want. This is typically done by splitting them into test suites based on high level features.

Testlink is a popular open source test case management tool that is easy to use and allows nesting of test suites inside one another.

Regardless of whether automation happens or not I'd organize the test cases first.

1

7000 steps in single excel sheet without any hierarchy, itself pretty difficult to maintain . I don’t intend to comment on competency of team which developed it, but practically it is difficult to keep it updated, and validated. So you need to propose organization to put additional investment. Basically unorganized test cases is one kind of technical debt and it needs to be paid of before it gets converted into land mine for you some time in future.

If project has enough life to justify additional investment, you should put additional team for separating out steps to cases and cases to folder structure. The folder structure can be of your choice of division say for example feature wise or functionality wise .

If you can identify some repetitive pattern in steps, think of writing a script to separate test steps into test cases. Once testcases are ready, quite some effort aka expert members bandwidth needs to be budgeted for review. You can choose excel, xml or for that matter word also to complete this exercise.

Once testcases are ready , it should be pretty easy to import them in test management tool of your choice. Safe approach is to use management tool which is already in use within organization, unless it is known that it is not scalable enough to handle these testcases.

Regarding Automation, Although test can not ever be non-sapient, you need to consider and evaluate return on investment see what all parts can be automated. Remember maintaining automation itself can be costly if team is not competent enough in automating. If ROI is justified, go ahead propose automation for the portions which are feasible to automate.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.