Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

My question is intended to those QA shops which follow the paradigm of writing manual tests and automating them. Though I know that many QA shops don't agree to this paradigm at all.

Given the situation that you don't have access to fancy test management tool and you are automating manual tests written in spread sheet, Google Doc etc. How do you keep track of test which have been automated? Do you add a news column in spread sheet with automated, not automated status. Or any other mechanism which makes it easy for others to have a glance and find out the if a certain test case is automated or not.

Hence the objective is not to find how many test cases have been automated but to be able to quickly detect whether a certain test case is automated or not. For example if login with right user name and password does not work then how quickly can some one find whether this test case is automated or not?

share|improve this question
add comment

5 Answers

up vote 2 down vote accepted

In such situation, when the tests are written somewhere in Excel or Google docs, it will be a big headache for automation.

Sure, to track the automated/not automated tests, you could assign some IDs for manual and automated tests and then track the tests by IDs.

And of course you can mark the tests as “automated” in the spreadsheet but after some time the headache comes…

  1. The other testers will not rely on the automation and they will execute the “automated” tests manually.
  2. The automation based on manual tests will have a lot of messy code, because the “manual” structure is not always good for automation.
  3. The manual and automation tests will have their own live and after several months those two branches will be completely unsynchronized.

The way to solve this problem is making the manual tests automation driven. As an approach you could use the Specification by Example. (And here is my collection on this topic). As the tools, you could look at FitNesse, Cucumber, Concordion.

Sure, you have a huge legacy of manual tests. But why not to start with this approach for new tests and new product functionality?

share|improve this answer
    
You hit nail on the head –  Tarun Mar 13 '12 at 6:00
add comment

If you're using C# you can include test case titles, descriptions, etc. in XML in the automated test case itself, then compile with /doc. The compiler will search for all XML tags in the source code and create an XML file.

Another approach is to use meta-tags in your automated tests and use those to produce a list of automated tests.

There are several ways to accomplish this without the overhead and constant maintenance of manually updating a spreadsheet everytime you automate another manual test. The main point being don't create more work than necesary. An automated test is documentation...use it.

Updating a spreadsheet could get messy and out-of-date, esp. if more than 1 person writing automation.

Of course, you could really get fancy and compare the list of automated tests with the list of all manual tests and automatically mark which are automated on a daily basis.

share|improve this answer
    
This is exactly what I am looking for - without the overhead and constant maintenance of. Is it possible to elaborate more on your approach, probably in language agnostic manner. –  Tarun Mar 6 '12 at 17:07
    
Excellent Idea Bj! We could develop a tool that automatically searches and reads the "tags" and exports that to a excel document for tracking –  Aruna Mar 7 '12 at 0:52
add comment

One approach I've used previously, with a spreadsheet, is for each tab to cover a different execution method, i.e. one tab for manually executed tests and another for automated tests. Whilst not the most efficient way of organising your tests it is effective for a small set of tests.

Hope this helps,

Steve.

share|improve this answer
add comment

In our cases too, we add a new column with 'automated' / 'not automated' status but at the same time we setup conditional color coding for the row selection as well, so that if anyone reviewing test cases sheets, it is easily visible at a glance, which test cases are automated and which test cases are not automated.

share|improve this answer
add comment

Do you add a news column in spread sheet with automated, not automated status.

That's the approach I use.

I typically have an overall document (either Word or Excel) which lists test cases, and indicates either the document to be used when they are manual test cases, or the script to be executed when they are automated.

share|improve this answer
    
How do you pass on this information to the new testers joining the team? Do we need to pass the excel sheet/path to excel sheet each time? When you move to another project and a new resource joins the team this approach could cause issues –  Aruna Mar 13 '12 at 3:39
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.