Approach for tracking automated test coverage:
- Testers first write manual test cases. Each test case template has standard columns.
Add two more columns::
Automated? (with Yes and No options)
A sub field AutomatedScriptID appears, if Yes is selected in above field)
Revisited? (with Yes, No and Not Applicable options). Default value is Not Applicable.
Once manual test cases have been created and complete requirement coverage has been ensured, automation testers start updating the manual test case sheet and select the value for Automated? field for which the default value is No.
Once Yes is selected in Automated? field, automation tester has to mention the automated script id in AutomatedScriptID field.
Now, as the testers test the functionality, sometimes test cases fail because of bugs. Test team mention the BugId in Comments column (which is part of every standard testing template) and fail the test case. And also sets the Revisited? column to No.
This Revisited? column will be updated by automation testers only. Even if the manual test case passed.
Now even if all the test cases have passed during manual testing. It can clearly be identified that there are test cases that need to be revisited by the automation testers as they might have to update automation scripts because of the code changes made in order to fix the bugs reported for those test case.
Once the automation testers revisits a test case that had associated bugs, he checks the corresponding AutomatedScriptId and analyze it if any change in the script is required on not. And then he takes the appropriate action. After that, the Revisited? column is set to Yes; which means that the corresponding automation script is up to date now.
Charts are created on the first sheet to show:
A. What all test cases have been covered/not covered/partially covered by automation testing?
B. How many test cases are pending to be revisited by the automation testers?
Note: Using this data, one can create more charts as per your project needs.
- At any point in time, the team is aware of the status of complete coverage of test cases in manual scripts and complete tracking is taken care of.
If using excel, automation tester has to be informed that a particular manual test case have failed because of bugs. But, if the team moves to some tool, this is taken care of as the tools sends a notification to the team as soon as any new bug is created or test case fails. So, all the stakeholders are updated. Some of our testing teams are using Jira and they are fine with it.