We use Meliora Testlab to get test results from automated tests which are run on Selenium. We use the requirements module to track the user stories. For manual tests this works great - I need to see how user stories in each sprint are tested, but now that we are planning to replace some of the manual test cases with automated ones, I'm wondering how I should interpret the results. Should I handle the automated and manual tests cases as equal? Or is it just easier to start using two different reports. I suppose we will not get so much failing automated tests as manual ones, so what will I learn?
Add a comment
|
1 Answer
I suppose we will not get so much failing automated tests as manual ones, so what will I learn?
Automated tests are not there to detect new bugs. They serve the purpose of certifying (checking ) the intended functionality is working as per the verification rules incorporated in the test suite.
Manual testing is actually intelligent testing and can find more bugs going beyond what is written in test case descriptions. Automated cases save effort/resources when you are repeating in every sprint faithfully doing the job of checking every time.
In way think automation more from point of view of a quality gate. Of course one can keep on investing and improving this quality gate by learning from failed manual case and incorporating the new rule of verification in all cases.
I'm wondering how I should interpret the results. Should I handle the automated and manual tests cases as equal? Or is it just easier to start using two different reports.
Since the purpose of manual (intelligent) testing to discover bugs and automated testing to do the checking they should be treated differently. It's not only easier to use two different reports but its useful while doing root cause analysis for understanding and planning focus areas and improvements.
