I am working on a project to automate legacy code, and I'm looking for a good metric to track my team's progress and estimate how long it will take to complete the project.
My thought was to take all the automatable test cases we have identified, and use the percentage of these that have been automated as the main way to report progress. I also thought that since many of our automatable test cases are about the same size, we could use an average time to automate each test case to give us some idea how big the project is and how much time we will need to complete it.
The problem with this idea is that many of the test cases have multiple parameters. I don't want the metrics I use to measure progress and estimate time to give the wrong idea of how many legacy features are left to be automated or how much time we will need to finish the project.
For instance, assuming my project is a custom browser that supports multiple search engines. Should the search feature using each of these engines be considered a separate 'automatable' test case as from automation perspective each of these is just a parameter of may be a single test method?
So if in the test automation code I have a parameterized test, for tracking purposes of an Agile project, should I consider it to be many test cases as there are search engines?
What if I am just counting test cases and not automatable cases, then should I treat separately?
Also, should my test cases account for implementation differences? Let's say my app supports my browser and Chrome, should I account for Chrome test cases separately?
If there is an industry standard practice here, I would like to follow it.
My main concern is that I choose metrics that will give me results that are not distorted by being too large or too small.