The startup I work for is looking into outsourcing our manual testing to an unaffiliated testing lab called Applause (formerly uTest). Right now the Operations team is doing all the manual labor, and none of them have training as testers.
Applause offered us a one-month trial period, after which we would enter into a 12-month contract provided they meet a set of conditions agreed upon by both parties. What criteria should I ask for that would be fair to both parties? I feel as though I cannot ask for measurements like "x amount of bugs".
Here are some example criteria that Applause has used in the past:
- Correctly identify the severity of defects (to within 80% accuracy) which they provided
- Issues logged as always occurring are in fact reproducible using the steps, and environment provided 80+ %
- Issues contain appropriate level of detail and information (i.e. log files, screen shots, etc) required 80+ %
- Test cases and environments covered and completed within the time requested