I’ve just spent the last 6 months replacing a couple of old frameworks one using enterprise toolsets with a shiny new appium/webdriver framework. Everything has gone well with good buy in from both the testers and management. Our test runs which previously were taking 40 hours + are now far more manageable. We are in the process of planning for the next year and determining the milestones for the Test Engineering team. Management has seen the value in reducing the run time of the nightly run, using open source tools and creating an easier to use framework, but they have asked me now to provide them with metrics or the value that will come from some of the tools/processes that I've seen implemented at other companies.
This might not be the best place to ask all of these, but I’m hoping people here might have gone through something similar. Does anyone have any way of valuing:
- Comparison reports/run history with a combined log file from multiple sources to enable testers/devs to faster diagnose failures
- Running tests more regularly/on checkin rather than just nightly, or running just the affected tests
- Reducing flaky tests or rerunning flaky tests
- Bots to automatically segment tests and raise defects
- More regular security/performance testing rather than at release
- Star increases for apps
- Code coverage stats
- Dashboards
Cheers, James