What points should be covered in an Automation POC?

Question

Background : We are planning to automate our web application, for that our manager asked for the Proof of Concept (POC).

Overview : After studying and analyzing all factors of application, our team decided to choose Selenium WebDriver using Java as we have hands-on experience. We have designed one complete Framework structure using POM, TestNG, Maven, BitBucket, reports and all for Login module.

Problem : We haven't conducted any automation proof of concepts before, so we do not have any idea what should we write in POC document that seems good document and the manager is able to understand. I have browsed through the internet but different information creating confusion.

Can someone suggest some good points that we can use to improve our POC?

Answer 1

Your POC report is destined for the manager who wants to know the outcome of your investigation. More importantly, he'll have to decide whether to continue with the POC results, do another one (with different technology, for example) or stop investing in test automation at all.

So, try to make your POC report as informative and comprehensive as possible. When looking at it from this angle, you might want to address points like the following. Don't forget, they are estimations. Also, honesty is key, so list your negative experiences as well as the positive aspects.

• Effort to create a fully functional framework and pipeline?
• Effort and possibility to integrate in continuous integration cycle?
• How long does it take to automate one regression test case (and by extension the entire range of existing regression tests)?
• How much work will be needed to keep tests up to date?
• How much time will you save once the regression suite has been automated?
• How easy is it to learn (for technical and/or non-technical staff)?
• Who is owner of the framework and tests? Do your testers need help from the developers for everything they do, or the contrary?
• How do you see quality in general improve by using this?
• What are the (dis)advantages of the chosen technology/language?
• Are there alternatives your team really thinks are worth investigating?

Answer 2

1. The first and foremost target of automation testing is increasing the productivity in long run with different utilities. So the main point needs to be added is the forecast productivity. How the ROI( return on investment) is going to be increased release to release as the test case number will increase. A rough estimation you can mention in your doc.
2. Some open source utilities you can introduce for webservice testing through mobile. A good light weight webservice testing tool is ResTer -Rest API Tester. Open source utilities always attracts project needs.
3. Also take some prior reference from developers if possible.

Answer 3

All managers are interested in results. And your POC should influence your manager by showing some positive results. Some of the results your manager will be interested in will be:

a) Code Coverage - How much of the application can be automated with this framework? 100% automation is next to impossible. I'm guessing 70-80% is a good figure.

b) CI Integration - Integration with CI tools like Jenkins etc. to promote rapid build deployment and testing.

c) Time saving - How much time will it really save. Will it be worth the time and effort taken to develop it?

d) Re-usability: Can you re-use the whole code or part of it for testing other web applications /services?

e) Robustness: How robust and efficient is your framework? Does it run without fail each time it is initiated? Does it give out false positives and negatives?

f) Scalability: The ability of your framework to support new features and system integration over time.

g) Resources and Cost: Ideally your company would be having dedicated machines and servers for automation. How many such machines and servers need to be setup? What's the cost of maintaining those machines and servers?

That and many others. But the ones mentioned above sum it up at the most. That and a lot of other factors.