Feasibility of automation testing

Question

We are currently working on a new web site and I've been asked to investigate automation and make recommendations. Right now, I'm leaning heavily towards Selenium+Web Driver for our testing. But, I'm trying to figure out the resources we need.

I believe we need to bring in an experienced automation developer to get our infrastructure in and then we need at least one dedicated person to do nothing but maintain and improve the automated testing. I'm concerned that I won't get a dedicated resource and they would eventually need to work on manual QA testing because of some emergency and eventually would become a manual tester and automation would fall by the wayside.

Have you seen QA developers split between automation and manual testing work well? Were there any special things that were done to help ensure the automation tests were kept up?

Answer 1

Selenium/WebDriver will do just fine. WebDriver is W3C standard for browser automation.

Automation of testing is on a spectrum, from almost fully manual to 100% automated.

Test automation is manual testing aided with some automation. Preferably close to 100% but you will start with less than 100% :-)

Automated tests are full suites of tests run automatically - continuous integration style. It is not easy to get there, but you don't have to. Start with automation of the most repetitive tasks in testing.

Split between manual and automated testing does not work well, but reason is that type of personality who are good at manual testing does not make (in my personal experience) good developers, and vice versa. Good developer looks all the time what process is repetitive and can be refactored and automated. Good manual tester looks what is different and suspicious.

Also, automated test is wrong name for the process. It should be "automated checks". Because automated tests can (and do) only check for conditions expected by test engineer, but will blatantly ignore anything unusual, if checking that unusual condition was not coded by test engineer. In comparison, unusual stuff will be picked by any decent manual tester.

So start with (some) test automation, and grow from there. But even before that, you need strong automated suite of unit tests (written and maintained by developers). System level test (e2e) without extensive unit test covering most of the code make little sense. Unit tests are cheaper to write and closer to errors.

Answer 2

I'd like to also add to what Peter said above. Automating UI interaction, in this case using Selenium, can eat up quite a bit of time maintaining tests. If your team really wants to invest in automation, first you need an agreement that development will have a focus on making their product testable and expose elements in a manner that is easy to automate against. What does this mean? Using unique identifiers in the HTML for example that will never change and that are human-readable. If every time a new version of your site is released for test and ids are changed, no matter how well you've written your tests you will need to spend time to update them before they provide value once again.

Before writing automated tests that drive a browser, dev needs to be aware of what they need to do in order to facilitate writing new tests and maintaining those that already exist.