What is a good automated tester interview technical test scenario?

Question

We have a role which requires an automated tester. Mainly this will take the form of writing automating tests using Selenium Webdriver written in Java.

I am trying to develop a technical test ensure the candidates

1. Can read and understand code
2. Can write code using good practice

I want the test to be balanced toward a automated tester (as opposed to a developer). Part 1 I have accomplished by writing a (complicated) test myself, printing out the source code and the candidate has to follow the steps on a browser and telling me if the test passes or if any of the asserts fail.

How would you recommend I accomplish part 2? Ideally I would benefit from scenarios and what skills you think they highlight in the candidate.

Answer 1

For second part I can suggest a way based on what we have used in our organization

You could give a example of a simple web application i.e - Simple form -> Thank You Page ..

Then ask to design and develop a automation script for the given scenario . Then you can see whether he/she is using Page Object model for the design and also coding standards are followed or NOT

Answer 2

Since a candidate experienced in selenium web driver should be familiar with frameworks, show an already built simple framework and ask him/her to explain it and see how well he/she can articulate the selenium webdriver concepts and java concepts.

Answer 3

Give the candidate a one hour practical test. Ask for as many test points / validations as possible. You will know if you like what you see. Setup shouldn't be hard, because you don't need to develop an entire framework to run a few tests that output pass / fail to the console.

If Selenium is hard to get going with in an hour, let them use HtmlUnit - it is in Java, and whether you assert or test, it's the same. You want to prove that the candidate can implement the principle.

I had a case where a candidate spent the entire hour trying to write scaffolding code and didn't test anything. That is a tell tale sign.

A goal could be: do 20 equivalence class tests on this edit box; or: here are manual steps - automate them. Those have a clear pass/fail outcome.You have to decide what you want in a candidate and test for that. Knowing Selenium doesn't imply knowing testing. I am wary of tool jockeys. I employ people who fundamnentally understand what they are doing and the tool is just a tool of of a certain fit-for-purposeness. For that case HtmlUnit, Splinter, Selenium, WATiR etc. all essentially do the same if you understand the problem.

An automator often has to fix tests that don't work any more. Give the candidate a broken test (in your harness/framework, whatever) and get her to fix the test. Or let the candidate create a simple test and export it to Java (it's been a while), then parameterise with data from a csv file.

Answer 4

Thanks for everyone's input.

I have created a test scenario which has a number of areas requiring resolution. I have made it freely available on GitHub so it can benefit others.

Some of the areas it helps assess the candidate are;

1. Did they follow the guidelines set out in the documentation
2. Are the test errors appropriate to explain the cause of the failure(s)
3. Do they write appropriate test descriptions
4. Can they make appropriate judgments on the validity of tests which "pass"