2

We have a role which requires an automated tester. Mainly this will take the form of writing automating tests using Selenium Webdriver written in Java.

I am trying to develop a technical test ensure the candidates

  1. Can read and understand code
  2. Can write code using good practice

I want the test to be balanced toward a automated tester (as opposed to a developer). Part 1 I have accomplished by writing a (complicated) test myself, printing out the source code and the candidate has to follow the steps on a browser and telling me if the test passes or if any of the asserts fail.

How would you recommend I accomplish part 2? Ideally I would benefit from scenarios and what skills you think they highlight in the candidate.

2

For second part I can suggest a way based on what we have used in our organization

You could give a example of a simple web application i.e - Simple form -> Thank You Page ..

Then ask to design and develop a automation script for the given scenario . Then you can see whether he/she is using Page Object model for the design and also coding standards are followed or NOT

  • setting up a simple form page is simple enough in advance. However setting up an IDE and designing making the base of a suite could take a bit long for an interview. I am not sure its realistic (or fair) to have a technical test that takes hours to accomplish. How long would you envision giving the candidate to accomplish the task? – ECiurleo Nov 11 '15 at 11:30
  • 1
    Usually we have 90 mins practicals test for all dev positions , test automation follows the same thing . They are allowed refer internet as well , I think 90 mins would be enough cover two page web application with happy path /primary work flow automated , Below are the things we are expecting 01) Page Object Pattern is used | 02) At least Primary work flow is automated properly (with relevant Assertions ..etc ) | 03) Common coding standards are followed in Page classes as well as in Test Fixtures – DasunB Nov 13 '15 at 4:46
1

Since a candidate experienced in selenium web driver should be familiar with frameworks, show an already built simple framework and ask him/her to explain it and see how well he/she can articulate the selenium webdriver concepts and java concepts.

  • I quite like that idea, but how much extra would that bring me that what I have gained from scenario 1? Do you think that 1 and 2 could in fact be combined? – ECiurleo Nov 11 '15 at 13:18
  • By "1 and 2 combined" do you mean the two points that you have mentioned in point form in your question? – ilm Nov 11 '15 at 14:28
  • Yes, 1 and 2 as defined in the question – ECiurleo Nov 12 '15 at 6:22
  • I think it should not be combined. As for testing the candidate on good coding practice, I suppose as ou_ryperd suggested, here are manual steps - automate them would be a good approach. – ilm Nov 12 '15 at 11:08
0

Give the candidate a one hour practical test. Ask for as many test points / validations as possible. You will know if you like what you see. Setup shouldn't be hard, because you don't need to develop an entire framework to run a few tests that output pass / fail to the console.

If Selenium is hard to get going with in an hour, let them use HtmlUnit - it is in Java, and whether you assert or test, it's the same. You want to prove that the candidate can implement the principle.

I had a case where a candidate spent the entire hour trying to write scaffolding code and didn't test anything. That is a tell tale sign.

A goal could be: do 20 equivalence class tests on this edit box; or: here are manual steps - automate them. Those have a clear pass/fail outcome.You have to decide what you want in a candidate and test for that. Knowing Selenium doesn't imply knowing testing. I am wary of tool jockeys. I employ people who fundamnentally understand what they are doing and the tool is just a tool of of a certain fit-for-purposeness. For that case HtmlUnit, Splinter, Selenium, WATiR etc. all essentially do the same if you understand the problem.

An automator often has to fix tests that don't work any more. Give the candidate a broken test (in your harness/framework, whatever) and get her to fix the test. Or let the candidate create a simple test and export it to Java (it's been a while), then parameterise with data from a csv file.

  • Sometimes good foundations are the most important part ;) So, to clarify, what do you want the candidate to accomplish? I tend to find offering an open question will not give the candidate enough direction. I don't want to prove they can't accomplish a task I want them to have a clear question/scenario to achieve. If the candidate is familiar with Selenium (why they are in the interview) what benefit would it be to ask them to use something they are unfamiliar with? – ECiurleo Nov 11 '15 at 12:05
  • I offered HtmlUnit as alternative because of what you said: "setting up an IDE and designing making the base of a suite could take a bit long for an interview" – ou_ryperd Nov 11 '15 at 12:47
  • A goal could be: do 20 equivalence class tests on this edit box; or: here are manual steps - automate them. Those have a clear pass/fail outcome.You have to decide what you want in a candidate and test for that. Knowing Selenium doesn't imply knowing testing. I am wary of tool jockeys. I employ people who fundamnentally understand what they are doing and the tool is just a tool of of a certain fit-for-purposeness. For that case HtmlUnit, Splinter, Selenium, WATiR etc. all essentially do the same if you understand the problem. – ou_ryperd Nov 11 '15 at 12:57
  • but as per the question, this is for the technical part of the interview. Ie can they actually use the tool effectively or are they just Saying they can. – ECiurleo Nov 11 '15 at 13:20
  • An automator often has to fix tests that don't work any more. Give the candidate a broken test (in your harness/framework, whatever) and get her to fix the test. Or let the candidate create a simple test and export it to Java (it's been a while), then parameterise with data from a csv file. <shrugs> – ou_ryperd Nov 11 '15 at 13:33
0

Thanks for everyone's input.

I have created a test scenario which has a number of areas requiring resolution. I have made it freely available on GitHub so it can benefit others.

Some of the areas it helps assess the candidate are;

  1. Did they follow the guidelines set out in the documentation
  2. Are the test errors appropriate to explain the cause of the failure(s)
  3. Do they write appropriate test descriptions
  4. Can they make appropriate judgments on the validity of tests which "pass"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.