16

A junior automation developer I work with who has around two years of experience interviewed for a QA position. He was asked to write some basic programs for whiteboard exercises like checking if a number is prime, determining if a string is a palindrome, etc. That was the first technical interview, so he was surprised that he was being asked technical questions that seemed more relevant to application development despite applying to be a tester.

I don't understand why people conduct automation tester interviews as though they were interviewing application developers. They are two different roles. My question is, is it considered a good practice to conduct developer & automation tester interviews in the same way? If not, then how should an automation tester be interviewed?

Improve this question
5
  • 34
    Automation testers are developers. They develop software which tests another software.
    – Embedded
    Commented May 23, 2018 at 9:01
  • 6
    Whilst we're on the subject of not understanding interview processes, I can't fathom how anyone thinks that the ability to write a function to determine if a word is a palindrome or if a number is prime is a good measure of someone's ability to solve practical problems and write good quality code that is understandable and maintainable. It's like showing a brick to an architect and asking him to tell you the percentage of silica it contains.
    – ESR
    Commented May 24, 2018 at 5:10
  • 2
    @EdmundReed I believe that the purpose of this is to exclude candidates who can't write code, rather than to find good coders. It's like showing a brick to an architect and asking them to tell you what it is. If they don't say "it's a brick" then you know they're lying about being an architect. You would be surprised how many people say that they're programmers, but can't write a simple function. Luckily there aren't as many these days, now that the "dot com" hype has long gone, but they still exist; hence the need for silly programming interview questions :-)
    – Aaron F
    Commented May 25, 2018 at 8:37
  • @AaronF the way I see it, it's like trying to hire a writer/content designer, and testing their writing ability by getting them to write a palindrome that contains an example of the superlative case. It's just not a good measure of whether they will be good at what you will ultimately be paying them to do.
    – ESR
    Commented May 27, 2018 at 7:44
  • @ESR - One can ask programming questions, but must ask other types of questions also. Ask them to write tests for their programming problem. Maybe develop the code with TDD. Then, ask them about designing software. If they can't even do simple programming, even when given plenty of time and cordial interview, then how can they do more complex programming at work ? IMO, it should be a wholesome approach.
    – MasterJoe
    Commented May 4, 2020 at 3:25

6 Answers 6

Reset to default
29

I have had about 4 QA automation jobs, about 10 interviews, and was on the other side of the interview table about 10 times.

QA Automation involves writing automated testing projects. These are software projects which can be very complex, often requiring:

  • Managing deployments of applications
  • Starting and stopping application servers
  • Setting up test data, and clearing directories and files after
  • Catering for various operating systems
  • Gathering, formatting, displaying results
  • Interacting with the app in various, often complex, ways
  • Requests, Responses, Selenium, Page Objects, Use cases, Story/feature files
  • A sound structure, following best practices, efficient use of design patterns
  • Project should be easy to follow, maintainable, flexible

With this in mind, if someone is applying for a QA automation role, then they need software skills. If they are going to be working on the framework extensively, or creating a new one, then they should have very good programming skills, understand build tools, software architecture, design patterns, and how to use the features of the language very well.

I've often seen people with insufficient coding skills getting hired. As a result, the quality of the automation project suffers greatly, becomes flaky and hard to maintain - and loses its value.

To finish, I think that a QA automation interview should involve programming questions to demonstrate that the applicant does have the necessary coding skills. In addition, there should be QA specific questions, such as:

  • Explaining different test types and give their experience with each
  • Have they had issues communicating with unwelcoming developers and how to deal with that
  • How would they test a certain application (to see that they are methodical and thorough)
  • etc...
Improve this answer
4
  • 9
    I've often seen people with insufficient coding skills getting hired. As a result, the quality of the automation project suffers greatly, becomes flaky and hard to maintain - and looses it's value. I wish I could upvote this answer 1000 times. I just started a new job where this is the current status of their automation framework. The devs tried to take it over and fix it but they gave up because it was too complex.
    – tehbeardedone
    Commented May 23, 2018 at 16:00
  • 2
    While most of this answer is completely true, judging someone by how well they can perform some task off the top of their head without outside assistance is a terrible test of skill. I'll never be able to identify prime numbers without access to the appropriate documentation but let me decide how the test automation should be done and you'll get a logical, easily understood framework which is as easy to maintain as possible...the concept also applies for full-on devs if you ask me.
    – Cronax
    Commented May 24, 2018 at 14:10
  • @Cronax - Then choose better problems to ask. Ask them to write tests for the problem.
    – MasterJoe
    Commented May 4, 2020 at 3:29
  • +1 to this answer. We had to process the JSON of a tree like structure. Generally, the data we want is at 3rd level in the tree. Rarely, it can be at the 4th or 5th level. I saw automation which had 3, 4 & 5 level nested loops to handle this. If not found at level 3, try 4th & so on. All this extra, confusing code could be replaced with recursive code that could find the required data no matter which level it was located in. Such bad coders actually exist and you'll find them in well known companies. Resume fraud combined with bad interview process are often responsible.
    – MasterJoe
    Commented May 4, 2020 at 3:32
6

What's going on here is that the company has decided that application engineers and automation engineers are both programmers and as such both roles need to know programming principles.

Notice for instance the tasks you list such as Palindrome, Prime Number, etc. are also programming tasks that the developers being hired will likely not actually do in their potential job either, but the test is intended to see how they approach common (often easy problems). Do they write code first?, tests first?, do they use red-green-refactor, etc.

So that's why the companies are doing it...
As to whether that is a good idea, approach, etc:

I think there is some value to this approach however I see this as useful information but I do not like to use these tests as the basis of simple gating, pass/fail testing. I prefer to add this information to interviewing in person/video which is where I learn the most about a person and whether they will fit in with my organization. I have found that many people can learn specific technical requirements on the job as needed, so I try to gain an understanding of their soft skills and how they approach learning rather than just what specific technical skills they have right now.

Improve this answer
5
  • +1 for "I do not like to use these tests as the basis of simple gating, pass/fail testing. Smart people can learn these things as needed."
    – Nitin Rastogi
    Commented May 23, 2018 at 13:26
  • 4
    @NitinRastogi - sadly this is not true: There are plenty of smart people who cannot learn these things - i.e., coding - as needed. For a simple proof, look at cseducators.stackexchange.com where a lot of professional computer science educators who have found it difficult to teach smart people how to code hang out and assist each other trying to figure it out.
    – davidbak
    Commented May 23, 2018 at 20:13
  • 1
    You could select developers on the same premise: take anyone from the street, pair him or her with senior dev and give some time to learn development. It might take time. But who is going to pay for that the time, when newcomer is mistakes and learning from those mistakes, and his mentor is teaching him instead of doing his job?
    – dzieciou
    Commented May 24, 2018 at 9:05
  • @davidbak and dzieciou - good points. I have updated my answer to explain better
    – Michael Durrant
    Commented May 24, 2018 at 10:37
  • @MichaelDurrant - There is no way one could do tdd, refactor etc. when all you have is HackerRank, Coderpad.io instead of a modern IDE like Intellij IDEA. Even harder when you have 1 hour and interview anxiety. The anxiety is mainly the problem of the interviewee, but it is an obstacle.
    – MasterJoe
    Commented May 4, 2020 at 3:39
5

I once wrote a blog post on why it is hard to find technical testers who can program, and why candidates fail such technical interviews. We needed testers with developers' skills for two reasons:

  1. Writing maintainable tests and test tools requires engineering skills. I have interviewed testers who claimed they can program, because they wrote Selenium scripts, but when tasked with writing simple loops and if/then clauses, they failed.

    Useful libraries require stronger design principles than just avoiding the repetition of code.[…] We have reviewed test suites built with hodge-podge libraries on several occasions. The results are never pretty.

  2. In Agile teams, the responsibilities of testers and developers often overlap and same task can be performed by both testers and developers. Isolating root causes of bugs is one of those tasks, and understanding system architecture by the tester may be useful here. During interviews I asked many testers whether they know how their previous system works, what's its architecture, etc. Many said it wasn't necessary: it was the responsibility of developers to understand that. They have never paired with devs on troubleshooting bugs. Testing is often a collaborative task, where each part can contribute with their skills, but both need to understand some details of the work done by the other part. It helps.

    In most of the organizations I’ve observed, I believe the testers should have been doing more bug isolation, especially for severe bugs.

All quotes come from "Lessons Learned in Software Testing: A Context-Driven Approach" and "How to Make your Bugs Lonely: Tips on Bug Isolation"

Improve this answer
4

What you call "automation testers" and "developers" are both the developers. Both of those two classes should understand basic concepts of the language they are going to use (what is often referred as core), both should think algorithmically effective and both should be aware of most popular architectural patterns.

The difference between "automation tester" and "developer" is that the developers usually work on more complex systems which introduce more complex problems and hence "developers" should be more deeply involved in algorithmic issues (the tasks you mentioned like a palindrome, etc, should be a piece of cake for both).

Since "developers" usually involved in a more complex systems development they have to know much more (nearly dozens) frameworks and libraries then "automation testers" have to know (which is often limited by Selenium, Cucumber and some libraries for making REST calls).

Thus, there is nothing special in asking a candidate some basic algorithmic questions and core concepts of the language they are supposed to use in automation.

Improve this answer
3

also to add all above, the trend is shifting test to left which means testers /test involving earlier in development pace. Due this testers specially automation testers need to have knowledge of programming.

Improve this answer
1
  • Good point, because lower level tests are more technical.
    – dzieciou
    Commented May 26, 2018 at 8:53
-2

Basic logic understanding is expected from everyone directly associated with software development, so it's necessary that these questions should be asked as a preliminary filter to Dev, QA and managers as well.

Also asking a basic programming question does not necessarily mean, that somebody has to code on day to day basis but to make sure that one meets the bottom line and can think logically and contribute in solving(or at least understanding) technical/domain problems along with the team.

I take these interviews lot of times, and ask these questions frequently without stressing on concrete code but to understand their approach how they understand & try to solve problems logically.

I think any reasonable interviewer will be fine even with just the pseudo code or high level algorithm(in plain English) provided by the candidate if it is logically correct without accompaniying any actual code.

Improve this answer
1
  • 1
    A scrum master, manual QA, project manager, and even CEO of a software development company are strongly associated with software development - yet there's typically no need for them to know basic programming. They need to know about the higher levels of abstraction. Also you're not answering either question in the original post.
    – DtotheK
    Commented May 24, 2018 at 10:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.