I'm trying to hire a QA engineer to come in and relieve some technical debt on our test suite. I'm having a hard time testing their knowledge of ruby and testing scripts in ruby.
Any suggestions on a good problem to give the candidates?
I'm not the most experienced guy when it comes to interviews, but this is my opinion on the subject.
I like to separate personal interviews and assignments.
For me personal interviews are mainly about two things:
The first one is quite obvious. Is the person a good fit to the team? Is he/she willing to learn? Will he/she enjoy the job?
The second one might be a bit tricky. Normally asking something that would require them to think. Something not necessarily tech related. Maybe start off with simple math problems, or anything that the person is capable of solving with his/her background. The point is to see how the person would solve a new problem. Something they've possibly never met before.
Of course for a senior role there should be also some questions about the technologies that are in use. But I don't really see the point of asking really specific programming questions throughout the whole interview.
For the assignment you can give something similar that will be in use for their actual work. For example if you're looking for someone who will mostly work on UI testing, ask them to create a dummy application to test and create a UI test framework for that. Specify the technologies that should be used and maybe a few requirements. If you specify the requirements on the application, you can already see how they write test cases for example.
After submitted, talk with them about their solution, ask them why they chose to do it the way they did. What could be improved upon, etc.. From this you can tell much more about the technical knowledge of people.
I suggest to give a test requirement and ask for automate it. Then the candidate should have to explain the how and why he/she have made every decision. Pros and cons should be described.