I’ve been involved in recruiting software engineers for a long time, to the point of job interviews taking up a significant part of my time. So I’ve done enough interviews to notice patterns about what works and what does not. The single best thing I changed about how we interview is to iinclude a code review as a standard part of the technical interview.

Why deal with code in an interview?

The most important factor for a developer is, unsurprisingly, that they are able to effectively work with code. What’s less obvious is that there are a lot of people who nominally have the right sort of experience and background by e.g. having completed a degree in computer science, but are hopelessly lost doing actual software development. Others have a different tech background than what you are using,so it might go either way – they might have to relearn their whole way of developing, or just transfer their skills to a new syntax. And the single most important thing in an interview is to not just talk about projects and tech, but get down to actually using it.

What’s the alternatives?

Asking questions about technology often comes down to a quiz about random trivia. But whether a candidate knows a particular library or function by heart is irrelevant, since in a real work situation you could always just look it up. Besides, it’s too easy to mistake “candidate happens to have worked with the same tools as I have, so knows the same details I do” with real, generalized skills.

A lot of companies do whiteboard coding, i.e giving candidates a programming challenge in the interview, and having them write the code to solve it on a whiteboard during the interview. Which is a nice test of stress resistance, but nothing else. Nobody does any real programming without an IDE (or at least a decent text editor) and access to language or library docs. You can improve by providing a laptop having all that preinstalled – but the most fundamental problem with coding tests is that the 30 minutes or so allotted to them is way too little to understand a non-trivial problem, find a solution to it, code it, test it, and bring it into production-ready shape. At best, you are testing how quickly they can whip up a first prototype. At worst, you are only testing if they can come up with the right algorithm under time pressure.

So you might instead give candidates a more significant programming exercise as a take-home exam, to complete on their own time. Which is a reasonable test for programming aptitude – except you don’t know whose, since they might have had someone else solve it for them. Or brute-forced something that’s really too hard for them by spending a whole weekend on something that you’d expect a competent candidate to finish in two or three hours. And those competent candidates will either not bother to do several hours of free work for you, or by the time they get around to it have several competing offers on the table.

A variant on this that seems to work is to take a small but real development task from a real project, have prospective candidates do it in their own time, and pay them for the time spent. This way you get a real work sample, the candidate does not have to work without pay, and both sides experience what it would be like to work with each other. I’d be a little concerned about not being able to compare candidates, since each candidate gets a different task. But if you are like any other software company, you’d probably like to hire as many good developers as you can get your hands on, not just fill a single position. So “How easy is it to work with him? Can he do the job?” are much more interesting questions to answer than “Is candidate x better than candidate y?”. The problem, and the reason we do not do this, is that you need to come up with enough development tasks that are suitable. They need to be small, not time-critical, relatively isolated from other tasks, and require neither significant setup time nor product- or project-specific knowledge – to find all those criteria in a single task basically never happens if you’re doing custom development.

Code reviews to the rescue

So for us doing a code review, by presenting candidates with a medium-sized piece of pre-written code, and asking them to review it, has turned out to be the best way to evaluate technical skills. It does not rely on google-able memorization, and is flexible in how much time you spend on it. And since it is a task that most developers have probably done in their job before, it actually measures real-world job performance rather than training for something interview-specific.

It took a few tries to get it right – so next time I’ll share some of what we’ve learnt about how to do a good code review in a job interview.