A 10-question quiz on your technical interviewing
Is your interviewing truly world-class?
Over on our website, we’ve placed a short quiz designed to test your organization’s technical interviewing. The fact is that many companies have holes in their process, but there are some simple things you can do to plug those holes.
After completing the quiz and getting your score (go ahead, it takes less than 2 minutes), you can come back here for some commentary on the questions and what they reveal about your process.
1. How many interviews have you conducted?
There’s no substitute for reps. Interviewing is a difficult process, and no one is born a great interviewer. Much of what you learn as you become a more-skilled evaluator comes by learning from mistakes you’ve made in the past. It’s hard to become a great pianist practicing for an hour every six months, and the same is true of interviewing.
2. Do you know your false-positive (hiring a bad candidate) and false-negative (rejecting a good candidate) rates?
These two statistics are the cornerstone of evaluating any pass-fail system. It’s fairly easy to detect when you’ve hired someone you shouldn’t have, though even there it’s hard to understand how to change your process to prevent these false-positives.
It’s even harder, however, to evaluate false-negatives: candidates who would have made great employees but your process filtered them out. But it can be done.
3. Do you track candidate-evaluation consistency metrics?
Imagine you had two identical copies of a candidate, each sitting in a separate interview room. Two different interviewers, A and B, asking the same questions, interview the two copies. How similar are the results of the evaluations going to be? Maybe A’s expectations are higher, or is looking for something different with the question that B does. Unless you’ve thought about and started tracking consistency metrics, you don’t even know the answer to this consistency question. So how are candidates supposed to get a fair shake, exactly?
4. How much training have you received in evaluating technical ability?
In many tech companies, the process of becoming an interviewer consists of either asking or being told that you’re going to start interviewing candidates. Maybe you get some token training, but you’re basically left to your own devices on what you ask and how you evaluate candidates. Sound familiar?
World-class technical interviewing processes require extensive interviewer training. As I said, this stuff is hard, and no one is born a great interviewer. Not only that, but everyone needs to be on the same page on what you’re looking for and how to evaluate it. Otherwise, you’re left with a hodgepodge of opinions without much sense of what it all means.
5. Do you receive training in legal and compliance issues with interviewing and hiring?
“Hey, what prescription are those glasses anyway?” Congratulations, your interviewer may have just exposed your firm to legal liability.
Recruiting, interviewing and hiring is a legal minefield. Different jurisdictions have different laws and regulations, and these change over time. Without training, your interviewers could be exposing your firm to liability if they ask the wrong question or base their evaluations on the wrong factors.
One of the best ways to mitigate both implicit and explicit bias is to train interviewers in these issues, and to create a process that respects the candidates and the legal environment.
6. How frequently do you seek and receive feedback on your evaluations of candidates?
The best predictor of how good an interviewer you end up becoming is how much feedback on your own performance you seek out and receive. So create a process where interviewers naturally receive this sort of feedback from other interviewers. Have other people listen in or sit in. Create an interviewer-grading system.
The benefits of this are enormous: interviewers get better at interviewing, you develop consensus on what matters in your interviewing process, and you end up with better evaluations of candidates. Don’t forget: candidates are evaluating you too! Good candidates can tell which are the good places to work by seeing how they were evaluated during the process.
7. How do you decide which technical questions to ask?
8. How do you decide what constitutes a good question, and a good answer to a question?
9. Do you know if your questions appear on job-seeker websites like Glassdoor?
10. Do you keep data on question performance over time?
It’s easy to get lazy with interviewing. It’s a software developer job so just ask linked list cycle detection. Problem solved, right? Or maybe ask the most fiendishly difficult question imaginable, so that candidates are made to feel unworthy.
The point is to find good co-workers, not to make yourself feel smart. Or to evaluate whether candidates can memorize the most famous interview questions and answers. The questions you ask should target the skills you need, and the only way to do that is to come up with them yourself. Then ask your co-workers for their opinion, then calibrate them with candidates over time.
Then check if some candidate leaked your prize question online! You always need to be coming up with new questions, if only for your own sanity.
Final thoughts
For most tech companies, hiring is the most important on-going decision they make. The people who walk in the door every day make the difference between success and failure over the long term. So why aren’t you putting as much effort into finding the people who do the work as you do on the work itself? Believe me, the difference between a company with world-class hiring and an average one is like night and day. You can get better at this. It takes work, but the benefits are yours forever.
