Most technical managers find the interviewing skills of their recruiters woefully inaccurate. They contend this is largely due to their inability to accurately assess technical competency.
Most recruiters find the interviewing skills of their technical hiring managers equally inaccurate. They attribute this to a narrow focus on technical skills while missing the candidate's ability to successfully achieve consistent results doing the real tech work required including working collaboratively on project teams.
As a recruiter I found I could determine technical competency by understanding the recognition the person received as a result of his or her technical accomplishments. Due to the uncanny predictability of this approach, I was asked to speak to literally hundreds of various business groups describing the technique. A lawyer at one of these presentations suggested I must have borrowed the method directly from Sherlock Holmes. In fact, he mentioned he took a six-month course on how to conduct a deposition the same way that I explained in 15 minutes.
While the Sherlock connection was coincidental the metaphor is appropriate. The approach starts by rethinking how non-techie recruiters (or any non-technical person) can interview technical people for technical skills using the same approach Sherlock uses to gather evidence. (Here's a video lesson covering the process in step-by-step fashion.)
The investigative technique starts by asking the candidate to describe the projects he or she has been assigned 90-120 days after starting any new job. During the fact-finding I find out if it was a stretch project or an important project, who the person was working with and if the task was at or below the person's current skill level. This simple approach quickly reveals what the person's supervisor thinks of the candidate. I then ask what projects were assigned to the person as a result of his or her performance on the first project.
The degree of recognition on a technical project provides clues to the quality of the person's work and upside potential. Recognition could be in the form of a pat on the back, some special award or letter, a promotion or simply being asked to handle another critical task or being assigned to an important project team. Most often the tasks assigned reveal the person's dominant strengths or an area the candidate's manager wants to strengthen or leverage. Getting evidence like this for 3-4 jobs indicates if the person is in the top half or top quartile of his/her peer group. (Note: For customer-facing jobs, like sales, consulting or public accounting, ask about the clients the person was assigned to work with soon after joining the team. The best people in these roles are assigned to the most important and/or the most difficult clients to work with ahead of their peers.)
However, this is only half the solution. The other half is ensuring any technical interviewer, including the hiring manager, assesses the person properly. Too often these interviewers focus on depth of technical skills and the person's ability to cleverly answer some unrelated technical problem-solving question. This is how fully qualified candidates often get excluded due to improper interviewing. To address this huge problem, I use one of my favorite assessment techniques inspired by Charlie Rich's epic song, Behind Closed Doors.
How to Control the Interview Behind Closed Doors
To make sure the technical interviewer conducts a proper interview, have the candidate summarize in two or three paragraphs his or her most significant technical accomplishment most closely related to the requirements of the job.
Then have the hiring manager review the technical accomplishment for fifteen minutes at the start of the interview and reject the person outright if the accomplishment doesn't meet the requirements for technical competency. If it does, the hiring manager then needs to ask the candidate to describe one or two other major technical accomplishments related to real job needs. For example, "Now tell me about your biggest accomplishment related to achieving six sigma yield." Getting examples of major accomplishments most comparable to real job needs not only increases assessment accuracy but also minimizes the impact of first impression and the tech interviewer's natural tendency to box check skills and ask brain teasers. (In this Sherlock Holmes Interviewing lesson you'll find the ideal format for the write-up and the fact-finding approach the hiring manager needs to use to validate the information presented and the role the person played.)
When it comes to interviewing, it's important to ensure that what goes on Behind Closed Doors is an accurate and objective assessment of the person's technical ability to handle the actual requirements of the job. This starts by first understanding how the person will actually use his/her technical skills on the job and then finding out what recognition the person received for doing comparable work. Once you do this a few times you'll then want to thank Mr. Holmes for giving you a better approach for assessing any technical competency.