I was talking with a very smart guy last week, who told me that weather prediction has gotten 50% better in the last 10 years. This involved a lot of modeling and AI in combination with hundreds of data scientists looking at every variable that affects the weather within 2-3 degrees in your zip code at any time of the day within 30-60 minutes. This is pretty cool and pretty exciting stuff, yet predicting on-the-job performance is still done with pseudo-science, bias, gut feelings and thumbs.
What the heck is happening? I’ve been thinking about hiring when I first become a hiring manager 45 years ago and 1,500 hires later (as a recruiter). Mostly I’ve been thinking about why hiring processes are still as outdated as they were when I became a recruiter (in 1978). Here’s what I find puzzling:
- Why do recruiters need to review 150 resumes to make one decent hire, but only 3-4 referred candidates to make one great hire?
- Why do we hire people we know based on their past performance and potential, but we hire strangers based on their past experience?
- Why do hiring managers want to hire candidates who can hit the ground running, but the best people who can hit the ground running want to run on different tracks?
- People who see the job as a career move don’t need as big a salary increase compared to people who are accepting ill-defined lateral transfers. So, why do we negotiate the salary, the location and job title as the condition for having a conversation with someone?
- While we all want to hire more diverse people, why do we expect them to have the same skills and experiences and look and sound like everyone we’ve already hired
- Why is it we still can’t measure quality of hire?
- Why do people in the top half have less or different experience than those in the bottom half? That’s how they got into the top half.
- Why do we design the candidate experience largely for candidates we’re not going to hire, rather than for the candidates we do want to hire?
- Why do recruiters and tech vendors get excited about doing the wrong things more efficiently?
- Why are we still fighting the war for talent?
Since I've had a chance to track the performance of hundreds of people who were and weren't hired for many years after the hire, one problem is that there are just too many cooks and too many recipes being used on how to hire people. Based on this tracking, I've summarized the best and worst predictors of on-the-job performance in this table. (Click here for a popup of this table with the explanations.)
While the list is near perfect, getting everyone to use an evidence-based process to collect these best predictors and avoid the worst ones turns out to be the biggest challenge. (I'll be discussing this concept at my next webcast.)
Predicting the weather requires the same insight and technical ability as rocket science. Predicting quality of hire requires a year or two of statistics, a good understanding of human nature, recognizing the difference between steps and systems, and an agreement by everyone who is involved in making the yes/no decision that evidence trumps bias, emotions, thumbs and gut feelings.
Last week, I was talking with a senior technical director who handles major defense contracts for advanced weapon systems. He lamented that he could not find enough top hardware and software engineers because the recruiters he is using are not strong enough to assess technical skills.
The situation was worsened because he wasn’t permitted to discuss the projects with the recruiters other than in general terms. He went on to say his engineering leaders were part of the problem, because all they did when interviewing candidates was box check skills and experience and use random brain teasers to separate the skills-competent engineers into hirable and non-hirable categories.
I offered a simple solution to both problems that I’ve been using successfully for years.
How to assess candidates technical skills
The first problem to address is helping non-techie recruiters interview technical people. I also refer to this as the Sherlock Holmes interviewing technique.
The core of this solution involves asking the candidate to describe the projects he or she has been assigned 90-120 days after starting any new job or being assigned to any project team. During the fact-finding for each assignment I find out if the project was a stretch project or an important project or if it was at or below the person’s current skill level. This quickly tells you what the person’s supervisor thinks of him or her. I then ask what projects were assigned to the person as a result of his or her performance on the first project.
Part of detective work involves finding out if the person received any special recognition for doing a good job. Recognition could be in the form of a pat on the back, some special award or letter, a promotion or simply being asked to handle a critical task. The task assigned usually uncovers the person’s dominant strengths. Doing this for 3-4 jobs quickly reveals if the person is in the top half or top quartile of his/her peer group. (Note: For customer-facing jobs, like sales, consulting or public accounting, ask about the clients the person was assigned to work with soon after joining the team. The best people are quickly assigned to the most important and/or the most difficult clients to work with.)
The assessment point here is that the best people get assigned by those who know them to ever increasing important assignments. They’re given exposure to important people including company executives and clients early in their tenure with a new company and are asked to lead or join critical project teams. As critical, for the strongest people this happens on a regular basis in different companies with different leaders. While non-technical interviewers can’t fully assess technical depth and knowledge, using this approach they can accurately assess the results of the person’s technical, project management and team skillset.
The senior technical director was blown away by this approach and agreed it would be a great method for his recruiters to use. Then he asked what he could do to improve his technical leaders’ ability to assess these same people. I then described one of my favorite assessment techniques inspired by Charlie Rich’s epic song, Behind Closed Doors.
How to work with hiring managers on assessing technical candidates
Over the years, I’ve arranged interviews with hundreds of technical people with hiring managers who were not very good at interviewing. In the beginning, I wasn’t doing any training, but I came up with a technique that worked wonders. It started by asking the candidate to summarize in two or three paragraphs his or her most significant technical accomplishment most closely related to the requirements of the job. This could range from working with marketing on developing the product roadmap and preparing the detailed specs, designing a high pressure valve for use in deep water drilling or leading the UX design for a new e-commerce application.
I’d then send this write-up along with the candidate’s resume to get the hiring manager to agree to interview the candidate. But here’s the kicker: I asked the hiring manager to review in detail the major accomplishment as soon as the interview started and reject the person outright if the accomplishment wasn’t good enough to meet the manager’s requirements for job success. Managers loved this approach. More importantly, the approach minimized the impact of first impression bias and the manager’s tendency to box check skills and ask brain teasers.
The senior technical director excitedly stood up, laughing, screaming “Yes, yes, yes, that would work!” and gave me a virtual high five on his Zoom info conference screen.
While you can’t control what goes on Behind Closed Doors you can certainly set the stage using inference and deduction to assess candidates and by giving both the candidate and interviewer a script to follow. And if you listen closely you’ll hear some screaming behind those doors. It’s likely Sherlock Holmes.
To access this article you must be a registered user/member of this site. Click the appropriate link below to continue.
You are not enrolled in this course. Please go to our Courses page to enroll now.
Please login / Shop Courses Now.
If you understand real job needs, you only need to ask two questions to predict on-the-job success. It doesn't matter what you ask if you don't know job needs, you'll either be wrong, or lucky.
It took me 30 years and 2,500 interviews to make this table showing the best and worst predictors of on-the-job success. Since I've had the opportunity to work with people I've placed after they were hired and with candidates who were excluded, I can say with the utmost confidence that most interviewers are biased and few have a clue on how to interview properly.
The problem starts by using laundry lists of skills and competencies to screen candidates and then asking a hodgepodge of supposedly clever questions to confirm the interviewer's initial biased reaction to the candidate. These emotional feelings are then voted on using thumbs with the biggest thumb winning the yes/no decision contest.
This rather cynical view is justified if you've ever been involved in these types of ego contests masquerading as judgment.
There is a better way.
First, throw away the laundry list of skills and competencies and define the job as a series of 5-6 performance objectives describing the task, the action required to complete the task and some measure of success. For example, "Reduce scrap on the widget line from 5% to less than 1% by year-end," is a lot better than saying, "Must have 10+ years of high-volume widget production experience, an engineering degree from a top school, a results-oriented attitude and enough EQ to get hugs from the entire team."
Once you put your list of performance objectives in priority order ask the candidate this question, "Can you describe your most significant career accomplishment related to (reducing scrap)?" Spend about 15 minutes on this question, gaining insight into the results achieved, the competencies and skills used, the environment and culture, and the process used to achieve the results. Then ask this same question again for all of the other performance objectives. This will determine fit with the job and if you plot the accomplishments over time the trend line reveals consistency, growth and potential.
The second question is entirely different. It goes something like this:
"One of the biggest challenges in this job is (provide short description). If you were to get the job, how would you go about solving it?"
For the widget example, the question might be, "We have a big scrap problem. Can you walk me through how you would figure out the root cause and put together a solution?"
Asked properly this question uncovers a critical ability of all top performers: job-related problem-solving skills. The best candidates I've met in my 35 years in executive search all have the ability to anticipate the needs of the job before starting it. They can figure out very quickly what's wrong or what's necessary to accomplish a task, what they need to do to implement a solution, and what resources they need to do it. Even better, they "see" the problem, the solution, and the steps needed to get there. They also know what they don't know and are confident enough to tell you how they'll get this information.
When you ask this problem-solving question it's important to turn off the spotlights and shift the conversation into a more natural give-and-take discussion about real job needs. This way the meeting is no longer an interview but a business-like discussion with a team member trying to work together to figure out a solution to a real problem. Once you get comfortable with this style of interviewing, you'll be able to assess the following four dimensions of thinking skills.
The Four Dimensions of Thinking and Problem-solving
Depth. Determine if the reasoning is complex, advanced or superficial. The best candidates demonstrate a good understanding of the cause and effect of a problem and can determine how to find the root cause. Superficial reasoning is evidenced by a bunch of seemingly unrelated or more generic ideas.
Focus. Is the focus technical, tactical, or strategic? Candidates with a pure technical focus get into process or concept details. Those with a tactical bent address the results and outcomes more. A strategic focus is represented by a longer time horizon with consideration to the implications and the unintended consequences.
Team or individual emphasis. Understand if the candidate's ideas and approaches involve others or if the person is more individual or self-focused. This is an important consideration if the person will be managing others or involved in a number of team projects.
Functional or multifunctional perspective. The best candidates understand the implications of their job on other people and other functions. Listen for this as the candidate plans out his or her tasks and asks questions.
Use the Anchor and Visualization Technique to Increase Interviewing Accuracy
Often people can talk a good game but can't deliver the results. To address this concern, just ask this question after the candidate has answered the problem-solving question, "Now can you tell me about something you've accomplished that's most related to what we need done?"
I refer to this two-question combination as the Anchor and Visualize approach. A track record of past performance and the ability to visualize the future is a great indicator of ability. When combined with a clear understanding of real job needs using a performance-based job description, these two questions are all you need to ensure you're hiring a great person.
You can't solve a three-dimensional problem with a one-dimensional solution.
Every year about this time I get interviewed by some magazine about what I think will happen in the upcoming year to improve quality of hire. This year is no different. And neither is my answer. There will be no difference.
The reason why is explained in the table showing the best and worst predictors of on-the-job performance. Too many interviewers focus on the wrong predictors of success and even worse, most interviewers fail to fully consider the fit factors. These are the essential predictors described in the bottom of the table - fit with the job, fit with the hiring manager and fit with the company culture.It took me 30 years to make this chart, observing hundreds of candidates after I interviewed them and tracking their subsequent performance for years. Part of this tracking included working with candidates who were not hired at one company but were hired at another one. Many became my hiring manager clients. This offered me a unique perspective on the entire sourcing, interviewing and recruiting process.
As a result of all of this, I can say with the utmost confidence that most interviewers are biased and few have a clue on how to interview properly.
Eliminating 150% of all hiring mistakes might seem a bit odd on the surface but there is a rational explanation. A third of the errors in hiring are attributed to not hiring the best person, either because the person didn’t find the job attractive, the person was improperly recruited or the person was box-checked out on factors that don’t predict success. Another third of the errors are due to the use of flawed interviewing practices. And the final third are caused by an inability to close the offer due to weak negotiating skills on the part of the recruiter and/or hiring manager.
In most cases these problems are caused by using job descriptions that emphasize skills, experiences and competencies. Few top people – whether they’re active or passive – find these jobs worth considering so they don’t even apply or return calls to recruiters reaching out directly to them.
To start attracting and hiring stronger talent throw away your traditional job descriptions and describe the work that needs to be done. Every job can be defined as a series of 5-6 performance objectives describing the task, the action required to complete the task and some measure of success. For example, “During the first year develop a non-destructive test to determine the maximum speed a big brick can be sucked through a 12-foot diameter tunnel before exploding.” This is much more accurate and more exciting a job requirement than saying the person must have 10+ years of experience understanding the quantum physics of turbulent airflow, a PhD from a top school and strong communication skills.
If you can prove the person can do this work and is passionate about doing it, he/she will have exactly the skills, experiences, motivation, competencies and academic credentials needed AND the compensation/location/title will be less important. As part of the performance-based interviewing process I advocate, I ask candidates to provide in-depth examples of accomplishments for each of the required performance objectives. This approach provides all the evidence needed to assess all of the strong and essential factors shown in the table.
It’s important to recognize that even if a person possesses all of the strong predictors of success, the person will underperform if the essential predictors (the Fit Factors) aren’t carefully considered. The person must be passionate about doing the actual work required and the person must be able to work closely with the hiring manager. Often these factors are only superficially considered. As part of the interviewing process, I ask how the person’s different managers impacted his/her performance and where the person naturally goes the extra mile. This provides great insight on these two fit factors. Assessing cultural fit is relatively simple as long as you recognize that company culture is driven largely by the growth rate of the company, its decision-making process, the politics and its financial performance. As you dig into the person’s accomplishments ask about these points to ensure a strong culture fit.
You need to appreciate the fact that top people will get counteroffers and they will get better offers from other companies. So if you can’t close the deal within your compensation range you’ll lose out on hiring the person even if you see the best people and conduct a very accurate interview. That’s why I urge talent leaders and business executives that they need to be 150% better to hire better people. This is a three-pronged problem and why piecemeal solutions are ineffective. It all boils down to the fact that you can't solve a three-dimensional problem with a one-dimensional solution.