Learn a new word: The Optimal Stopping Problem

Monday, May 22, 2017 at 5:43PM

Steve in HR, Learn a new word, Recruiting, data, statistics

Steve in HR, Learn a new word, Recruiting, data, statistics

I caught an interview over the weekend with one of the authors of *Algorithms to Live By* (can't recall which of the two co-authors I heard, but it doesn't matter. Kind of like it doesn't matter which of the two guys in Daft Punk plays a particular instrument on any given track. But that is another story.), and wanted to share a new word I learned from the interview that has some relevance to HR/Recruiting.

For this installment of Learn a new word I submit ** The Optimal Stopping Problem**.

From our pals at Wikipedia:

In mathematics, the theory of optimal stopping or early stopping is concerned with the problem of choosing a time to take a particular action, in order to maximise an expected reward or minimise an expected cost. Optimal stopping problems can be found in areas of statistics, economics, and mathematical finance (related to the pricing of American options). A key example of an optimal stopping problem is thesecretary problem. Optimal stopping problems can often be written in the form of a Bellman equation, and are therefore often solved using dynamic programming.

I bolded the *'secretary problem*' which, despite its dated-sounding kind of name, is the example most commonly cited when discussing optimal stopping, and as luck would have it, is directly tied to HR/Recruiting.

The secretary problem is essentially, the question of 'Given X number of job candidates for a given position, and also given you have to make a 'hire/decline' decision on each candidate before moving to the next one, how many candidates do you need to interview in order to maximize your probability of identifying the best candidate, while minimizing the risk of making a 'bad' hire, (say by waiting too long, rejecting too many candidates, and having to settle for a candidate that is left).

Let's say you have 10 candidates for a position. You probably wouldn’t offer the job to the first candidate you interview, because you have no idea how that candidate compares to anyone else, or the general caliber of the candidates overall . But you probably don't want to wait until the 10th candidate, because if they’re the only one left you’re going to be forced to offer them the job (or keep it unfilled), regardless of how strong a candidate they are. Somewhere in the middle of the process there must be an ideal place to stop interviewing more candidates just to see what they’re like, and make a selection. But where to stop?

Enter the Optimal Stopping Problem. You can dig into the math here, but it turns out there is an ideal place to stop interviewing candidates, (or dating different people in order to try and choose who to marry), and it's after you have interviewed (or dated), **37%** of the contenders. After you get to 37%, make a note of the 'best' candidate you have seen so far, (let's call her Mary Jane). Then, continue interviewing candidates and when you find the first one that is 'better" than Mary Jane, stop all further interviews and immediately offer that person the job.

How it works is related to the math behind estimating where the best candidate could be in the lineup. This number, expressed as 1/e, where 1/e eventually approaches 0.368, or about 37%. By analyzing the possible distribution of talent, it also turns out that if you interview the first 37 percent of candidates then pick the next one who is better than all the people you’ve interviewed so far, you have a 37 percent chance of getting the best candidate.

It's a really interesting way of looking at the hiring decision making process, (as well as other processes that involve trying to make the 'best' choice amongst a number of alternative). But it makes sense somehow, even if only on an anecdotal level.

How many times have you slogged endlessly through an interview process where after some point candidate after candidate seem the same, and certainly no better than one you saw two weeks ago?

Or how many of us have, (maybe even privately), thought about a past boyfriend or girlfriend that 'got away' and for some reason has never been eclipsed by the series of people that you have subsequently dated?

Knowing when to stop, and understanding the probability that you have seen the best, or close enough to it, in any decision process is an enormously valuable thing.

In the secretary problem, and in probably a bunch of other problems too, the answer seems pretty clear - once you hit 37% you have seen enough, you won't learn much if anything else useful, and you know how to make your decision.

It is easy to apply in a job vacancy with 10 candidates.

It is a little tougher to estimate just how many people you are willing/able to date in order to know when to apply the 37% cutoff.

**Have a great week!**

Article originally appeared on Steve's HR Technology (http://steveboese.squarespace.com/).

See website for complete article licensing information.