Should you give the assessment if you don't care about the results?

Last week America's second most popular sporting spectacle took place. No, not the beginning of the NBA playoffs, but rather the annual National Football League player draft, an incredible three days of televised talent assessment, evaluation, and selection. The NFL draft, once a largely behind the scenes administrative event, has grown over the years into a multi-day, multi-media extravaganza, with an entite cottage industry of draft 'experts' and advisors seemingly making a really good living not actually evaluating players for the actual teams, but rather appearing on TV to inform and share with fans and viewers their opinions of draft-eligible players, offer their speculation on which players will be selected by which teams, and comment more generally on how well or poorly each team's talent evaluators did in making their player selections.

Making the 'right' selections from among the large pool of eligible talent, (almost all American college football players that have graduated from school, exhausted all of their college eligibility, or have declared themselves 'eligible' to be selected), like talent selection in any business, is challenging, complex, and incredibly important. On a good year, anywhere from 10-15% of a team's total active roster can be supplied via that year's draft. 'Hitting' or making the right picks, like finding a rare or overlooked talented player in later draft rounds, or avoiding 'missing', by bypassing players that later turn out to have unsuccessful playing careers often eventually means the difference in overall organizational success or failure.

All the teams know how important the draft process is, and thus, over the years more and more steps and components have been introduced to the pre-draft player evaluation process. From intense study of college game video, to a battery of physical tests and measurements, and more recently, even formalized tests of a potential player's cognitive and reasoning capability, in the form of what is called the Wonderlic test. The Wonderlic consists of 50 questions to be answered in 12 minutes, and is meant to give teams a general feeling for the overall thinking and reasoning capability of a player, as well as provide a means of comparison with all the other potential players who also take the test.

Most years the draft process ensues without much mention of the Wonderlic test as a part of the player evaluations, except only, and as happened this year, when a particularly high-profile and anticipated top draft choice caliber player gets a really low Wonderlic score. This year Morris Claiborne from LSU, regarded as one of the Top 10 available players in the draft reportedly scored a 4 (out of a possible 50) on the Wonderlic. A score of 4 is really, really bad, according to ESPN it was the lowest reported score in more that 10 years, (for comparison, an average score is about 21).

Despite the alleged poor score, Claiborne was indeed selected by the Dallas Cowboys with the 6th overall selection. So apparently the disastrous Wonderlic score did not impact Claiborne's standing and attractiveness as a candidate for the NFL. In fact, Dallas Cowboys owner Jerry Jones stated the test score was 'not an issue at all', and Cowboys coach Jason Garrett remarked, 'We talk about the test scores, but we also talk about 'What's his football IQ', also seemingly dismissing the value of the Wonderlic as a means to predict future performance as an actual football player.

Now of course the Cowboys reps might be trying to defend their selection of Claiborne and downplaying the significance of the Wonderlic score is certainly in the team's self-interest, but the ESPN story linked above also refers to Claiborne's view that the test was essentially meaningless and not at all important in determining his ability to actually play football at the highest level. He is quoted as saying -  "I mean, I looked on the test and wasn't nothing on the test that came with football, so I pretty much blew the test off."

Sort of an odd situation, the player, (candidate), and the team, (employer), both essentially admitting that one of the common if not primary assessment tools given to all players doesn't have anything to do with the actual job, and as soon as the assessment results don't fit with what our more traditional and time-tested evaluations tell us, (like actually watching the candidate play football), they will essentially be discarded from consideration. Seems like a big waste of eveyone's time.

Now sure, you can argue with me that Caliborne, as a top player in this year's draft was not ever going to be impacted by his score, (good or bad), on the Wonderlic, and that the test is really meant for use as a supplementary measure or data point for players whose football talents are more questionable, and that it can be used to help make decisions between closely related prospects.

But the league made Caliborne, and other 'top talent' take the test. And I bet, if you look closely at your organization's recruiting practices as well, you might find similar examples of making 'top talent' run through hoops or perform silly, eventually meaningless, exercises because 'that's just our process.'

Claiborne didn't really have an option to decline the test, the NFL has an effective monopoly on professional football in America. But any 'top talent' you might be recruiting? Well they likely have plenty of options. You probably want to make sure your process understands that.

The Big Picture Thinker, or Making Candidates Tap Dance

When trying to find the best candidate for the job, how many interviews are too much?

When do your standard questions become a little insulting or the screening surveys you have carefully crafted go too far, and in the process turn away candidates with the background and qualifications you are seeking, but feel taken aback by having to prove themselves during your application process?

I started to think about this while reading a recent post on The Daily WTF blog, a site normally centered around tales of dodgy computer programming, clueless end users, and mostly amusing but not really cruel hijinks and frivolity for the geeky set. Every so often The Daily WTF shares a job interview story, and while normally kind of fun, the 'Big Picture Thinker' yarn is one of the best I have seen.

So the story goes something like this:

After an in-person technical interview for an unnamed development, (or possibly managerial position), the company sends a standard, (but simple), technical aptitude test to the candidate. The test is meant to help gauge written communication skills. But in this case, not only did the hiring manager forget to attach the test to his email, he surprisingly found himself dealing with a candidate that clearly did not feel it necessary to 'prove' himself by taking the test. Take a look at the candidate's response:

----------------------------------------------------------

From: Thomas B-------
Sent: Friday, April 08, 2011 10:37 AM
To: James S------
Subject: RE: Written Test


When a big picture thinker with nearly 20 years of experience in 
IT sends you a resume and cover letter like mine and says that he 
can help you win a client that is pulling in 1.3 Billion per year, 
here's what you don't do:  

  1. Set up an interview with a couple of in-the-box thinking 
     Microsoft drones with questions on minutia.  

  2. Hand him a test to see what his "style", attention to 
     detail, and problem solving approach is.  
     
Here's my style: I am certain that I can run circles around your 
best developers with my own, original, incredibly efficient model; 
but more importantly, I am a director that can help them run 
circles around their own current misguided misconceptions.  But I 
am thankful for this lesson, as I have learned that I need to add 
a cover to my cover letter that reads:  If you are an in-the-box 
thinking Microsoft house, and you find yourself regurgitating 
terms like OOP, MVC, TDD, BDD, Cucumber, etc..., without really 
understanding what it all means and how much it is actually 
costing your company to have bought into that industry pushed 
bullshit, then DO NOT contact me.  I'd save you too much money, 
and you obviously do not want that.

So the question now is:  Did I pass the test?

The answer is: Fuck yes I did.

Thomas B-------

PS. You forgot to attach the quiz.  

Do this: Print out a copy of it, ball it up, and throw it at 
your own forehead, because that's what I would do if I were 
there.

-----------------------------------------------------------------------------------------

Classic, and kind of instructive. Sure, Thomas B. the candidate in question is quite likely a pompous jerk, and doesn't seem like the type of employee that would be a great addition to the team. But it is also likely that he probably did possess the basic technical qualifications for the job, and that his experience and resume details would have borne that out. 

I get the need for organizations to be careful, thorough, and sure, (or as sure as you can be), before pulling the trigger on a new hire. The stakes are high, the pressure to find top talent is palpable, and the costs of making a bad hire are high. But at the same time making candidates unnecessarily jump through hoops, answer incredibly basic questions, and otherwise put them into a kind of disrespected and subservient position is not really warranted either.

So the next time you are about to administer that 'test', think about whether or not you too should 'print out a copy, ball it up, and throw it at your own forehead.'

Bench Pressing and Basketball

With the National Basketball Association player draft fast approaching, fans, observers, and pundits alike love to speculate and predict the player draft order, and imagine the glorious future for their favorite team once this years' version of young Timmy 'The Flint Assasin' Sackett, or some other such prospect joins the squad.

Readers of this site, along with my pieces on Fistful of Talent, know that sports, and in particular how the talent evaluation and assessment processes that professional sports teams undertake as they consider which players to draft, recruit as free agents, trade, and compensate; make for some compelling stories and often illuminate applicable lessons for those of us with concerned with more mundane but similar workplace conundrums. None of the 'Sports and HR' parallels are more clearly illustrated than annual player drafts that all the major USA professional sports leagues conduct.

The purpose of these drafts is to help 're-stock' the talent pools in the league with new players, ones that have the capability and potential to raise the overall talent profile of the league and the individual teams. Essentially each season, younger, more talented players (or at least ones judged to have potential to be good players), enter the league while older and/or less skilled/more expensive players exit. It is a kind of a cool, virtuous 'Lion King' style circle of life, but will louder music and more tattoos.

The trick for talent evaluators and people in charge of player personnel decisions in the draft is how to assess the complex combination of a prospect's performance on the court to date (usually in college basketball, but sometimes just high school, or international play), the player's physical attributes, their personality and character, and finally whether or not that elusive 'fit' between style, physical traits, and mental make-up exists between the prospect and the team.

You will often see quotes from NBA or other sports execs talking about players they select as being 'Our kind of player', or 'His style fits how we like to play'. These quotes are as much about cultural and organizational fit as they are about hitting jump shots or ability to rebound the basketball. The rules of the game are the same for every team, but how they go about assembling the team and their philosophies about how to best accomplish the universal goal of winning the championship are all unique.

So in sports, like in most every other line of business, talent assessment and selection is really hard. So NBA teams have come to increase or expand the variables they assess and measure when it comes to the talent evaluation process for potential draftees. One of these variables is the number of times the prospect can successfully bench press 185 lbs, a moderate amount of weight for a well-conditioned athlete, certainly not a power lifter or bodybuilder burden, but also a weight that could present a challenge. The 185 pound bench press is meant to give a generalized assessment of the player's upper body strength, that at least in theory could translate to effectiveness on the court. But bench pressing isn't really basketball, they don't roll out a bench and some barbells in the 4th quarter of a close game. The other advantage to teams in using the bench press test, (and a myriad of other fitness and strength tests they use), is that every prospect takes the same assessments, thereby giving the teams a common data set across the entire talent pool from which to make comparative judgments.

But the data itself offers a team no competitive advantage - every team in the league has access to the same information. The trick is knowing how to interpret the 'measurables' (bench press, vertical jump, etc.), with the 'intangibles', (character, coachability, likeability), and finally a frank assessment of 'Can this guy actually play?'; in order to make the best talent selections. 

But back to the bench press, which is the reason I wrote this piece. Yesterday I noticed a tweet from Chad Ford, one of ESPN's basketball writers and analysts commenting on the bench press test results from a few of this year's current NBA draft prospects.  The tweet is below:

The implication of the tweet is a kind of red flag or warning about those few players unable to successfully bench press 185 pounds. That teams considering drafting these players may pause, and fans of teams that eventually do take these players might need to be concerned that their lack of demonstrable upper body strength (doing something that isn't actually playing basketball), portends poorly for their future performance as NBA players.

It is hard to say for sure if this poor performance on the test will actually hurt these players draft position, it certainly won't help it, but I think the larger point is about data collection in general. Whether it is an NBA team evaluating a power forward, or a software company assessing the background and skills of a candidate for a development job, our abiliity to collect reams of data about background, capability, demonstrable skills, and even mental make up has never been greater. We have access to powerful analytics tools to crunch the data and perhaps eventually to construct detailed and predictive 'success' models.

It could very well be the success on the bench press test does suggest future success on an NBA team. Or failure on the test predicts failure on the court.

But even if we can create those kinds of models, for basketball players or software developers, they will never be fool proof, as people and performance are ultimately likely too unpredictable to ever understand absolutely. We have to be open-minded enough to ignore our own models from time to time.

You may, even if you are not a basketball fan, have heard of a player called Kevin Durant. He is a star player for the Oklahoma City Thunder, has led the league in scoring, led the USA team to the Gold Medal in the World Basketball Championship last summer.

In 2007, when Durant declared himself eligible for the NBA draft, he was unable to bench press 185 a single time. 

And we know how Durant has worked out. 

Sure collect, assess, analyze, correlate, model - it's important. But don't forget, bench pressing is not basketball.

Grading Talent the Big Tuna Way

Last night ESPN ran an interesting behind-the-scenes look at how American professional teams typically evaluate talent, with special guest former National Football League executive and head coach Bill 'Big Tuna' Parcells. The context of the show was the league's upcoming college player draft, the annual exercise where the league's teams assess, grade, and ultimately select from 5-10 players each to 're-supply' the talent on their teams. It is a massive, high-stakes, expensive, and critically important recruiting, assessment, and alignment exercise.

Parcells' resume and achievements as a successful coach, and talent evaluator are solid - he served in very senior roles at several NFL organizations, winning two Super Bowl Championships as the Head Coach of the New York Football Giants. 

In the show Parcells' shared some of the talent selection criteria and thought processes that organizations that he was a member of, and in general, most other teams tend to follow when making player selections in the league's annual college player draft. Some of the criteria and processes were fairly obvious, and would apply generally to any talent selection or recruiting context, (players who had been kicked off their college team for disciplinary reasons should be avoided), but some of the other concepts Parcells discussed perhaps are not so apparent to casual observers, and just might have some additional applicability to more conventional talent selection processes.

Here are three Talent Evaluation ideas straight from the Big Tuna:

1. Understand the predictors of success (some are not so obvious)

In NFL football every team measures and grades the basic and easily understood physical characteristics of potential draftees, (height, weight, strength, speed), but during the show Parcells mentioned a few not-so-obvious keys he assesses, (e.g. for the position of cornerback, length of the player's arms). For potential quarterback prospects, Parcells insisted he only wanted players that actually graduated from college, as he felt it demonstrated intelligence, and more importantly commitment. 

The larger point is every competitor has access to the same talent pool, the basic and obvious assessment criteria are widely known and universally adopted, so finding the less clear and more predictive evaluation criteria that other teams may not have discovered is one of the ways to claim some advantage and make better selection decisions than the competition.

2. Make sure everyone involved in Talent selection understands these predictors

Once the criteria is established, and a process to collect and assess these criteria developed, Parcells emphasized the critical need for everyone involved in the talent selection process to understand the criteria, and consistently grade to the criteria. From scouts, to assistant coaches, to even the team owner, the definition of what a top candidate looks like has to be understood by everyone. There are so many players to assess, that no one member of the organization can possibly 'know' every candidate, so the selection process becomes a team effort, and the talent selection team has to have that common ground for any chance of success. Talent is talked about in the common language of the team's assessment ratings, and no conversation about talent fails to reference these assessments.

3. Know yourself

Parcells described a common acronym used in football draft processes, NFU, which means 'Not For Us'. This term is assigned to players that the strict adherence to positional capability assessments or past production in the college game might indicate are good candidates and should be considered in the selection process. But these NFL players have raised some concern off the field, of their attitude, style, work ethic somehow will not be a cultural match to what the organization is looking for. Parcells strongly advises teams to know themselves, know the style they want to play, the kinds of mental makeups that players need to have to 'fit' on the team, and to avoid the temptation of selecting players with fantastic physical skills that might not 'fit' otherwise. These kinds of gambles rarely work out, and they are the ones that get coaches and talent evaluators fired.

But in the end, despite incredibly detailed and complex processes for physical measurement, tests of intelligence, and well-documented and easily reviewed past performance in college football, selecting players for NFL teams is still and imperfect process. So-called 'can't miss' top prospects often fail to live up to expectations, while others deemed marginal prospects once vetted by the traditional processes end up as star players.

Having a system and some ground rules to follow, to find ways to uncover predictors your competition may have missed, and perhaps most importantly a deep and confident organizational self-awareness are a few ways our pal the Big Tuna offered up to try and land more Peyton Mannings and less Ryan Leafs (inside football reference, Google it).