Questions to ask before letting an algorithm make HR decisions

Nearing the halfway mark in 2018 and I am ready to call it right now - the topic/trend that has and will continue to dominate the HR and HR technology discussion this year is Artificial Intelligence or AI.

I will accept my share of the responsibility and blame for this no doubt. I have hit the topic numerous times on the blog, I have programmed at least seven sessions (or more) featuring AI topics for the upcoming HR Technology Conference, and the subject comes up on just about every HR Happy Hour Podcast at one point or another. In fact, one of my favorite HR Happy Hour Shows this year was the conversation I had with author and professor Joshua Gans on his new book titled Prediction Machines: The Simple Economics of Artificial Intelligence.

So if you are thinking that everyone in HR and HR tech is all in on AI you'd probably be right. And yet even with all the attention and hype, at some level I still wonder if we are talking about AI in HR enough. Or more specifically, are we talking about the important issues in AI, and are we asking the right questions before we deploy AI for HR decision making?

I thought about this again after reading an excellent piece on this very topic, titled 'Math Can't Solve Everything:Questions We Need to be Asking Before Deciding an Algorithm is the Answer' on the Electronic Frontier Foundation site. In this piece, (and you really should read it all), the authors lay out five questions that organizations should consider before turning to AI and algorithms for decision support purposes.

Let's take a quick look at the five questions that HR leaders should be aware of and think about, and by way of example, examine how these questions might be assessed in the context of one common 'AI in HR' use case - applying an algorithm to rank job candidates and decide which candidates to interview and consider.

1. Will this algorithm influence—or serve as the basis of—decisions with the potential to negatively impact people’s lives?

In the piece on EFF, the main example or warning cited when AI-driven processes might negatively impact people's lives is in the use of an algorithm called Compas, which has been used to predict convicted criminals likelihood to become repeat offenders. The potential danger is when the Compas score influences a judge to issue a longer prison sentence to someone the algorithm suggests is likely to repeat offend. But what if Compas is wrong? Then the convicted offender ends up spending more time than they should have in prison. So this is a huge issue in the criminal justice system.

In our HR example, the stakes are not quite so high, but they still matter. When algorithms or AI is used to rank job candidates and select candidates for interviews, those candidates who are not selected, or are rated poorly, are certainly negatively impacted by the loss of the opportunity to be considered for employment. That does not mean the AI is 'wrong' or bad necessarily, but just that HR leaders need to be open and honest that this kind of AI will certainly impact some people in a negative manner.

With that established, we can look at the remaining questions to consider when deploying AI in HR.

2. Can the available data actually lead to a good outcome?

Any algorithm relies on input data, and the 'right' input data, in order to produce accurate predictions and outcomes. In our AI in HR example, leaders deploying these technologies need to take time to assess the kinds of input data about candidates that are available and that the algorithm is considering, when determining things like rankings and recommendations. This is when we have to ask ourselves additional questions on correlation vs. causation and whether or not one data point is a genuine and valid proxy for another outcome.

In the candidate evaluation example, if the algorithm is assessing things like educational achievement or past industry experience of a candidate, are we sure that this data is indeed predictive of success for a candidate in a specific job? Again, I am not contending that we can't know which data elements are indeed predictive and valid, but that we should know them, (or at least have really strong evidence they are likely to be valid).

3. Is the algorithm fair?

At the most basic level, and the one that has the most applicability for our AI in HR example, HR leaders deploying AI have to assess whether or not the AI is fair - and the simplest way is to review if the algorithm is treating like groups similarly or disparately? Many organizations are turning to AI-powered candidate assessment and ranking processes to try to remove human bias from the hiring process and attempt to ensure fairness. HR leaders, along with their technology and provider partners have the challenge and responsibility to validate this is actually happening. 'Fairness' is a simple concept to grasp, but can be extremely hard to prove, but one that is inherently necessary in order for AI and algorithms to drive organizational and even societal outcomes. There is a lot more we can do to break this down, but for now, let's be sure we know we have, in HR, to ask this question early and often in the AI conversation.

4. How will the results (really) be used by humans?

If you deploy AI and algorithms for the purposes of ranking candidates, how will you use the AI-generated rankings? Will they be the sole determinant of which candidates get called for interviews, advance in the hiring process, and ultimately have a chance at an offer? Or will the AI rankings be just a part of the consideration and evaluation criteria for candidates, to be supplemented by 'human' review and judgement?

One of the ways the authors of the EFF piece suggest to ensure that human judgement is still a part of the process, is to engineer the algorithms in such a manner that they don't produce a single numerical value, like a candidate ranking score, but rather a narrative report and review of the candidate that a human HR person or recruiter would have to review. In that review, they would naturally apply some of their own human judgement. Bottom line, if your AI is meant to supplement humans and not replace them, you have to take active steps to ensure that is indeed the case in the organization.

5. Will people affected by these decisions have any influence over the system?

This final question is perhaps the trickiest one to answer for our AI in HR example. Job candidates who are not selected for interviews as a result of a poor or lower relative AI-driven ranking, will almost always have very little ability to influence the system or process. But rejected candidates often have valid questions as to why they were not considered for interviews and seek advice on how they could work to strengthen their skills and experiences in order to improve their chances for future opportunities. In this case, it would be important for HR leaders to have enough trust and visibility into the workings of the algorithm in order to precisely understand where any given candidate was ranked poorly. This ability to see the levers of the algorithm at work, and be able to share them in a clear and understandable manner is what HR leaders have to push their technology partners on, and be able to provide when needed.

As we continue to discuss and deploy AI in HR processes, we have to also continue to evaluate these systems and ask these and other important questions. HR decisions are big decisions. They impact people's lives in important and profound ways. They are not to be taken lightly. And if some level of these decisions are to be trusted to an algorithm, then HR leaders have to hold that algorithm (and themselves), accountable.

Have a great week!

#HRTechConf Update: Submissions Open for Awesome New Technology and Discovering the Next Great HR Tech Company

NOTE: I had an important HR Technology Conference update that I posted yesterday over on the the Conference's HR Tech Insiders blog, but I did want to cross-post here too, to make sure any and all interested HR technology companies and solution providers had the news. Thanks!

From HR Tech Insiders...

Attention HR Technology Solution Providers: Submissions to be considered for the annual HR Technology® Conference and Exhibition "Awesome New Technologies for HR" and "Discovering the Next Great HR Technology Company" sessions are being accepted and can be submitted on the HR Tech website HERE.

In case you are new to these sessions, here is what they are, how they work, and who is eligible for consideration for each session.

"Awesome New Technologies for HR" showcases larger, more established HR tech solution providers, (publicly traded, been in the market for several years, maybe running TV spots on CNBC, etc.), who are invited to submit their latest, most innovative solutions for consideration. These can be new modules for an existing platform, a reinvention of one or more of their solutions, or something totally new and unique in the HR tech market. During the summer, I will review, arrange demonstrations, and select 5 or 6 solution providers to present for 10 minutes on our main stage at HR Tech and be recognized as an "Awesome New Technology for HR" for 2018.

"Discovering the Next Great HR Technology Company" is for the startups, less-established, or emerging HR tech solution providers in the space, and works a little differently than "Awesome New Technologies." From the submissions we receive on our website, HR Tech works with a group of industry experts -  George LaRocque, Principal Analyst and Founder of HRWins , Madeline Laurano, Founder and Principal Analyst of Aptitude Research Partners., Ben Eubanks, Principal Analyst of Lighthouse Research & Advisory, and Lance Haun, Practice Director for The Starr Conspiracy to select eight semi-finalist HR tech solution providers.

Then, during the summer our analyst coaches will work with the eight semi-finalists to hone their messaging and demonstrations, and will be posting videos and additional information about the semi-finalist startups.

In July and August we will be looking to you, the HR Tech Insiders audience, to vote online on the HR Tech Insiders site and help us select from these eight semi-finalists, the four finalists that will get to present to the audience at the conference in Las Vegas in September. And in a new wrinkle for 2018, the four finalists will be joined by a fifth company - the winner of the 1st Annual HR Technology Conference Pitchfest which will take place during the Conference. Finally, this will culminate in live demonstrations from the five finalists on our main stage after which Conference attendees will select the Next Great HR Technology Company for 2018 live in Vegas!

We encourage all interested HR technology solution providers for either session to submit an entry for consideration here. The application deadline is Friday, June 29th, so don't wait too long to submit.

I can't wait to review the submissions and see all the incredible HR technology innovation I know is out there!

Emotional surveillance - coming to a workplace near you?

I am going to submit today's dispatch from the HR Happy Hour Home Office without much commentary, as like many tech-driven developments we hear about, this one is probably too extreme to have much of an effect in the US or any of the other places where readers of this blog reside, (Hi Canada!).

From one of my favorite sources on all things going on in business in China, the South China Morning Post, here is a little bit of a piece titled 'Forget the Facebook leak: China is mining data directly from worker's brains on an industrial scale':

Workers outfitted in uniforms staff lines producing sophisticated equipment for telecommunication and other industrial sectors.

But there’s one big difference – the workers wear caps to monitor their brainwaves, data that management then uses to adjust the pace of production and redesign workflows, according to the company.

The company said it could increase the overall efficiency of the workers by manipulating the frequency and length of break times to reduce mental stress.

Hangzhou Zhongheng Electric is just one example of the large-scale application of brain surveillance devices to monitor people’s emotions and other mental activities in the workplace, according to scientists and companies involved in the government-backed projects.

Concealed in regular safety helmets or uniform hats, these lightweight, wireless sensors constantly monitor the wearer’s brainwaves and stream the data to computers that use artificial intelligence algorithms to detect emotional spikes such as depression, anxiety or rage.

The technology is in widespread use around the world but China has applied it on an unprecedented scale in factories, public transport, state-owned companies and the military to increase the competitiveness of its manufacturing industry and to maintain social stability.

Wow, pretty wild, fairly extreme - even by the looser standards for what is ok and not ok in the workplace that still prevail in most of China.

But here's the interesting thing, we all have already come to accept certain kinds of monitoring in the workplace. We make hourly workers punch in and punch out every day, (and remind them to be sure to punch out before taking lunch). All kinds of call center representatives have their calls and interactions with customers reviewed and even listened to in real time by supervisors. Warehouse workers are often subjected to really close and detailed kinds of monitoring - how fast they find items for an order, how many errors they make per shift, and how closely they achieve "goal" performance each week.

Ever white collar jobs are subject at times to really close monitoring and supervision. Most lawyers and consultants are still billing by the hour, so they must keep and have reviewed detailed time and activity logs. Many organizations require receipts for every dollar spent on employee travel in order for the employee to get reimbursed. Are you sure you had that Dunkin' coffee for $2.65? Even the rise and increasing popularity of workplace chat apps like Slack have created more environments where your 'status', i.e. are you currently working, is visible to everyone and monitored by most.

The point being that sure, this idea of monitoring employee brainwaves in real time, or as one Chinese official described it, conducting 'emotional surveillance' seems ludicrous, it can also be seen as just the next, tech-enabled step on a path that lots of organizations are already walking. And the deployment of these kinds of technologies for workers in dangerous, important roles like airline pilot or high-speed train operator could offer another level of safety for the public - a pilot judged to be in an emotional state prior to takeoff could be pulled from the flight as a precaution.

I don't have a great, insightful conclusion to this story at the moment only to say that while it is inevitable that technologies will continue to advance, and offer better, more, and more personal information about workers, it is (hopefully), going to be the role of smart HR people to help guide organizations as to the best, fairest, and 'right' use of these kinds of tool. The pilot on the above flight is not just a pattern of brainwaves after all. He/she is an actual human.

Have a great day!

HRE Column: Getting Personal with HR Tech

Once again, I offer my semi-frequent reminder and pointer for blog readers that I also write a monthly column at Human Resource Executive Online called Inside HR Tech that can be found here.

This month, after having attended several HR technology solution provider user conferences, I look at the major ideas, themes, and directions these providers, (and others) are talking about, using as design cues for their latest innovations, and are becoming more important for HR leaders to think about and understand. I am not looking directly at individual bits of functionality or capability, but rather more fundamental and cross-product ideas and concepts in HR tech.

These are some of the major themes that we will be focusing on for the next HR Technology Conference - the nature of the most innovative HR technology solutions are putting user experiences, personalization, and embedding more intelligent recommendations to users and HR leaders.

In the piece, I talk about each of these themes in some detail.

Here's an excerpt from this month's piece in HRE Online:

A major part of the planning process in creating the program for the upcoming HR Technology Conference and Exposition^® is attending as many industry and HR technology solution provider customer conferences as I can. The primary benefit for attending these events are the conversations: with product executives about their current and future plans, with HR leaders who are using these products in their organizations, and with industry analysts and influencers about what they are seeing in the HR tech market.

In the last month or so, I have had the opportunity to attend three such events: Ultimate Software’s Connections conference in Las Vegas, IBM’s Think event (also in Las Vegas) and Oracle HCM World in Dallas. (As an aside, while I do believe Las Vegas is the best place for any large conference, kudos to Oracle for choosing a location with great weather and great barbecue.)

Rather than producing an event report for each conference, I thought I’d take this opportunity to highlight some common themes across all three events. While every HR technology provider approaches new trends, technologies and customer challenges in its own way, it is useful to assess what kinds of technology developments and “big picture” considerations are being seen across the industry, as these tools and developments are likely to shape much of the HR technology conversation for both solution providers as well as in customer organizations this year.

Here are the three major themes I took from those recent conferences.

AI-powered Solutions

At HR Tech last year, artificial intelligence was the one theme that seemed to emerge out of almost every conversation with a solution provider. This is a good thing for HR leaders, but potentially problematic as well. While the promise of AI and AI-powered HR technology is amazing, the confusing blend of terminology, technology and marketingspeak can make AI in HR tech a difficult concept to grasp, as well as challenging to understand its practical applications.

At the recent events I have attended, one major theme seems to be communicating more clearly in this emerging HR technology area. Currently, the primary way this technology is being deployed in HR tech is in the form of using AI to create more specific and tailored recommendations to support HR and HCM processes (for example: job matching, targeted employer value proposition messaging to specific, desired candidates, and recommended actions for managerial coaching and development opportunities for current employees). These and other AI-powered capabilities are demonstrating how this advanced technology can be put to work by HR and organizations without having to “learn” how the AI really works or hire AI-savvy HR staffers.

Expect to see more AI usage, and more examples of AI becoming the “fabric” of HR technology platforms as this technology evolves and organizations become more comfortable with AI-powered HR tools. From what I heard at the three recent conferences, AI offers HR leaders tremendous opportunity and promises to dominate the discussion in HR tech in 2018 and beyond.

Read the rest at HR Executive online...

If you liked the piece you can sign up over at HRE to get the Inside HR Tech Column emailed to you each month. There is no cost to subscribe, in fact, I may even come over and plant your spring garden, take your dog for a walk, or change the oil in your lawn mower.

Have a great day!

Is every company soon to be an 'Artificial Intelligence' company?

A few years back the quote 'Every company is a technology company' made the rounds on social media and in presentations on the workplace, the future of work, and in probably too many TED talks to try and compile.

But while some work and workplace sayings, at least to me, don't necessarily become any more true just because they are repeated all the time, ('Culture eats strategy for breakfast', I am looking right at you), this notion of just about every kind of organization becoming much more reliant, dependent, and committed to more and more advanced technologies as a means to survive, compete, and thrive still seems valid to me.

Can you think of any business, small, medium, or large, that has not had its processes, products, services, communications, administration, customer service, and marketing significantly impacted by new technology in the last decade? Aside from perhaps a few of the very smallest, local service businesses, I can't really think of any. And even those kinds of places, say like a local barbershop or pizza joint, are likely to have a 'Follow us on Facebook' or a 'Find us on Yelp' sticker in the window.

I thought about this idea, of every company being a technology company, again recently when I saw this piece on Business Insider - 'Goldman Sachs made a big hire from Amazon to lead its Artificial Intelligence efforts'. While it isn't surprising or revealing at all to think of a giant financial institution like Goldman being transformed by technology like so many other firms in all industries, this specific focus on AI technology is I think worth noting.

Here's an excerpt from the piece:

Goldman Sachs has hired a senior employee from Amazon to run the bank's artificial-intelligence efforts.

Charles Elkan has joined Goldman Sachs as a managing director leading the firm's machine learning and AI strategies, according to an internal memo viewed by Business Insider.

Elkan comes from Amazon, where he was responsible for the Artificial Intelligence Laboratory at Amazon Web Services, according to the memo. He previously led the retailing giant's Seattle-based central machine-learning team.

"In this role, Charles will build and lead a center of excellence to drive machine learning and artificial intelligence strategy and automation, "Elisha Wiesel, Goldman Sachs' chief information officer, wrote in the memo. "Charles will work in partnership with teams across the firm looking to apply leading techniques in their businesses."

The key element I think to the announcement of Goldman's new AI hire is meant to work with groups across the entire business in order to find ways to apply AI and Machine Learning technologies. Almost as if Goldman is not looking to create the 'AI Department' akin to the classic 'IT Department' that exists in just about every company, but rather to find ways to infuse specific kinds of tech and tech approaches all over the company.

And thinking about AI in that way, much differently to how most companies have looked at most of the major technological advances in the past is what leads me back to the question and title of the post. If the Goldman, (and plenty of other companies too) example of looking for ways to embed AI technology and techniques all across their businesses, then it is not really a stretch to suggest at least in some ways they are seeking to become 'an AI company' at their core.

What's been the most significant single technology advance in the last 25 years or so that has done more to change how work and business get done?

Email?

The web?

Mobile phones?

Probably some combination of these three I would bet. And has any company you have known decided to 'brand' or consider themselves 'an email company?' Or a 'mobile phone' company? 

Not really, these were just tools to try and get better, more efficient, more profitable being whatever kind of company they really were.

So I think the answer to the 'AI question' for Goldman, or for anyone else going all in with AI at the moment is 'No', we aren't really trying become an Artificial Intelligence company. We probably should just consider AI and its potential as just another set of tools that can be leveraged in support of what it is we are really trying to do.

Even if it is tempting to try and create the latest management/workplace axiom.

Have a great week!