Enter your email address:

Delivered by FeedBurner


E-mail Steve
This form does not yet contain any fields.
    Listen to internet radio with Steve Boese on Blog Talk Radio

    free counters

    Twitter Feed

    Entries in Technology (239)


    On trusting algorithms, even when they make mistakes

    Some really interesting research from the University of Pennsylvania on our (people's) tendency to lose faith and trust in data forecasting algorithms (or more generally, advanced forms of smart automation), more quickly than we lose faith in other human's capabilities (and our own capabilities), after observing even small errors from the algorithm, and even when seeing evidence that relative to human forecasters, the algorithms are still superior.

    From the abstract of Algorithm Aversion: People Erroneously Avoid Algorithms After Seeting Them Err:

    Research shows that evidence-based algorithms more accurately predict the future than do human forecasters. Yet, when forecasters are deciding whether to use a human forecaster or a statistical algorithm, they often choose the human forecaster. This phenomenon, which we call algorithm aversion, is costly, and it is important to understand its causes. We show that people are especially averse to algorithmic forecasters after seeing them perform, even when they see them outperform a human forecaster. This is because people more quickly lose confidence in algorithmic than human forecasters after seeing them make the same mistake. Participants who saw the algorithm perform were less confident in it, and less likely to choose it over an inferior human forecaster. This was true even among those who saw the algorithm outperform the human.

    Let's unpack that some. In the research conducted at Penn, the authors showed that even when given evidence of a statistical algorithm's overall superior performance at predicting a specific outcome (in the paper it was the likelihood of success of MBA program applicants that the humans and the algorithm attempted to predict), most people lost faith and trust in the algorithm, and reverted to their prior, inferior predictive abilities. And in the study, the participants were incentivized to pick the 'best' method of prediction: They were rewarded with a monetary bonus for making the right choice. 

    But still, and consistently, the human participants more quickly lost and faith and trust in the algorithm, even when logic suggested they should have selected it over their (and other people's) predictive abilities.

    Why is this a problem, this algorithm aversion?

    Because while algorithms are proving to be superior at prediction across a wide range of use cases and domains, people can be slow to adopt them. Essentially, whenever prediction errors are likely—as they are in virtually all forecasting tasks—people will be biased against algorithms, because people are more likely to abandon an algorithm than a human judge for making the same mistake.

    What might this mean for you in HR/Talent?

    As more HR and related processes, functions, and decisions become 'data-driven', it is likely that sometimes, the algorithms we adopt to help make decisions will make mistakes. 

    That 'pre-hire' assessment tool will tell you to hire someone who doesn't actually end up beign a good employee.

    The 'flight risk' formula will fail to flag an important executive as a risk before they suddenly quit, and head to a competitor.

    The statistical model will tell you to raise wages for a subset of workers but after you do, you won't see a corresponding rise in output.

    That kind of thing. And once these 'errors' become known, you and your leaders will likely want to stop trusting the data and the algorithms.

    What the Penn researchers are saying is that we have much less tolerance for the algorithm's mistakes than we do for our own mistakes. And maintaining that attitude in a world where the algorithms are only getting better, is, well, a mistake in itself.

    The study is here, and it is pretty interesting, I recommend it if you are interested in making your organization more data-driven.

    Happy Tuesday.


    A feature that Email should steal from the DMV

    In New York State, and I suspect in other places as well, when you visit a Department of Motor Vehicle (DMV) office to get a new license, register your sailing vessel, or try to convince the nice bureaucrats that you did in fact pay those old parking ticket fines, there is generally a two-step process for obtaining services.

    You first enter the office and wait in line to be triaged by a DMV rep, and once he/she determines the nature of your inquiry, you receive a little paper ticket by which you are assigned a customer number, and an estimated waiting time until you will be called by the next DMV agent. You then commence waiting until your number is announced and you can complete your business. 

    That little bit of information, the estimated wait time, is the aspect of the DMV experience that I think has tons of potential for in other areas, most notably in Email communications. The DMV estimates your wait time, (I imagine), in a really simplistic manner. It is a function of the number of customers waiting ahead of you, the number of DMV agents available, and the average transaction time for each customer to be served. Simple math, and probably is pretty accurate most of the time.

    The Email version of the 'Estimated Wait Time' function would be used to auto-reply to every (or selected) incoming email messages with a 'Estimated Response Time' that would provide the emailer with information about how long they should expect to wait before receiving a reply. 

    How would this work, i.e., what would the 'Estimated Response Time' algorithm need to take into account? Probably, and at least the following data points.

    1. The relationship between the sender and the recipient - how frequently emails are exchanged, how recent was the last exchange, and what has been the typical response time to this sender in the past

    2. The volume of email needing action/reply in the recipient's inbox at the time the email is received, and how that volume level has impacted response times in the past

    3. The recipient's calendar appointments (most email and calendar services are shared/linked), for the next 1, 3, 12, 24, etc. hours. Is the recipient in meetings all day? Going on vacation tomorrow? About to get on a cross-country flight in two hours?

    4. The subject matter of the email, (parsed for keywords, topics mentioned in the message, attachments, etc.)

    5. Whether the recipient is in the 'To' field or in the 'CC' field, whether there are other people in the 'To' and 'CC' fields, and the relationship of the recipient to anyone else receiving the email

    And probably a few more data points I am not smart enough to think of in the 20 minutes or so I have been writing this.

    The point?

    That a smart algorithm, even a 'dumb' one like at the DMV, could go a long way to help manage communications, workflow, and to properly set expectations. When you send someone an email you (usually) have no idea how many other emails they just received that hour, what their schedule looks like, the looming deadlines they might be facing, and the 12,458 other things that might influence when/if they can respond to your message. But with enough data, and the ability to learn over time, the 'Expected Response Time' algorithm would let you know as a sender what you really need to know: whether and when you might hear back.

    Let's just hope once the algorithm is in place, we all don't get too many "Expected Response Time = NEVER" replies.

    Now please Google, or Microsoft, or IBM get to work on this.


    It's hard in the modern world: A DisruptHR Cleveland Preview

    Next week I will have the great pleasure of attending and presenting at the DisruptHR Cleveland event to be held Thursday, February 5 at 5:30PM at the Music Box Supper Club  - (event details and registration here).

    The presentations at the DisruptHR events follow the popular 'Ignite' format - each presenter has 20 slides that auto-advance at 15 seconds per slide resulting in a total of 5 minutes to tell their story. It is a fun and exciting, (if a little bit frightening) format for both speakers and the audience.

    My little talk, (and it is almost complete, please relax DisruptHR Cleveland organizers, I will get it to you soon), has a working title of 'It's hard in the modern world'; or, 'A 5-minute review of humanity's relationship with technology'

    As I said, the presentation is not 100% complete, so I won't post it here yet, but I did want to share the central theory behind the talk, and also solicit some ideas and feedback if readers are so inclined that I may consider as I finalize the slides.

    Here it is:

    While 'modern' advances in technology seem incredibly disruptive, the entirety of human history has been nothing but a series of (mostly), technology driven disruptions. Fire, the wheel, metallurgy, farming - these and many more tech advances were just if not more disruptive to humanity than Candy Crush Saga.

    At the end there will be some profound conclusions/recommendations/wisecracks to help sum up and interpret that assertion, but that is the basic idea behind the talk.

    My questions to you, dear readers, are these:

    Are we really in the most technology-driven disruptive period in (at least recent) human history?

    Are things really different now?

    Do I have a chance of convincing the good people of Cleveland that the modern age of technology is not more disruptive than the transition to the Bronze Age from the 'Run or be eaten alive age?'

    Hope to see lots of folks out in Cleveland next week!


    Diversity, testing, and how bugs often go unnoticed

    Recently I was talking to a friend who told me that he was in the market for a new car. My friend, who has been a loyal driver of a particular brand of vehicle for many years, for argument's sake let's call it Lexura (not the actual brand, but since I don't want to get hassled by any PR folks, I am making up this brand). When I asked him if he was considering the latest Lexura luxury model, a brand new one for 2015 that has generated significant buzz and some really stellar initial reviews, my friend surprised me by saying 'No', that he had soured on the Lexura brand.

    When I asked why the conversation went more or less as follows:

    Him: 'I like Lexura, I really do. But the last two Lexura's I owned have had the exact same problem. When it is raining, and I have the windshield wipers on, the water comes right inside the driver's window. I am always getting wet.'

    Me: 'Why would you have the window open if it is raining so much that you have to have the wipers on? That seems a little odd. Most people you know, close the windows when it's raining.

    Him: 'Well, usually I do. But sometimes I am smoking. And I like to keep the window open when I am smoking in the car. And then if it is raining the water comes right in off the wipers.'

    A sort of odd story, and immediately after hearing the 'when I am smoking in the rain the wipers get me wet' take from my friend I starting thinking about software, (and really any other kind of product), that is developed, tested (significantly), and then is released into the wide, strange, and harsh world of customers and users.

    My friend should not, on paper anyway, be driving in the rain, wipers on, driver's side window rolled down, getting wet. It does not really make much sense to most of us. It's raining. Roll up the window, dummy.

    But if you are a smoker, then it is pretty common that when you are smoking and driving that you want/need the window down, at least part of the way. I guess even smokers don't really want to be trapped in a small, enclosed area with their own second hand smoke. So they open the window. And most of the time it works out just fine. Unless it's raining and your Lexura has the bad habit of directing water off of the windshield wipers right into the open window and in your face.

    So let's spin this back to technology and testing and think about how/why 'bugs' like the Lexura spitting water back through the driver's window (let's just assume that it is actually, a bug for now), can make it past the testers and developers and engineers and make it into the world. Could it be, perhaps, that no one on the Lexura design/dev team was a smoker? That driving in the rain with the wipers on and window open was never actually tested, as it would have never occurred to the non-smokers at Lexura that this was actually a thing that would be important to some customers? That this possible lack of 'diversity' in the makeup of the Lexura team led to a bug that was only likely to be experienced by customers whose specific issues were not adequately represented at Lexura?

    This is kind of a odd story, as I mentioned above, but I think there is something important here nonetheless.

    First, it is almost impossible in the design, development, and testing processes of software, hardware, or products of any sort to test everything, every potential use case that is possible. It just cant' be done. Bugs will results, often from customers using the product in a way the builders never considered or even could have reasonably imagined.

    And second, 'diversity', at least the way we usually think about it, is often a very incomplete way to frame the noble notion of ensuring all important and representative voices are heard. Because every time you think you have incorporated ideas and points of view from all the necessary constituencies, one new one you never thought about raises a hand, and wants to be heard.

    Even smokers who drive in the rain with the windows open.

    Have a great week.


    SURVEY: Depressingly, Email remains the most important technology at work

    One of my go-to places for news, data, and research on technology adoption, usage, and trends is the Pew Research Internet Project. Towards the end of 2014, the folks at Pew released a short research report titled Technology's Impact on Workers, a look at how and which kinds of technologies are effecting work and workforces. It is a pretty interesting and easily digestible report, but since I know you are really busy and might not have time to read the entire research report, I wanted to call out one data point, and then we can, together, pause, reflect, and lament for a moment.

    First the data point, take a look at the chart below that displays survey responses to the question of which technologies workers (separated into office workers and non-office workers), consider 'very important' to their jobs:

    Two things stand out from this data. First, and the obvious one (and still exceedingly depressing), is that email remains the most important type of technology cited by office workers for helping them perform their jobs. Despite its relative maturity (and that is putting it nicely, as far as technology goes, email at about 30 years old should have been brought out behind the barn and put out of its misery decades ago), email continues to hold its vise-like death grip on modern office work. I hope I live (or at least work) long enough to see email finally disrupted from this position, but so far alternate workplace communication and collaboration options have not been able to accomplish what (ironically), almost everyone desires - the end of being slaves to email all day.

    The other bit of data from the Pew survey comes from the bottom portion of the chart - the kinds of technologies that workers find not 'very important' to them in getting their jobs done. And in a result that will make the social networking aficionados cringe (and many CIOs who would prefer to block these kinds of things from corporate networks happy), social networking sites like LinkedIn, Twitter, and Facebook were cited as 'very important' by a measly 7 percent of office workers and 2 percent of non-office workers. Now that doesn't mean that these networks are 'not important', based on the way the question was phrased, but certainly the vast disparity in the stated importance of social networks in getting work done compared to email, (general) internet availability, and phones paints a pretty clear picture. For most folks, technology use at work is dominated by email, with web access and phones, (land and mobile), rounding out about 90% of the technology picture.

    I will close with a quote from the Pew report, and then sulk away with my head bowed, dreaming of a better future for our children...

    This high standing for email has not changed since Pew Research began studying technology in the workplace. Email’s vital role has withstood major changes in other communications channels such as social media, texting, and video chatting. Email has also survived potential threats like phishing, hacking and spam and dire warnings by commentators and workplace analysts about lost productivity and email overuse.


    Happy Wednesday.