All principles

    42 · Automating

    Misuse and Disuse

    People either lean on automation too much or refuse it, and both cost them.

    Evidence: Holds, with limits

    What we do with it

    1. 01

      Expect both in the same room. Some people won't touch the tool, and some won't question it.

    2. 02

      One visible mistake can turn people off a tool that does better than they do. Show its track record, not just its latest miss.

    3. 03

      False alarms teach people to ignore alarms. Keep them rare, or the real ones go unanswered.

    4. 04

      Automate for the people using it. Handing them whatever was left over after automating is the mistake the research calls abuse.

    Examples

    Getting it wrong, getting it right.

    Don't

    A security warning that fires on every sign-in from a new browser, so people learn to click past it.

    Do

    A warning that fires rarely, says why, and is right when it does.

    The evidence

    What the research says.

    Evidence: Holds, with limits

    Parasuraman and Riley's review found that false alarms are a common cause of disuse: an alarm that keeps going off for nothing teaches people to ignore it. Both directions show up in later experiments. Berkeley Dietvorst, Joseph Simmons and Cade Massey found in 2015 that people who saw a forecasting algorithm make mistakes were less likely to choose it, even when it had beaten a human forecaster. Jennifer Logg, Julia Minson and Don Moore found the reverse in 2019: lay people took advice more readily when they thought it came from an algorithm, while experienced forecasters leaned on it less, and their accuracy suffered. Which way people err depends on who they are and what they've just seen.

    How our view changed

    We don't assume people will over-trust a tool or under-trust it. We find out which, for these users and this task.

    A second look

    Is your product breaking one of these?

    Most teams can't tell from the inside. Send us the page or the flow that's not working, and we'll tell you which of these we'd look at first.

    Email hello@synergyprod.com