41 · Researching
The Say-Do Gap
What people say they do, or would do, often isn't what they actually do.
Evidence: Holds, with limitsWhat we do with it
- 01
Watch people use the thing. Ask questions afterward to understand what you saw.
- 02
Ask about the last time it happened. Questions about what people usually do, or would do, get guesses.
- 03
Treat a feature request as a clue to a problem, and go find the problem.
Examples
Getting it wrong, getting it right.
Don't
A survey asks whether people would use a new feature. Most say yes. It ships, and almost nobody turns it on.
Do
Watch a handful of people try a rough version of it first, then decide.
The evidence
What the research says.
Nisbett and Wilson's review was a landmark, and it has critics. K. Anders Ericsson and Herbert Simon showed in 1980 that people can report what they're thinking while they do a task fairly reliably. What doesn't hold up is asking afterward why they did something, or what they'd do in the future. Thomas Webb and Paschal Sheeran pooled 47 experiments in 2006 and found that a medium-to-large change in what people intended (d = 0.66) produced only a small-to-medium change in what they did (d = 0.36). That's why usability tests ask people to think aloud as they go. Morten Hertzum and colleagues checked in 2009 that classic think-aloud changes little about how people behave, apart from making tasks take longer. The chattier, relaxed kind changes more.
How our view changed
We trust what people say while they work, and treat their reasons and predictions afterward as leads to check.
Sources
- Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3).
- Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3).
- Webb, T. L., & Sheeran, P. (2006). Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132(2).
- Hertzum, M., Hansen, K. D., & Andersen, H. H. K. (2009). Scrutinising usability evaluation: Does thinking aloud affect behaviour and mental workload? Behaviour & Information Technology, 28(2).
- Nielsen, J. (2001). First rule of usability? Don't listen to users. Alertbox column, August 2001.
Last reviewed November 7, 2026. How we keep this current