Representativeness and base rates

Lesson 4 · Thinking Fast and Slow × AI · about 10 minutes

Mission: stop AI answers that fit a stereotype from overriding how common things actually are.

The core idea

The representativeness heuristic judges probability by how well something matches a mental prototype. Kahneman and Tversky linked it to base rate neglect: people underweight how common a category is when a description seems to fit a type. Their 1973 engineer-lawyer study showed participants leaning on a personality sketch over the known group proportions (overview, lower trust, check the book for the original).

This matters for AI in two ways. The model can produce a stereotyped answer that fits a prototype, and you can accept a judgment about a person or case because the description sounds typical while ignoring how rare the outcome is. Researchers also debate how far these results generalize, so treat the effect as a useful caution, not a law.

Practice

Choose the bias the scenario shows.

1. An AI screening tool calls a candidate "probably a nurse" because the résumé sounds caring. Very few applicants in the pool are nurses.
Correct. The description fit a prototype and the base rate was ignored.
Not quite. The judgment rests on how well the candidate fits a stereotype, not on how common nurses are among applicants.
2. A test is 95 percent accurate. The condition affects 1 in 1,000 people. Your result is positive, and you conclude you are 95 percent likely to be sick.
Correct. Most positives come from the large healthy group, so the chance you are sick is much lower than 95 percent.
Not quite. The rarity of the condition is the base rate, and ignoring it is base rate neglect.
3. You want the AI to judge a case fairly. What should you ask for first?
Correct. The base rate sets the starting point before the description is weighed.
Not quite. Ask for the base rate first, before the vivid description shapes the answer.

Takeaway

When an AI judgment depends on how well a case "fits," ask how common the outcome is in the relevant population, and whether the fit information actually predicts it.

Try it this week: pick a prediction the AI made about a person, customer, or risk. Ask it for the base rate, then compare its answer to its first one.

Sources

Questions? Stuck on any idea? Ask your teacher in this conversation. Previous: Lesson 0003. Next: Lesson 0005, framing.
← Previous Next →