Framing: same facts, different answers

Lesson 5 · Thinking Fast and Slow × AI · about 10 minutes

Mission: notice when my wording, rather than the facts, is steering what the AI tells me.

The core idea

A framing effect occurs when logically equivalent descriptions lead to different choices. Kahneman and Tversky's classic example compares a 90 percent survival rate with a 10 percent mortality rate: the same risk, very different reactions. Kahneman discusses framing in the Part IV chapters on choices (see Sources).

Language models are also sensitive to wording. A January 2026 preprint tested 14 LLM judges with paired prompts that differed only in whether the framing was positive or negative. Every judge showed framing bias, and in one example the same response was judged "Yes" under both framings (When Wording Steers the Evaluation, arXiv 2601.13537). A second study found that changing the framing of comparison questions shifted answers across GPT, Claude, and Qwen models (More or Less Wrong, arXiv 2506.03923).

Practice

Choose the bias the scenario shows.

1. You ask, "Is this plan risky?" and get a list of dangers. You later ask, "Is this plan safe?" and get a list of strengths.
Correct. The same plan was described as risky or safe, and the emphasis changed.
Not quite. The facts did not change, only the wording did. That is framing.
2. "80 percent succeed" and "20 percent fail" describe the same plan. Your preference changes depending on which one you read.
Correct. The percentages are the same, and the wording of the outcome changed the answer.
Not quite. The two options carry the same information, so the change comes from framing.
3. You want to know if an AI verdict depends on how you phrased the question. What is the best test?
Correct. Asking the opposite framing reveals whether your answer depends on wording.
Not quite. A second run with the opposite framing is the test that exposes framing dependence.

Takeaway

Before trusting an AI judgment, rephrase the question in the opposite direction and compare. If the answer flips, the wording is doing the work.

Try it this week: take one recurring question you ask AI, such as "Is this a good idea?", and ask it in both positive and negative form. Note any difference.

Sources

Questions? Stuck on any idea? Ask your teacher in this conversation. Previous: Lesson 0004. Next: Lesson 0006, fluency.
← Previous Next →