Two modes of thinking when you use AI

Lesson 1 · Thinking Fast and Slow × AI · about 10 minutes

Mission: notice when fast thinking is accepting or dismissing AI output without checking it.

The core idea

System 1 is fast, automatic, and intuitive. System 2 is slow, effortful, and deliberate. System 2 is supposed to check System 1, but it is lazy and often accepts whatever System 1 proposes (overview).

AI output arrives fluent, confident, and instant. That invites System 1 to accept it without a second look. Research on automation bias finds people over-rely on automated recommendations, and the tendency rises with workload (Goddard et al. systematic review).

Cognitive surrender

Improving term

Cognitive surrender: at Improving, we treat cognitive surrender as a risk to watch for whenever we use AI.

The term comes from Shaw and Nave at the Wharton School, who describe it as AI output accepted with minimal scrutiny while human reasoning is no longer engaged (Shaw and Nave, 2025). Our working definition draws on industry summaries (Insights, Workday).

It differs from cognitive offloading. When you use a calculator or a calendar, the tool handles a routine sub-task and your own reasoning stays in charge. Cognitive surrender hands the core judgment and the analytical steering to the machine (World Economic Forum). The passage above describes the conditions that make surrender likely: fluent output, a fast System 1 response, and high workload.

Shaw and Nave report three experiments with 1,372 participants. When the AI gave faulty answers and people consulted it, most followed the wrong answer: 79.8 percent in their first study, and 73.2 percent across all studies on those trials (Shaw and Nave, 2025; figures as summarized by an interactive explainer, to be checked against the paper). They also report that the AI makes people more confident even when it is wrong. The authors call the third system AI, which sits outside the brain and shapes how System 1 and System 2 operate.

Two ways to go wrong

An error of commission is following a wrong AI recommendation. An error of omission is missing a problem the AI did not flag. Both come from letting the system's output stand in for your own check.

Practice

For each scenario, choose the mode most likely driving the response.

1. The AI summary of a 40-page contract says "no termination penalty." It reads cleanly, so you forward it to legal without opening the contract.
Correct. Quick, fluent acceptance is System 1 at work.
Not quite. Notice how fast the response happened and whether any checking occurred.
2. The AI gives a confident fact that contradicts something you know, and you stop to check the source before deciding which to believe.
Correct. Stopping to check a conflict is System 2 at work.
Not quite. Stopping to check a source takes effort, which is the System 2 signature.
3. You reject an AI draft in one second because the first sentence sounds generic, without reading the rest.
Correct. A snap rejection on a single sentence is System 1.
Not quite. A snap judgment from one sentence is System 1.

Takeaway

The fluency of AI output is a System 1 signal, not evidence of accuracy. Before acting on AI output, name the one claim that matters most and verify it against a source you control.

Try it this week: pick one AI answer you would normally accept, and check its single most important claim.

Sources

Questions? Stuck on any idea? Ask your teacher in this conversation to explain it another way or give a new example. Next: Lesson 0002, anchoring.
← Course index Next →