Left on the Plate

The Cookie Aroma and the Non-Replication

Eli: In 1998, psychologist Roy Baumeister's team baked chocolate-chip cookies inside the lab, filling it with the aroma of warm cookies. They called it a taste study.

Mara: And the cookies weren't alone: there were chocolate candies, with red and white radishes sitting alongside them.

Eli: The students had been asked to miss a meal. They were also supposed to go at least 3 hours without eating.

Mara: One group could have the radishes, but had to leave the chocolate alone.

Eli: Our listener asks: Why did Roy Baumeister's 1998 ego-depletion radish experiment fail to replicate in massive multi-lab trials? With all those papers, I expected independent labs to find something.

Mara: Today's angle is why Roy Baumeister's 1998 ego-depletion radish experiment failed to replicate. Except those big trials didn't repeat the radish setup. Ego depletion says willpower works like a limited fuel supply. Use some on a tough task, and you temporarily have less for the next.

Eli: So the cookies weren't even part of the test?

Mara: In 2016 they used computer tasks. Those came from Chandra Sripada, Daniel Kessler and John Jonides's 2014 study.

Eli: So fewer differences between labs. And less involvement from whoever ran the experiment.

Mara: They started with a letter-crossing test. A computerized thinking task followed. Unsafe Science reports the team wanted a quick, uniform procedure that international labs could run with few changes.

Eli: Yet that original paper built a whole research field. It picked up more than 4,800 citations.

Mara: More than 600 independent studies followed. Over 2,000 researchers worked on the effect.

Eli: Thousands of researchers. How did those cookies and radishes become the case for willpower as a fuel tank?

The Willpower Muscle

Mara: The 1998 trial began with 67 undergraduates earning course credit, seated individually beside fresh chocolate-chip cookies and raw radishes.

Eli: And someone was watching through a one-way mirror, making sure the radish eaters did not touch the forbidden sweets.

Mara: Next came unsolvable geometric line puzzles. The participants assigned to radishes quit after roughly eight minutes.

Eli: Eight minutes, whereas the cookie eaters held out more than twice as long.

Mara: And the no-food control group held out even longer.

Eli: The radish eaters rang their quit bell ten minutes before the cookie eaters called it quits.

Mara: That drop anchored Baumeister's strength model. He claimed restraining an impulse depletes the limited willpower battery required for later persistence.

Eli: Treating human restraint just like a muscle tiring from exertion?

Mara: Exactly. Resisting temptation wears down your willpower muscle, leaving less fuel for the next hurdle.

Eli: That sounded persuasive, and by 2010, Martin Hagger and Nikos Chatzisarantis synthesized nearly two hundred study estimates.

Mara: They tracked Cohen's d, a statistical ruler measuring effect size: 0.2 marks a subtle nudge, while 0.5 is noticeable.

Eli: And 0.62 is a clear, heavyweight difference, which was the overall score Hagger reported.

Mara: Across the published articles, that sizable effect made the whole theory seem rock solid.

Eli: Wait. How did a grand theory of human willpower convince everyone when each test condition averaged barely 27 participants?

Mara: Groups that small dragged estimated statistical power down between 0.31 and 0.69. That is the probability that a study will successfully detect a genuine effect if one actually exists.

Eli: Way below the scientific benchmark of 0.80.

Mara: To reliably catch a modest effect, an experiment actually required 252 participants per condition.

Eli: So a huge consensus rested on fragile footing. What happened when independent laboratories finally put this willpower claim to an unbending, preregistered test—publicly committing to the study plan and statistical analyses before collecting data?

The Zero-Effect Reckoning

Mara: It hit a wall. When outside teams finally tested the willpower muscle under strict conditions, the effect completely vanished.

Eli: In 2016, Martin Hagger organized a Registered Replication Report spanning twenty-three independent laboratories. They tested 2,141 participants on a computerized sequential task.

Mara: Over 2,100 people across twenty-three sites.

Eli: Replication researchers reported an average effect on reaction-time variability of just d equals 0.04. That measures how inconsistently someone responds, where fluctuating speeds signal that self-control is slipping. Only two labs out of the twenty-three observed a statistically significant outcome.

Mara: It flatlined. That happened because the consortium required preregistration: filing your research recipe and math plans in a public registry before collecting data, so you cannot change the rules after seeing the results.

Eli: So researchers could not adjust their criteria after looking at the numbers?

Mara: Not at all. Every lab logged its design on the Open Science Framework, where editor Alex O. Holcombe vetted it against the protocol beforehand, and preregistration ensured every single result had to be reported.

Eli: And that was not an isolated project. Replication researchers also analyzed the Many Labs 3 dataset.

Mara: The one running the classic Stroop task?

Eli: Right, naming ink colors while overriding the printed word. In that reanalysis of over two thousand participants, finishing the exhausting task had no measurable impact on anagram persistence.

Mara: Not even a slight drop?

Eli: Zero. Psychologist Miguel Vadillo explained that this sample size would have caught a real effect, adding that 'the published record of ego depletion was too good to be true.'

Mara: As psychologist Michael Inzlicht observed, finding no discernable effect seems to contradict two decades of published papers.

Eli: I have to admit, I assumed that massive published consensus guaranteed a real signal. But replication researchers found an effect indistinguishable from zero, which Martin Hagger and Nikos Chatzisarantis noted raises considerable doubts. If the effect vanishes when tested rigorously, how did hundreds of published studies manage to find it in the first place?

The Ghost in the Funnel Plot

Mara: Those earlier findings were steered by publication bias. Simply put, scientific journals eagerly print exciting, positive breakthroughs while leaving messy or null results buried in desk drawers.

Eli: Any trial that stumbled simply disappeared from sight.

Mara: In 2014, Evan Carter and Michael McCullough mapped the field using a funnel plot. That graph displays sample size against outcomes: when smaller, imprecise trials lean solely one way and the other side is empty, unpublished studies were filtered out.

Eli: The entire counterweight had vanished.

Mara: Once Carter's group adjusted for that distortion with statistical regression models, they found the true effect was indistinguishable from zero, dealing a heavy blow to the resource idea.

Eli: Carter and McCullough only started scrutinizing those papers after failing to produce ego depletion in their own lab.

Mara: They were not alone. Michael Inzlicht noted that researchers had spent the 2000s quietly whispering about how difficult the effect was to replicate.

Eli: In fact, Jordan Peterson explicitly cautioned Inzlicht back then to avoid the entire topic because his own replication attempts kept failing.

Mara: Still, positive papers flooded journals. In 2022, Ulrich Schimmack evaluated over 1,800 published hypothesis tests across the ego-depletion literature to map the distribution of evidence.

Eli: And the findings piled up right against the standard significance border, at p equals.05. Point zero five is the conventional threshold researchers use to declare statistical significance.

Mara: Based on their statistical power, Schimmack calculated that just 13% of those tests should have succeeded. That enormous discrepancy suggested up to 34% of positive claims were false alarms.

Eli: Were hundreds of researchers actively fabricating numbers, or just caught up in perverse career incentives?

Mara: It was loose methodology. Researchers ran one-tailed tests—evaluating an effect in only one direction, which makes it easier to clear the threshold for statistical significance—added unjustified control variables, or rounded borderline p-values down to claim a discovery.

Eli: And journal gatekeepers encouraged it. Roy Baumeister admitted an editor instructed him to drop flatter experiments "so as not to use up a lot of journal space with mediocre results." He defended deleting the weaker studies to conserve page count—nothing polishes an empirical theory quite like saving paper.

Mara: Carter and McCullough emphasized that this distortion reached far beyond ego depletion. In the landmark 2015 Reproducibility Project, out of 97 psychology studies retested, only about 36% held up.

Eli: And across social psychology experiments, just 25% replicated.

Mara: That reckoning turned ego depletion into the defining poster child of psychology's replication crisis, fueling a credibility revolution focused on transparency and open records. But it leaves the core dilemma: was self-control depletion an illusion all along, or did the replication consortia bungle their test?

The Battery That Wasn't

Eli: Roy Baumeister argued the massive trials missed the point entirely. In retrospect, he and Kathleen Vohs called the 2016 design foolish because crossing out letters lacked a habit-forming phase.

Mara: Right, without building that initial habit, they insisted participants had nothing to override. And defenders pointed out that the replication's manipulation check showed no meaningful difference in fatigue between conditions.

Eli: A manipulation check is just an internal test to verify that an experiment really provoked the psychological state researchers intended. Baumeister reported that tapping keys on a screen could never match resisting warm cookies.

Mara: The sensory aroma was powerful, but Vohs decided to run her own preregistered test. In 2021, she coordinated thirty-six labs using tasks picked by proponents.

Eli: And the result barely nudged above zero.

Mara: Her confirmatory analysis, testing a pre-specified hypothesis strictly according to a predetermined plan, reported an effect of just point zero six. The collected data were four times more likely under the assumption of no effect.

Eli: On figure tracing alone, depleted volunteers did give up sooner. And in real life, And proponents point to Katherine Milkman's research finding that compliance erodes during demanding shifts.

Mara: She tracked healthcare staff across thirteen million compliance records. Handwashing started declining after about two hours on duty.

Eli: Two hours is a long stretch. A 2025 study led by Junhua Dang found that stretching laboratory tasks to forty minutes produced clear self-control slips.

Mara: Yet Michael Inzlicht notes that doesn't prove an energy reservoir ran dry. If self-control failure came from losing a biological fuel, mental shifts couldn't suddenly restore it.

Eli: Inzlicht still believes depletion is real, but he argues the dip reflects changing priorities and plain fatigue. Willpower isn't a battery that empties; your attention simply drifts.

Mara: That reframing collapses the strength model. When motivation or priorities shift, self-control bounces right back.

Eli: Still, Baumeister refused to budge, entitling a 2022 paper with the claim that depletion was social psychology's best-replicated finding. As late as 2024, he maintained that stance.

Mara: So to answer our listener, the radish experiment never failed in the big replication trials because nobody actually repeated it. Researchers swapped the food for standardized software, and the phantom resource vanished.

Eli: And that is why the radish experiment became the face of a scientific collapse it technically never took part in.

Mara: Look into ego depletion.

Eli: And follow science with us to see which established theories survive testing.

Mara: That's it for today. I'm Mara.

Eli: And I'm Eli. Thanks for listening, and we'll catch you next time on Angle.