Pattern perception and the clustering illusion

Why noticing patterns in random-looking data is usually adaptive, not irrational — and why a famous 'debunking' of one such pattern was later revised.

Established Supported by convergent, high-quality evidence.

Plain-language answer

People are very good at spotting patterns, and that includes spotting patterns that aren’t really there. Truly random sequences — coin flips, shuffled cards, independent events in a life — produce runs, clusters, and streaks far more often than intuition expects, and a mind tuned to notice structure will flag those clusters as meaningful. This isn’t a simple flaw: missing a real pattern is often costlier than imagining a fake one, so a trigger-happy pattern detector can be the better bet even when it misfires some of the time.

Why it matters

Almost every coincidence story starts with this exact mechanism: something in a stream of ordinary events gets flagged as a pattern worth noticing. The question worth asking is not “why did my brain notice that?” — noticing is what pattern-detecting minds do — but “is the pattern unusual given what random, unconnected events actually look like?” Those two questions have different answers far more often than intuition suggests.

Worked example: the “hot hand” and its own reversal

The best-known test case is the “hot hand” in basketball: the belief that a player who has just made several shots in a row is more likely to make the next one. A famous 1985 study analysed real shooting records and found no statistical evidence for it — makes and misses were about as clustered as a fair coin toss would produce, suggesting that “hot streaks” were a clustering illusion rather than a real phenomenon.(Gilovich, Vallone & Tversky, 1985)

That finding became one of the most cited examples of a cognitive bias in the research literature. It has also since been revised. In 2018, economists identified a subtle statistical bias in how the original analysis — and its many replications — measured “streakiness”: a bias that systematically understated any real hot-hand effect. Correcting for it reverses the original conclusion, recovering evidence consistent with a real, if modest, hot hand.(Miller & Sanjurjo, 2018)

This is worth sitting with rather than smoothing over. The clustering illusion is real and well documented as a general tendency — but whether any specific streak (in basketball shooting, or in your own life) reflects a real pattern or an illusory one is an empirical question that can be revisited, and sometimes overturned, on better analysis. That is exactly the stance this site tries to take toward coincidence claims generally: treat a surprising pattern as worth investigating, not as self-evidently either real or illusory.

Common misconception

“You only think that’s a pattern because of a cognitive bias” is sometimes used as if it settles the question. It doesn’t — it’s a hypothesis about why something was noticed, not a finding about whether a real pattern exists. The hot-hand reversal above is a case where the “it’s just a bias” explanation was itself the one that needed correcting.

Limits and open questions

The corrected hot-hand analysis is still debated in parts of the sports analytics literature, and the underlying statistical issue — selection bias in measures of streakiness within a fixed-length sequence — turns out to be subtle enough that it escaped notice for over three decades. That is a useful caution about how long a plausible, well-cited finding can stand before a methodological wrinkle is found in it.

  • The taxonomy’s “repetition/clustering” lens asks exactly the question this page is about: is a clustering unusual under the process that’s actually generating it?
  • How to evaluate a coincidence claim applies the same “don’t settle for the first plausible explanation” discipline to individual cases.

Key takeaways

  • Spotting patterns in random-looking data is a normal, often adaptive feature of how minds work — not evidence of irrationality by itself.
  • Whether a specific noticed pattern is statistically real is a separate, empirical question from why it got noticed.
  • Even a famous, decades-old “this was just an illusion” finding can turn out to rest on a subtle statistical error — a reason for humility in either direction, not a reason to stop asking the question.

Sources

  1. Gilovich, Vallone, Tversky (1985). The Hot Hand in Basketball: On the Misperception of Random Sequences. Cognitive Psychology, 17(3), 295-314. https://doi.org/10.1016/0010-0285(85)90010-6 ↩
  2. Miller, Sanjurjo (2018). Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers. Econometrica, 86(6), 2019-2047. https://doi.org/10.3982/ECTA14943 ↩