Plain-language answer
Showing that an event is consistent with chance under a reasonable model is not the same as showing that chance is what actually happened, and it is not the same as proving that no other cause was involved. This site spends most of its effort showing why events that feel impossible are often statistically unremarkable — but that project has a limit worth stating plainly: “not statistically surprising” is not the same conclusion as “definitely no hidden cause,” and readers deserve that distinction stated directly rather than glossed over.
Why it matters
A statistical test that fails to find something unusual is answering a narrower question than it might appear to: “is this result more surprising than an ordinary chance process would produce?” A “no” answer to that question does not mean “chance is confirmed as the explanation” — it can also mean the test lacked the power to detect a real effect, that the wrong model was used as the comparison, or that important information was simply never included in the analysis. Statisticians have a compact name for this gap: absence of evidence is not evidence of absence.(Altman & Bland, 1995)
Worked example: two different kinds of “we didn’t find anything”
Imagine a study designed to detect a modest effect, using a sample too small to reliably detect an effect of that size even if it were real. A null result from that study is genuinely uninformative about whether the effect exists — the study simply wasn’t equipped to tell. Compare that with a large, well-powered study specifically designed to detect an effect of that size, which also finds nothing: that result is much stronger evidence that the effect, if it exists at all, is smaller than the study could detect. Both studies report “no significant effect found,” but they support very different conclusions, and confusing the two is a common way that weak evidence for “it’s just chance” gets treated as if it were strong evidence.
The same caution applies directly to coincidence claims analyzed on this site: showing that a specific pattern is not unusually rare under a stated model (say, once multiple opportunities and dependence are properly accounted for) is a real, useful finding — but it is a claim about that specific model’s predictions, not a comprehensive proof that every possible alternative explanation, including ones nobody has proposed or tested, is ruled out.
Common misconception
“You’ve shown this could have happened by chance, so you’ve shown it did happen by chance” overstates what a probability calculation can establish. A probability estimate under a stated model shows that the model can account for the observed event without needing an additional explanation — it does not, on its own, rule out every other possible explanation, particularly ones the model never considered in the first place.
Limits and open questions
This page is not an invitation to treat every explained coincidence as secretly unexplained after all — most of the specific analyses on this site give good, well-sourced reasons to prefer the chance-based explanation over untested alternatives. The point is narrower: state findings with the precision they actually support (“consistent with chance under this model,” “no evidence found for X given this test’s power”) rather than overstating them into a certainty (“proven to be nothing but chance”) that the underlying method cannot actually deliver.
Related
- What is a coincidence? makes the same distinction at the definitional level — “no established causal connection” does not mean “no cause exists.”
- Selection effects covers one common reason a chance-based analysis can miss something real: the visible data was already filtered in a way the model didn’t account for.
Key takeaways
- Showing an event is unsurprising under a stated chance model is a real, useful finding — it is not the same as proving no other cause exists.
- “Absence of evidence is not evidence of absence”: a null result can mean there is truly nothing there, or it can mean the test wasn’t equipped to detect what was there — these require different follow-up.
- Precise, appropriately hedged language about what a probability analysis does and doesn’t establish is part of this site’s editorial standard, not an occasional caveat.