Plain-language answer
A probability calculated for a pattern that was only defined after seeing which pattern happened to occur is not a meaningful measure of how surprising that pattern really was. This is sometimes called the “Texas sharpshooter” problem, after the joke about a marksman who fires at a barn wall and then paints a target around wherever the bullet holes happen to cluster. The shots were not aimed at that target — the target was drawn to fit the shots.
Why it matters
Coincidence stories almost always involve a similar move: a match is noticed first (two names are similar, two dates line up, two lives share a detail), and only afterward is a precise “how unlikely is this?” criterion constructed around exactly that match. The resulting probability can look small and impressive, but it describes the odds of a target that was custom-fitted to the outcome — not the odds that were actually running before anyone knew what to look for.
Worked example: HARKing in published research
The clearest documented version of this problem outside coincidence-hunting is what research-methods scholars call HARKing — Hypothesizing After the Results are Known: presenting a hypothesis that was actually formed by looking at the data as though it had been predicted in advance. A methodological analysis names several distinct forms of this practice and reports survey evidence that at least some forms are both common in published psychology research and widely regarded as improper once identified.(Kerr, 1998) The core problem HARKing describes is structurally identical to a “painted-after-the-fact” coincidence claim: a hypothesis (or a matching criterion) that was actually chosen to fit an already-observed result gets presented with the same confidence-inspiring language (“as predicted…”) as a hypothesis that genuinely came first.
Common misconception
“I found the pattern first, and only afterward worked out how unlikely it was — but the math itself is correct, so the conclusion holds” mistakes arithmetic correctness for logical validity. The calculation can be entirely correct as a description of “how likely is this exact pattern, defined this precisely” and still be the wrong number to treat as evidence of anything, because the criterion was fitted to the data rather than specified independently of it. The fix is not better arithmetic — it is asking whether the match was defined before or after the outcome was known, and if after, treating the resulting probability as illustrative rather than evidentiary.
Limits and open questions
Not every post-hoc pattern is meaningless — genuine discoveries are often noticed informally before being tested rigorously, and serendipitous observation is a real, valuable part of how research (and everyday life) generates hypotheses worth investigating further. The distinction that matters is between using a post-hoc observation to generate a hypothesis that then gets tested against fresh, independent data, versus treating the original, pattern-fitted observation itself as if it were confirmatory evidence.
Related
- Multiple opportunities and selection effects covers the closely related effect of choosing which comparison, out of many possible ones, gets reported.
- Selection effects covers the case where the bias operates on which cases are collected rather than which pattern is chosen to describe them.
- How to evaluate a coincidence claim puts “was the match defined before or after the events?” at the center of its checklist.
Key takeaways
- A probability computed for a pattern defined after the outcome was already known describes a target painted around the result, not genuine advance improbability.
- HARKing — presenting a post hoc, results-informed hypothesis as if predicted in advance — is a documented, named version of this same error in published research, not just an informal coincidence-hunting mistake.
- The key diagnostic question for any striking match is simple to state and easy to forget to ask: was this exact criterion specified before or after the events that supposedly satisfy it?