Plain-language answer
“What’s the probability of that?” is never a complete question, because it never says probability relative to what group of comparable cases. A woman named Jane who died at 41 might be compared with all women, all people who died at 41, all people with her specific medical history, or all people who share her exact birth date — and each comparison group (each reference class) can give a different answer, from the same underlying facts. Deciding which class is the right one is a judgment call, not a calculation.
Why it matters
Coincidence claims routinely quote a striking-sounding probability without naming the reference class that produced it. “The odds of two strangers wearing the same unusual outfit to the same event were a thousand to one” — compared with what? All outfit pairings that day? All pairings involving that specific outfit? All pairings the observer would have found equally notable, including different-but-similarly-unusual matches? Each choice changes the number, sometimes by orders of magnitude, and the choice is almost always made after the event, by whoever found it striking.
Philosophers call this the reference class problem, and it turns out to be more than a technical worry about frequentist statistics: it recurs, in some form, across essentially every interpretation of what a probability even is.(Hájek, 2007) There is no algorithm that hands you the one correct reference class for a unique, real-world event — only a set of reasonable candidates that can disagree.
Worked example: one person, several honest answers
Suppose someone asks, “what is the probability that I would run into my old university roommate at a conference in another country?” Reasonable reference classes include:
- all conference attendees who have an old roommate (a huge, generic class — gives a tiny probability for this specific pairing);
- all pairs of people who attended the same university at overlapping times and now work in the same profession (a much smaller, more relevant class — gives a larger probability, because shared professional networks make such reunions far less rare);
- all instances in this person’s life where they attended an event with more than, say, 200 other people (a class chosen to explain why some reunion eventually happened, even if not this exact one).
None of these is “wrong.” They answer different questions. The mistake is picking whichever class makes the number look most dramatic and presenting it as the probability of the event, rather than as one estimate under one explicit model.
Common misconception
“There’s an objectively correct reference class, we just have to find it” is false for most real, one-off events. Hájek’s analysis shows that even in-principle rigorous approaches to probability run into a version of the same problem: any event can be described as a member of many overlapping, equally legitimate classes, and nothing forces a unique choice.(Hájek, 2007) The honest response is not to give up on quantifying anything, but to state the reference class explicitly and show what happens under a couple of reasonable alternatives.
Limits and open questions
Narrower reference classes usually feel more “personal” and relevant, but narrowing a class too far runs out of data and starts smuggling in details that were only noticed because of the outcome — which quietly turns a reference-class choice into another form of the post-hoc matching problem. There is a real tension between specificity and having any usable base rate at all, and no formula resolves it automatically.
Related
- Probability is a model, not a verdict covers the closely related idea that a probability is always relative to a stated model.
- Base rates looks at what happens when people ignore the reference class’s underlying frequency altogether.
Key takeaways
- Every “probability of this happening” claim implicitly picks a comparison class, and different reasonable choices give different numbers from the same facts.
- There is usually no single correct reference class for a unique real-world event — only more or less defensible candidates.
- When a coincidence story quotes long odds, ask what group of events those odds were computed against, and whether a differently framed group would give a very different answer.