Reference-class worksheet

Plain-language answer #

Step 6 of how to evaluate a coincidence claim — “choose the reference class” — is where most informal coincidence analysis goes wrong, because it’s easy to skip past without noticing a choice was even made. This worksheet turns that step into an explicit exercise: writing down two or three candidate reference classes side by side, so the choice (and its consequences for the final answer) is visible rather than hidden.

Why a worksheet, not just a rule #

There is no formula that outputs “the” correct reference class for a unique, real event — the choice is a judgment call, and different defensible choices can produce very different numbers from the same facts.(Hájek, 2007) The most useful discipline available is not finding the one right answer, but making the candidates and their consequences explicit enough that a reader can evaluate the choice for themselves. See reference classes for the underlying concept this worksheet applies.

The worksheet #

For a specific coincidence claim, fill in each row:

1. State the event exactly. What precisely happened, in plain, literal terms, without any interpretation yet attached.
2. List two or three candidate reference classes. For each one, name the group of comparable events or people it compares against. Aim for at least one narrow, “personal” class and at least one broad, “population” class.
3. For each candidate, estimate (even roughly) how often the event type occurs within that class. Mark clearly which estimates are well-grounded (from real data) and which are guesses.
4. Note what each choice implicitly assumes. A narrow class often assumes details that were only noticed because of the outcome (see post-hoc probability); a broad class may average over relevant differences the narrow class would have captured.
5. State which reference class you’re using going forward, and why. There is no need to defend it as objectively correct — only as a reasonable, clearly stated choice that a reader could disagree with and substitute their own.
6. Redo the calculation, if there is one, under at least one alternative reference class. If the conclusion changes substantially, say so explicitly rather than presenting only the version that supports your preferred conclusion.

Common misconception #

“I picked the reference class that seemed most natural, so I don’t need to mention the others” defeats the purpose of the exercise. The worksheet’s value is in showing your work — a reader who sees only one reference class has no way to judge whether a different, equally reasonable choice would have changed the answer.

Limits and open questions #

This worksheet cannot manufacture population data that doesn’t exist — for many everyday reference classes (how often do people think of a friend right before that friend calls?), no one keeps a systematic count, and step 3’s estimates will sometimes have to stay honestly labeled as guesses. That is a legitimate outcome of the worksheet, not a failure of it: knowing precisely which numbers are solid and which are guesses is more useful than a single unlabeled estimate.

Related #

• How to evaluate a coincidence claim — this worksheet expands step 6 of the full twelve-step method.
• Reference classes covers the underlying concept in more depth.

Key takeaways #

• Write down at least two candidate reference classes for any coincidence claim, not just the one that feels most natural.
• Label which frequency estimates are grounded in real data and which are guesses.
• If the choice of reference class changes the conclusion, say so explicitly rather than presenting only the answer you prefer.