Why is it that restaurants famous for great food always seem to be tucked away down a back alley or in some out-of-the-way spot? We also often hear that good-looking celebrities can't act, while actors who can act have ordinary looks. As experiences like these pile up, it is easy to conclude that two strengths rarely come together. But this conclusion may be an illusion created not by the world but by the sample we happened to see. This phenomenon is ‘Berkson's Paradox.’
Put simply, Berkson's Paradox means ‘the phenomenon in which a false inverse relationship appears between two unrelated traits when you look only at a group that has been filtered by some criterion.’ In statistics it is also called Berkson's Bias or Collider Bias, and it is a type of selection bias that arises from the way a sample is chosen.
In this piece we will look in turn at the hospital story where Berkson's Paradox was first discovered, how false relationships arise, examples from everyday life, and ways to avoid the illusion.
The False Relationships Created by Berkson's Paradox
What Is Berkson's Paradox?
This phenomenon was first systematically pointed out by Joseph Berkson, a biostatistician at the Mayo Clinic in the United States, in a paper he published in 1946 in the journal Biometrics Bulletin. A researcher who had studied physics, medicine, and statistics, he focused on the pitfall in studies that try to determine the causes of disease using hospital data alone.
(1) A Relationship Seen Only Inside the Hospital
The example Berkson gave was diabetes and cholecystitis (inflammation of the gallbladder). If you gather hospitalized patients, those without diabetes must have come to the hospital for some other reason, so they are more likely than the general public to have another illness such as cholecystitis. As a result, inside the hospital the two diseases appear to avoid each other, even if in the outside world they have no relationship at all.
(2) Similar Names
Even for the same phenomenon, the name differs slightly from field to field.
- Berkson's Bias: A name that refers to the same phenomenon as a problem with the sample.
- Collider Bias: The distortion that arises when a sample is filtered by an outcome variable that two causes both influence.
- Endogenous Selection Bias: The name mainly used in sociology and economics.
How a False Inverse Relationship Arises
The key is ‘a door you can enter only if you have at least one of the two.’ Even if two traits are independent of each other, once you gather only people who have at least one of them, someone who lacks one trait becomes more likely to have the other. That is why, within the filtered group, the higher one is, the lower the other seems to be. Conversely, even if two traits actually move together, within the filtered group that relationship may appear weaker or vanish altogether.
(1) Working It Out with 1,000 Stamps
Suppose a collector has 1,000 stamps, of which 300 are pretty, 100 are rare, and 30 are both. Since 10% of all the stamps and 10% of the pretty stamps are rare, prettiness and rarity are unrelated. But if only the 370 stamps that are pretty or rare are put on display, all 70 of the not-pretty stamps in the display case are rare. Someone who sees only the display case is fooled into thinking that ‘the less pretty a stamp is, the rarer it is.’
(2) What the Name ‘Collider’ Means
In the outcome ‘was put on display,’ the arrows of two causes, prettiness and rarity, collide together. A variable where two causes converge like this is called a collider, and the moment you filter a sample by this variable, a relationship that did not exist before appears.
Berkson's Paradox in Everyday Life
Berkson's Paradox is common outside the hospital too, because most of the groups we see are people or things that have already passed some criterion.
[Case 1] Why Are Good-Looking People Rude?
American mathematician Jordan Ellenberg used dating partners as an example. If you only date people whose combined score for kindness and looks exceeds a certain level, a less kind person has to be better-looking to pass the bar. So among the people you have dated, the impression forms that ‘good-looking people are rude,’ yet in the world as a whole the two traits may have no relationship at all.
[Case 2] A Warning from COVID-19 Research
In 2020, researchers warned about this pitfall in COVID-19 research as well. They pointed out that if you analyze only people who were tested or hospitalized, the many factors that influence testing and hospitalization become tangled together, and risk factors different from reality can appear. It is a case showing that how a sample was gathered can change the conclusion.
[Case 3] The Great Restaurant in a Remote Alley
To survive, a restaurant needs at least one of two things: good food or a good location. A restaurant in a prime spot can hold on thanks to passing customers even if its food is ordinary, but a restaurant in a remote alley stays open only if its food is outstanding. So if you look around only at the restaurants open today, you see the relationship ‘the more remote the location, the tastier the food.’ That is because the restaurants that have already closed have dropped out of the sample.
How to Avoid Berkson's Paradox
To avoid this illusion, you need to check the door the sample came through before looking at the conclusion. If survivorship bias, a similar pitfall, is the problem of looking only at what survived, Berkson's Paradox is the problem of misreading the relationships among things that were filtered.
(1) Ask About the Filter First
Examine who this data was gathered from and by what criterion. If there is a condition that both traits influence, such as hospitalization, acceptance, or fame, you should suspect a false relationship.
(2) Compare with an Unfiltered Group
If possible, compare with data from a group that has not passed through any criterion, such as the general population. If the relationship appears differently in the two groups, it is very likely an illusion created by the process of selecting the sample.
Berkson's Paradox reminds us that the world we see is a world that has already been filtered once. Relationships that seem to run against each other, like taste and location or looks and personality, may look completely different when we open the door of the sample and look again.
Asking first how a group was gathered rather than focusing on the relationship before your eyes is the first step in statistical thinking that is not fooled by numbers.
