If you toss a coin ten times, getting heads seven times is not all that rare. But if you toss the same coin ten thousand times, the proportion of heads comes remarkably close to half. Each individual toss is completely unpredictable, yet when many are put together, the result becomes astonishingly stable. The principle that explains this phenomenon is the law of large numbers.
Put simply, the law of large numbers is a fundamental theorem of probability theory stating that the more times independent trials are repeated under the same conditions, the closer the observed average or proportion gets to the theoretical expected value. Jacob Bernoulli was the first to prove it, in work published in 1713, and he called it his ‘golden theorem.’ Today it underpins almost every field that deals with risk in numbers, from calculating insurance premiums to opinion polls and running casinos.
This article explains, in easy-to-understand terms, what the law of large numbers means and its basic principle, the history of how the law came about, the misconceptions people commonly fall into, and examples of how it is used in everyday life.
How the Law of Large Numbers Works and How It Is Used
What Is the Law of Large Numbers?
When you roll a die once, there is no knowing which face from 1 to 6 will come up. In theory, however, the average of a die's faces, that is, the expected value, is (1+2+3+4+5+6) ÷ 6 = 3.5. Over the first few rolls the average may be 2 or 5, but the more hundreds or thousands of times you roll, the more the average of the faces stays close to 3.5.
(1) Sample mean and expected value
The average of the values actually observed is called the sample mean. The law of large numbers says that as the number of trials n grows, the probability that the sample mean strays far from the expected value approaches zero.
(2) Two forms
In mathematics, the law is described in two strengths.
- Weak law of large numbers: If the number of trials is large enough, the probability that the sample mean lies close to the expected value approaches 1.
- Strong law of large numbers: The stronger statement that if the trials go on forever, the sample mean almost surely converges to the expected value.
How Did the Law of Large Numbers Come About?
People have long known from experience that if you try something many times, the results even out. The first person to prove this intuition mathematically was the Swiss mathematician Jacob Bernoulli.
(1) Bernoulli's golden theorem
Bernoulli worked on the problem for about 20 years, and his results were published in 1713, after his death, in Ars Conjectandi (The Art of Conjecturing). He was so proud of the result that he called it his ‘golden theorem.’
(2) Poisson gives it a name
The name ‘law of large numbers’ was first used in 1837 by the French mathematician Siméon Denis Poisson. Chebyshev later offered a rigorous proof in 1846, and in the 20th century Borel and Kolmogorov completed the strong law of large numbers, making it a pillar of modern probability theory.
Common Misconceptions About the Law of Large Numbers
The law of large numbers is often misinterpreted. The most typical example is the gambler's fallacy.
(1) The gambler's fallacy
In August 1913, black came up 26 times in a row on a roulette wheel at the Monte Carlo Casino in Monaco. Convinced that red was now due, people bet more and more money and lost heavily. But a roulette ball has no memory of previous results, so the probability of red stays the same every time.
(2) Diluted, not made up for
The law of large numbers does not mean that earlier lopsided results get paid back later. As the number of trials grows, the initial imbalance is simply diluted in an enormous number of new results, so the proportion approaches the expected value. That is why the difference between the number of heads and tails can actually grow.
(3) The law of small numbers
The psychologists Tversky and Kahneman pointed out, calling it the ‘law of small numbers,’ that people tend to believe a handful of results represents the whole. Jumping to conclusions from a few cases amounts to using the law of large numbers in reverse.
Where Is the Law of Large Numbers Used?
The law of large numbers makes things that are unpredictable for an individual calculable for a group as a whole. Thanks to it, many systems that manage uncertain risks in numbers have become possible.
- Insurance: There is no knowing when any one person will have an accident, but with many policyholders, the total number of accidents becomes predictable, so a fair premium can be set.
- Opinion polls: The larger a randomly drawn sample, the closer the sample's approval rating comes to the actual approval rating of the whole population.
- Casinos: A customer may win a single game, but as countless games add up, the expected value that favors the casino shows up directly as profit.
- Monte Carlo simulation: A calculation method that estimates the answer to a complex problem by repeating random experiments on a computer countless times.
However, for the law to hold, each trial must be independent and conducted under the same conditions, and the sample must not be skewed. If data are collected in a biased way, no matter how much you collect, they will only converge on the wrong value.
The law of large numbers shows that order lies hidden even in a world full of chance. Rather than swinging between joy and despair over a single result, we need to learn to see the trends that emerge once enough experience has accumulated.
An attitude of not rushing to judgment from a few cases: that is the first step toward sound statistical thinking.
