Life is uncertain. None of us know what is going to happen. We know little of what has happened in the past or is happening now outside our immediate experience. Uncertainty has been called the “conscious awareness of ignorance”—be it of the weather tomorrow, the next Premier League champions, the climate in 2100 or the identity of our ancient ancestors.
In daily life, we generally express uncertainty in words, saying an event “could,” “might” or “is likely to” happen (or have happened). But uncertain words can be treacherous. When, in 1961, the newly elected U.S. president John F. Kennedy was informed about a CIA-sponsored plan to invade communist Cuba, he commissioned an appraisal from his military top brass. They concluded that the mission had a 30 percent chance of success—that is, a 70 percent chance of failure. In the report that reached the president, this was rendered as “a fair chance.” The Bay of Pigs invasion went ahead and was a fiasco. There are now established scales for converting words of uncertainty into rough numbers. Anyone in the U.K. intelligence community using the term “likely,” for example, should mean a chance of between 55 and 75 percent.
Attempts to put numbers on chance and uncertainty take us into the mathematical realm of probability, which today is used confidently in any number of fields. Open any science journal, for example, and you’ll find papers liberally sprinkled with P values, confidence intervals and possibly Bayesian posterior distributions, all of which are dependent on probability.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
And yet any numerical probability, I will argue—whether in a scientific paper, as part of weather forecasts, predicting the outcome of a sports competition or quantifying a health risk—is not an objective property of the world but a construction based on personal or collective judgments and (often doubtful) assumptions. Furthermore, in most circumstances, it is not even estimating some underlying “true” quantity. Probability, indeed, can only rarely be said to “exist” at all.
Chance interloper
Probability was a relative latecomer to mathematics. Although people had been gambling with astragali (knucklebones) and dice for millennia, it was not until French mathematicians Blaise Pascal and Pierre de Fermat started corresponding in the 1650s that any rigorous analysis was made of “chance” events. Like the release from a pent-up dam, probability has since flooded fields as diverse as finance, astronomy and law—not to mention gambling.
To get a handle on probability’s slipperiness, consider how the concept is used in modern weather forecasts. Meteorologists make predictions of temperature, wind speed and quantity of rain, and often also the probability of rain—say, 70 percent for a given time and place. The first three can be compared with their “true” values; you can go out and measure them. But there is no true probability for the last to compare with the forecaster’s assessment. There is no “probability-ometer.” Either it rains or it doesn’t.
What’s more, as emphasized by philosopher Ian Hacking, probability is “Janus-faced”: it handles both chance and ignorance. Imagine I flip a coin and ask you the probability that it will come up heads. You happily say “50–50” or “half” or some other variant. I then flip the coin, take a quick peek at it, cover it up and ask, “What’s your probability it’s heads now?”
My argument is that any practical use of probability involves subjective judgments. The objective world comes into play when probabilities are tested against realities.
Note that I say “your” probability, not “the” probability. Most people would now hesitate to give an answer before grudgingly repeating “50–50.” But the event has now happened, and there is no randomness left—just your ignorance. The situation has flipped from aleatory uncertainty, about the future we cannot know, to epistemic uncertainty, about what we could but currently do not know. Numerical probability is used for both these situations.
There is another lesson in here. Even if there is a statistical model for what should happen, it is always based on subjective assumptions—in the case of a coin flip, that there are two equally likely outcomes. To demonstrate this concept to audiences, I sometimes use a two-headed coin to show that even their initial opinion of “50–50” was based on their trust in me. Such trust can be rash.
Subjectivity and science
My argument is that any practical use of probability involves subjective judgments. This idea doesn’t mean I can put any old numbers on my thoughts—I would be proved a poor probability assessor if I claimed with 99.9 percent certainty that I can fly off my roof, for example. The objective world comes into play when probabilities, and their underlying assumptions, are tested against reality, but that doesn’t mean the probabilities themselves are objective.
Some assumptions that people use to assess probabilities will have stronger justifications than others. If I have examined a coin carefully before it is flipped, and it lands on a hard surface and bounces chaotically, I will feel more justified with my 50–50 judgment than if some shady character pulls out a coin and gives it a few desultory turns. But these same strictures apply anywhere that probabilities are used—including in scientific contexts, in which we might be more naturally convinced of their supposed objectivity.
Here’s an example of genuine scientific, and public, importance. Soon after the start of the COVID-19 pandemic, researchers working on the RECOVERY trials started to test therapies in people hospitalized with the disease in the U.K. In one experiment, more than 6,000 people were randomly allocated to receive either the standard care given in the hospital they were in or that care plus a dose of dexamethasone, an inexpensive steroid. Among those on mechanical ventilation, the age-adjusted daily mortality risk was 29 percent lower in the group allocated dexamethasone than in the group that received only standard care (95 percent confidence interval of 19 to 49 percent). The P value—the calculated probability of observing such an extreme relative risk, assuming a null hypothesis of no underlying difference in risk—can be calculated as 0.0001, or 0.01 percent.
This is all standard analysis. But the precise confidence level and P value rely on more than just an assumption of the null hypothesis. They also depend on all the assumptions in the statistical model, such as that the observations are independent—that there are no factors that cause people treated more closely in space and time to have more similar outcomes. But there are many such factors, whether it’s the hospital in which people are being treated or changing care regimes. The precise values also rely on all of the participants in each group having the same underlying probability of surviving for 28 days. But this probability will differ for all kinds of reasons.
None of these false assumptions necessarily mean the analysis is flawed. In this case, the signal is so strong that a model allowing, say, the underlying risk to vary among participants will make little difference to the overall conclusions. If the results were more marginal, however, it would be appropriate to do extensive analysis of the model’s sensitivity to alternative assumptions.
To exercise the much quoted aphorism, “all models are wrong, but some are useful.” The dexamethasone analysis was particularly useful because its firm conclusion changed clinical practice and saved hundreds of thousands of lives. But the probabilities that the conclusion was based on were not “true”—they were a product of subjective, if reasonable, assumptions and judgments.
Down the rabbit hole
But are these numbers then our subjective, perhaps flawed estimates of some underlying “true” probability that is an objective feature of the world?
I will add the caveat here that I am not talking about the quantum world. At the subatomic level, the mathematics indicates that causeless events can happen with fixed probabilities (although at least one interpretation states that even those probabilities express a relationship with other objects or observers, rather than being intrinsic properties of quantum objects). But equally, it seems that this possibility has negligible influence on everyday observable events in the macroscopic world.
I can also avoid the centuries-old arguments about whether the world, at a nonquantum level, is essentially deterministic and whether we have free will to influence the course of events. Whatever the answers, we would still need to define what an objective probability actually is.
Many attempts have been made to do so over the years, but they all seem either flawed or limited. These efforts include frequentist probability, an approach that defines the theoretical proportion of events that would be seen in infinitely many repetitions of essentially identical situations—for example, repeating the same clinical trial in the same population with the same conditions over and over again. This seems rather unrealistic. British statistician Ronald Aylmer Fisher suggested thinking of a unique dataset as a sample from a hypothetical infinite population, but this scenario seems to be more of a thought experiment than an objective reality. Or there’s the semimystical notion of propensity—that there is some true underlying tendency for a specific event to occur in a particular context, such as my having a heart attack in the next 10 years. This idea seems practically unverifiable.
There are a limited number of well-controlled, repeatable situations of such immense complexity that, even if they are essentially deterministic, fit the frequentist paradigm by having a probability distribution with predictable properties in the long run. These include standard randomizing devices such as roulette wheels, shuffled cards, spun coins, thrown dice and lottery balls, as well as pseudorandom number generators, which rely on nonlinear, chaotic algorithms to produce numbers that pass tests of randomness.
In the natural world, we can throw in the workings of large collections of gas molecules, which, even if following Newtonian physics, obey the laws of statistical mechanics. And in genetics, the tremendous complexity of chromosomal selection and recombination gives rise to stable rates of inheritance. It might be reasonable in these limited circumstances to assume a pseudo-objective probability—“the” probability, rather than “a” (subjective) probability.
In every other situation in which probabilities are used, however—from broad swaths of science to sports, economics, weather, climate, risk analysis, catastrophe models, and so on—it does not make sense to think of our judgments as being estimates of “true” probabilities. We are simply attempting to express our personal or collective uncertainty in terms of probabilities on the basis of our knowledge and judgment.
Matters of judgement
This all just raises more questions. How do we define subjective probability? And why are the laws of probability reasonable if they are based on stuff we essentially make up? These and related questions have been discussed in the academic literature for almost a century with, again, no universally agreed-on outcome.
One of the first attempts was made in 1926 by mathematician Frank Ramsey of the University of Cambridge. He ranks as the person in history I would most like to meet. He was a genius whose work in probability, mathematics and economics is still considered fundamental. He worked only in the mornings, devoting his remaining hours to a wife and a lover, playing tennis, drinking and enjoying exuberant parties while laughing “like a hippopotamus” (he was a big man, weighing in at 108 kilograms). He died in 1930 at just 26 years old, probably, according to his biographer Cheryl Misak, from contracting leptospirosis after swimming in the River Cam.
Ramsey showed that all the laws of probability could be derived from expressed preferences for specific gambles. Outcomes have assigned utilities, and the value of gambling on something is summarized by its expected utility, which itself is governed by subjective numbers expressing partial belief—that is, our personal probabilities. This interpretation does, however, require an extra specification of these utility values. More recently, it’s been shown that one can derive the laws of probability simply by acting in such a way as to maximize their expected performance when using a proper scoring rule, such as the ones used in some popular online quizzes.
Attempts to define probability are often rather ambiguous. In his 1941–1942 paper “The Applications of Probability to Cryptography,” for example, English mathematician Alan Turing uses the working definition that “the probability of an event on certain evidence is the proportion of cases in which that event may be expected to happen given that evidence.” This definition acknowledges that practical probabilities will be based on expectations—human judgments. But by “cases,” does Turing mean instances of the same observation or of the same judgments?
The latter has something in common with frequentist definition of objective probability, just with the class of repeated similar observations replaced by a class of repeated similar subjective judgments. In this view, if the probability of rain is judged to be 70 percent, it is placed in the set of occasions in which the forecaster assigns a 70 percent probability. The event itself is expected to occur in 70 percent of such occasions. This definition is probably my favorite. But the ambiguity of probability is starkly demonstrated by the fact that, after nearly four centuries, there are many people who won’t agree with me on that.
Pragmatic approach
When I was a student in the 1970s, my mentor, statistician Adrian Smith, was translating Italian actuary Bruno de Finetti’s Theory of Probability. De Finetti had developed ideas of subjective probability at around the same time as Ramsey but entirely independently. (They were very different characters: in contrast to Ramsey’s staunch socialism, in his youth de Finetti was an enthusiastic supporter of Italian dictator Benito Mussolini’s style of fascism, although he later changed his mind.) That book begins with the provocative statement “probability does not exist,” an idea that has had a profound influence on my thinking over the past 50 years.
In practice, however, we perhaps do not have to decide whether objective “chances” really exist in the everyday nonquantum world. We can instead take a pragmatic approach. Rather ironically, de Finetti himself provided the most persuasive argument for this approach in his 1931 work on “exchangeability,” which resulted in a theorem that now bears his name. A sequence of events is judged to be exchangeable if our subjective probability for each sequence is unaffected by the order of our observations. De Finetti brilliantly proved that this assumption is mathematically equivalent to acting as if the events were independent, each with some true underlying chance of occurring, and that our uncertainty about that unknown chance is expressed by a subjective, epistemic probability distribution. This finding is remarkable: it shows that when starting from a specific but purely subjective expression of convictions, we should act as if events were driven by objective chances.
It is extraordinary that such an important body of work, underlying all of statistical science and much other scientific and economic activity, has arisen from such an elusive idea. And so I will conclude with my own aphorism: in our everyday world, probability probably does not exist—but it is often useful to act as if it did.
This article is reproduced with permission and was first published on December 16, 2024.