How to Change Your Mind: Bayes' Theorem and the Mathematics of Evidence

Imagine you’re playing a game with a friend. They have a bag of marbles—you don’t know how many, or what colors. They reach in and pull out a red marble. Then another red marble. Then a third.

At what point do you start thinking, “Maybe most of the marbles in that bag are red”?

This is a puzzle about evidence and belief. How do you decide what to believe? How much should a single observation change your mind? These questions might seem like the stuff of everyday guessing games, but they’ve been at the center of a huge debate in philosophy, statistics, and science for over 250 years.

The debate revolves around a simple-looking formula called Bayes’ Theorem, named after an 18th-century British minister named Thomas Bayes. He worked it out to solve a puzzle about probability, but it turned out to be much bigger than that. Some people think Bayes’ Theorem is the key to understanding how knowledge itself works. Others think it’s useful but not the whole story. And a surprising number of people simply get it wrong.

Here’s what you need to know.

What’s the Big Idea?

Let’s start with an example that’s less abstract than marbles. Suppose a new disease has been discovered. It’s rare—only 1 in 1,000 people have it. A test for the disease is pretty good: if you have the disease, the test will say “positive” 99% of the time. If you don’t have the disease, it will say “negative” 95% of the time.

You take the test. It comes back positive. How worried should you be?

Most people guess something like “I probably have it, since the test is pretty accurate.” But the real answer is surprising. Out of 1,000 people, only 1 actually has the disease. That person will almost certainly test positive (99 times out of 100). But among the 999 healthy people, 5% will get a false positive—that’s about 50 people. So for every 51 people who get a positive test, only 1 actually has the disease.

If you get a positive result, your chance of actually having the disease is about 1 in 51—roughly 2%. Not 99%.

This is not just a trick. It happens all the time in medicine, in criminal trials, in scientific experiments. People get confused because they forget to ask: “How rare is this thing to begin with?” Bayes’ Theorem is basically a formula that forces you to take that into account.

The theorem itself is simple. It says:

The probability that a hypothesis is true, given that you’ve seen some evidence, is equal to your initial probability for the hypothesis, multiplied by how well the hypothesis predicts the evidence, divided by how likely the evidence was to begin with.

That’s it. In symbols, it looks like this:

P(H given E) = P(H) × P(E given H) / P(E)

But the notation isn’t what matters. What matters is the basic insight: what you should believe after seeing evidence depends on what you believed before, and on how surprising the evidence is.

Where Do Prior Beliefs Come From?

This is where things get interesting—and controversial. In the disease test example, your “prior” probability (what you believed before the test) was 1 in 1,000. That came from knowing how rare the disease was in the general population.

But what about situations where you don’t have population statistics? Suppose you’re trying to figure out whether a friend is lying to you. You have some general sense of how honest they are, but it’s just your subjective judgment. Philosophers who use Bayes’ Theorem call this kind of thing a “subjective probability”—it’s not a fact about the world, but a measure of your personal confidence.

Some philosophers argue that this is fine—that all probabilities are ultimately personal, reflecting what we happen to believe. Others think that science requires more objectivity, and that personal hunches don’t belong in a serious account of evidence.

Here’s a strange thing about this debate: the people who disagree most sharply can still agree on many practical points. They disagree about what probabilities are, but they often agree about how to calculate them. Bayes’ Theorem works the same way regardless of whether you think probabilities are in your head or out in the world.

The Three Big Problems with Simple Probability

If Bayes’ Theorem is so useful, why isn’t everyone using it all the time? Because applying it turns out to be genuinely difficult.

First problem: Where do your prior probabilities come from? In the disease test, we had statistics. But most real-life situations don’t come with neat numbers attached. When a detective considers whether a suspect is guilty, what’s the “base rate” of guilt? When a scientist tests a new theory, what’s the “prior probability” that the theory is true? You might have a hunch, but a hunch isn’t a number.

Second problem: What counts as the “right” reference class? In the disease example, we used the rate in the general population. But maybe you’re in a high-risk group. Should your prior be higher? How do you decide which group to use? This is called the “reference class problem,” and it hasn’t been solved.

Third problem: What about hypotheses you haven’t thought of? Bayes’ Theorem compares the hypothesis you’re considering to everything else—the catch-all “not-H.” But what if the truth is something you haven’t imagined? The catch-all category includes all possible alternatives, but you have to assign it a probability without knowing what those alternatives are. That’s tricky.

These problems don’t mean Bayes’ Theorem is wrong. They mean it’s hard to apply in real situations. And that’s part of why philosophers still argue about it.

How Bayesians Think About Learning

Let’s shift to a bigger question. How do you learn from experience? When you see something new, how should you change your beliefs?

The simplest answer is called conditioning. If you become certain of something new (say, you see with your own eyes that it’s raining), then your new belief about any hypothesis should be exactly what you used to think about that hypothesis assuming the rain was true. In other words, you take your old conditional probability and make it your new absolute probability.

This makes intuitive sense. If you thought “if it’s raining, the ground will be wet” with high confidence, and then you discover it’s raining, you should now be highly confident that the ground is wet.

But what if you don’t become completely certain? What if you just become more confident? Maybe you hear a weather report that makes you think rain is likely, but not certain. Philosophers call this Jeffrey conditioning (after Richard Jeffrey, who worked it out in the 1960s). It’s more complicated, but the basic idea is the same: your new beliefs should be a mixture of what you would think if the evidence were true and what you would think if it were false, weighted by your new confidence in the evidence.

Some philosophers have argued that these conditioning rules follow directly from Bayes’ Theorem combined with a simple principle: if two people start with different beliefs but see the same evidence, the person whose prior beliefs better predicted the evidence should end up more confident in their view. This sounds reasonable, but it’s surprisingly powerful. Combined with the theorem, it forces you to update your beliefs by conditioning.

What’s at Stake?

Why does any of this matter? Because people make decisions based on evidence all the time, and they get it wrong surprisingly often.

Doctors misdiagnose patients because they ignore base rates. Juries convict innocent people because they misinterpret forensic evidence. Scientists overestimate their results because they don’t account for how unlikely a true discovery actually is. Bayes’ Theorem offers a way to do better—not by being complicated, but by being systematic.

The debate isn’t settled. Some philosophers think Bayesian reasoning is the foundation of all rational thought. Others think it’s a helpful tool but not a complete theory of evidence. Still others think it’s fundamentally flawed because it relies on personal probabilities that can’t be objectively justified.

But everyone agrees on one thing: if you understand Bayes’ Theorem, you’ll make fewer mistakes about evidence. And that’s not nothing.

Appendices

Key Terms

Term	What it means in this debate
Prior probability	What you believe about a hypothesis before seeing new evidence
Posterior probability	What you should believe after taking the new evidence into account
Conditional probability	The probability of one thing given that another thing is true
Base rate	How common something is in the general population
Likelihood	How probable the evidence would be if your hypothesis were true
Conditioning	The rule for updating beliefs when you become certain of new information
Jeffrey conditioning	The rule for updating beliefs when you just become more confident in something

Key People

Thomas Bayes (1702–1761): An English minister and mathematician who worked out the theorem that now bears his name. He didn’t publish it in his lifetime; a friend found his notes after his death and published them.
Richard Jeffrey (1926–2002): An American philosopher who showed how to update beliefs when evidence is uncertain. His “probability kinematics” (later called Jeffrey conditioning) made Bayesian reasoning more realistic.

Things to Think About

Suppose you flip a coin and it lands heads ten times in a row. Does that make you think the coin is probably rigged? What if your friend says “they’re all fair coins in this bag, I promise”? How should your prior belief affect your conclusion?
If two people disagree about something (like whether a defendant is guilty), and they both see the same evidence, should they end up agreeing? Why or why not? Bayes’ Theorem says they might not, if their prior beliefs were different enough.
In the disease test example, the test was “pretty good” but not perfect. How good would a test need to be for a positive result to mean you almost certainly have the disease? Does the answer depend on how rare the disease is?
Some philosophers think “subjective probabilities” are too personal to be the basis for science. But if you reject subjective probabilities, what do you use instead? Can you do science without any personal judgments at all?

Where This Shows Up

Medical testing: Every time you get a test result, doctors (should) use Bayes’ Theorem to interpret it. Many diagnostic errors happen when they don’t.
Criminal justice: Forensic evidence is often misinterpreted because jurors don’t think about base rates. The “prosecutor’s fallacy” is a famous example of getting Bayes’ Theorem backwards.
Machine learning: Spam filters, speech recognition, and recommendation algorithms all use Bayesian reasoning to make predictions from data.
Sports analytics: When a player makes an incredible shot, Bayes’ Theorem helps figure out whether they’re actually that good or just lucky.