Reasoning under uncertainty: Bayes' rule
Update beliefs with Bayes' rule and compute a posterior from a small Bayesian network by enumeration.
def posterior(prior, sensitivity, false_positive):
evidence = sensitivity * prior + false_positive * (1 - prior)
return sensitivity * prior / evidence
print(round(posterior(0.01, 0.9, 0.09), 4), round(posterior(0.10, 0.9, 0.09), 4))
P_RAIN = 0.2
P_SPRINKLER = {True: 0.01, False: 0.4}
P_WET = {(True, True): 0.99, (True, False): 0.9, (False, True): 0.8, (False, False): 0.0}
def joint(rain, sprinkler, wet):
p = P_RAIN if rain else 1 - P_RAIN
p *= P_SPRINKLER[rain] if sprinkler else 1 - P_SPRINKLER[rain]
p_wet = P_WET[(sprinkler, rain)]
return p * (p_wet if wet else 1 - p_wet)
numerator = sum(joint(True, s, True) for s in (True, False))
evidence = sum(joint(r, s, True) for r in (True, False) for s in (True, False))
print(round(numerator / evidence, 4), round(evidence, 4))0.0917 0.5263 0.3577 0.4484
Explanation
Logic gives true or false answers, but real knowledge is often uncertain: tests are imperfect and sensors are noisy. Probability theory provides a principled way to represent degrees of belief and to update them. Bayes' rule says P(H | E) = P(E | H) · P(H) / P(E): the posterior belief in a hypothesis H after evidence E depends on the prior P(H) and the likelihood of the evidence if H is true.
Base rates matter. In the first part, a disease has a prior of 1 percent, the test detects it 90 percent of the time and gives a false positive in 9 percent of healthy people. A positive result gives a posterior of only about 9.2 percent, because most positive tests come from the large healthy majority. With a prior of 10 percent the posterior is 52.6 percent. Ignoring the prior is called the base-rate fallacy.
A Bayesian network encodes a joint distribution compactly with a graph of conditional probability tables. The second part uses a tiny network where rain influences the sprinkler and both can wet the grass. It computes P(Rain | WetGrass) by enumeration: sum the joint probability over the hidden variable for the numerator, and over all variables consistent with the evidence for the denominator. Enumeration is exact but exponential in the number of variables, which is why larger models use variable elimination, sampling, or approximate methods.
Explore the dry run
def posterior(prior, sensitivity, false_positive): evidence = sensitivity * prior + false_positive * (1 - prior) return sensitivity * prior / evidence print(round(posterior(0.01, 0.9, 0.09), 4), round(posterior(0.10, 0.9, 0.09), 4)) P_RAIN = 0.2P_SPRINKLER = {True: 0.01, False: 0.4}P_WET = {(True, True): 0.99, (True, False): 0.9, (False, True): 0.8, (False, False): 0.0} def joint(rain, sprinkler, wet): p = P_RAIN if rain else 1 - P_RAIN p *= P_SPRINKLER[rain] if sprinkler else 1 - P_SPRINKLER[rain] p_wet = P_WET[(sprinkler, rain)] return p * (p_wet if wet else 1 - p_wet) numerator = sum(joint(True, s, True) for s in (True, False))evidence = sum(joint(r, s, True) for r in (True, False) for s in (True, False))print(round(numerator / evidence, 4), round(evidence, 4))Follow the example
Start to inspect the source, values, and output. These prepared steps explain the published sample; grouped loop steps describe multiple iterations.
View the complete dry-run table
| Step / line | What happens | State after step | Output so far |
|---|---|---|---|
| The probability of a positive test: true positives plus false positives (the law of total probability). | P(positive) = 0.0981 for a 1% prior | — | |
| A positive test raises the 1% prior to only 9.2%, and the 10% prior to 52.6%: the base rate matters. | P(positive) = 0.0981 for a 1% prior | 0.0917 0.5263 | |
| Numerator: the total probability of wet grass and rain, summing over the hidden sprinkler variable. | P(positive) = 0.0981 for a 1% prior | 0.0917 0.5263 | |
| Denominator: the probability of wet grass over all combinations of rain and sprinkler. | P(positive) = 0.0981 for a 1% prior | 0.0917 0.5263 | |
| Seeing wet grass raises the belief in rain from 20% to 35.8%. P(wet grass) is 0.4484. | P(positive) = 0.0981 for a 1% prior | 0.0917 0.5263 0.3577 0.4484 |
Cost and common mistakes
Enumeration over n Boolean variables is O(2ⁿ). The first calculation is O(1).
- Confusing P(positive | disease) with P(disease | positive) is the most common probabilistic reasoning error. Bayes' rule links them.
- Enumeration cost is exponential in the number of variables. It only suits very small networks.
- The numbers in the sprinkler network are illustrative, not measured. A conclusion is only as good as its probability tables.
Try it yourself
Why is the posterior after a positive test so much lower than the 90% sensitivity?
Reveal answer and reasoning
Because the disease is rare. With a 1% prior, the 9% false-positive rate applied to the 99% healthy people produces far more positive tests than the 90% detection rate applied to the 1% who are ill.