Conditional probability and Bayes

New evidence arrives and beliefs should move. Without a disciplined way to update, base rates vanish and headlines lie. Medical tests, spam filters, and diagnostics all need the probability of a cause given a symptom, not only the probability of a symptom given a cause.

Thomas Bayes's essay on inverse probability was published posthumously; Pierre-Simon Laplace and others expanded the program. MacTutor sketches both careers. Steal the ethic: intellectual honesty about prior belief and evidence. Show how you update; do not pretend the data alone spoke with no assumptions.

This chapter is the mathematics of updating for a modern rereader: restrict attention to the world where evidence happened, then reweigh hypotheses. Posterior proportional to likelihood times prior is the heartbeat you will meet again in classifiers and uncertainty estimates.

Conditional probability of $A$ given $B$ (with $P(B) > 0$):

Restrict attention to the world where $B$ happened, then weigh $A$ inside that world.

Bayes' rule:

Posterior from likelihood times prior, normalized by evidence.

Independence means $P(A \mid B) = P(A)$ when defined: learning $B$ tells you nothing new about $A$.

$P(\text{disease}\mid\text{positive test})$ is not $P(\text{positive test}\mid\text{disease})$. Clinics, spam filters, and courtroom rhetoric all trip on that swap.

Walk a rare-disease example slowly. Suppose a condition hits 1 in 1,000 people. A test is 99% sensitive and 99% specific in a simplified sketch. Among 100,000 people, about 100 are sick; the test catches about 99 of them. About 99,900 are healthy; about 999 false positives appear. Among positives, most are still healthy. Base rates matter. Intuition fails without Bayes. Always track the prior.

Email: $P(\text{spam}\mid\text{words})$. Medicine: $P(\text{condition}\mid\text{marker})$. Draw prior → likelihood → posterior until it feels inevitable.

Algebrica's Bayes theorem page is the structured companion.

Swapping conditionals. $P(A\mid B)$ and $P(B\mid A)$ answer different questions. Headlines that swap them manufacture panic or false comfort.

Ignoring base rates. A mostly-accurate test for a rare event can still yield many false alarms. Accuracy slogans without prevalence are incomplete.

"The data speak for themselves." Likelihoods need priors (explicit or smuggled) to become posteriors. Silent priors are still priors.

Independence as "unrelated forever." Independence is a modeling claim about information. Dependence can be weak or strong; zero mutual information is a special case, not the default.

Updating twice on the same evidence. Double-counting $E$ inflates posteriors. Evidence gets used once unless it is truly new.

Treating $P(E)$ as optional. The normalizer matters. Proportional reasoning is fine mid-calculation; final probabilities need care.

Thomas Bayes's essay on inverse probability was published posthumously, quiet and profound. Laplace and others expanded the program into a working method for updating beliefs with evidence. MacTutor's Bayes biography is the short reliable map.

The modern slogan is Bayesian updating; the human habit is older than the formula: revise when the world talks back. Steal the honesty about priors even when you use frequentist tools elsewhere.

Medical tests, spam filters, and diagnostics all need $P(\text{hypothesis}\mid\text{evidence})$, not only $P(\text{evidence}\mid\text{hypothesis})$. Legal rhetoric that cites test accuracy without base rates is a conditional-probability failure mode with human costs.

Weather forecasts and sensor fusion systems update continuously: yesterday's posterior becomes today's prior.