Path
Probability & information · Lesson 1
Chance and probability
Probability assigns weight to outcomes. In the finite equal-outcome world it is favorable over total. Deeper theories extend that idea to richer spaces.
Core ~27 minDeep study ~78 minWhat is a probability?What does random mean here?
How this idea was born
Games of chance forced an honest question: if the future is not pinned down, how should you weigh the possibilities? Dice, cards, and unsettled bets made shrug-level uncertainty expensive. Certainty is a luxury; probability is the language for graded possibility.
Blaise Pascal, with Pierre de Fermat, answered gambling questions with combinatorial clarity in the seventeenth century and helped found probability as a mathematical subject. MacTutor sketches Pascal's intensity about truth and meaning; attribute the founding partnership carefully rather than inventing a single tavern myth. Steal the ethic: take uncertainty seriously enough to compute, not casually enough to shrug. Later, Kolmogorov would give the subject a measure-theoretic spine; this chapter starts with the finite equal-outcome world (favorable over total).
For a modern rereader, A/B tests and weather forecasts still speak that language. Insist on humility about models, tails, and what "random" is claiming.
First principles
In a finite set of equally likely outcomes,
favorable over total.
Probabilities lie between 0 and 1. Impossible gets 0, certain gets 1. Disjoint outcomes add (finite additivity in the classical counting setting; countable additivity arrives with measure-theoretic foundations later associated with Kolmogorov).
Two useful readings:
- Long-run frequency: repeat the experiment; the fraction of times happens approaches under ideal assumptions.
- Degree of belief: a coherent assignment of weights to possibilities (subjective Bayesian reading).
Both help if you know which hat you are wearing.
Worked intuition
"Fair die" is a model: six outcomes, equal weight. Reality might be a shaved die. Probability mathematics tells you what follows from the model. Data and humility tell you whether the model deserves trust.
A fair six-sided die has . A biased die does not. Data estimates the weights. A/B tests and weather forecasts speak the same graded language with messier outcome spaces.
Algebrica's probability page and discrete random variables page add careful definitions beside the counting intuition.
Fat tails are what happen when your model understates extremes. Classroom normals are polite. Markets, storms, and some empirical losses are jumpier. Probability becomes wisdom when it stays attached to assumptions.
Common confusions
Equally likely as a fact of nature. Often it is a modeling choice. Symmetry arguments can justify it; data can reject it. Do not smuggle fairness in without noticing.
Probability as prophecy of a single trial. A probability 0.3 does not mean "will happen a little." Frequency readings talk about long runs; belief readings talk about coherent weights. Single-trial drama needs careful language.
Ignoring the sample space. is meaningless without (or a sigma-algebra in richer settings). Change the frame of possible outcomes and the number can change.
Thin tails by default. Assuming light tails because the formula is pretty is a modeling sin with real costs. Strogatz's fat-tail warning is an ethics of surprise.
Gambler's fallacy. Independent fair flips do not "owe" you a heads. Memoryless models stay memoryless.
Confusing unlikely with impossible. Probability events happen in large enough worlds. Rare is not never.
History & stories
Pascal, with Fermat, answered gambling questions with combinatorial clarity and helped found probability as a mathematical subject. MacTutor's Pascal biography is the standard short reference. Their correspondence treated fair games with counting arguments: the combinatorial seed of the frequency-friendly reading.
Later, Laplace and others systematized classical probability; Kolmogorov's twentieth-century axioms gave the measure-theoretic spine. You will meet Kolmogorov again with expectation. Here, keep Pascal's intensity: uncertainty deserves computation.
Real world
Dice, cards, A/B tests, and weather forecasts all speak probability. Insurance and epidemiology live on weighted outcomes. Every dashboard percentage is a modeling claim wearing a confident font.
Most "fair coin" talk is modeling. Real coins can be biased; real markets have fat tails; real storms cluster. Ask of every probability number: what assumptions minted it?
Open when you want the machine-learning connection. Skip freely.
Field notes, extra examples, and glossary live here when you want more than the core path.
Carry this forward
- Probability weights outcomes inside a model; equally likely is often a choice.
- Frequency and belief readings both help when labeled honestly.
- Next: updating beliefs with evidence (Bayes).
Read more
Go deeper when curiosity hits - videos, essays, and primary trails.
- referenceMacTutor - Blaise Pascal
- video3Blue1Brown - Bayes theorem (intuition warmup)
- videoKhan Academy - Probability
- bookSteven Strogatz - The Joy of XChance chapters, including fat tails and why extremes matter.
- articleAlgebrica - ProbabilityClassical probability with careful definitions and figures.
Check your understanding
A short learning loop - try a few, learn from misses, mark complete when you have engaged. No timer, no scoreboard.
Try at least 3 core prompts, or choose I'll return later.