Introduction to Probability

1. Why probability? (measuring uncertainty)

Probability is a number between 0 and 1 that says how likely something is: 0 means it can't happen, 1 means it's certain, and ½ means it's as likely as not. We often write it as a percentage, from 0% to 100%.

Life is full of uncertainty: will it rain, will the bus be late, which side will a tossed coin land on? We can't know the answer in advance, but we can measure how likely each answer is. A forecast of "70% chance of rain" doesn't promise rain; it says that rain is more likely than not, so an umbrella is a good idea.

The words we use

Our first examples use a coin and a die. A coin has two sides, called heads and tails; a fair coin lands on each side with the same chance. A die (plural: dice) is a small cube whose six faces show the numbers 1 to 6. When you roll it, one face ends up on top, and on a fair die each of the six numbers has the same chance.

See it settle: the law of large numbers

Flip a fair coin a few times and the share of heads jumps around. Flip it many times and the share settles close to 50%. Try it with a coin, or switch to the die and watch the share of sixes settle near 1/6:

Three ways to find a probability

From here on we write the probability of an event in short as P(event). For example, P(heads) means "the probability of heads", and P(late) means "the probability that the bus is late". The sign ≈ means "about".

A weather forecast mixes these ideas: forecasters look at how often it rained on past days with similar conditions (frequency), using computer models and expert judgement.

Where it shows up

• Weather forecasts, insurance prices and medical risk estimates.
• Games of chance, sports predictions and quality control in factories.
• Data science and AI, where models output probabilities, like "90% likely to be spam" (unwanted junk email).

2. Sets and sample spaces (the language of events)

The sample space, written Ω (the Greek letter omega), is the set of all possible outcomes of an experiment. A set is simply a collection of things, written between curly brackets: {2, 4, 6} is the set of the numbers 2, 4 and 6. An event is a part of the sample space, a subset (a set inside a bigger set). For a die roll, Ω = {1, 2, 3, 4, 5, 6} and the event "even" is the subset {2, 4, 6}. Thinking of events as sets lets us combine them with three simple operations: "or", "and" and "not".

Think of the sample space as a room with everyone in it. An event is a circle drawn on the floor: some people stand inside, the rest outside. Combining events is just asking who stands inside one circle, inside both, or outside.

Union, intersection and complement

We name events with capital letters, like A and B. In the table, A is "the roll is even" = {2, 4, 6} and B is "the roll is greater than 3" = {4, 5, 6}.

A Venn diagram draws Ω as a box and each event as a circle, so these operations become regions you can see. The demo has three examples: a die roll; a class of 30 students, where some take math and some take physics; and a die roll split into even and odd numbers. Pick an example and an operation:

Mutually exclusive events

Two events are mutually exclusive (or disjoint) when they can't happen together: their intersection is empty, A ∩ B = ∅. "Even" and "odd" on one die roll are mutually exclusive; "even" and "greater than 3" are not, because a 4 or a 6 is both. In the diagram, mutually exclusive events are circles that don't overlap (try Even vs odd above).

Where it shows up

• Surveys: customers who bought product A, product B, both or neither.
• Search and databases: "cats AND dogs" (intersection), "cats OR dogs" (union), "NOT cats" (complement).
• Every probability rule in the next lessons is written with these three operations.

3. Counting (how many possibilities?)

Many probabilities are "favourable outcomes ÷ all outcomes", where the favourable outcomes are the ones that belong to our event, so we need to count outcomes, often too many to list. Three tools do it: the multiplication rule, permutations (arrangements where order matters) and combinations (selections where it doesn't).

Think of ordering a meal: 3 starters, 4 mains and 2 desserts. You don't need to write out every menu; each starter can go with each main and each dessert, so there are 3 × 4 × 2 = 24 menus.

The multiplication rule

If a choice happens in steps, with a options in the first step, b in the second and c in the third, the total is a × b × c. A 4-digit PIN has 10 choices for each digit, so there are 10 × 10 × 10 × 10 = 10,000 PINs.

Permutations and combinations

In the meal example, each step chose from a different list: a starter, then a main, then a dessert. Often we instead choose several items from the same group, for example 3 people out of a group of 5. Then something changes: a person who has already been picked can't be picked again, so each pick has one choice fewer than the pick before it.

When we pick from one group like this, there are two different questions we can ask, and they have different answers:

• Does the order of the picks matter? For example, the first person picked gets gold, the second silver, the third bronze. Then each ordered pick is called a permutation.
• Does only the group matter? For example, the 3 people form a team, and nobody has a special place. Then each group is called a combination.

Let's count both, using the same group of people.

Example: 5 people, called A, B, C, D and E. In how many ways can we pick 3 of them, one after another?

1. First pick: any of the 5 people, so 5 choices.
2. Second pick: the first person is already taken, so 4 are left.
3. Third pick: two people are taken, so 3 are left.

Each first choice can go with each second choice and each third choice, so by the multiplication rule there are 5 × 4 × 3 = 60 ordered picks. You can check it: the picks that start with A are AB_, AC_, AD_ and AE_, each with 3 people left for the last place, so 4 × 3 = 12 of them. Each of the 5 people can be first, and 5 × 12 = 60.

These 60 ordered picks are the permutations of 3 people from 5. The order counts, so ABC (A first, B second, C third) and CBA (C first, B second, A third) are two different permutations. This is the right count when the places mean something, like gold, silver and bronze medals.

Now suppose we only want a team of 3, where nobody has a special place. The six orders ABC, ACB, BAC, BCA, CAB and CBA all describe the same team: A, B and C. So the 60 ordered picks count each team several times. How many times? That's the number of ways to put 3 people in order, and it is the multiplication rule again: 3 choices for who goes first, then 2, then 1, so 3 × 2 × 1 = 6.

This "count down to 1" product has a name: the factorial, written n! and read "n factorial". n! = n × (n − 1) × … × 1 is the number of ways to put n things in order, so 3! = 6.

Every team appears 3! = 6 times among the 60 ordered picks, so the number of different teams is 60 ÷ 6 = 10. These 10 teams are the combinations of 3 people from 5: the order doesn't count.

How the three tools fit together: each one is built from the one before it.

The demo below has two modes. Outfits builds outfits from 3 shirts, 2 pairs of trousers and 2 pairs of shoes, using the multiplication rule (3 × 2 × 2 = 12). Team of 3 repeats the example above: it lists the 60 ordered picks (permutations), groups the ones with the same 3 people, and ends with the 10 teams (combinations).

Where it shows up

• Passwords and PINs: each position can hold any allowed character, and characters can repeat (1111 is a valid PIN). The order matters too: 1234 and 4321 are different PINs. So we don't remove any choices; we use the plain multiplication rule. A 4-digit PIN has 10 choices for each of its 4 positions: 10 × 10 × 10 × 10 = 10,000 PINs. A 6-digit PIN has 10 multiplied 6 times = 1,000,000, which is 100 times more: each extra position multiplies the total by the number of choices. A password of 8 small letters has 26 choices per position: 26 multiplied 8 times ≈ 209 billion. Allow capital letters and digits as well (26 + 26 + 10 = 62 choices per position) and 8 characters give about 218 trillion, around 1,000 times more. You can check the method with a 2-digit PIN: 10 × 10 = 100, and the list is 00, 01, 02, … up to 99.
• Handshakes: 10 people meet, and each person shakes hands once with every other person. How many handshakes? Count ordered pairs first: 10 choices for the first person, then 9 for the second, so 10 × 9 = 90. But A shaking hands with B is the same handshake as B shaking hands with A, so every handshake was counted 2! = 2 times. That leaves 90 ÷ 2 = 45 handshakes, a combination of 2 people from 10. You can check the method with 4 people, A, B, C and D: 4 × 3 ÷ 2 = 6, and the list is AB, AC, AD, BC, BD and CD.

4. Basic probability rules (combining chances)

A few simple rules let us work out the probability of "A or B" and "not A" from probabilities we already know. When all outcomes are equally likely, a probability is just counting: favourable outcomes ÷ all outcomes.

Think of a prize draw: 100 numbered tickets go into a box and one is drawn at random. If you hold 5 of the tickets, your chance of winning is 5/100 = 5%. The rules below are common sense written down, so you can use them even when the numbers get bigger.

The three basic rules (axioms) in plain words

1. Between 0 and 1: every probability is at least 0 and at most 1.
2. Something happens: the whole sample space has probability 1, P(Ω) = 1.
3. Separate events add up: if A and B are mutually exclusive (can't happen together), P(A ∪ B) = P(A) + P(B).

Everything else follows from these three axioms.

The examples below use a standard deck of playing cards. In case you haven't used one: a deck has 52 cards, split into 4 suits (groups, each with its own symbol): ♥ hearts and ♦ diamonds are red, ♠ spades and ♣ clubs are black. Each suit has the same 13 ranks: A (ace), 2, 3, 4, 5, 6, 7, 8, 9, 10, J (jack), Q (queen) and K (king). J, Q and K show a picture of a person, so they are called face cards: 3 in each suit, 12 in the whole deck. Every card appears exactly once, so 4 suits × 13 ranks = 52 cards.

In practice, the rules you'll use most are:

Adding probabilities: when events can't happen together

Recall from lesson 2: two events are mutually exclusive when they can't happen at the same time, so they share no outcomes (A ∩ B = ∅). A spinner shows this clearly. It is a circle split into 4 equal sections, numbered 1 to 4, with a needle in the middle: you spin the needle, and the section where it stops is the result. Let A = "the needle lands on 1" and B = "the needle lands on 3". In the demo, section 1 is blue and section 3 is green. Press Next, Back or Reset to move through the steps.

The same idea works on the die from lesson 2. Take one roll with A = "even" = {2, 4, 6} and B = "odd" = {1, 3, 5}. In the picture, the blue circle is A and the green circle is B, and they don't touch.

"A or B" means the roll lands in either circle. Because the circles don't overlap, every outcome in "A or B" is in exactly one of them, so we can count each circle on its own and add: 3 outcomes + 3 outcomes = 6 outcomes. As probabilities: P(A or B) = P(A) + P(B) = 3/6 + 3/6 = 6/6 = 1. That's correct: every roll is either even or odd. This is the third axiom above ("separate events add up"), seen on a picture.

The addition rule: when events overlap

Now keep A = "even" = {2, 4, 6} but change B to "greater than 3" = {4, 5, 6}, the events from lesson 2. These two can happen together: a roll of 4 or 6 is both. In the picture, the blue circle is A, the green circle is B, and the purple region is their overlap A ∩ B = {4, 6}.

1. Just add, as before: P(A) + P(B) = 3/6 + 3/6 = 6/6 = 1. But that's too big: a probability of 1 would mean every roll is even or greater than 3, yet a roll of 1 or 3 is neither.
2. Find the mistake: counting A gives 2, 4, 6 and counting B gives 4, 5, 6, so 4 and 6 were counted twice, once in each circle (the orange ×2 marks).
3. Fix it: subtract the double-counted overlap once: P(A) + P(B) − P(A ∩ B) = 3/6 + 3/6 − 2/6 = 4/6.
4. Check by counting: A ∪ B = {2, 4, 5, 6}, which is 4 of the 6 outcomes: 4/6 ✓.

So the addition rule is P(A ∪ B) = P(A) + P(B) − P(A ∩ B), and the mutually exclusive case is just the special case where the overlap is empty: P(A ∩ B) = 0, so there is nothing to subtract. The two die examples give the same total, 3/6 + 3/6 = 1, but it is right for "even or odd" (no overlap) and wrong for "even or greater than 3" (overlap). Always ask first: can both happen at once?

The same thing happens with cards. The cards that are both hearts and face cards (J♥, Q♥, K♥) are counted once in "hearts" and once in "face cards", so the addition rule subtracts them once: 13/52 + 12/52 − 3/52 = 22/52. See it on the deck:

The complement rule: the chance that something doesn't happen

Roll a die once, and let A = "a six" = {6}, so P(A) = 1/6. "Not A" is written Aᶜ, the complement from lesson 2: every outcome that is not in A, here {1, 2, 3, 4, 5}. In the bar below, each cell is one face of the die (count the dots): the orange cell is A and the blue cells are Aᶜ.

A and Aᶜ can't happen together (a roll is either a six or not), so they are mutually exclusive and we can add them, as in the even and odd example above. Together they fill the whole bar, which is the whole sample space, and P(Ω) = 1 (the second axiom). So P(A) + P(Aᶜ) = 1, which gives the complement rule: P(Aᶜ) = 1 − P(A). Here P(not a six) = 1 − 1/6 = 5/6. Check by counting: 5 of the 6 faces are not a six ✓.

The complement rule is most useful when "not A" is easier to count than A, as the next example shows.

The complement trick: "at least one six"

Here is the complement rule at its most useful: an event that is hard to count directly, but whose opposite is easy. We use the same die, but now roll it several times.

The question: roll a die 4 times. What's the chance that at least one of the 4 rolls shows a six? "At least one" means one six, two sixes, three sixes or four sixes: any number of sixes except zero.

Counting each of those cases is long. The opposite is much simpler: the opposite of "at least one six" is "no six at all". By the complement rule, P(at least one six) = 1 − P(no six). So we only need P(no six), and the counting tools from lesson 3 give it:

1. All possible results: each roll can show any of 6 faces, so by the multiplication rule 4 rolls can give 6 × 6 × 6 × 6 = 1,296 different sequences (like 3, 6, 1, 1). On a fair die they are all equally likely.
2. Results with no six: each roll must show one of the other 5 faces (1, 2, 3, 4 or 5), so there are 5 × 5 × 5 × 5 = 625 sequences with no six.
3. P(no six) = 625 ÷ 1,296 ≈ 0.482. This is the same as (5/6) × (5/6) × (5/6) × (5/6): for each roll, 5 of the 6 faces avoid a six. Multiplying 5/6 by itself 4 times is written (5/6)⁴, and for n rolls it is (5/6)ⁿ.
4. The answer: P(at least one six) = 1 − 0.482 = 0.518, a little more than half.

Move the slider to change the number of rolls. The green bar is P(at least one six), the orange bar is P(no six), and the two always add up to 1.

Where it shows up

• Card and board games: the chance of drawing the card you need.
• Reliability: the chance that at least one of several backup systems works is 1 − P(all of them fail).
• Risk: the chance of at least one accident, failure or win over many tries.

5. Conditional probability and independence (when you know something)

Conditional probability P(A | B), read "the probability of A given B", is the chance of A once we know that B happened. We zoom into B: only the outcomes in B count now, and we ask what share of them is also in A.

The chance that a random person is wearing a coat might be 30%. But if you know it's snowing outside, the chance is much higher. The extra information doesn't change the people; it changes which group you're looking at.

Why P(A | B) = P(A ∩ B) ÷ P(B)

Start with the die from lesson 2: one roll, with A = "even" = {2, 4, 6} and B = "greater than 3" = {4, 5, 6}. The demo below works out P(A | B) one step at a time. Press Next to go forward, Back to go back a step, and Reset to start again. In the picture, the blue circle is A, the green circle is B, the purple region is where they overlap (A ∩ B), and outcomes that are ruled out fade.

So conditioning just rescales the overlap by the size of B: P(A | B) = P(A ∩ B) ÷ P(B) = (2/6) ÷ (3/6) = 2/3. Turned around, it gives the multiplication rule for probabilities: P(A ∩ B) = P(B) × P(A | B) = 3/6 × 2/3 = 2/6.

Zoom in: spam and the word "free"

Now the same zoom-in with real data. Spam means unwanted junk email, such as adverts or scams. Email programs try to spot it, and one clue is the word "free". Here are 100 emails. Orange ones are spam, blue ones are not, and a ★ marks emails that contain the word "free". Zoom in and count:

Notice that P(spam | free) = 77% and P(free | spam) = 80% are different questions with different answers. Swapping A and B in P(A | B) changes the meaning, a common mistake (lesson 6 shows how to go from one to the other).

The formula and the multiplication rule

The numbers below come from the demo: of the 100 emails, 30 are spam, 31 contain "free", and 24 are both spam and contain "free". So P(spam) = 0.30, P(free) = 0.31, P(spam and free) = 0.24, and P(free | spam) = 24/30 = 0.80.

Careful: this is a different rule from the counting rule in lesson 3, even though both are called a multiplication rule. Lesson 3 multiplies numbers of choices; this one multiplies probabilities. The second row is just the first row turned around (multiply both sides by P(B)), exactly as in the last step of the die demo above.

Two draws from a box: the multiplication rule as a tree

A box holds 3 red and 5 blue balls, 8 in total. We draw two balls, one after the other, without replacement: the first ball stays out, so the box changes before the second draw. Let A = "the first ball is red" and B = "the second ball is blue". What is P(A and B)?

The demo shows the box before each draw (red and blue dots are the balls) and a tree diagram: each branch is one possible draw, labelled with its chance, and each path from left to right is one possible result of both draws. The orange path is the one we follow. Press Next, Back or Reset to move through the steps.

So for events that happen one after the other, multiply along the path: the chance of the first step, times the chance of the second step given the first. This is the multiplication rule from the table above, P(A ∩ B) = P(B) × P(A | B), just written in the order things happen: when A comes first, condition on A, so P(A ∩ B) = P(A) × P(B | A). Both give the same P(A ∩ B).

Notice the last step: when the first ball is put back (with replacement), the box is the same for the second draw, so the first draw no longer changes the second one: P(B | A) = P(B) = 5/8. Events like that have a name, which is the next idea.

Independent events

Two events are independent when knowing one doesn't change the chance of the other: P(A | B) = P(A). In words: the chance of A is the same whether or not we know that B happened, so B gives no information about A.

Example: toss a coin and roll a die at the same time. Let A = "the coin shows heads" and B = "the die shows an even number" (2, 4 or 6). Step through the demo: the grid has one cell for every possible result, with the coin as the row (H = heads, T = tails) and the die as the column. Press Next, Back or Reset to move through the steps.

Why independence makes "and" so simple. The multiplication rule from the tree says P(A ∩ B) = P(A) × P(B | A). For independent events, P(B | A) is just P(B), as step 4 showed, so we can swap it in: P(A ∩ B) = P(A) × P(B). For the coin and die: 1/2 × 1/2 = 1/4, which is 3 of the 12 cells. Compare the emails above, where knowing "free" raised the chance of spam from 0.30 to 0.77, and the box of balls, where the first draw changed the second unless the ball was put back: those events are not independent, so you must use P(B | A).

For example, the chance of two heads in two flips is ½ × ½ = ¼, because the second flip doesn't care about the first.

The gambler's fallacy

After five heads in a row, many people feel tails is "due". It isn't: a coin has no memory, so each flip is still 50/50. This mistake is called the gambler's fallacy. Check it with a big simulation:

Where it shows up

• Spam filters and recommendations: the chance of something (spam, a product you'd like) given the words or clicks we've seen.
• Medicine: the chance of a disease given a symptom or a test result (next lesson).
• Insurance: the chance of a claim given age, location or history.

6. Total probability and Bayes' theorem (reasoning backwards from evidence)

Often we know how likely the evidence is if something is true, but we want the reverse: how likely it is to be true, given the evidence. Bayes' theorem turns P(evidence | cause) into P(cause | evidence). Its partner, the law of total probability, adds up all the ways the evidence can happen.

Think of a smoke alarm. It almost always beeps when there's a fire, but it also beeps for burnt toast. When it beeps, how likely is a real fire? It depends on how often each cause happens, and burnt toast is far more common than fires.

The law of total probability

The idea: slice the sample space into pieces, find how much of the event sits in each piece, then add the pieces up. It helps when an event is hard to count directly but easy to count inside each piece. We'll see it first on the die from lesson 2, then use it on a real example.

The slices must form a partition. Like the slices of a cake, they don't overlap (every outcome is in exactly one slice), and together they cover everything (nothing is left out). Here the event is B = "the roll is even" = {2, 4, 6}, and the slices are A₁ = {1}, A₂ = {2, 3, 4} and A₃ = {5, 6}; the small numbers just number the slices. In the demo, the blue outline is B, and the orange, green and pink strips are the three slices. Press Next, Back or Reset to move through the steps.

Now a real example: the smoke alarm from the start of this lesson. Here the event is a beep, and the slices are the three things that can happen on an evening: a fire, burnt toast, or a quiet evening. Picture 1,000 evenings in one kitchen:

• A real fire happens on 2 evenings, and the alarm beeps every time: 2 beeps.
• Burnt toast happens on 100 evenings, and its smoke sets the alarm off 30% of the time: 30% of 100 = 30 beeps.
• On the other 898 evenings nothing happens and the alarm stays quiet. (To keep it simple, a fire and burnt toast never happen on the same evening.)

How often does the alarm beep? A beep comes either from a fire or from toast, never from both, so we count each way on its own and add them: 2 + 30 = 32 beeps in 1,000 evenings, so P(beep) = 32 ÷ 1,000 = 0.032.

The same steps with probabilities. "The alarm beeps every time there's a fire" is a conditional probability: P(beep | fire) = 1. To get the chance of a fire and a beep on an evening, use the multiplication rule for probabilities from lesson 5 (also called the product rule), P(A ∩ B) = P(B) × P(A | B):

• P(beep ∩ fire) = P(fire) × P(beep | fire) = 0.002 × 1 = 0.002
• P(beep ∩ toast) = P(toast) × P(beep | toast) = 0.10 × 0.30 = 0.030

Adding the two ways gives P(beep) = 0.002 + 0.030 = 0.032, the same as counting. The table below shows the same steps.

In general: if something can happen in several separate ways, its total probability is the sum over those ways: P(B) = P(B | A₁) × P(A₁) + P(B | A₂) × P(A₂) + …. Here B is the event we care about (a beep), and A₁, A₂, … are the separate situations it can come from (a fire, burnt toast, a quiet evening). The small numbers just number them. They must be mutually exclusive and cover every case; a situation that never causes B, like a quiet evening, simply adds 0.

Bayes' theorem

Bayes' theorem links two events:

• The cause (A) is the possible explanation we want to find out about but can't see directly, such as "there is a fire".
• The evidence (B) is what we actually observe, such as "the alarm beeps".

We usually know how likely the evidence is for each possible cause, P(evidence | cause). We want the reverse, P(cause | evidence).

The formula is P(A | B) = P(B | A) × P(A) ÷ P(B). In words: how likely the evidence is if the cause is true, times how common the cause was before we saw any evidence, divided by how common the evidence is overall.

For the smoke alarm: the alarm beeps; how likely is a real fire? Here A = "there is a fire" and B = "the alarm beeps", and the three numbers come from the total probability example above:

• P(B | A) = P(beep | fire) = 1: if there's a fire, the alarm always beeps.
• P(A) = P(fire) = 0.002: how common fires are before we hear anything (the base rate).
• P(B) = P(beep) = 0.032: how often the alarm beeps overall, from the law of total probability.

So P(fire | beep) = 1 × 0.002 ÷ 0.032 ≈ 0.06 = 6%. Check it with the counts: of the 32 beeps, only 2 were a fire, and 2 ÷ 32 ≈ 6%. The formula and the counting give the same answer.

The easiest way to use Bayes is with counts of people instead of percentages. Take a medical test for a disease. A test result is positive when the test says "you have it", and negative when it says "you don't". Tests make mistakes both ways. The demo has three sliders:

• how common: the share of people who really have the disease (the base rate),
• test finds: the share of sick people the test correctly flags as positive,
• false alarms: the share of healthy people the test wrongly flags as positive.

With the starting values: 1% of 1,000 people = 10 are sick, and the test finds 90% of them, so 9 true positives. 9% of the 990 healthy people ≈ 89 false alarms. That's 9 + 89 = 98 positive results, but only 9 of them are sick: P(sick | positive) = 9 ÷ 98 ≈ 9%. Follow it step by step on 1,000 people:

Where it shows up

• Medical screening: why a positive result is usually followed by a second, more accurate test.
• Spam filters: the chance an email is spam, given the words it contains (a classic use of Bayes).
• Quality control and fraud detection: which machine, account or process is the likely cause of a problem.

7. Random variables (outcomes as numbers)

A random variable is a number that depends on the outcome of a random experiment, like your winnings in a game or the number of customers in an hour. We usually call it X. Its distribution says which values X can take and how likely each one is.

A shop owner can't know how many customers will come in the next hour, but after many hours she knows the pattern: usually 8 to 12, rarely more than 20, and about 10 on average. A random variable and its distribution describe exactly that kind of pattern.

Discrete and continuous

A simple dice game

Roll a die. On a 6 you win 3 points; on a 4 or 5 you win 1 point; on 1, 2 or 3 you lose 1 point. Your winnings X is a discrete random variable with three values: −1, +1 and +3. Is the game worth playing? We'll answer that after working out the expected value below.

First, the PMF of X: each value with its chance. A fair die has 6 equally likely faces, so count the faces that give each value:

The demo draws this PMF as bars: one bar per value, and each bar's height is that value's chance, so the three heights add up to 1. The CDF button shows instead the chance of getting that value or less: 3/6 for −1, 3/6 + 2/6 = 5/6 for +1 or less, and 6/6 = 1 for +3 or less. Press Play to roll the die many times: the line underneath shows your average winnings per game so far, next to the dashed line at the expected value.

Expected value, variance and standard deviation

Working it out for the dice game. First the chance of each value: a 6 is 1 face of 6, so P(win 3) = 1/6; a 4 or 5 is 2 faces, so P(win 1) = 2/6; a 1, 2 or 3 is 3 faces, so P(lose 1) = 3/6.

• Expected value: multiply each value by its chance and add up: 3 × 1/6 + 1 × 2/6 + (−1) × 3/6 = (3 + 2 − 3) ÷ 6 = 2/6 ≈ 0.33.
• Variance: for each value, take its distance from 0.33, square it (multiply it by itself), and weight it by its chance: (3 − 0.33)² × 1/6 + (1 − 0.33)² × 2/6 + (−1 − 0.33)² × 3/6 ≈ 1.185 + 0.148 + 0.889 ≈ 2.22. Squaring makes every distance positive, so values below and above the average don't cancel out.
• Standard deviation: the square root of the variance. √ means "square root": the number that, multiplied by itself, gives 2.22. That's about 1.49, since 1.49 × 1.49 ≈ 2.22. It brings the spread back to the same units as the winnings (points).

So you win about a third of a point per game on average, but any single game varies by about 1.49 points around that. The expected value is not a value you'll see in one game (you never win exactly 0.33 points); it's what the average settles to over many games, just like the law of large numbers in lesson 1. In the same way, "on average 10 customers per hour" is the expected value of the hourly count.

Continuous variables: probability is area

For a continuous variable, like how long you wait for a call, the chance of any exact value (exactly 3.000… minutes) is 0. Instead we ask about ranges, and the probability is the area under the PDF curve over that range. The total area under the curve is 1.

Where it shows up

• Games and insurance: expected value tells whether a bet or a policy pays off in the long run.
• Business planning: expected sales or customers per hour, and their spread, to plan stock and staff.
• Risk: standard deviation measures how uncertain an outcome is, such as the return on an investment.

8. Common distributions (ready-made models)

Many real situations follow a handful of well-known patterns, called distributions. Each one has a recognisable shape and one or two numbers, its parameters, that set the details. Once you recognise the situation, you get its probabilities, average and spread for free.

Think of clothing sizes: you don't design a new shirt for every person; you pick the right ready-made pattern and adjust the size. Distributions are ready-made patterns for chance: pick the one that matches the situation, then set its parameter.

Explore them

Pick a situation. The matching distribution appears; move the slider to change its parameter and watch the shape, average and spread change.

The seven distributions at a glance

How to choose

Where it shows up

• Call centres and shops plan staff with Poisson counts and Exponential waiting times.
• Quality control uses the Binomial (faulty items in a batch) and the Normal (measurement errors).
• Grading, heights, blood pressure and many measurements are summarised with a mean and a standard deviation.

9. More than one variable (how two quantities move together)

Often we care about two random variables at once: study hours and grades, temperature and ice cream sales. Their joint distribution gives the chance of each pair of values, and covariance and correlation measure whether they tend to rise and fall together.

Think of two dancers. If one steps forward whenever the other does, they move together (positive correlation). If one steps back whenever the other steps forward, they move in opposite directions (negative). If they ignore each other, there's no pattern.

A joint table

For two discrete variables, a table lists the probability of every combination. Here: weather and ice cream sales on a random summer day.

The totals at the edges give each variable on its own: P(sunny) = 0.60, P(high sales) = 0.55. Are they independent? If they were, P(sunny and high) would equal 0.60 × 0.55 = 0.33. It's 0.45, so no: sunny days make high sales more likely, just like dependent events in lesson 5. Two random variables are independent when every cell equals the product of its two totals.

The sum rule and the product rule

A joint table makes two rules easy to see. We write P(X, Y) for "X and Y together", one cell of the table. From these two rules you can derive the other rules for combining probabilities in this course, including conditional probability, the law of total probability and Bayes' theorem. They are also the starting point of many AI methods. Σ with a small Y under it means: add up over every value of Y (here, over "sales high" and "sales low").

The same two rules as pictures. The whole box is all days, with area 1, and the area of each region equals its probability. In the first picture the box is split by the dashed line into two columns, one per sales level. Each column's width is its chance, from the bottom row of the table: high sales 0.55, low sales 0.45. Inside each column, the top piece is the sunny days and the bottom piece the rainy days, so the four pieces are the four cells of the table: 0.45 and 0.10 in the high-sales column (0.45 + 0.10 = 0.55), 0.15 and 0.30 in the low-sales column (0.15 + 0.30 = 0.45).

Here we read 0.75 from the table, so step 3 only takes us back to 0.45: a check that the rule works. The product rule is most useful the other way round, when you don't have the whole table: if a forecast says P(sunny) = 0.60 and past experience says sales are high on 75% of sunny days, the rule gives P(sunny and high) = 0.60 × 0.75 = 0.45 straight away.

The totals at the edge of the table are called marginal probabilities, because they sit in the margins; that's where "marginalization" comes from. You've met both rules already: the product rule is the multiplication rule for probabilities from lesson 5, and the law of total probability in lesson 6 is the sum rule combined with the product rule.

Covariance and correlation

A joint table works when each variable has only a few values, like sunny or rainy. For numbers that can take many values, like hours of study and exam grades, we draw a scatter plot instead: each dot is one student, placed across by their study hours and up by their grade. The question is the same as before: when one number is high, does the other tend to be high too?

Covariance answers this with a simple recipe. Take 4 students:

1. Find each average: hours (2 + 4 + 6 + 8) ÷ 4 = 5; grades (50 + 60 + 70 + 80) ÷ 4 = 65.
2. Measure each distance from its average: above the average is positive, below is negative.
3. Multiply the two distances for each student. If both are above their averages, or both below, the product is positive: the two numbers go together. If one is above and the other below, the product is negative: they go in opposite directions.
4. Average the products: (45 + 5 + 5 + 45) ÷ 4 = 100 ÷ 4 = 25. It's positive, so more hours go with higher grades. (Statistics tools usually divide by one less than the count, 3 here, which gives about 33; the sign and the meaning are the same.)

The sign of the covariance gives the direction, but its size is hard to read, because it depends on the units: with grades out of 10 instead of out of 100, every grade distance is 10 times smaller and so is the covariance, even though the link is exactly the same. Correlation, written r, fixes this. It rescales the covariance so it always lies between −1 and +1: +1 means the dots lie exactly on a rising straight line, −1 exactly on a falling straight line, and 0 means no straight-line pattern at all. Our 4 students lie exactly on a rising line (every 2 extra hours add 10 points), so for them r = +1.

As formulas. E[ ] means "the average of", as in lesson 7, and σ_X is the standard deviation of X (σ_Y the same for Y):

The standard deviations use the same distances as the recipe: square each distance (9 = 3 × 3, and so on), average them, and take the square root, just as for the dice game in lesson 7. Dividing by both standard deviations cancels the units, which is why r always stays between −1 and +1, whatever units you measure in. And if you put the same variable in twice, the covariance becomes the variance from lesson 7: Cov(X, X) = Var(X).

The demo below has two examples:

• 📚 Study hours vs grade: 40 made-up students. The slider sets how strongly hours and grades are linked in this class, from opposite (−1) to together (+1). It is only there to show what each kind of pattern looks like: in real life, more study usually goes with higher grades, so a negative correlation here would be unusual. The dashed lines mark the two averages. Dots at the top right (both above average) or bottom left (both below) push the correlation up; dots at the top left or bottom right push it down. The bar under the plot shows r: it grows to the right in green when r is positive, and to the left in orange when it is negative.
• 🍦 Ice cream vs drownings: 40 summer days across a whole country. Each dot is one day: how much ice cream was sold, and how many people drowned. Look at the correlation first, then press Show the hidden cause to colour each day by its temperature.

One more link: if two random variables are independent, their covariance and correlation are 0, because neither tells you anything about the other.

Where it shows up

• Finance: investments that don't rise and fall together (low or negative correlation), like shares in very different companies, are combined so a loss in one is often balanced by another, which reduces risk.
• Health and social research: spotting links (like exercise and blood pressure), then testing whether one actually causes the other.
• Data science: checking which measurements move together before building a model.

Glossary & formulas

The key terms and formulas from the course on one page. Filter by topic to revise before the final quiz.

Glossary

Formula sheet

Final quiz & certificate

This is the last step of the course: one list of 10 questions on the main concepts of every lesson. After each answer you'll see the correct option and why it's right. Score 80% or more (8 of 10) to earn a certificate with your name.