Join · scan
OTTO-DS4
ottolearn.org/join
Module 1 · Descriptive Statistics · DS-4
Maya's sleep story is out: "CEGEP students average 6.8 hours." Then a reader comments: "Not in my program — we pull all-nighters and crash on weekends." Maya checks her data. The reader's program averages… exactly 6.8. Both are right — and today's numbers explain how.
Live session · join with code OTTO-DS4 · ottolearn.org/join
Same rules as always: vote before I explain, every poll is anonymous, and changing your mind after discussion is the goal.
Today · the follow-up: "the average hides the all-nighters"
The range — and its trap
Variance & standard deviation
Quartiles & the IQR
Outlier fences
The coefficient of variation
Every question is a number Maya needs before the follow-up can run — and a poll you'll answer first.
Recall · last class · 45 seconds
Warm-up · 60 seconds · anonymous
Hold your answer. By the end of class you'll have four different numbers for "how spread out" — and know which one belongs in the follow-up story.
Core concept 1 · C1
The reader and the average are both right — so the follow-up needs a number for spread. Tool #1 is the one you'd invent yourself: measure the full width.
Biggest value minus smallest. Steady program: \(7.3 - 6.3 = 1.0\) h. The reader's program: \(9.8 - 3.8 = 6.0\) h. Six times wider — the follow-up has its first number.
\(\text{Range} = \max - \min\)The range answers in the data's own units — hours — and you can read it off a sorted list in seconds. That speed is its whole appeal.
It uses the two most extreme values and ignores everything between them. One wild value — a single 9.9 h crasher joins the steady program — and its range more than triples.
Fast, readable — and fragile. Worse: two datasets can have the SAME range and be nothing alike. See it:
See it · C1
Predict first: both brackets span 60. Once the dots appear, will the two datasets look the same? Step through — and watch what the range cannot see.
C2 · Predict first · ConcepTest · vote → discuss → re-vote
Core concept 2 · C2 · why that happened
Deviations sum to zero by definition — so square each one first. Squares are never negative; nothing cancels. Their average is the variance.
\(\sigma^2 = \dfrac{\sum (x_i - \mu)^2}{N}\)Data for the whole population → divide by \(N\): that's \(\sigma^2\). Only a sample (Maya's 5 students) → divide by \(n - 1\): that's \(s^2\).
\(s^2 = \dfrac{\sum (x_i - \bar{x})^2}{n-1}\)A sample rarely catches the population's extremes, so its raw spread runs small. Dividing by the slightly smaller \(n - 1\) inflates the estimate just enough to be honest on average.
Careful: \(n - 1\) lives in the variance formula ONLY. The mean still divides by \(n\) — Bessel corrects spread, not everything.
I do · watch me measure the swing
Hours: \(3.8,\ 5.3,\ 6.8,\ 8.3,\ 9.8\); \(\bar{x} = 6.8\). Deviations: \(-3,\ -1.5,\ 0,\ +1.5,\ +3\). (Sum: 0 — exactly as the poll promised.)
\(9,\ 2.25,\ 0,\ 2.25,\ 9\) — the minus signs are gone, and the biggest deviations now count extra.
\(\sum (x_i - \bar{x})^2 = 9 + 2.25 + 0 + 2.25 + 9 = 22.5\)
Five students are a sample of the program, so Bessel applies: \(s^2 = \dfrac{22.5}{5 - 1} = 5.625\).
The reader's program: \(s^2 = 5.625\) hours-squared. Nobody sleeps in squared hours — one fix coming (C3). But first, see what those squares really are:
See it · C2
Each squared deviation is literally a square: side \(|x_i - \bar{x}|\), area \((x_i - \bar{x})^2\). Predict first: drag one dot far out — what happens to the average square? Then toggle \(\div N\) vs \(\div (N-1)\) to watch Bessel's correction work.
Drag any dot left or right. Each coloured square has area = (deviation)².
Core concept 3 · C3
Take the square root of the variance. The reader's program: \(s = \sqrt{5.625} \approx 2.37\) h — back in honest, headline-ready hours.
\(s = \sqrt{s^2}\)The SD is the typical distance from the mean. A typical student in that program sleeps about 2.4 h away from the 6.8 average — all-nighter one day, crash the next.
Variance \(\leftrightarrow\) SD by squaring / rooting. Careful below 1: a variance of \(0.25\) is an SD of \(0.5\) — there the SD is the bigger number.
One number for "how consistent" — but the root has a reason. Watch me pull the reader's program back into real hours:
I do · watch me get the units back
Full support — watch me undo the squaring and read the SD in context.
C2 left the reader's program at \(s^2 = 5.625\) hours-squared. The number is real, but nobody sleeps in squared hours — I can't print it.
Squaring is exactly what wrecked the units, so I undo it with its opposite — the square root: \(s = \sqrt{5.625}\).
\(\approx 2.37\) h. Now it reads as a typical distance from the 6.8 mean — headline-ready, and exactly what the reader asked for.
The reader's program: \(s \approx 2.37\) h. Same move now on the steady program — you take the root:
We do · you finish it
Support fades — the variance is set up; you take the root and call the comparison.
Steady program, sample of 5 (hours): \(6.3,\ 6.6,\ 6.8,\ 7.0,\ 7.3\), with \(\bar{x} = 6.8\).
What is the steady program's standard deviation \(s\) — and what does the comparison now say?
Practice · C2–C3 · variance → SD
On your own now — fresh data, every step yours. Re-roll as many as the room needs.
See it · C3
Every point lives in a zone: within 1 SD of the mean, between 1 and 2, or beyond 2. Predict first: drag one point far out — what happens to the ±1s band everyone else is judged by?
Core concept 4 · C4
Sort, then: \(Q_2\) is the median; \(Q_1\) is the median of the lower half; \(Q_3\) the median of the upper half. Like the median, quartiles are positions — they may fall between data values.
The span of the middle half of the data. The lowest 25% and the highest 25% — where outliers live — never touch it.
\(\text{IQR} = Q_3 - Q_1\)The SD feels every value, so an outlier inflates it (squared, no less). The IQR ignores the tails entirely. SD is to IQR what the mean is to the median (DS-3).
Five numbers — min, \(Q_1\), \(Q_2\), \(Q_3\), max — summarize any dataset. Drawn as a picture, they're a box plot. Watch one get built:
I do · watch the build
The five-number summary drawn step by step: sorted values → \(Q_2\) → \(Q_1\) and \(Q_3\) → the box → the whiskers → and, in step 6, what happens when an outlier crashes the party.
Press Next to begin.
See it · C4
Predict first: drag the maximum far to the right. Which numbers chase it — range? SD? — and which one holds still? That's "sensitive vs resistant", live.
Which pair goes in print? · mixes DS-2 + DS-3
Practice · C4 · quartiles & IQR
Core concept 5 · C5
Lower fence: \(Q_1 - 1.5 \times \text{IQR}\). Upper fence: \(Q_3 + 1.5 \times \text{IQR}\). Any value outside the fences is a potential outlier.
\(Q_1 - 1.5\,\text{IQR}, \quad Q_3 + 1.5\,\text{IQR}\)Convention, not physics. On bell-shaped data, \(1.5 \times \text{IQR}\) lands near \(\pm 2.7\) SDs — so only about 0.7% of ordinary values get flagged by pure chance.
A fence flags a value for investigation. A typo gets fixed; a genuine extreme stays in — sometimes the outlier IS the story.
A negative lower fence just means nothing can be flagged on the low end — that's information, not an error. Don't "fix" it to 0.
See it · C5
Predict first: how far must the dot travel before it turns red? Then drop the multiplier to 0.5 — how many perfectly ordinary values suddenly get flagged?
Quick check · C5 · 45 seconds
Practice · C5 · fence check
Core concept 6 · C6
Maya's two beats: rents (\(s \approx \$1{,}963\)) and sleep (\(s = 2.37\) h). Which is "more variable"? Dollars vs hours — the raw SDs cannot be compared at all.
Divide the SD by the mean. The units cancel — what's left is spread as a percentage of the centre, comparable across any two datasets.
\(\text{CV} = \dfrac{s}{\bar{x}} \times 100\%\)The CV needs a true zero (dollars, hours, grams — not °C), and a mean near 0 makes it explode. Check both ingredients before quoting it.
Reader's program: \(2.37 / 6.8 \approx 35\%\). The rent listings: \(1963 / 1650 \approx 119\%\) — relative to its centre, the rent market is over three times wilder than even the all-nighter program.
See it · C6
Predict first: keep \(s\) fixed and drag the mean toward zero — what happens to the CV? Why does a small centre make the same spread "relatively" huge?
Drag a mean slider toward zero — the coloured 0→x̄ segment shrinks while the green ±s bracket keeps its size, so the spread becomes huge relative to the centre and CV explodes.
Practice · C6 · which varies more?
Production · partners · one sentence
Exit ticket · muddiest point · anonymous
Wrap
Compute — deviations → square → average → \(\sqrt{\ }\): the \(s\) pipeline (divide by \(n-1\) for samples); sort → quartiles → \(\text{IQR} = Q_3 - Q_1\) for the resistant version.
Match the pair — symmetric data: mean + SD. Skewed or outlier-ridden: median + IQR. Centre and spread travel together, sensitive with sensitive, resistant with resistant.
Compare with the CV — across units or scales, divide \(s\) by \(\bar{x}\): spread as a percentage of the centre.
Next class (DS-5): the average is 6.8 and the spread is 2.4 — so is YOUR 5.5 hours unusual or ordinary? Locating one value inside a distribution.
Sleep-program samples and rent figures are illustrative, constructed for this class — not survey results.