L-REG-2: Solutions — Linear Regression

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Problem 1 — Build the Equation and Check It

Compute the slope with the response spread on top, then the intercept:

Check the line at the point of means. Substituting must give :

If it does not, an earlier step has an arithmetic slip. The check is necessary, not sufficient: every line through passes it, whatever its slope.

Problem 2 — Predict an Extreme Case

Work in standard units.

1. How far from the mean is , in standard deviations?

2. The prediction is only times as far:

3. Convert back to the units of :

The value is the SD line: it predicts a second value just as extreme as the first, which ignores . The least-squares prediction is closer to the mean whenever .

Problem 3 — Say What the Slope Means

The sentence has five decisions: per unit of (not per unit of , and not per standard deviation); predicted (not actual); the direction from the sign of ; the size (not , not the intercept); and on average (not “for every individual”, not “because of”). Example: “For each additional hour of weekly study time, the predicted final exam score increases by 2.40 points, on average.”

Problem 4 — One Residual

Predict at the observation’s , then subtract observed minus predicted:

A positive residual places the point above the line (the model under-predicted it); a negative one, below.

Problem 5 — Where Do the Misses Go?

For each blank row, compute and . The printed line is the exact least-squares line for the six points, so the residuals add to exactly :

Squaring each residual and adding gives the SSE:

Adding the residual sizes without their signs is a different, wrong quantity: it is neither nor the SSE.

Problem 6 — On Your Own, with Steps on Request

Compute and as in Problem 1, then substitute the requested into . The common wrong turns are a slope with , using as the intercept, and the SD-line slope without .

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

From here on, keep unrounded (at least four decimal places) and round only the answers you enter.

Problem 1 — Prediction and Residual

1. , to four decimal places.
2. , computed with the unrounded . Rounding to two decimals first can move by a few hundredths; the feedback names that path.
3. at the observation’s .
4. , then the point’s position from the sign of .

Problem 2 — Diagnose the Write-Up

Check each step against its rule, using the numbers the student wrote:

1. Slope: . A ratio written , or used as the slope, is the first error.
2. Intercept: . Adding is the first error.
3. Prediction: substitute the stated into the student’s own line.
4. Residual: observed minus predicted. is the first error.
5. Conclusion: a predicted change, on average. “Causes” is the first error.

The student carries an earlier error forward consistently, so later steps can be internally correct and still wrong: name the first step whose rule is broken.

Problem 3 — Could This Be the Least-Squares Line?

Here and , so the least-squares line is , in whatever form it is written (, or ).

• Lines with slope 1.5 but another intercept (, , ) miss .
• Lines through with the wrong slope: swaps the standard deviations, is itself, flips the sign, and is the SD line.

Problem 4 — Is the Intercept Meaningful?

The two cafés share the equation and the claim “at 0 °C we predict about 42”. Québec City’s observed days run from −8 °C to 24 °C, so 0 °C is inside the data and the claim is a supported prediction. Miami’s run from 12 °C to 33 °C, so 0 °C is outside them and the intercept only positions the line. The observed range is the single fact that changes the answer.

Problem 5 — Predict the Other Way Round

The line that predicts from minimises horizontal misses:

Round to three decimals; compute from the unrounded and round it to two. Solving the printed -on- line for gives slope , which is wrong unless ; reusing is also wrong.

Problem 6 — Synthesis

Compute , and as in Problem 1. The slope sentence says “predicted” and “on average”. Then compare the requested with the stated observed range: inside it, the prediction stays within the data; outside it, the line is being extended beyond the data (extrapolation, which L-REG-3 takes up), so the prediction should be treated with caution.

Retrieval — Is a Straight Line the Right Summary?

A least-squares line is a fair summary for two numerical variables on independent cases with a roughly straight pattern and no single point steering it. A U-shaped pattern, a categorical response, an influential point, repeated readings on one person, and scatter that widens with each need a closer look first.

Mixed Review

Review Problem 1 (L-DS-4). Compute , the squared deviations and their sum, then

Dividing by instead, reporting the variance , or averaging the absolute deviations are the named wrong turns.

Review Problem 2 (L-REG-1). is times itself, and it is never negative. For the sprinters, : the fitted line accounts for about 42% of the variation in race times. Correlation does not establish causation; event specialty is one confounder that drives both training kilometres and 100 m times.

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Question 1 — Feynman Test

A strong answer says that a residual is a vertical miss, observed minus predicted; that least squares makes the sum of squared misses as small as possible, so a line that hits many points but misses the rest badly loses; and that the slope is a predicted change per unit of , on average, not a guarantee or a cause.

Question 2 — Which Line Is Least Squares?

Square every residual in each row and add. The line with the smallest total is the least-squares line. The line through three of the points has two large misses that make its total larger; the line whose residuals also add to zero passes through with the wrong slope, and its total is larger too.

Question 3 — The Equation from Untidy Summaries

Keep unrounded when you compute .

Question 4 — Does the Intercept Mean Anything?

The intercept is a supported prediction only for a data set whose observed -range reaches 0. Check each range: one, both or neither may reach 0.

Question 5 — A Residual from a Printed Line

Use the coefficients exactly as printed: , then , and read the position from the sign of .

Question 6 — The Other Way Round

Solving the printed line for , using the slope , or dropping (the SD line) are the named wrong turns.

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Both missions are regression to the mean in disguise: people were noticed, or enrolled, because one measurement was extreme.

Path A — ✈️ The Praise Paradox

1. Predict each cadet’s second landing from the first with the line for all cadets:

2. Compare each actual second landing with its prediction, actual minus expected.

Both cadets moved toward the average, and both moves were predicted before anyone said a word. Against those predictions the gaps are small, so the two landings are no evidence for the coach’s claim; they are not proof of no effect either. A fair test randomly assigns cadets to praise or to no feedback.

Path B — 🩺 The Clinic

Fictional data; a statistics exercise, not medical advice.

1. Predict the group’s average second reading with no supplement, from the line for all patients, at the group’s average first reading.
2. Compare the actual average with that prediction.

Nearly all of the drop was predicted without any supplement, because the group was chosen for high first readings. Only a randomly assigned comparison group chosen the same way can show whether the supplement adds anything.

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Challenge 1 — Two Lines, One Product

The reciprocal of is about , not . The lines coincide only when .

Challenge 2 — Sensitivity of the Slope

With fixed, is proportional to : multiply each correlation by the ratio. The units stay units of per unit of , and a stronger association still says nothing about cause.

Challenge 3 — Why Is the Slope in Standard Units Equal to ?

This is smallest at , where it equals : the line leaves the fraction of the variation unexplained and accounts for the fraction .