Cette leçon n'est pas encore traduite. Elle est présentée en anglais. Retour au cours

L-DS-1: Statistical Vocabulary and Sampling

Module 1 · Descriptive Statistics

Section 1: Introduction

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

In 1936, a major American magazine called the Literary Digest mailed 10 million questionnaires asking people who they planned to vote for in the upcoming presidential election. Nearly 2.4 million people responded — an enormous number by any standard. Based on those responses, the magazine confidently predicted that Alfred Landon would defeat incumbent President Franklin Roosevelt in a landslide.

Roosevelt won by one of the largest margins in American electoral history.

How? The Literary Digest had two related selection problems. Its mailing frame drew heavily from telephone directories and automobile registration records, so it did not cover all voters evenly. More importantly, only about 24% of recipients returned the questionnaire, and later evidence indicates that Roosevelt supporters were much less likely to respond. The imperfect frame and the differential non-response both mattered, but non-response was probably the larger cause of the failed prediction. Neither problem could be repaired by collecting millions of responses through the same process.

That story illustrates a truth that runs through everything in statistics: the quality of your conclusions can never exceed the quality of your data collection. Before we can talk about calculating averages, drawing graphs, or testing hypotheses, we need to get the foundations right — the vocabulary, the concepts, and the methods that determine whether our data is trustworthy in the first place.

This lesson is where all of statistics begins.

After this lesson, you will be able to:

Distinguish a population from a sample and a parameter from a statistic, using the correct notation for each
Classify variables as nominal, ordinal, discrete, or continuous and explain the clue that determines the type
Compare common sampling methods, identify likely sources of bias, and judge what conclusions a sample can support
Evaluate basic survey questions for leading, double-barrelled, or ambiguous wording

This is Lesson L-DS-1 — the first lesson of the entire course. Everything that follows — probability, confidence intervals, hypothesis tests, regression — builds on the concepts you’ll learn here. Spend time with the vocabulary. It’s the language the rest of the course is written in.

Section 2: Prerequisites

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Before diving into the mechanics of data collection, ensure you are comfortable with the basic mathematical language used to describe groups and values.

Entry numeracy — sets and subsets: You already know that a subset sits inside a larger set. Today that relationship will help you separate the group a study targets from the smaller group it actually measures.
Entry numeracy — fractions, proportions, and percentages: You can convert a count such as out of into . You will use that skill to read response rates and sample composition.
Everyday data language — characteristics and observations: You can distinguish a characteristic such as height from one recorded value such as . Today that distinction becomes the formal language of variables and values.
Entry numeracy — rounding: You can round a decimal to a requested place, such as . Later lessons will use that habit when reporting statistics.

Entry Numeracy Check

This checks arithmetic you already know; it does not test the new statistical vocabulary.

Before continuing, check that you can:

Recognize that a subset belongs inside a larger set. Convert a count out of a total into a fraction or percentage. Tell a characteristic from one observed value of that characteristic.

Bridge to the new language: In everyday speech, “the people surveyed” may sound like the whole group, and any reported number may sound like “the result.” Statistics is more precise: the target group and measured subset have different names, as do numbers that describe each one. Section 3 builds those roles from the familiar set-and-subset relationship.

Section 3: Core Concepts

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Your C1–C6 route through the lesson:

C1: identify the target group and the measured subset;
C2: identify the number that describes each group;
C3: classify what was recorded;
C4: determine how the sample was selected;
C5: judge whether selection can distort the conclusion; and
C6: inspect how survey wording can distort responses.

C1 — Population and Sample

Every statistical study begins with a question about a group. That group is called the population. But studying an entire population is usually impossible — it’s too large, too expensive, or simply impractical. So instead, we study a subset of it: a sample.

Population

The population is the complete set of all individuals, objects, or measurements of interest in a study. It is the group we want to draw conclusions about.

Notation: population size is written N (capital letter).

Examples: all adults in Quebec; every manufacturing component produced last month; all registered voters in Canada.

Sample

A sample is a subset of the population actually selected for study. We observe the sample and use it to make inferences about the population.

Notation: sample size is written n (lowercase letter).

Examples: 400 adults randomly selected from Quebec; 50 components pulled from last month’s production; 1,200 registered voters contacted by phone.

The key idea: we study the sample, but our goal is always to say something about the population. The sample is a means to an end — the population is what we care about.

A common error is to define the population as “the people I surveyed” — that’s the sample. The population is the group you want to generalize to, whether you’ve reached them yet or not. Ask yourself: “Who do I want my conclusions to apply to?” That’s your population.

C2 — Parameter and Statistic

Now that we have a population and a sample, we need to describe them numerically. This is where a critical distinction enters — one that the entire course depends on.

Parameter

A parameter is a numerical value that describes a characteristic of the population. Parameters are usually unknown — we can’t measure the whole population.

Common parameters:

Population mean: (the Greek letter “mu”)
Population proportion:
Population standard deviation: (the Greek letter “sigma”)
Population size: N

Statistic

A statistic is a numerical value computed from sample data. Statistics are known (we calculated them), but they vary from sample to sample.

Common statistics:

Sample mean: (read “x-bar”)
Sample proportion: (read “p-hat”)
Sample standard deviation:
Sample size: n

The relationship to remember: statistics can estimate parameters when the sampling design and analysis support population inference. In that setting, we compute from the sample to estimate the unknown . Without that support, still describes the observed sample, but it is not automatically a representative estimate of .

Memory trick: Parameter → Population (both start with P). Statistic → Sample (both start with S). Greek letters (, ) go with the population; Roman letters (, ) go with the sample.

The most common error in this lesson: Treating a sample statistic as if it were the population parameter. For example, reporting “the average score is ” is fine — but then concluding “so the population mean is ” is an overstatement. The statistic estimates the parameter; it is not equal to it. The difference matters for everything that comes later.

Notation focus: Use and for means; use and for proportions. The symbols (population standard deviation) and (sample standard deviation) are introduced above so the notation is familiar when you meet them in L-DS-2.

Figure 1: Where a statistic comes from. Draw a random sample of 10 people from the population and watch the selected values produce x̄. A new sample can produce a new x̄, while the population and its μ stay fixed.

The simulation can reveal because it contains every value in its small, artificial population. In a real study, the population parameter still exists, but it is usually hidden. The next figure keeps the same population–sample structure while showing how it transfers to different research settings.

Figure 2: The four-element framework across contexts. Population parameters describe the whole group and are usually unknown; sample statistics are computed from observed data and used to estimate them. Take another sample to see whether the statistic moves while the fixed parameter stays put; earlier statistics remain as smaller dots. The census scenario shows the important exception: measuring everyone reveals the parameter directly.

C3 — Types of Variables

Before collecting data, you need to know what kind of data you’re dealing with. The type of variable determines which graphs, which summaries, and which statistical tests are appropriate. Using the wrong tool for the wrong type of variable is one of the most common mistakes in applied statistics.

Qualitative (Categorical) Variables

A qualitative variable classifies individuals into categories. The values are labels or names, not numbers you can do arithmetic with.

Nominal: Categories have no natural ranking. You can only check equality, not order.

Examples: eye colour (brown, blue, green); country of birth; favourite music genre; blood type (A, B, AB, O).

Ordinal: Categories have a meaningful order, but the gaps between ranks are not necessarily equal.

Examples: education level (high school < college < university < graduate); customer satisfaction (poor / fair / good / excellent); pain scale (1 through 10 as labels).

Quantitative (Numerical) Variables

A quantitative variable takes numerical values where arithmetic comparisons make sense. Differences are meaningful on interval and ratio scales; ratios are meaningful only when the scale has a true zero. For example, is 10 degrees warmer than , but it is not “twice as hot.”

Discrete: Takes only countable values — usually whole numbers. You can list them (at least in principle).

Examples: number of children in a household; number of defects per batch; number of goals scored in a game.

Continuous: Is modelled as able to take any real value in an interval, even when the recorded measurement is rounded. Measured, not counted.

Examples: height (171.3 cm, 171.31 cm, …); temperature; time to complete a task; blood pressure.

Trap 1: A variable coded with numbers is not automatically quantitative. Postal codes are numbers, but you can’t compute “average postal code.” Phone numbers, student ID numbers, and jersey numbers are nominal.

Trap 2: Pain scores rated 1–10 are often treated as quantitative, but technically they’re ordinal — the gap between “3 out of 10” and “4 out of 10” may not equal the gap between “7 out of 10” and “8 out of 10.” Context matters for this distinction.

Trap 3: Age in years (18, 19, 20…) looks discrete, but age is really continuous — you’re always some fractional number of years old. The measurement convention creates the appearance of discreteness.

Ordinal data and arithmetic don’t mix. When a rating scale is coded as numbers (1 = Terrible, 5 = Excellent), it is tempting to compute an average. But ordinal gaps are not guaranteed to be equal — the jump from “Poor” to “Okay” may not represent the same real-world distance as the jump from “Good” to “Excellent.” Throughout this course, Likert-scale and ranked-category responses are treated as ordinal. A more honest summary uses the median category or a frequency table. You will see exactly why this matters in L-DS-2.

Figure 3: The variable-type hierarchy. Select any example below the tree to trace its classification path. Each leaf node shows a distinct colour used consistently in this course. The ⚠ examples are traps — variables that look like one type but are actually another.

C4 — Sampling Methods

How you select your sample determines whether it will accurately represent the population. There are many methods — each with different strengths, weaknesses, and appropriate contexts.

Simple Random Sampling (SRS)

Every possible sample of size n has an equal probability of being selected. Consequently, each individual has the same inclusion probability, .

How: Number every individual in the population 1 through N; use a random number generator or lottery to select n distinct individuals without replacement.

Strength: The usual sample mean and sample proportion are unbiased estimators under SRS; the design produces the mathematical properties statistical inference relies on.

Weakness: Requires a complete sampling frame — a list of every individual in the population — which is often unavailable or expensive to build.

Stratified Sampling

Divide the population into non-overlapping subgroups called strata (e.g., age groups, departments, regions). Then draw a simple random sample from each stratum. For greater precision, designers often choose strata whose members are similar with respect to the outcome, but similarity is a design goal rather than the definition.

Strength: Guarantees representation of important subgroups; can produce more precise estimates than SRS for the same total sample size.

Weakness: Requires knowing the strata boundaries in advance; more complex to execute than SRS.

Systematic Sampling

Number every individual, choose a random starting point, then select every k-th individual. In the simplest case, is an integer and ; non-integer intervals require a fractional or circular systematic procedure.

Example: To sample 50 from 500, select a random start between 1 and 10, then take every 10th person after that.

Strength: Easy to implement; spreads the sample evenly across the population.

Weakness: A periodic pattern aligned with the interval can make a particular sample badly unrepresentative and greatly increase sampling variance. With a genuinely random start in the exact case, periodicity alone does not make the usual sample mean biased.

Cluster Sampling

Divide the population into naturally occurring groups called clusters (e.g., schools, city blocks, hospitals). Randomly select a few clusters, then survey every individual in the selected clusters. Efficient cluster designs prefer clusters that each resemble a small version of the population, but the selection mechanics define the method.

Strength: Very cost-effective when the population is geographically dispersed and there is no complete list of individuals.

Weakness: Often has higher sampling variance than an equally sized SRS when individuals within the same cluster have similar outcomes, as commonly happens in schools, households, and neighbourhoods.

Multistage Sampling

A combination of methods applied in stages. Common in large national surveys.

Example: Stage 1 — randomly select provinces. Stage 2 — within each province, randomly select cities. Stage 3 — within each city, randomly select households. Stage 4 — interview one adult per household.

Strength: Practical for very large populations spread across wide areas.

Weakness: Complex to design; errors can compound across stages.

Convenience Sampling

Select whoever is easiest to reach — the first n people who walk by, the first respondents to an online survey, volunteers who self-select.

Strength: Fast and cheap.

Weakness: High risk of selection bias and no known inclusion probabilities. The people who are easiest to reach may differ systematically from the rest of the population.

Cluster ≠ Stratified. Both divide the population into groups, but the sampling mechanics run in opposite directions. In stratified sampling, you sample individuals from every group. In one-stage cluster sampling, you randomly select some groups and include everyone in those groups. Good designs often aim for similarity within strata and diversity within clusters, but those are efficiency goals, not guarantees.

Figure 4: A population of 56 people arranged in a grid. Each tab shows which people get selected under that sampling method. For Stratified sampling, dot colour shows stratum membership; for Cluster, it shows cluster membership.

C5 — Bias in Sampling

A sample is biased when it systematically favours certain outcomes over others — when it consistently misrepresents the population in one direction. Bias is not random error. Increasing the sample size through the same biased selection mechanism may reduce random variation, but it does not by itself remove the bias. A biased sample of 10,000 people can be less trustworthy than an unbiased sample of 100.

Common Sources of Sampling Bias

Undercoverage: Some segments of the population have a lower probability (or zero probability) of being selected. Example: An online survey excludes people without internet access.
Voluntary response bias: People who feel strongly about an issue are more likely to respond. Example: Calling in to a radio poll about crime — only the angriest listeners call.
Non-response bias: People selected for the sample don’t respond, and those who don’t respond differ systematically from those who do. Example: A survey on work satisfaction — disengaged employees may ignore it.
Convenience-sampling risk: Whoever is easiest to reach receives a larger or unknown chance of inclusion. The ordinary unweighted estimate is biased when those inclusion chances are associated with the outcome being measured.

Bias affects the validity of conclusions: a biased sample may give us an accurate picture of who responded, but a distorted picture of the population we actually care about.

Figure 5: Each dot is one sample's estimate of μ (mean weekly exercise hours). The four quadrants cross bias (centred on μ vs. off-target) with precision (tight vs. spread) — because the two are independent. Read the columns: the left (unbiased) column centres on μ whether tight or spread; the right (biased) column misses μ either way. The top-right cell is the trap — a precise, confident-looking process that remains systematically centred off target. Bias is not random error: a bigger sample shrinks the spread but never pulls a biased cloud back onto μ.

C6 — Survey Design Principles

Enrichment — beyond the assessed standards. Survey-question design is valuable real-world knowledge, but it isn’t part of this lesson’s graded standard (S-DS-1 covers vocabulary, classification, sampling methods, and sampling bias — i.e. who you ask, not how you word the question). Read this to become a sharper consumer of surveys; it won’t appear on a standards assessment.

Even with a perfectly selected sample, a poorly designed survey can still produce bad data. The questions themselves can introduce error.

Sources of Response Error in Surveys

Leading questions: Wording that pushes respondents toward a particular answer. Bad: “Do you agree that the current tax rate is too high?” Better: “Do you think the current tax rate is too high, about right, or too low?”
Double-barrelled questions: Two questions combined into one. Bad: “Are you satisfied with the price and quality of the product?” (What if you love the quality but not the price?)
Social desirability bias: Respondents give answers they think are socially acceptable, not honest ones. Example: Self-reported exercise frequency, hours spent studying, or income.
Ambiguous wording: Questions that different respondents interpret differently. Bad: “Do you eat regularly?” (What does “regularly” mean?)
Order effects: The order of questions influences answers. Asking about satisfaction with a specific product before asking about overall satisfaction changes the overall satisfaction rating.

Figure 6: Four common survey design flaws. For each, the problematic word or phrase is highlighted in the original question. Click a flaw type to examine it — then compare the original against the corrected version.

Good survey design rules of thumb:

Ask one thing per question
Use neutral wording — no loaded language
Pilot test the survey with a small group before the real study
Keep the questionnaire as short as possible (longer surveys = more non-response)
Guarantee anonymity where possible (reduces social desirability bias)

Section 4: Worked Examples

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Let’s work through some examples together — classification and identification problems that look exactly like what you’ll see in practice. I’ll narrate my thinking at each step, not just the final answer.

Example 1 — The Four Elements

Scenario: A nutritionist wants to know the average daily sugar intake of Canadian adults. She recruits 250 adult volunteers from three cities, records their sugar intake for a week, and computes an average of 87 g/day.

Identify: (a) the population, (b) the sample, (c) the parameter of interest, (d) the statistic.

Step 1 — Identify each element.

Let’s work through each piece systematically.

(a) Population: “Canadian adults” — the entire group the nutritionist wants to draw conclusions about. Not just adults in three cities; she wants to generalize to all Canadian adults. This is a large, defined group that was never fully studied.

(b) Sample: The 250 adult volunteers recruited from three cities. This is the subset actually measured. Note: these are volunteers, which raises concerns about bias (voluntary response) — we’ll come back to that.

(c) Parameter of interest: The average daily sugar intake of all Canadian adults — written . This is unknown. We didn’t measure every Canadian adult, so is never directly observed.

(d) Statistic: The sample average of 87 g/day — written . This was computed from the 250 participants and directly describes them. Because they were volunteers recruited from only three cities, the design does not justify treating g/day as a representative estimate of for all Canadian adults.

Key takeaway: We know for these participants because we computed it from their data. The population mean remains unknown, and this voluntary-response design provides no probability-based justification for claiming . The statistic could happen to be close to , but the selection process does not establish that it is representative.

Example 2 — Which Sampling Method?

Scenario: A university wants to survey its 8,000 students about library resources. Here are three different approaches they could use:

Approach A: Assign each student a number 1–8000, generate 300 distinct random numbers without replacement, and survey the students with those numbers.
Approach B: Divide students by faculty (Science, Arts, Engineering, Business, etc.). Randomly select 50 students from each faculty.
Approach C: Post a survey link on the university’s social media page and survey whoever responds.

Classify Approaches A, B, and C. For each one, record the sampling method and the key clue that identified it. Then recommend Approach A or B for this university and explain the tradeoff behind your choice.

Pause and write. Record your answer and reasoning on paper or in your notes before continuing. Even if you are unsure, committing your thinking helps you see what you understood and what you need to revisit.

Show Solution

Approach A — Simple Random Sampling (SRS). Every possible group of 300 students has the same chance of being selected because the design chooses 300 distinct numbers without replacement from the complete numbered list. The defining clues are the complete frame, random selection, and no repeated student numbers.

Approach B — Stratified Sampling. The strata are the faculties (homogeneous subgroups). We sampled randomly from every stratum. Key clue: “divide students by faculty” (the strata) + “randomly select from each” (the sampling step).

Approach C — Convenience Sampling (voluntary response). Posting a link and surveying whoever responds is the definition of voluntary response bias. Only students who feel motivated to click will respond — probably not a representative cross-section. Key clue: “whoever responds.”

Which approach would you recommend? Approach B uses equal allocation: every faculty contributes 50 students, regardless of faculty size. It guarantees representation from every faculty and is especially useful for comparing faculties. It is not proportional unless the faculties are the same size, and it is not automatically more precise than SRS; precision depends on how responses vary within and between faculties. For a population-wide estimate, unequal faculty sizes would require appropriate weights. Approach A is also defensible, but it does not guarantee a particular number from every faculty. Approach C is problematic because students with strong opinions are more likely to respond.

Example 3 — Variable Types in the Wild

Scenario: A hospital collects the following data for each patient admitted to the emergency room:

Patient’s primary language
Triage level (1 = critical, 2 = urgent, 3 = less urgent, 4 = non-urgent, 5 = minor)
Time (in minutes) from arrival to first physician contact
Number of previous ER visits in the past year
Discharge status (discharged / admitted / transferred)

Classify all five variables as nominal, ordinal, discrete, or continuous. Beside each classification, record the clue you used.

Hint: For each variable, ask yourself (1) Is it a category label or a number you can do arithmetic with? (2) If it’s a category, does it have a natural order? (3) If it’s a number, can you get fractional values?

Show Solution

(a) Primary language → Qualitative, Nominal. Languages are category labels with no natural order. “Spanish” is not greater than or less than “Mandarin.”

(b) Triage level → Qualitative, Ordinal. The levels 1–5 have a meaningful order (1 is more severe than 5), but the gaps between levels aren’t equal. The difference in severity between levels 1 and 2 is not the same as between levels 4 and 5. Ordinal: ordered categories with unequal gaps.

(c) Time to physician contact → Quantitative, Continuous. Time can take any value in a range — theoretically 14.3 minutes, 14.37 minutes, 14.371 minutes. It’s measured (not counted) and any decimal value is possible.

(d) Number of previous ER visits → Quantitative, Discrete. You count visits — you can’t have 2.7 ER visits. It takes whole number values and the gaps are equal.

(e) Discharge status → Qualitative, Nominal. Three categories with no natural order — “admitted” is not greater than “transferred.” Category label, no ranking.

Example 4 — Spotting Hidden Bias

Scenario: A municipal government wants to know whether residents support a new cycling infrastructure project that would repurpose some car lanes. They commission a survey and mail questionnaires to 2,000 randomly selected registered voters. They receive 380 completed responses.

The results: 71% of respondents support the project. A city councillor says: “Great news — 71% of voters support this.”

Decide whether the councillor’s claim is justified. Record your strongest concern, then write one sentence that reports the result more honestly.

Show Solution

This is a real example of how a technically “random” sample can still produce unreliable conclusions.

Problem 1 — Non-response bias: Only 380 of 2,000 responded — a 19% response rate. The 81% who didn’t respond may have very different opinions. Who doesn’t bother to mail back a survey about cycling infrastructure? Possibly people who rarely cycle and have no strong motivation to engage with the topic. We simply don’t know why they didn’t respond — and that uncertainty is the problem.

Problem 2 — Wording matters: We don’t know how the question was phrased. “Do you support new cycling infrastructure?” might get different responses than “Do you support repurposing car lanes for cyclists?” Both describe the same project, but one activates different associations.

Problem 3 — The 71% is a statistic, not the parameter: The councillor said “71% of voters support this.” But the 71% is the sample proportion, , not the unknown population proportion . Generalizing to all voters requires accounting for the sampling design and non-response; the low response rate makes that inference doubtful.

The honest summary: “71% of the 380 respondents support the project. However, non-response bias may make this estimate unreliable as a measure of the full voter population’s opinion.”

Section 5: Guided Practice

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Work through each problem below. The dropdown will tell you immediately whether you’ve chosen the right answer — and if not, it explains exactly why the other choices are wrong.

Problem 1 — The Four Elements (C1 + C2)

A sports analytics company wants to know how many minutes per week professional basketball players spend on strength training. They contact 60 players from the NBA and find an average of 280 minutes per week.

Step 1 — What is the population?

Step 2 — What is the statistic?

Step 3 — Which notation best represents the parameter of interest?

Problem 2 — Notation Match (C1 + C2)

Match each description to the correct notation.

(a) The average salary of all 4,500 employees at a company is $67,200.

Which notation correctly represents the reported average salary?

(b) A researcher randomly selects 80 apartments in Montreal and finds the average rent is $1,450/month.

Which notation correctly represents the reported average rent?

Problem 3 — Bias Identification and Direction (C5)

For each scenario, identify (a) the type of bias present and (b) the direction of the bias — whether it likely makes the estimate too high or too low.

Scenario A: A television station invites viewers to text their opinion on whether a proposed tax increase should pass. 78% of the 4,200 responses say “No.”

What type of bias is most clearly present?

Which direction does this bias most likely push the estimate?

Problem 4 — Variable-Type Rematch (C3)

Classify the variable from its meaning, not from how it happens to be recorded. Commit to one category before reading the feedback, then generate another context until the decision rule feels automatic.

Section 6: Independent Practice

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Try these problems on your own. Solutions are hidden — attempt each one before revealing. The problems are intentionally interleaved across all six concepts.

Problem 1 — Choose the Sampling Design (C4)

The context changes on every draw. Select the method that matches the actual selection mechanics; feedback appears only after you commit.

Problem 2 — Parameter or Statistic? (C2)

For each study, decide whether the reported number comes from the whole defined population or from a sample. Then match that role to the correct symbol.

Problem 3 — Bias in a News Story (C4 + C5)

Read the following excerpt, then answer the questions.

“A new study proves that people who work from home are more productive than those who work in offices. Researchers from TechTrend International surveyed 1,200 fully remote employees at five tech companies. 84% reported that they were more productive when working from home.”

Which study design can be identified from this excerpt?

Now analyze the conclusions:

(a) Is the conclusion (“people who work from home are more productive”) justified by the study? Explain. (b) Identify at least two specific sources of bias. (c) Who is excluded from the population, and how might this affect the conclusion?

Show Solution

(a) No — the conclusion dramatically overstates what the study shows. The study surveyed remote employees who self-reported their productivity. There is no control group (in-office workers measured the same way), no objective productivity measure, and no documented probability sample from a broadly defined population of workers. “Proves” is inappropriate — this is one survey with significant limitations.

(b) Sources of bias:

Social desirability bias: Remote workers asked whether they’re productive while working remotely may feel pressure to say “yes” — admitting low productivity could feel like justifying an end to remote work privileges.
Undercoverage: Only remote employees at five tech companies are included. Workers in industries where remote work is impossible (manufacturing, healthcare, retail) are entirely excluded.
Unspecified selection: The excerpt does not say how the companies or employees were chosen. If employees volunteered, voluntary-response bias would be a concern; another selection process would require a different analysis.

(c) The population is implicitly “all workers” or “all people who work from home,” but only tech workers at companies that already embraced remote work are included. This excludes:

Workers in industries where remote work is impractical
Employees who tried remote work and returned to the office (survivorship bias)
Workers who want remote work but haven’t had the option

The conclusion would need a much broader, randomized study design to support such a sweeping claim.

Now drill the bias types themselves. Name the bias in each scenario — the reveal defines the whole taxonomy so you can tell the confusable types apart.

Problem 4 — Variable Type (C3)

Classify the variable by what its values mean. The generated contexts deliberately include rounded measurements, numeric codes, ordered labels, and true counts.

Problem 5 — Population or Sample? (C1)

Identify the target group or the measured subset. The distractors deliberately mix in the variable and the numerical result, so use the role of each phrase rather than a memorized keyword.

Problem 6 — Survey Question Critique (C6)

Enrichment — not assessed. This problem practices survey-question design (C6), which is beyond the graded standards for this lesson. Do it to sharpen your judgment — it won’t be on a standards check.

Each question below has a flaw. Identify the problem with each question and rewrite it to fix the flaw.

”Don’t you agree that our school needs better sports facilities?"
"Are you satisfied with the price and the speed of our delivery service?"
"How often do you exercise regularly?” (with response options: Never / Sometimes / Often / Always)
“Given that most experts agree climate change is a serious problem, do you support carbon taxes?”

First, practice identifying the wording flaw in a fresh question. Then write your four improved questions on paper or in your notes and compare them with the models below.

Show Solution

(a) Problem: Leading question. “Don’t you agree” frames the question to pressure respondents toward a “yes.” Fix: “Do you think our school’s sports facilities need improvement?” (or offer “Yes / No / No opinion”)

(b) Problem: Double-barrelled. A respondent might love the price but be unhappy with speed — the question forces one answer for two separate things. Fix: Split into two questions: “How satisfied are you with the price of our delivery service?” and “How satisfied are you with the speed of our delivery service?”

(c) Problem: Ambiguous wording. “Regularly” means different things to different people — some might define it as once a week, others as daily. The response options (Sometimes, Often) are equally vague. Fix: “In an average week, on how many days do you exercise for at least 30 minutes?” (Response: 0 / 1–2 / 3–4 / 5–6 / 7)

(d) Problem: Leading question with a loaded premise. “Most experts agree climate change is a serious problem” primes respondents toward agreement before the actual question is asked. Fix: Remove the preamble: “Do you support the introduction of carbon taxes in Canada?” (Yes / No / Unsure)

Section 7: Mastery Check

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

These questions ask you to recall and apply what you’ve learned independently.

Question 1 — Feynman Test

A classmate missed today’s lesson. Explain in your own words:

What is the difference between a population and a sample?
Why do statisticians use samples instead of studying the whole population?
What is the difference between a parameter and a statistic, and why does the distinction matter?

Write your explanation as if you’re texting your classmate — clear, direct, and with no jargon you haven’t defined.

Compare with a model answer

A population is the full group you care about — like all students at your school. A sample is just a portion you actually study — like 50 randomly chosen students. We use samples because studying everyone is usually too expensive or time-consuming.

A parameter is a number that describes the population (like the true average GPA of all students — we might never know it exactly). A statistic is a number computed from your sample (like the average GPA of your 50 students — you computed it directly). When the sampling design supports population inference, the statistic can estimate the parameter. Otherwise, it directly describes only the observed sample.

Key check: Does your answer clearly state that parameters are usually unknown while statistics are computed from data? That’s the heart of why this distinction matters.

Question 2 — Apply (C3 + C4)

A coffee chain with 240 store locations across Canada wants to measure customer satisfaction. Their data team proposes two approaches and asks you to evaluate them.

The chain has four regions: Atlantic (20 stores), Quebec (60 stores), Ontario (80 stores), Western Canada (80 stores).

Plan 1: Randomly select 30 stores. Within each selected store, randomly sample 2% of that store’s transactions from the previous month and survey the corresponding customers.

Plan 2: From each region, randomly select stores proportional to the region’s number of locations (Atlantic: 5 stores, Quebec: 15 stores, Ontario: 20 stores, Western: 20 stores). Within each selected store, randomly sample 2% of that store’s transactions from the previous month and survey the corresponding customers.

Evaluate the plans.

(a) Which sampling method does Plan 1 use?

In one sentence, what is Plan 1’s main weakness in this context?

(b) Which sampling method does Plan 2 use?

How does Plan 2 address the weakness you named in (a)?

(c) Customer satisfaction rating is measured as: Very Dissatisfied / Dissatisfied / Neutral / Satisfied / Very Satisfied.

What type of variable is the customer satisfaction rating?

Why does that variable type matter for how you’d summarize the data?

Show model answers (the written parts)

(a) weakness: Because the 30 stores are drawn without regard to region, the realized sample can over-represent one region and include few or no Atlantic stores. Plan 1 is a two-stage cluster design: stores are selected first, then transactions are selected within those stores.

(b) how Plan 2 fixes it: Treating the four regions as strata and selecting the same fraction of stores from each region guarantees proportional store representation. Sampling the same fraction of transactions within each selected store also avoids giving low-traffic stores the same customer weight as high-traffic stores.

(c) why the type matters: Because the rating is ordinal, you can report the mode or median category and the percentage in each level, but a numerical mean (“average satisfaction = 3.7”) is questionable — the gaps between levels aren’t guaranteed equal, so the scale isn’t interval-level.

Question 3 — Find the Error (C4 + C5)

A student analyses an informal social-media poll in four claim-steps. Exactly one claim contains an error — find it, then read the feedback. Each new problem plants a different mistake, so cycle through several.

Self-Assessment

How confident are you with the material in this lesson?50% confident

Still unsureFully confident

If you’re under 70% confident, revisit Section 3 (Core Concepts). Focus on the concept that felt shakiest in this section — that’s the one most worth reviewing before you continue.

Section 8: Boss Fight

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

You’ve built the vocabulary. You’ve practiced the classifications. Now it’s time to put it all together.

Choose your path. Both cover the same concepts — they just ask you to use them differently.

🔬 The Analyst

A national survey was conducted. Your job is to dissect it — identify what was done well, what was done wrong, and what conclusions are and aren’t justified.

Emphasis: critical reading, classification, identifying bias

🏗️ The Architect

A research question has been posed. Your job is to design the study — choose sampling methods, write good survey questions, and justify every decision.

Emphasis: methodology, survey design, justification

🔬 Path A — The Analyst: Dissecting a Real Survey

Below is a summary of a survey conducted by a Canadian media outlet. Read it carefully — there are problems hiding in every paragraph.

The Survey Report:

“We asked Canadians about their screen time habits. Our team emailed a survey to 5,000 subscribers of our newsletter, and 1,847 completed it — a 37% response rate. Results showed the average daily screen time was 6.4 hours, which we report as the true average for all Canadian adults ( h). Among respondents, we identified three groups by profession — students, office workers, and other — and found students averaged 8.1 h/day, office workers 6.0 h/day, and others 5.2 h/day. Since we collected professional background, this constitutes stratified sampling. The screen time question asked: ‘How many excessive hours do you spend on screens daily?’ Respondents entered a whole number from 0 to 24 (if they entered ‘3,’ this means 3 hours). The variable ‘daily screen time’ is qualitative-ordinal because responses are recorded as whole numbers.”

Your Analysis Tasks

Task A1. Audit the report’s chain of evidence. Identify the two claims that most seriously overreach what was actually done, name the statistical ideas needed to evaluate them, and explain each verdict.

Task A2. Rewrite the headline conclusion so it says exactly what the data support. Include the correct symbol for the observed 6.4-hour value and name the broader quantity the report wants to learn about.

Task A3. Identify at least two threats that could make the reported value differ systematically from the target value. State a direction only where the scenario supports one; otherwise explain why the direction is unknown.

Task A4 (survey wording is enrichment; variable classification is assessed). Classify (i) the underlying daily screen-time duration and (ii) the recorded whole-hour response. Explain what information the recording rule loses, then rewrite the question to remove its strongest wording problem while preserving a suitable numerical response.

Show Full Solution

A1 — The two most serious overreaches: First, the report calls post-hoc profession groups “strata.” Stratified sampling requires dividing the target population into strata and randomly sampling from every stratum before data collection. Here, newsletter subscribers formed a convenience frame and respondents then self-selected. Second, the report calls the respondents’ mean the true mean for all Canadian adults. The sample cannot establish that population value because the frame and response process do not represent all Canadian adults.

A2 — Defensible rewrite: “Among the 1,847 newsletter subscribers who responded, the sample mean daily screen time was hours. This statistic may be used cautiously to investigate the unknown population mean for Canadian adults, but this survey design does not support a representative national estimate.”

A3 — Threats to the estimate:

Undercoverage (direction unknown): Newsletter subscribers differ from Canadian adults who are not subscribers. The frame excludes part of the target population, but the scenario does not establish whether subscribers have more or less screen time.
Potential non-response bias (direction unknown): Only 37% responded. Bias occurs if respondents and non-respondents differ in screen time; the scenario gives no benchmark that identifies the direction.
Question wording (plausibly downward): Calling screen time “excessive” signals that high values are undesirable and can encourage underreporting.

A4 — Measurement and wording: The underlying duration of daily screen time is quantitative and continuous. The recorded response is quantitative and discrete because the instrument accepts only the whole numbers 0 through 24. Rounding to whole hours loses within-hour detail, but it does not turn a numerical measurement into an ordinal category. A neutral version is: “During the past seven days, approximately how many hours per day did you spend looking at screens (phone, tablet, computer, and television combined)? Enter a value from 0 to 24, rounded to the nearest tenth of an hour.”

Reflection: What was the most challenging part of this analysis? What would you check first if you were a journalist reviewing this survey before publication?

🏗️ Path B — The Architect: Design the Study

You’re working as a research consultant for the Quebec Ministry of Education. They need to understand student mental well-being across all CEGEP institutions in Quebec.

Context: Quebec has 48 public CEGEPs. For this study, define the population as all students enrolled in regular education at those institutions during the selected reference term. Enrollment varies substantially across institutions. The ministry wants to survey 3,000 students. Mental well-being will be measured with a pre-specified, validated multi-item instrument whose documented scoring rule produces a numerical composite score. The ministry’s research question is: “What is the average composite mental-well-being score of students in Quebec’s public CEGEPs, and do scores differ by institution size?” Any enrollment totals and size categories used in the final design must come from a dated ministry roster for that reference term.

Your Design Tasks

Task B1. Translate the ministry’s question into a precise target group, a quantity to estimate, and the sample evidence that could estimate it. Use notation where it clarifies the distinction.

Task B2. Design a probability-based selection procedure that supports both a province-wide estimate and credible comparisons by institution size. Specify the frame, every selection stage, allocation, and any weighting; justify the design against two plausible alternatives.

Task B3. Anticipate one way the realized data could still misrepresent the target group. Propose a mitigation and state what evidence you would inspect before claiming that the mitigation worked.

Task B4 (question wording is enrichment; variable classification remains assessed). Describe the measurement instrument and write one example neutral, time-bounded well-being item. Classify the individual item response, the instrument’s validated composite score, and the institution-size variable required by the research question. Explain how to derive institution size from the sampling roster without asking students for an extra demographic. State two privacy protections required before collection.

Note: The survey-wording portion of this task is enrichment (C6). Identifying the variable type of each question (C3) is still on-standard.

Show Full Solution

B1 — Target, quantity, and evidence: Population = all students enrolled in regular education at Quebec’s 48 public CEGEPs during the named reference term. Parameter = their population mean composite score on the pre-specified mental-well-being instrument. Sample = the 3,000 students selected through the probability design below; the weighted sample mean estimates .

B2 — Recommended method: Stratified sampling by institution size. Since the research question specifically asks about differences by institution size, use the dated enrollment roster to classify every institution into mutually exclusive small, medium, and large strata. Treat those cut points as documented design choices, not timeless facts.

Give each size stratum a minimum sample large enough for comparison, allocate the remaining surveys in proportion to enrollment, and randomly select students from the complete student lists within each stratum. Because this allocation deliberately oversamples some strata, use inverse-probability weights when estimating the overall provincial mean.

Why not SRS? A student-level SRS could include too few students from the small-institution stratum for a stable comparison. Why not convenience? Easily reached volunteers can differ systematically from students who do not volunteer. Stratification ensures every size group contributes enough observations for the planned comparison.

B3 — Residual threat and check: Non-response can remain even after probability selection. Send consistent reminders and offer the survey in French and English, then compare respondent characteristics with the sampling frame by institution-size stratum. Adjust weights only when the available frame variables justify the adjustment; do not assume reminders removed the bias.

B4 — Measurement, questions, summaries, and privacy:

Use a validated multi-item instrument exactly as documented: preserve its wording, response options, scoring direction, missing-data rule, and composite-score calculation. Validation of the full scoring procedure—not the appearance of numbered response options—is what supports treating the resulting composite score as quantitative and summarizing it with a weighted mean.

Example well-being item: “Over the past two weeks, how often have you felt able to handle the challenges of daily student life?” Options: Never / Rarely / Sometimes / Often / Always Item-response type: Qualitative — Ordinal. Report category frequencies and a median or modal category for this item; its labels alone do not guarantee equal gaps. If this wording is not part of the chosen validated instrument, treat it as an illustrative draft rather than silently adding it to that instrument.
The validated composite score is quantitative because the instrument’s documented scoring procedure supports numerical interpretation and a mean. Do not infer that property from the numbered item options alone.
Institution-size stratum (small / medium / large) is qualitative — ordinal. Derive it by linking each sampled student’s institution to the dated enrollment roster, then retain only the derived stratum needed for analysis. Do not ask for program, age, gender, or another personal characteristic unless a pre-specified, approved research purpose actually requires it.

Because mental-well-being responses are sensitive personal information, replace direct identifiers with a random study ID and keep any temporary linkage file separately with tightly limited access. Report only aggregates with small-cell suppression, provide support resources, delete linkage data when no longer needed, and complete the applicable research-ethics and Quebec privacy review before collection.

Reflection: What aspect of this design was hardest to decide? Is there a sampling method you considered and rejected? What would you do if the ministry’s budget only allowed for 500 surveys instead of 3,000?

Section 9: Challenge Problems

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Ready for more? These go beyond the lesson objectives. They’re for students who want to think more deeply — or get a preview of where this course is headed.

Challenge 1 — The “Perfect” Sample? (C2)

A worker cooperative has exactly eight employees. Their hourly wages are:

The cooperative reports that the mean wage for all eight employees is $50 per hour. A journalist takes a simple random sample of two employees and happens to select the employees earning $20 and $80. The sample mean is also $50 per hour.

The journalist’s editor makes three claims:

“Both $50 values are parameters because they have the same value.”
“The matching means prove that the two-person sample perfectly represents the workforce.”
“This one exact match proves that a simple random sample of two gives an unbiased estimate of the population mean.”

Fact-check all three claims. Your response must:

use correct notation for both means;
determine how many of the possible two-person samples have a mean of exactly $50 and how likely this match is;
explain what matching means do—and do not—show about representativeness; and
show, without calculating every sample mean individually, whether the mean of all possible two-person sample means equals the population mean.

Before writing the full fact-check, verify the two numerical sub-results on fresh, structurally similar wage lists.

Show Solution

Claim 1 — False: equal values can represent different quantities.

The mean computed from the entire population is a parameter:

The mean computed from the two sampled employees is a statistic:

The statistic happens to equal the parameter, but its notation and meaning still depend on where the data came from.

Claim 2 — False: matching one feature does not mean matching the population.

The sample contains only the lowest-paid and highest-paid employees. It omits six of the eight wage levels, so its distribution is not the population’s distribution. In fact, this sample matches both the population mean and the population range, yet it still does not reproduce how wages are distributed between those endpoints. A sample can match one—or even two—summary values by coincidence without being a miniature copy of the population.

There are four two-person samples with a mean of exactly $50:

$20 and $80
$25 and $75
$30 and $70
$35 and $65

The total number of distinct two-person samples is:

Therefore, the probability of an exact match under simple random sampling is:

The match is possible, but it is not evidence of perfect representation.

Claim 3 — The conclusion is correct, but the editor’s evidence is not.

An estimator is unbiased when its average over all possible random samples equals the population parameter. One lucky sample cannot establish that property.

We can check all 28 samples without calculating their means one at a time. Each of the eight wages appears in exactly seven different two-person samples. Therefore, the sum of all 28 pair totals is:

The sum of the 28 sample means is:

Their average is:

So the sample mean is unbiased under this simple random sampling procedure—but because the full collection of possible sample means balances around , not because this particular sample landed exactly on it. Accuracy describes one estimate; unbiasedness describes a sampling procedure over repeated samples.

Challenge 2 — Concentrated vs. Spread Multistage Designs (C4)

A national research institute wants to estimate the proportion of high school students in Canada who spend more than 3 hours/day on social media. Canada has approximately 3,200 high schools and 1.5 million high school students. The data-processing budget permits at most 5,000 completed questionnaires. For this simplified comparison, assume every selected school has at least 100 eligible students, both selection stages use valid probability samples, and the designs are otherwise comparable. Also assume responses have positive within-school correlation: students at the same school tend to be more alike than students from different schools.

(a) Design A randomly selects 30 schools, then randomly selects 100 students within each selected school. Calculate the total number of students surveyed. Does it fit the stated questionnaire cap?

(b) Design B randomly selects 60 schools, then randomly selects 50 students within each selected school. Verify its total sample size and name the two sampling stages.

(c) Both designs survey the same number of students. Under the stated positive within-school correlation, which design should generally produce the more precise estimate? Which design requires fewer school agreements? Explain the tradeoff using their common total sample size.

(d) The true proportion of students who spend more than 3 h/day on social media is an unknown parameter.

Which symbols should represent the population proportion and the sample proportion?

Cold-transfer rematch: The context below changes on every draw. Choose the design that best meets all stated constraints before reading its feedback.

Show Solution

(a) 30 schools × 100 students/school = 3,000 students. Because , Design A fits the stated questionnaire cap.

(b) Design B is a two-stage multistage plan:

Stage 1: Randomly select 60 schools from the 3,200 high schools across Canada (using SRS or a documented provincial stratification plan).
Stage 2: Within each selected school, randomly select 50 students from the school’s enrollment list.
Total: 60 × 50 = 3,000 students, so Design B also fits the 5,000-questionnaire cap.

(c) Design B should generally be more precise under the stated positive within-school correlation because the same 3,000 observations are spread across 60 school contexts rather than 30. Design A is operationally simpler in one specified respect: it requires agreements with only 30 schools rather than 60. Both fit the questionnaire cap, so the supported tradeoff is broader school coverage versus fewer school agreements; no broader cost or practicality claim follows without additional information.

(d) The parameter is the population proportion . The sample proportion computed from the 3,000 selected students is . Both symbols were introduced in C2 of this lesson.

Challenge 3 — When Convenience Sampling Creates Bias: A Proof Sketch (C5)

This problem asks you to reason carefully about the conditions under which an unweighted convenience sample produces a biased estimate.

Consider the following fixed-size model: a population has N individuals, and the procedure ultimately selects exactly n of them. Each individual has some characteristic value x (say, hours of exercise per week). The true population mean is .

Let be the probability that individual i enters this size- sample. Because exactly people are selected, . Under SRS, for every individual. In a convenience process, the values are often unequal or unknown. Unequal probabilities alone do not guarantee bias: in this fixed-size model, the ordinary sample mean is biased when the inclusion probabilities are associated with the measured values. Probability-sampling designs can also use unequal probabilities deliberately and recover population estimates with appropriate weights.

(a) Suppose a gym posts a sign-up sheet for a study on exercise habits and closes registration after exactly people enroll. Give one plausible reason frequent exercisers could have higher and one plausible reason low exercisers could have higher . What evidence would you need before choosing a direction?

(b) For parts (b)–(d), assume inclusion probability is positively associated with exercise time: frequent exercisers have higher than low exercisers. Under that explicit assumption, in what direction will the sample mean be biased relative to ?

(c) Why would increasing the number of gym volunteers fail to repair this particular selection mechanism?

(d) Within this fixed-size model, write a short proof sketch explaining why an association between and biases the ordinary, unweighted sample mean.

Show Solution

(a) Either direction is plausible from the sign-up mechanism alone. Frequent exercisers may notice the gym posting more often or feel more interested in the topic, which could raise their . Low exercisers may be more motivated by support, incentives, or a desire to change, which could instead raise theirs. Evidence about who saw the invitation and response rates by prior exercise level would be needed to choose a direction.

(b) Under the stated assumption that high exercisers have higher inclusion probabilities, will be systematically higher than — the ordinary sample mean will overestimate the true population mean exercise time.

(c) A larger sample drawn through the same gym sign-up mechanism still undercovers non-gym-goers. It can reduce random variation around the mean of the reachable volunteer pool without making that pool represent the full population. Repair requires changing the selection process or using justified adjustment information, not merely adding more volunteers.

(d) Proof sketch: Because the sample size is fixed at , the expected unweighted sample mean is:

Subtracting the population mean gives:

Under SRS, every , so every term in the last sum is zero and . In the gym sign-up process, larger exercise values are paired with larger inclusion probabilities, so the positive terms receive more weight and the expected unweighted mean lies above . The unequal probabilities matter because they are associated with ; unequal probabilities unrelated to the outcome do not force bias in this fixed-size model, and a probability design with known unequal probabilities can use appropriate weights.

Section 10: Solutions Reference

▾

A tag such as (C2 + C3) after a title names the Core Concepts parts the problem draws on, so you know where to look back.

Exact solutions for fixed problems and current reasoning guides for regenerable problems in Sections 5–9 are available on the solutions page. The guides explain decision rules, common mistakes, and interpretation without pretending every generated scenario has one permanent answer.

View Solutions and Reasoning Guides →

If you’re stuck: Re-read the relevant Core Concept in Section 3, then find the Worked Example that maps to that concept (e.g., Example 1 maps to Concept 1). The solutions page shows the reasoning behind every step, not just the final answer.

Quick-Reference Notation

Measure	Population (Parameter)	Sample (Statistic)
Size
Mean	(mu)	(x-bar)
Proportion		(p-hat)

Variable Type	Sub-types	Examples
Categorical (Qualitative)	Nominal	Eye color, Postal code
	Ordinal	Letter grade, Survey rating
Numerical (Quantitative)	Discrete	Number of children, Shoe size
	Continuous	Height, Weight, Time

Sampling Method	How it works
Simple Random (SRS)	Every possible sample of size has an equal chance.
Stratified	Divide into groups (strata), take SRS from every group.
Cluster	Divide into groups (clusters), randomly select some groups, sample everyone in those.
Systematic	Pick every -th individual from a list.
Convenience (Biased)	Pick whoever is easiest to reach.