Sampling and the Central Limit Theorem

From Sample to Population

Statistics uses a sample — a subset of data actually collected — to infer properties of a population — the whole group of interest. A population characteristic (like the true mean $\mu$) is a parameter; the corresponding sample quantity (like $\bar{x}$) is a statistic that estimates it. The gap between them is what statistical inference must bridge.

For the sample to represent the population, it should be a random sample — every member equally likely to be chosen — avoiding bias (systematic error from non-random selection, like polling only one neighbourhood). No amount of clever analysis fixes a biased sample; representativeness comes first.

Because a sample is random, a statistic like $\bar{x}$ is itself a random variable that varies from sample to sample. Its distribution — the sampling distribution — describes how the estimate fluctuates. Two facts about the sample mean's sampling distribution:

• Its centre equals the population mean $\mu$ (the sample mean is unbiased).
• Its spread is $\frac{\sigma}{\sqrt{n}}$ (the standard error): larger samples give more precise estimates, but only as $\sqrt{n}$, so quartering the error needs four times the data.

The crowning result is the Central Limit Theorem (CLT): for a large enough sample, the sampling distribution of the mean is approximately normal, regardless of the population's own shape. Averages of many independent draws become bell-shaped even when the underlying data are skewed. This is why the normal distribution is everywhere, and why we can attach confidence intervals and tests to sample means.

Common pitfall: thinking the Central Limit Theorem says the raw data become normal, or that larger samples make the population itself normal. The CLT is about the sampling distribution of the mean — the averages become approximately normal as $n$ grows, even if the individual data are skewed. The population's shape does not change; it is the distribution of the statistic (the mean), not the data, that turns normal.