Correlation and Regression

Relationships Between Variables

When two numerical variables are measured together, we want to know how they relate. Two tools answer this.

Correlation measures the strength and direction of a linear relationship with a single number, the correlation coefficient $r$, which always lies in $[-1, 1]$:

• $r$ near $+1$ — strong positive linear relationship (both rise together).
• $r$ near $-1$ — strong negative linear relationship (one rises as the other falls).
• $r$ near $0$ — no linear relationship (though a nonlinear one may still exist).

Correlation only detects linear association; a perfect parabola can have $r = 0$ despite a clear relationship.

Linear regression goes further: it fits the line of best fit $\hat{y} = mx + b$ through the data, chosen by least squares — minimising the sum of squared vertical distances (residuals) from the points to the line. The line lets you predict $y$ from $x$ and interpret the slope as "the average change in $y$ per unit change in $x$." Regression and correlation are linked: $r$ measures how tightly the points cluster around that best-fit line.

The single most important caution in all of statistics: correlation does not imply causation. A strong $r$ means the variables move together, not that one causes the other. The association could run the other way, or arise from a confounding variable driving both (ice-cream sales and drownings correlate — both are driven by hot weather, not by each other). Establishing causation requires controlled experiments or careful causal reasoning, never correlation alone.

Common pitfall: concluding that because two variables are correlated, one causes the other. Correlation measures only that they move together; a confounding variable may drive both, or the causal direction may be reversed, or the link may be coincidence. "Correlation does not imply causation" — causal claims require experiments or explicit causal reasoning, not a high $r$ alone.

A scatter plot with the accent least-squares line minimising the vertical residual segments from each point, the tightness of clustering illustrating the correlation strength.

$$-1 \le r \le 1,\quad \hat{y} = mx + b$$