Correlation vs Regression
When two numerical variables are measured together, we use two key tools to understand their relationship.
Correlation measures the strength and direction of a linear relationship using the correlation coefficient , where .
| Coefficient () | Meaning | Example |
|---|---|---|
| Near | Strong positive linear relationship | Both variables rise together |
| Near | Strong negative linear relationship | One rises as the other falls |
| Near | No linear relationship | Nonlinear patterns may still exist |
Linear regression goes further by fitting the line of best fit . This is chosen by least squares, which minimises the sum of squared vertical distances (residuals) from data points to the line.
Prediction and Pitfalls
The regression line allows you to predict from . The slope represents the average change in per unit change in , while measures how tightly points cluster around the line.
Common pitfall: Concluding that because two variables are correlated, one causes the other.
A high only means variables move together. A confounding variable may drive both (like ice-cream sales and drownings both driven by hot weather), the direction may be reversed, or it may be coincidence.
Causal claims require experiments, never correlation alone.