# Correlation and Regression

Mathematics I · Data Analysis and Probability · https://tryals.app/learn/mathematics-i/correlation-and-regression

## Correlation vs Regression

When two numerical variables are measured together, we use two key tools to understand their relationship.

**Correlation** measures the strength and direction of a linear relationship using the correlation coefficient $r$, where **$-1 \le r \le 1$**.

| Coefficient ($r$) | Meaning | Example |
| --- | --- | --- |
| Near $+1$ | Strong **positive** linear relationship | Both variables rise together |
| Near $-1$ | Strong **negative** linear relationship | One rises as the other falls |
| Near $0$ | No **linear** relationship | Nonlinear patterns may still exist |

**Linear regression** goes further by fitting the **line of best fit** $\hat{y} = mx + b$. This is chosen by **least squares**, which minimises the sum of squared vertical distances (residuals) from data points to the line.

## Prediction and Pitfalls

The regression line allows you to **predict** $y$ from $x$. The slope represents the average change in $y$ per unit change in $x$, while $r$ measures how tightly points cluster around the line.

**Common pitfall:** Concluding that because two variables are **correlated**, one **causes** the other. 

A high $r$ only means variables move together. A **confounding variable** may drive both (like ice-cream sales and drownings both driven by hot weather), the direction may be reversed, or it may be coincidence. 

**Causal claims require experiments**, never correlation alone.

## Practice questions

4 of this lesson's 13 practice questions, with answers. The full set is in the app.

### 1. A regression line has equation y = 3x + 5. Predict the value of y when x = 4.

**Answer:** 17 (within ±0.01)

**Why:** 3(4) + 5 = 17. The slope 3 means that on average y rises by 3 for each unit increase in x, an average change, not a promise about any individual data point.

Page: https://tryals.app/practice/mathematics-i/correlation-and-regression/a-regression-line-has-equation-y-3x-5-predict-the-value-of-y

### 2. A regression line has equation y = 2x + 1. For the observed data point (3, 10), compute the residual.

**Answer:** 3 (within ±0.01)

**Why:** The prediction is 7 and the observation is 10, so the residual is 10 - 7 = 3: the point sits 3 above the line. Least squares chooses the line that makes the sum of these residuals, squared, as small as possible.

Page: https://tryals.app/practice/mathematics-i/correlation-and-regression/a-regression-line-has-equation-y-2x-1-for-the-observed-data

### 3. Complete the definition of the least squares criterion.

**Answer:** Least squares chooses the line that minimises **the sum of the squared vertical distances from the points to the line**.

**Why:** The residuals are vertical gaps, and squaring them prevents cancellation while penalising large misses heavily. That squaring is also why a single outlier can drag a regression line noticeably out of position.

Page: https://tryals.app/practice/mathematics-i/correlation-and-regression/complete-the-definition-of-the-least-squares-criterion

### 4. A least-squares line can be fitted to any scatter of points, and its slope always has a value. Why is the line sometimes worth reporting and sometimes meaningless?

A. Because the slope is meaningless whenever the correlation is negative.
B. Because a line always describes the data adequately, so the question of meaning does not arise.
C. Because the procedure fits a line whatever the data looks like.
D. Because least squares is only valid for datasets with more than thirty points.

**Answer:** C. Because the procedure fits a line whatever the data looks like.

**Why:** Least squares is an optimisation, not a test: give it points on a circle and it returns the line minimising squared vertical distances, a well-defined answer to a question nobody should have asked. Deciding whether it means anything is a separate step done on the residuals, which is why r² alone is insufficient, as Anscombe's second dataset shows.

Page: https://tryals.app/practice/mathematics-i/correlation-and-regression/a-least-squares-line-can-be-fitted-to-any-scatter-of-points-and-its