# Hypothesis Testing

Mathematics I · Data Analysis and Probability · https://tryals.app/learn/mathematics-i/hypothesis-testing

## Hypothesis Testing Foundations

**Hypothesis testing** is a formal procedure for deciding whether data support a claim. It frames the question as a contest between two opposing hypotheses:

| Hypothesis | Role & Description |
| --- | --- |
| **Null hypothesis** ($H_0$) | The default, no-effect claim (e.g., the coin is fair) |
| **Alternative hypothesis** ($H_1$) | The claim you are gathering evidence for (e.g., the coin is biased) |

The logic mirrors a courtroom: $H_0$ is presumed true until evidence forces a rejection. You compute a **test statistic** from data, then find the **p-value**: the probability of observing data **at least as extreme** as yours **if $H_0$ were true**.

The threshold is the **significance level** $\alpha$ (commonly 0.05). If $p < \alpha$, you **reject $H_0$** in favor of $H_1$; if large, you **fail to reject** it.

## Errors and Interpretive Pitfalls

Because decisions rely on chance data, two types of errors are possible:

| Error Type | Definition | Probability |
| --- | --- | --- |
| **Type I error** | Rejecting a **true** $H_0$ (false positive) | $\alpha$ |
| **Type II error** | Failing to reject a **false** $H_0$ (false negative) | Noted as $\beta$ |

There is a direct trade-off: lowering $\alpha$ reduces false positives but raises false negatives.

**Common Pitfalls:**
- **p-value misinterpretation:** It is **not** the probability that $H_0$ is true. It is the probability of the **data** assuming $H_0$ holds.
- **Failing to reject:** This is **not** proof that $H_0$ is true. Absence of evidence against the null is just weak data, never a proof.

## Practice questions

7 of this lesson's 13 practice questions, with answers. The full set is in the app.

### 1. Two hundred independent tests are run at significance level 0.05, and in every case the null hypothesis is actually true. How many rejections should be expected?

**Answer:** 10

**Why:** Alpha is exactly the Type I error rate, so about 0.05 times 200 = 10 true nulls will be rejected by chance alone. This is why running many tests and reporting only the significant ones manufactures discoveries out of noise.

Page: https://tryals.app/practice/mathematics-i/hypothesis-testing/two-hundred-independent-tests-are-run-at-significance-level-0-05-and

### 2. Complete the definition of the p-value.

**Answer:** The p-value is the probability of **observing data at least as extreme as yours, assuming H0 is true**.

**Why:** The p-value assumes H0 and asks how surprising the data would be. It is not the probability that H0 is true, that would be a statement about the hypothesis, which this framework never assigns a probability to.

Page: https://tryals.app/practice/mathematics-i/hypothesis-testing/complete-the-definition-of-the-p-value

### 3. Ice cream sales correlate with drowning deaths at r = 0.9. What is the most useful thing that correlation tells you?

A. Nothing at all, since correlation carries no information
B. That ice cream consumption is dangerous near water
C. That something makes both move together, here summer heat
D. That the sample must be biased, since the variables are unrelated

**Answer:** C. That something makes both move together, here summer heat

**Why:** The correlation is genuine and points at a shared cause rather than a link between the two. Option 3 overcorrects into uselessness: correlation is how confounders get noticed in the first place, and 'find the third variable' is a productive instruction, it is only the automatic causal reading that fails.

Page: https://tryals.app/practice/mathematics-i/hypothesis-testing/ice-cream-sales-correlate-with-drowning-deaths-at-r-0-9-what-is

### 4. Order the steps of a hypothesis test.

**Answer:**

1. State the null and alternative hypotheses
2. Choose the significance level alpha before seeing the result
3. Compute the test statistic from the sample data
4. Find the p-value, the probability of data this extreme assuming H0
5. Compare the p-value with alpha and state the decision

**Why:** Choosing alpha after seeing the p-value would let you guarantee any conclusion you like, which is why it is fixed in advance. Everything after that is mechanical: statistic, p-value, comparison.

Page: https://tryals.app/practice/mathematics-i/hypothesis-testing/order-the-steps-of-a-hypothesis-test

### 5. A correlation of r = 0 means no LINEAR relationship. Why does that leave open the possibility of a perfect relationship between the two variables?

A. Because r measures only how well a straight line fits.
B. Because r measures correlation rather than causation, and causation may exist without correlation.
C. Because r = 0 means the variables are independent, which is compatible with an exact relationship.
D. Because r = 0 occurs only when the data is too noisy to reveal any pattern.

**Answer:** A. Because r measures only how well a straight line fits.

**Why:** Take y = x² over a symmetric range: every point is determined exactly and r is 0, because the positive association on the right cancels the negative on the left. So plot the data before trusting a correlation. Anscombe's quartet shares means, variances and r = 0.816 across a line, a curve and a cluster whose outlier does all the work.

Page: https://tryals.app/practice/mathematics-i/hypothesis-testing/a-correlation-of-r-0-means-no-linear-relationship-why-does-that

### 6. A result that is statistically significant at the 0.05 level is automatically significant at the 0.01 level as well.

**Answer:** False

**Why:** False. A p-value of 0.03 is below 0.05 but above 0.01, so it is significant at the looser level only. The implication runs the other way: significance at 0.01 does guarantee significance at 0.05, since the stricter threshold is harder to clear.

Page: https://tryals.app/practice/mathematics-i/hypothesis-testing/a-result-that-is-statistically-significant-at-the-0-05-level-is

### 7. Sort each outcome of a hypothesis test.

**Answer:**

- Type I error: Reject H0 when H0 is actually true
- Type II error: Fail to reject H0 when H0 is actually false
- Correct decision: Reject H0 when H0 is actually false, Fail to reject H0 when H0 is actually true

**Why:** Two of the four combinations are correct decisions and two are errors. Type I is claiming an effect that is not there, with probability alpha; Type II is missing one that is. Lowering alpha guards against the first at the cost of the second.

Page: https://tryals.app/practice/mathematics-i/hypothesis-testing/sort-each-outcome-of-a-hypothesis-test