Hypothesis Testing

Hypothesis Testing Foundations

Hypothesis testing is a formal procedure for deciding whether data support a claim. It frames the question as a contest between two opposing hypotheses:

Hypothesis	Role & Description
Null hypothesis ( $H_0$ )	The default, no-effect claim (e.g., the coin is fair)
Alternative hypothesis ( $H_1$ )	The claim you are gathering evidence for (e.g., the coin is biased)

The logic mirrors a courtroom: $H_0$ is presumed true until evidence forces a rejection. You compute a test statistic from data, then find the p-value: the probability of observing data at least as extreme as yours if $H_0$ were true.

The threshold is the significance level $\alpha$ (commonly 0.05). If $p < \alpha$ , you reject $H_0$ in favor of $H_1$ ; if large, you fail to reject it.

A real coin's null distribution, with the p-value as its own tail

Errors and Interpretive Pitfalls

Because decisions rely on chance data, two types of errors are possible:

Error Type	Definition	Probability
Type I error	Rejecting a true $H_0$ (false positive)	$\alpha$
Type II error	Failing to reject a false $H_0$ (false negative)	Noted as $\beta$

There is a direct trade-off: lowering $\alpha$ reduces false positives but raises false negatives.

Common Pitfalls:

p-value misinterpretation: It is not the probability that $H_0$ is true. It is the probability of the data assuming $H_0$ holds.
Failing to reject: This is not proof that $H_0$ is true. Absence of evidence against the null is just weak data, never a proof.

Hypothesis Testing Foundations

Errors and Interpretive Pitfalls

Practise this lesson

Data Analysis and Probability