The Test Statistic & P-Values
A test statistic (like a t or z-statistic) converts your data into a single standardized number measuring distance from what the null hypothesis predicts. It is compared against a critical value—the exact threshold it must exceed to reach statistical significance.
Depending on your research question, tests split into two distinct structures:
| Test type | Predicts | Rejection region |
|---|---|---|
| One-tailed | Directional effect | Entirely in one tail |
| Two-tailed | Non-directional effect | Split between both tails |
A two-tailed test is more conservative because it splits the alpha level, requiring a more extreme result in a single direction to achieve significance.
Confidence Intervals
An alternative to the p-value is the confidence interval (CI): a range of plausible values for the true population effect computed from sample data. CIs show both significance and the plausible size and precision of an effect.
A 95% confidence interval means that if the exact study is repeated many times, about 95% of the resulting intervals will contain the true population value.
Common Pitfall: Never say "there is a 95% probability the true value falls within THIS specific interval." The 95% refers to the long-run process across many repeated samples, not a single computed interval.