Statistical Power
Statistical power is a study's probability of correctly detecting a true effect when one exists (, where is the Type II error rate).
Power depends on four core factors:
| Factor | Effect on power | Why |
|---|---|---|
| Larger sample | Increases | More data points |
| Larger effect | Increases | Bigger true difference |
| Lenient alpha | Increases | Higher Type I error risk |
| Lower variability | Increases | Less "noise" in data |
Common pitfall: Treating statistical significance as a large effect. Huge samples make tiny, trivial effects statistically significant. Always report effect size alongside p-values.
Measuring Effect Size
Effect size measures the MAGNITUDE of a difference or relationship, independent of sample size. It complements p-values, which conflate magnitude with data quantity.
Cohen's d is a standardized effect size for comparing two group means in standard deviation units:
Jacob Cohen proposed these conventions for interpreting Cohen's d:
| Magnitude | Typical d Value |
|---|---|
| Small | Around 0.2 |
| Medium | Around 0.5 |
| Large | Around 0.8 |
These are general guidelines, not rigid rules. What is practically meaningful varies by research area.