# Power and Effect Size

Psychology I · Statistics · https://tryals.app/learn/psychology-i/power-and-effect-size

## Statistical Power

**Statistical power** is a study's probability of correctly detecting a true effect when one exists ($1 - \beta$, where $\beta$ is the Type II error rate).

Power depends on four core factors:

| Factor | Effect on power | Why |
| --- | --- | --- |
| Larger sample | Increases | More data points |
| Larger effect | Increases | Bigger true difference |
| Lenient alpha | Increases | Higher Type I error risk |
| Lower variability | Increases | Less "noise" in data |

**Common pitfall:** Treating statistical significance as a large effect. Huge samples make tiny, trivial effects statistically significant. Always report effect size **alongside** p-values.

## Measuring Effect Size

**Effect size** measures the MAGNITUDE of a difference or relationship, independent of sample size. It complements p-values, which conflate magnitude with data quantity.

**Cohen's d** is a standardized effect size for comparing two group means in standard deviation units:

$$d = \frac{M_1 - M_2}{SD}$$

Jacob Cohen proposed these conventions for interpreting **Cohen's d**:

| Magnitude | Typical d Value |
| --- | --- |
| **Small** | Around 0.2 |
| **Medium** | Around 0.5 |
| **Large** | Around 0.8 |

These are general guidelines, not rigid rules. What is practically meaningful varies by research area.

## Practice questions

3 of this lesson's 9 practice questions, with answers. The full set is in the app.

### 1. Sort each factor by whether it raises or lowers a study's statistical power.

**Answer:**

- Raises power: A larger true effect size, A more lenient alpha (0.10 instead of 0.01), Tighter, less variable measurements
- Lowers power: A smaller sample, Noisier, more variable data

**Why:** Larger effects, a more lenient alpha, and tighter measurements raise power; smaller samples and noisier data lower it. Power is fundamentally about making a real signal stand out from random noise.

Page: https://tryals.app/practice/psychology-i/power-and-effect-size/sort-each-factor-by-whether-it-raises-or-lowers-a-studys-statistical

### 2. A meta-analysis pools twelve published studies and reports a solid overall effect. What is the first thing a careful reader should ask?

A. Whether the studies used identical methods, without which pooling is invalid
B. Whether the total sample exceeds a thousand participants
C. Whether the studies were published in the same journal
D. Whether the twelve are all the studies that were run

**Answer:** D. Whether the twelve are all the studies that were run

**Why:** Pooling narrows the interval around whatever the inputs point at, so a literature missing its null results yields a confident wrong answer. That is why competent meta-analyses test for publication bias with funnel plots before reporting anything, the file drawer is the first threat, not the last.

Page: https://tryals.app/practice/psychology-i/power-and-effect-size/a-meta-analysis-pools-twelve-published-studies-and-reports-a-solid

### 3. A study with 50,000 participants reports a statistically significant difference with Cohen's d = 0.02. Why is that result significant and unimportant at the same time?

A. Because significance with a large sample is meaningless, so the p-value should be disregarded entirely.
B. Because a large enough sample distinguishes almost anything from zero.
C. Because a d of 0.02 shows the study contained an error, since real effects cannot be that small.
D. Because Cohen's d is unreliable for samples above a few thousand, so the effect size figure cannot be trusted.

**Answer:** B. Because a large enough sample distinguishes almost anything from zero.

**Why:** Detectability improves without limit as the sample grows, so with 50,000 people an effect of no consequence clears the threshold comfortably. Effect size asks how large, in units independent of n, and d = 0.02 means the distributions are almost entirely superimposed. The p-value is not meaningless; it is telling the truth about a question of limited interest.

Page: https://tryals.app/practice/psychology-i/power-and-effect-size/a-study-with-50-000-participants-reports-a-statistically-significant