# Hypothesis Testing and P-Values

Psychology I · Statistics · https://tryals.app/learn/psychology-i/hypothesis-testing-and-p-values

## The Test Statistic & P-Values

A **test statistic** (like a t or z-statistic) converts your data into a single standardized number measuring distance from what the null hypothesis predicts. It is compared against a **critical value**—the exact threshold it must exceed to reach statistical significance.

Depending on your research question, tests split into two distinct structures:

| Test type | Predicts | Rejection region |
| --- | --- | --- |
| **One-tailed** | Directional effect | Entirely in one tail |
| **Two-tailed** | Non-directional effect | Split between both tails |

A two-tailed test is more conservative because it splits the alpha level, requiring a more extreme result in a single direction to achieve significance.

## Confidence Intervals

An alternative to the p-value is the **confidence interval (CI)**: a range of plausible values for the true population effect computed from sample data. CIs show both significance and the plausible **size** and **precision** of an effect.

A **95% confidence interval** means that if the exact study is repeated many times, about 95% of the resulting intervals will contain the true population value.

> **Common Pitfall:** Never say "there is a 95% probability the true value falls within THIS specific interval." The 95% refers to the **long-run process** across many repeated samples, not a single computed interval.

## Practice questions

4 of this lesson's 11 practice questions, with answers. The full set is in the app.

### 1. Put the steps of a significance test in order.

**Answer:**

1. State the null and alternative hypotheses.
2. Choose an alpha level, such as 0.05.
3. Compute a test statistic from the sample data.
4. Compare the test statistic to the critical value (or the p-value to alpha).
5. Reject or fail to reject the null hypothesis.

**Why:** Hypotheses and alpha are set in advance, then the data yield a test statistic, which is compared to the threshold to reach a verdict. Choosing alpha before seeing the result is what keeps the false-positive rate honest.

Page: https://tryals.app/practice/psychology-i/hypothesis-testing-and-p-values/put-the-steps-of-a-significance-test-in-order

### 2. Select every statement that is TRUE about hypothesis testing.

A. A p-value is the probability that the null hypothesis is true.
B. A confidence interval that excludes the null value indicates a significant result.
C. Lowering alpha from 0.05 to 0.01 makes it easier to reject the null.
D. A two-tailed test is more conservative than a one-tailed test at the same alpha.
E. A Type I error is rejecting a null hypothesis that is actually true.

**Answer:** B. A confidence interval that excludes the null value indicates a significant result.; D. A two-tailed test is more conservative than a one-tailed test at the same alpha.; E. A Type I error is rejecting a null hypothesis that is actually true.

**Why:** Type I = rejecting a true null, two-tailed is more conservative, and a CI excluding the null means significance - all true. A p-value is not the probability the null is true, and a smaller alpha raises the bar for rejection, making it harder.

Page: https://tryals.app/practice/psychology-i/hypothesis-testing-and-p-values/select-every-statement-that-is-true-about-hypothesis-testing

### 3. Sort each research hypothesis by whether it calls for a one-tailed or a two-tailed test.

**Answer:**

- One-tailed test: The new drug will REDUCE anxiety, Group A will score HIGHER than group B
- Two-tailed test: The new drug will CHANGE anxiety, in some direction, The two groups will DIFFER, we do not predict which way

**Why:** Directional predictions ('reduce', 'higher') place the whole rejection region in one tail; non-directional predictions ('change', 'differ') split it across both tails. The two-tailed test is more conservative because it demands a more extreme result in the relevant direction.

Page: https://tryals.app/practice/psychology-i/hypothesis-testing-and-p-values/sort-each-research-hypothesis-by-whether-it-calls-for-a-one-tailed-or

### 4. Lowering alpha from 0.05 to 0.01 reduces false positives. Why is that not simply an improvement?

A. Because the two errors trade against each other
B. Because alpha does not affect the false positive rate, only the reported precision
C. Because 0.01 is too strict to ever reach significance in psychology
D. Because lowering alpha reduces the sample size required, weakening the study

**Answer:** A. Because the two errors trade against each other

**Why:** You are moving a line, and both kinds of error sit on it. Stricter alpha buys fewer false alarms and misses more real effects, which is right for a drug approval and wrong for a screening test where a miss is fatal. There is no setting that is simply correct.

Page: https://tryals.app/practice/psychology-i/hypothesis-testing-and-p-values/lowering-alpha-from-0-05-to-0-01-reduces-false-positives-why-is-that