# Advanced Methodologies

Psychology I · Statistics · https://tryals.app/learn/psychology-i/advanced-methodologies

## Synthesizing Evidence

When many studies address the same question, two methods synthesize the evidence. A **systematic review** comprehensively identifies, evaluates, and summarizes all available research using an explicit process to avoid cherry-picking.

A **meta-analysis** goes a step further, statistically combining numerical effect sizes into one precise overall estimate, pooling samples to increase statistical power.

| Method | Core Function |
| --- | --- |
| **Systematic review** | Summarizes existing research |
| **Meta-analysis** | Combines numerical results |

Both protect against bias and turn scattered findings into clear conclusions.

## Tracking Change Over Time

Studies tracking change use two contrasting designs. A **longitudinal design** follows the same individuals over time, directly measuring real change within each person.

A **cross-sectional design** compares different age groups at one point in time. It is faster and cheaper, but vulnerable to **cohort effects**: generational differences in life experience that masquerade as true age changes.

| Design | Tracks | Strength | Weakness |
| --- | --- | --- | --- |
| **Longitudinal** | Same group | Measures real change | Slow, costly, attrition |
| **Cross-sectional** | Different groups | Fast, cheap | Cohort effects |

> **Pitfall:** Treating a cross-sectional age difference as proof of aging. It often just reflects different historical eras.

## Practice questions

6 of this lesson's 11 practice questions, with answers. The full set is in the app.

### 1. Order the steps of conducting a meta-analysis.

**Answer:**

1. Define a precise research question.
2. Systematically search for all relevant studies using pre-set criteria.
3. Extract each study's numerical result, such as its effect size.
4. Statistically combine the effect sizes into one pooled estimate.
5. Interpret the overall, more precise combined estimate.

**Why:** Define the question, search comprehensively, extract each study's effect size, pool them statistically, and interpret the combined estimate. The systematic search up front is what prevents cherry-picking supportive studies.

Page: https://tryals.app/practice/psychology-i/advanced-methodologies/order-the-steps-of-conducting-a-meta-analysis

### 2. A cross-sectional finding that 70-year-olds score lower than 30-year-olds shows that reasoning declines with age.

**Answer:** False

**Why:** False. The groups differ in birth cohort as well as age — in schooling, nutrition and test familiarity — so the design confounds ageing with cohort. Longitudinal studies following the same people show far smaller declines, which is why the two designs disagree so consistently on this question.

Page: https://tryals.app/practice/psychology-i/advanced-methodologies/a-cross-sectional-finding-that-70-year-olds-score-lower-than

### 3. A longitudinal study follows the same people for thirty years; a cross-sectional study compares different age groups on one afternoon. Each design's main strength is the other's main weakness. What is the trade?

A. Longitudinal designs are always more valid, and cross-sectional designs are used only when funding is short.
B. Longitudinal holds the person constant; cross-sectional confounds cohort.
C. Cross-sectional designs measure change over time directly, while longitudinal designs only infer it.
D. Longitudinal designs avoid attrition because participants become committed to the study over time.

**Answer:** B. Longitudinal holds the person constant; cross-sectional confounds cohort.

**Why:** A longitudinal study compares each person with themselves, so generation cancels, it measures change rather than difference. The cost is not merely time: attrition is selective toward poorer health, so the surviving sample looks better than the cohort it started as, which can hide the very decline the study was built to detect.

Page: https://tryals.app/practice/psychology-i/advanced-methodologies/a-longitudinal-study-follows-the-same-people-for-thirty-years-a

### 4. Sort each strength or weakness under the design it describes.

**Answer:**

- Longitudinal: Directly measures real change within a person, Threatened by attrition over the years, Time-consuming and expensive
- Cross-sectional: Fast and cheap to run, Threatened by cohort effects

**Why:** Longitudinal directly measures change but is slow, costly, and prone to attrition; cross-sectional is fast and cheap but confounded by cohort effects. The trade-off is precision-about-change versus speed.

Page: https://tryals.app/practice/psychology-i/advanced-methodologies/sort-each-strength-or-weakness-under-the-design-it-describes

### 5. A cross-sectional study finds 70-year-olds score lower on reasoning than 30-year-olds. Why does the Flynn effect make this comparison nearly impossible to interpret?

A. Because the Flynn effect applies only to children, not to older adults
B. Because the Flynn effect means older adults were tested with different instruments
C. It does not affect it, the Flynn effect concerns absolute scores, not comparisons
D. Because scores rose about three points a decade

**Answer:** D. Because scores rose about three points a decade

**Why:** Age and birth cohort are perfectly confounded, and the Flynn effect puts a number on the confound: roughly twelve points of the gap is generational before ageing contributes anything. Cross-sectional designs therefore overstate cognitive decline systematically, which is why longitudinal and sequential designs exist.

Page: https://tryals.app/practice/psychology-i/advanced-methodologies/a-cross-sectional-study-finds-70-year-olds-score-lower-on-reasoning

### 6. A meta-analysis pools ten studies of 40 participants each and reports a precise overall effect. Why does pooling not automatically make the conclusion more trustworthy than the individual studies?

A. Because meta-analyses average effect sizes, and averaging always loses information.
B. Because the studies were conducted by different teams, so their results cannot legitimately be combined.
C. Because it inherits whatever is wrong with its inputs.
D. Because 400 participants is still too few for a meta-analysis to be informative.

**Answer:** C. Because it inherits whatever is wrong with its inputs.

**Why:** Pooling improves precision, which is not accuracy, narrowing an interval helps only if the estimate is aimed at the right place. If the ten available studies are the significant survivors of thirty, pooling locates the literature's inflated estimate with impressive confidence. The specific danger is that the garbage comes out looking more authoritative than it went in.

Page: https://tryals.app/practice/psychology-i/advanced-methodologies/a-meta-analysis-pools-ten-studies-of-40-participants-each-and-reports