Measuring Something That Moves
A social process is a patterned change over time. Studying one requires turning abstract concepts into measurable data. This process is called operationalisation.
Two properties of any measure must be kept distinct:
- Reliability: consistency across repeated measurements.
- Validity: whether the tool measures what it claims to measure.
A faulty scale can be reliable yet invalid. Sociological measures usually struggle with validity because concepts lack obvious countable units.
| Technique | Answers well | Answers badly |
|---|---|---|
| Probability survey | How common is X in a population | What X means to those who do it |
| In-depth interview | How people account for their conduct | How representative that account is |
| Ethnography | What actually happens | Population estimates |
| Documentary analysis | Change over long periods | Unrecorded questions |
| Experiment | Whether A causes B | Real-world generalisability |
Sampling and Indicators
Sample design matters more than raw sample size.
In 1936, the Literary Digest polled 2.4 million people (2.4 million returned ballots). It wrongly predicted victory for Alf Landon over Roosevelt. George Gallup polled 50 thousand and predicted correctly. A biased sample yields a precise estimate of the wrong quantity.
Watch for two methodological traps:
- ecological fallacy: inferring individual traits from aggregate data (Robinson, 1950).
- reactivity: subjects altering behaviour when observed, first noted at Western Electric.
Indicators track wider social change:
- Human Development Index: tracks income, health, and schooling (since 1990).
- Gini coefficient: summarises inequality from 0 to 1.
- at-risk-of-poverty: measures relative poverty rates.
Common pitfall: treating sample size as a guarantee of quality. Size reduces random error, but it never reduces systematic bias.