Reliability vs. Validity
Two distinct standards determine whether a measure is trustworthy. Reliability is CONSISTENCY: does the measure give the same result under the same conditions?
Validity is ACCURACY: does the measure actually capture what it claims to capture? A bathroom scale reading 5 lbs too heavy is reliable (consistent) but not valid (accurate).
| Concept | Core Question | Example Check |
|---|---|---|
| Reliability | Is it consistent? | Test-retest, inter-rater |
| Construct | Measures the right concept? | Does an anxiety scale tap anxiety? |
| Internal | Study justifies causal claims? | Are confounds ruled out? |
| External | Findings generalize? | Beyond this sample and setting? |
Types and The Golden Rule
Test-retest reliability checks if people score similarly later. Inter-rater reliability checks if independent raters agree. Internal consistency (via Cronbach's alpha) checks if scale items correlate.
Common pitfall: Assuming a reliable measure must be valid. Consistency is necessary but NOT sufficient for accuracy; a measure can be unfailingly consistent while measuring the wrong thing.
Remember the dartboard: a tight cluster far from the bullseye is reliable, not valid.