Practice question · Multiple choice
Two coders agree on 90% of segments, but nearly every segment belongs to one dominant category. Why does the raw agreement flatter them?
Hints
- Suppose 85% of segments are category A. What agreement would two coders reach by always guessing A?
- Ask what the raw figure is being measured against.
Show the answer
A. Because chance agreement is high when one category dominates
Why
The baseline moves with the category distribution: if 85% of segments are one label, blind agreement is already 85% and the coders have added five points. Cohen's kappa rescales against that floor, which is why raw percentage agreement is uninterpretable without knowing the scheme's balance.
Practise Qualitative Data Analysis
The app has 4 more questions on this lesson, and keeps your place in the course. Psychology I is free to start.
More questions on Qualitative Data Analysis
- Two researchers independently code 40 transcript segments and agree on 34. Why is the resulting 85% agreement…
- Inductive coding builds categories up from the data; deductive coding applies a framework decided in advance.…
- Put the stages of thematic analysis (Braun and Clarke) in order.
- Sort each activity under the coding approach it belongs to.
- Match each concrete analyst action to the concept it illustrates.
- Select every statement that is TRUE about qualitative data analysis.