Practice question · Multiple choice
Two researchers independently code 40 transcript segments and agree on 34. Why is the resulting 85% agreement figure a weaker guarantee than it appears?
Hints
- Suppose there were only two codes and both researchers assigned them by coin flip. What agreement would you expect?
- The figure has a floor built into it. What is it?
Show the answer
D. Because some agreement would occur by chance alone.
Why
Raw agreement has a floor that has nothing to do with skill: two coders assigning at random over two categories agree about half the time, so 85% is 85 against a baseline of 50. Kappa asks how much of the room above chance was actually captured, which is why the same 85% can be excellent or mediocre depending on the coding scheme's structure.
Practise Qualitative Data Analysis
The app has 4 more questions on this lesson, and keeps your place in the course. Psychology I is free to start.
More questions on Qualitative Data Analysis
- Inductive coding builds categories up from the data; deductive coding applies a framework decided in advance.…
- Put the stages of thematic analysis (Braun and Clarke) in order.
- Sort each activity under the coding approach it belongs to.
- Match each concrete analyst action to the concept it illustrates.
- Two coders agree on 90% of segments, but nearly every segment belongs to one dominant category. Why does the…
- Select every statement that is TRUE about qualitative data analysis.