Practice question · Multiple choice
In floating point, 0.1 + 0.2 == 0.3 is false. What does that reveal about how numbers are stored?
Hints
- Try writing 1/3 exactly in decimal. Now write 1/10 exactly in binary.
- The error is present before the addition.
Show the answer
D. That values are held in binary with finite precision
Why
Base 10 fractions are not generally base 2 fractions. This is why numerical code compares within a tolerance rather than for equality.
Practise Data representation and basic types
The app has 8 more questions on this lesson, and keeps your place in the course. Physics I is free to start.