Numbers That Aren't Quite Exact
Scientific computing runs on floating-point numbers — the computer's approximation of the real numbers. Because a machine has finite memory, it cannot store infinitely many digits, so most reals are stored approximately. Understanding this is the foundation of trustworthy numerical work.
A floating-point number is stored like scientific notation: a sign, a mantissa (significant digits), and an exponent (the scale), typically in the IEEE 754 standard (32-bit single or 64-bit double precision). This gives a fixed number of significant digits — about 15–16 for a double — and a huge range of magnitudes. But between representable values there are gaps: most real numbers fall between two floats and get rounded to the nearer one, introducing round-off error.
The consequences shape all numerical work:
- No exact equality.
0.1 + 0.2is not exactly0.3— each is rounded, and the tiny errors do not cancel. Never test floats with==; compare within a tolerance . - Limited precision. Only about 15–16 significant digits survive; beyond that is noise.
- Special values. Operations can produce infinity (overflow, or ) or NaN ("not a number," e.g. ), which propagate through further arithmetic.
Floating-point is not a flaw to be fixed but a reality to be managed. Every serious numerical result carries some error; the art of scientific computing is keeping that error small and knowing how large it might be.
Common pitfall: testing floating-point numbers for exact equality with==. Because reals are stored approximately and rounded, computations that "should" give the same value often differ in the last bits —0.1 + 0.2 != 0.3exactly. Always compare floats within a tolerance (), never with strict equality, or a correct computation will register as unequal.
A zoomed number line with discrete floating-point ticks and gaps between them; an accent real value falls in a gap and snaps to the nearest tick, the small remaining distance labeled the round-off error.