Representing the Real World
Scientific computing relies on floating-point numbers to approximate real numbers within finite memory. Stored via the IEEE 754 standard, each value uses a sign, a mantissa for significant digits, and an exponent for scale.
Because storage is finite, most real numbers fall between representable values and are rounded to the nearest tick, causing round-off error. Double precision grants roughly 15–16 significant digits, meaning anything beyond that is noise.
Arithmetic can also trigger infinity from overflow or division by zero, and NaN ("not a number") from invalid operations like , both of which propagate through code.
The Rules of Floating-Point
Numerical computing requires managing fixed precision and gaps instead of expecting exact math. Common issues include:
| Problem | Cause & Consequence | Solution |
|---|---|---|
| Exact Equality | 0.1 + 0.2 != 0.3 due to rounding bits. | Compare using tolerance: . |
| Limited Precision | Only 15–16 significant digits survive. | Track accumulated error limits. |
| Special Values | Infinity and NaN propagate through math. | Check inputs and handle edge cases. |
Common Pitfall: Testing floats with ==. Always use a tolerance () to avoid failing due to tiny rounding differences.