Managing Inevitable Error
Every numerical computation carries error; the discipline is knowing its sources and keeping it controlled. Two fundamental kinds of error arise:
- Round-off error — from representing numbers with finite precision (Lesson 1). It is inherent in floating-point arithmetic.
- Truncation error — from approximating an infinite process by a finite one: stopping a Taylor series after a few terms, using a finite step size in a derivative, or a finite number of rectangles in an integral. It is the error of the method, independent of the machine.
These often trade off. Making a step size smaller reduces truncation error (the method becomes more accurate) but can increase round-off error (subtracting nearly equal numbers amplifies the finite-precision noise) — so there is an optimal step size, not "smaller is always better."
Errors also propagate through a computation, and some ways of computing are far worse than others. Two dangers:
- Catastrophic cancellation — subtracting two nearly equal numbers destroys significant digits, leaving mostly noise. Reformulating to avoid the subtraction preserves accuracy.
- Error amplification — an unstable calculation magnifies small input errors into huge output errors.
An algorithm is numerically stable if small errors (round-off, input noise) stay small through the computation — they do not blow up. An unstable algorithm amplifies them, so even exact-looking inputs can give a wildly wrong answer. Two mathematically equivalent formulas can differ enormously in stability; choosing the stable one is a core skill. A related idea, conditioning, describes the problem itself: a well-conditioned problem has an answer insensitive to input changes, while an ill-conditioned one is sensitive no matter how good the algorithm.
Common pitfall: assuming that a smaller step size always gives a more accurate answer. Shrinking reduces truncation error but eventually increases round-off error (from subtracting nearly equal quantities), so accuracy improves only up to an optimal step size and then degrades. "Smaller is always better" ignores the round-off half of the trade-off — and, more broadly, an unstable algorithm can amplify tiny errors regardless of step size.