Sources and Trade-offs
Every numerical computation carries error. Two fundamental kinds arise:
- Round-off error: Inherent in floating-point arithmetic from representing numbers with finite precision.
- Truncation error: Error of the method from approximating an infinite process by a finite one, such as stopping a Taylor series or using finite steps.
These often trade off. Making a step size smaller reduces truncation error but can increase round-off error by subtracting nearly equal numbers.
| Error Type | Origin | Example |
|---|---|---|
| Round-off | Finite precision | Floating-point limits |
| Truncation | Finite approximation | Stopping a series early |
Common pitfall: Assuming a smaller step size always improves accuracy. Shrinking eventually increases round-off error, so accuracy peaks at an optimal step size.
Stability and Propagation
Errors propagate through computations. Two major dangers include catastrophic cancellation, where subtracting nearly equal numbers destroys significant digits, and error amplification from unstable calculations.
- Stable algorithm: Small errors stay small and do not blow up.
- Unstable algorithm: Amplifies small errors into wildly wrong answers.
- Conditioning: Describes the problem itself. A well-conditioned problem is insensitive to input changes, while an ill-conditioned problem is sensitive no matter the algorithm.
Two mathematically equivalent formulas can differ enormously in stability; choosing the stable one is a core skill.