# Numerical Errors and Stability

Mathematics I · Scientific Programming · https://tryals.app/learn/mathematics-i/numerical-errors-and-stability

## Sources and Trade-offs

Every numerical computation carries error. Two fundamental kinds arise:

- **Round-off error**: Inherent in floating-point arithmetic from representing numbers with finite precision.
- **Truncation error**: Error of the method from approximating an infinite process by a finite one, such as stopping a Taylor series or using finite steps.

These often trade off. Making a step size $h$ smaller reduces **truncation error** but can increase **round-off error** by subtracting nearly equal numbers.

| Error Type | Origin | Example |
| --- | --- | --- |
| **Round-off** | Finite precision | Floating-point limits |
| **Truncation** | Finite approximation | Stopping a series early |

**Common pitfall:** Assuming a smaller step size $h$ always improves accuracy. Shrinking $h$ eventually increases round-off error, so accuracy peaks at an optimal step size.

## Stability and Propagation

Errors propagate through computations. Two major dangers include **catastrophic cancellation**, where subtracting nearly equal numbers destroys significant digits, and **error amplification** from unstable calculations.

- **Stable algorithm**: Small errors stay small and do not blow up.
- **Unstable algorithm**: Amplifies small errors into wildly wrong answers.
- **Conditioning**: Describes the problem itself. A **well-conditioned** problem is insensitive to input changes, while an **ill-conditioned** problem is sensitive no matter the algorithm.

Two mathematically equivalent formulas can differ enormously in stability; choosing the stable one is a core skill.

## Practice questions

6 of this lesson's 11 practice questions, with answers. The full set is in the app.

### 1. Select every statement about cancellation and stability that is TRUE.

A. Conditioning describes the problem while stability describes the algorithm
B. Subtracting nearly equal numbers can leave mostly noise
C. Rewriting a formula to avoid such a subtraction can preserve accuracy
D. An unstable algorithm can always be fixed by using a smaller step size
E. Two algebraically identical formulas always have the same numerical accuracy

**Answer:** A. Conditioning describes the problem while stability describes the algorithm; B. Subtracting nearly equal numbers can leave mostly noise; C. Rewriting a formula to avoid such a subtraction can preserve accuracy

**Why:** Algebraically equal formulas can differ enormously in accuracy, that is precisely why rewriting helps, and instability is a defect of the method that no step size repairs. The other three statements are exactly as taught.

Page: https://tryals.app/practice/mathematics-i/numerical-errors-and-stability/select-every-statement-about-cancellation-and-stability-that-is-true

### 2. Order the steps of curing a formula that loses accuracy to cancellation.

**Answer:**

1. Identify where two nearly equal quantities are subtracted
2. Rewrite the expression algebraically to remove that subtraction
3. Check that the rewritten form is mathematically equivalent
4. Compare both forms on a case where the cancellation bites
5. Adopt the form whose error stays small

**Why:** Locate, rewrite, verify equivalence, test on a hard case, adopt. Skipping the equivalence check is how a 'more stable' formula quietly becomes a formula for a different quantity.

Page: https://tryals.app/practice/mathematics-i/numerical-errors-and-stability/order-the-steps-of-curing-a-formula-that-loses-accuracy-to

### 3. Sort each source of error by its kind.

**Answer:**

- Round-off error: Storing one third as a finite binary fraction, The last bits lost when two floats are added
- Truncation error: Stopping a Taylor series after four terms, Using 100 rectangles in place of a limit, Approximating a derivative by a difference quotient with h = 0.01

**Why:** Truncation error is the error of the METHOD, stopping an infinite process early, and is independent of the machine. Round-off is the error of the REPRESENTATION and would persist under a perfect method. Separating the two is what lets you attack each.

Page: https://tryals.app/practice/mathematics-i/numerical-errors-and-stability/sort-each-source-of-error-by-its-kind

### 4. Truncation error shrinks as the step size h decreases, while round-off error grows. Why does that leave an optimal h rather than "as small as possible"?

A. Because the total error is the sum of two terms moving oppositely.
B. Because round-off error is constant, so reducing h always helps until the computation becomes too slow.
C. Because truncation error stops decreasing below a threshold, after which further reduction is wasted.
D. Because very small values of h cannot be represented in floating point at all.

**Answer:** A. Because the total error is the sum of two terms moving oppositely.

**Why:** Truncation error falls with h and round-off grows like 1/h, because subtracting two nearly equal floats and dividing by a tiny h amplifies the representation error. The sum has a minimum — for a forward difference it sits near √(machine epsilon), about 10⁻⁸ — and pushing h to 10⁻¹⁵ makes the answer dramatically worse.

Page: https://tryals.app/practice/mathematics-i/numerical-errors-and-stability/truncation-error-shrinks-as-the-step-size-h-decreases-while

### 5. Complete the definition of numerical stability.

**Answer:** An algorithm is numerically stable when small errors **stay small** as the computation proceeds.

**Why:** Stability means errors do not get amplified, they stay small, they are never removed. An unstable algorithm magnifies them, so a computation on essentially exact input can still return a wildly wrong answer.

Page: https://tryals.app/practice/mathematics-i/numerical-errors-and-stability/complete-the-definition-of-numerical-stability

### 6. Subtracting two nearly equal numbers is called **catastrophic cancellation**. Why is it so much worse than other floating-point operations?

A. Because the leading digits cancel and the errors do not
B. Because it always produces exactly zero, losing all information
C. Because subtraction is implemented less accurately than addition
D. Because the result is close to zero, which floating point cannot represent

**Answer:** A. Because the leading digits cancel and the errors do not

**Why:** The absolute error is unchanged and the result is tiny, so the relative error explodes, accurate inputs give an answer that is mostly noise. It is why the quadratic formula is rearranged when b² ≫ 4ac, and why numerical analysts rewrite algebra that looks perfectly fine on paper.

Page: https://tryals.app/practice/mathematics-i/numerical-errors-and-stability/subtracting-two-nearly-equal-numbers-is-called-catastrophic