# Solving Linear Systems Numerically

Mathematics I · Scientific Programming · https://tryals.app/learn/mathematics-i/solving-linear-systems-numerically

## Linear Systems & Direct Methods

Solving $A\mathbf{x} = \mathbf{b}$ powers simulations and data fitting with millions of unknowns. Theory uses **Gaussian elimination**, reducing $A$ to triangular form before back-substitution.

The cost is about $\frac{2}{3}n^3$ operations for an $n \times n$ system. Because it is cubic, doubling $n$ multiplies the work eightfold. For huge sparse systems, **iterative methods** (Jacobi, Gauss-Seidel, conjugate gradient) refine guesses step-by-step instead.

Plain elimination fails on a zero pivot and amplifies round-off error on a small one. **Partial pivoting** swaps rows to place the largest available entry on the pivot, making the method **numerically stable** and safe for real-world use.

## Conditioning & Pitfalls

Even stable solvers fail if a system is inherently fragile. The **condition number** of $A$ measures solution sensitivity to input errors.

| System Type | Condition Number | Geometric Meaning | Robustness |
| --- | --- | --- | --- |
| **Well-conditioned** | Small | Nearly perpendicular | Robust to errors |
| **Ill-conditioned** | Large | Nearly parallel lines | Swings wildly |

> **Common pitfall:** Blaming the **algorithm** when the **problem is ill-conditioned**. A large condition number means the solution is hyper-sensitive. Even a perfect solver cannot rescue imprecise data; error lives in the problem, not the method.

## Practice questions

7 of this lesson's 11 practice questions, with answers. The full set is in the app.

### 1. An ill-conditioned linear system can give wildly wrong answers even when the algorithm is flawless and the arithmetic exact to 16 digits. Why is that a property of the problem rather than a failure of the method?

A. Because ill-conditioning only occurs when the matrix is singular, in which case the system is unsolvable.
B. Because ill-conditioned systems have no solution, so any answer produced is arbitrary.
C. Because the algorithm must be accumulating rounding errors, which is what ill-conditioning means.
D. Because conditioning measures how much the solution moves.

**Answer:** D. Because conditioning measures how much the solution moves.

**Why:** Two nearly parallel lines cross at a well-defined point that a hair's nudge sends a long way, so if the coefficients were measured or rounded the answer is simply not determined to the precision you wanted. Conditioning is a property of the problem and stability of the method, a stable algorithm on an ill-conditioned problem gives the answer the perturbed problem deserved.

Page: https://tryals.app/practice/mathematics-i/solving-linear-systems-numerically/an-ill-conditioned-linear-system-can-give-wildly-wrong-answers-even

### 2. A standard bound says the relative error in the solution can be as large as the condition number times the relative error in the data. With a condition number of 500 and data whose relative error is 0.0002, what is the largest relative error the solution may carry?

**Answer:** 0.1 (within ±0.001)

**Why:** 500 * 0.0002 = 0.1, a 10% possible error from data accurate to two parts in ten thousand. The condition number is exactly this amplification factor, which is why it, and not the solver, sets the ceiling on achievable accuracy.

Page: https://tryals.app/practice/mathematics-i/solving-linear-systems-numerically/a-standard-bound-says-the-relative-error-in-the-solution-can-be-as

### 3. Complete the description of partial pivoting.

**Answer:** Partial pivoting swaps rows so that the pivot is **the largest available entry in the column**.

**Why:** Choosing the largest available entry keeps every multiplier at most 1 in size, so round-off is not magnified. Taking the first nonzero entry instead is exactly the self-inflicted instability the lesson warns about.

Page: https://tryals.app/practice/mathematics-i/solving-linear-systems-numerically/complete-the-description-of-partial-pivoting

### 4. A condition number of 10⁸ is reported for a linear system solved in double precision, which carries about 16 digits. What should you conclude about the answer?

A. That the answer is wrong by a factor of 10⁸
B. That the system has no solution and the computation should be abandoned
C. That roughly half its digits may be meaningless
D. That the algorithm is unstable and should be replaced

**Answer:** C. That roughly half its digits may be meaningless

**Why:** Multiply 10⁻¹⁶ by 10⁸ and you have relative error around 10⁻⁸, about eight trustworthy digits. It describes the problem rather than the method, so a better algorithm will not recover the rest; what it buys you is knowing how much of the printout to believe.

Page: https://tryals.app/practice/mathematics-i/solving-linear-systems-numerically/a-condition-number-of-10-is-reported-for-a-linear-system-solved-in

### 5. Sort each 2 by 2 system by its conditioning, judging from how nearly parallel its two equations are.

**Answer:**

- Well-conditioned: x + y = 2 and x - y = 0, x = 1 and y = 1
- Ill-conditioned: x + y = 2 and x + 1.0001y = 2.0001, 2x + 3y = 5 and 4x + 6.0001y = 10

**Why:** In s1 and s3 the second equation is nearly a multiple of the first, so the lines are nearly parallel and their crossing point slides enormously under tiny data changes. s2 and s4 are perpendicular, so their intersection is pinned down firmly.

Page: https://tryals.app/practice/mathematics-i/solving-linear-systems-numerically/sort-each-2-by-2-system-by-its-conditioning-judging-from-how-nearly

### 6. Select every statement about solving Ax = b numerically that is TRUE.

A. Doubling n multiplies the direct method's work by about 8
B. Pivoting avoids dividing by a zero or tiny pivot
C. An ill-conditioned system can be rescued by a more stable algorithm
D. A large condition number means the solution is very sensitive to input error
E. Iterative methods reach the exact answer in a fixed finite number of steps

**Answer:** A. Doubling n multiplies the direct method's work by about 8; B. Pivoting avoids dividing by a zero or tiny pivot; D. A large condition number means the solution is very sensitive to input error

**Why:** Cubic cost, the purpose of pivoting and the meaning of the condition number are all exactly as taught. Iterative methods approach the answer rather than terminating at it, and conditioning is a defect of the problem that no algorithm can undo.

Page: https://tryals.app/practice/mathematics-i/solving-linear-systems-numerically/select-every-statement-about-solving-ax-b-numerically-that-is-true

### 7. Order the steps of solving a system by Gaussian elimination with partial pivoting.

**Answer:**

1. In the current column, find the largest entry and swap its row into the pivot position
2. Eliminate the entries below the pivot
3. Move to the next column and repeat
4. Arrive at an upper-triangular system
5. Back-substitute from the last equation upward

**Why:** Pivot selection must precede elimination, since it exists to make the division safe. Only once the whole matrix is triangular can back-substitution begin, the two phases are strictly sequential.

Page: https://tryals.app/practice/mathematics-i/solving-linear-systems-numerically/order-the-steps-of-solving-a-system-by-gaussian-elimination-with