# Linear Independence and Basis

Computer Science I · Algebra · https://tryals.app/learn/computer-science-i/linear-independence-and-basis

## Spanning and Independence

A **linear combination** is any sum $c_1 v_1 + \dots + c_k v_k$. Their **span** is the set of all such combinations—the entire region they can reach.

Vectors are **linearly independent** if no vector is a combination of the others. Equivalently, the only solution to $c_1 v_1 + \dots + c_k v_k = \mathbf{0}$ is all $c_i = 0$. Dependent means redundancy.

| Concept | Meaning | Pitfall / Edge Case |
| --- | --- | --- |
| **Spanning** | Reaches the whole space | May contain redundant vectors |
| **Independence** | Zero redundancy | May fail to reach the whole space |

## Basis and Dimension

A **basis** for a vector space is a set of vectors that is **linearly independent** AND **spanning**. It is a minimal spanning set with no wasted vectors.

The number of vectors in a basis is the **dimension** of the space. Every vector has a **unique expression** in a given basis, which forms the foundation of coordinates.

> **Common pitfall:** Confusing spanning with independence. A basis requires **both**. A spanning set with extra vectors is not a basis, nor is an independent set too small to span.

## Practice questions

5 of this lesson's 9 practice questions, with answers. The full set is in the app.

### 1. A model trained on 50 features has coefficients that swing wildly between refits while its predictions stay stable. What does that pattern indicate?

A. That the optimiser has not converged and needs more iterations
B. That the training data is too small, which affects predictions equally
C. That the model is underfitting and needs more features
D. Near-dependence among features, so many fits are almost equal

**Answer:** D. Near-dependence among features, so many fits are almost equal

**Why:** Multicollinearity: many weightings produce nearly the same predictions, so the fit is stable and the coefficients are arbitrary. The distinctive signature is exactly this split — reliable predictions, unreliable interpretation — and it is why regularisation exists to pick one solution from the near-degenerate family.

Page: https://tryals.app/practice/computer-science-i/linear-independence-and-basis/a-model-trained-on-50-features-has-coefficients-that-swing-wildly

### 2. A dataset with 50 correlated features can be well approximated using far fewer dimensions.

**Answer:** True

**Why:** True. Correlated features are partly redundant, so the data lies near a lower-dimensional subspace and a few principal components capture most of the variance. That is what PCA finds, and it is why 'number of columns' and 'intrinsic dimension' are different numbers.

Page: https://tryals.app/practice/computer-science-i/linear-independence-and-basis/a-dataset-with-50-correlated-features-can-be-well-approximated-using

### 3. Order the steps to test whether two plane vectors u and v are linearly independent.

**Answer:**

1. Ask whether one vector is a scalar multiple of the other
2. If yes, they are dependent
3. If no, they point in genuinely different directions
4. Conclude they are independent and form a basis for the plane

**Why:** For two vectors, independence is exactly 'neither is a multiple of the other'. Two independent vectors in R^2 also span the plane, so they form a basis.

Page: https://tryals.app/practice/computer-science-i/linear-independence-and-basis/order-the-steps-to-test-whether-two-plane-vectors-u-and-v-are

### 4. In the plane R^2, sort each set of vectors by what it does.

**Answer:**

- Spans the whole plane: (1, 0) and (0, 1)
- Spans only a line: (2, 2) alone, (3, 0) alone
- More than a basis (redundant): (1, 0), (0, 1) and (1, 1)

**Why:** Two independent vectors (s1) span the plane and form a basis. One vector (s2, s4) spans only the line through it. Three vectors in a 2D space (s3) cannot all be independent, one is redundant.

Page: https://tryals.app/practice/computer-science-i/linear-independence-and-basis/in-the-plane-r-2-sort-each-set-of-vectors-by-what-it-does

### 5. Select every set of vectors that forms a BASIS for the plane R^2.

A. (1, 0) and (0, 1)
B. (0, 1) and (2, 0)
C. (1, 0) alone
D. (1, 2) and (2, 4)
E. (1, 1) and (2, 1)

**Answer:** A. (1, 0) and (0, 1); B. (0, 1) and (2, 0); E. (1, 1) and (2, 1)

**Why:** A basis for R^2 is exactly two independent vectors. (1,0)&(0,1), (1,1)&(2,1), and (0,1)&(2,0) each qualify. (1,2)&(2,4) are dependent, and a single vector cannot span the plane.

Page: https://tryals.app/practice/computer-science-i/linear-independence-and-basis/select-every-set-of-vectors-that-forms-a-basis-for-the-plane-r-2