# Inner Product Spaces and Orthogonality

Mathematics I · Linear Algebra · https://tryals.app/learn/mathematics-i/inner-product-spaces-and-orthogonality

## Abstract Geometry and Orthogonality

The dot product of Unit 3 gave $\mathbb{R}^n$ its geometry. An **inner product** $\langle u, v \rangle$ generalizes this to any vector space. It must be symmetric, linear, and **positive definite** ($\langle v, v \rangle > 0$ for $v \neq \mathbf{0}$). From it come the **norm** $\|v\| = \sqrt{\langle v, v \rangle}$ (length) and the **angle** via $\cos\theta = \frac{\langle u, v \rangle}{\|u\|\|v\|}$.

Two vectors are **orthogonal** when $\langle u, v \rangle = 0$. An **orthonormal basis** is orthogonal and every vector has length 1. In such a basis, coordinates are trivial to compute: the $i$-th coordinate of $v$ is $\langle v, e_i \rangle$.

| Concept | Definition | Key Property |
| --- | --- | --- |
| **Orthogonal Set** | $\langle u, v \rangle = 0$ for $u \neq v$ | Automatically linearly independent |
| **Positive Definite** | $\langle v, v \rangle > 0$ for $v \neq \mathbf{0}$ | Ensures valid vector lengths |

## Projections and Gram-Schmidt

**Orthogonal projection** finds the closest point in a subspace $W$ to a given vector. The error $v - \operatorname{proj}_W v$ is **orthogonal** to all of $W$. This drives **least-squares approximation**: when $A\mathbf{x} = \mathbf{b}$ has no exact solution, projecting $\mathbf{b}$ onto the column space gives the best approximate fit.

The **Gram–Schmidt process** systematically turns any basis into an orthonormal one by subtracting components along chosen vectors.

> **Common pitfall:** Assuming an **orthogonal projection** is just truncating extra coordinates. Projection finds the **nearest** vector in $W$ via a geometric construction, and coordinates are trivial only in an **orthonormal** basis, not a general one.

## Practice questions

5 of this lesson's 11 practice questions, with answers. The full set is in the app.

### 1. Select every TRUE statement about inner products and orthogonality.

A. Two vectors are orthogonal exactly when their inner product is 0
B. The zero vector is orthogonal to every vector
C. A set of mutually orthogonal nonzero vectors is linearly independent
D. In an orthonormal basis, the i-th coordinate of v is the inner product of v with the i-th basis vector
E. Every orthogonal set is orthonormal

**Answer:** A. Two vectors are orthogonal exactly when their inner product is 0; B. The zero vector is orthogonal to every vector; C. A set of mutually orthogonal nonzero vectors is linearly independent; D. In an orthonormal basis, the i-th coordinate of v is the inner product of v with the i-th basis vector

**Why:** Orthogonality of nonzero vectors forces independence, and in an orthonormal basis coordinates need no system solving at all. But an orthogonal set becomes orthonormal only after each vector is divided by its own norm, that normalising step is exactly what Gram-Schmidt finishes with.

Page: https://tryals.app/practice/mathematics-i/inner-product-spaces-and-orthogonality/select-every-true-statement-about-inner-products-and-orthogonality

### 2. Order the steps of the Gram-Schmidt process applied to a basis v1, v2, v3.

**Answer:**

1. Take w1 = v1 as the first direction
2. Subtract from v2 its component along w1 to obtain w2
3. Subtract from v3 its components along both w1 and w2 to obtain w3
4. Divide each w by its own norm to obtain an orthonormal set

**Why:** Each step removes exactly the part of the new vector that the previous directions already reach, so what remains is orthogonal to all of them. Normalising last keeps the arithmetic clean; doing it early forces you to carry square roots through every subtraction.

Page: https://tryals.app/practice/mathematics-i/inner-product-spaces-and-orthogonality/order-the-steps-of-the-gram-schmidt-process-applied-to-a-basis-v1

### 3. Gram-Schmidt takes a perfectly good basis and replaces it with an orthonormal one. Why is that worth the computation, when the original basis already gives every vector unique coordinates?

A. Because each coordinate becomes a single inner product.
B. Because a general basis does not really give unique coordinates, and orthonormality is what makes them unique.
C. Because orthonormal bases can represent vectors that a general basis cannot reach.
D. Because Gram-Schmidt reduces the number of basis vectors needed, which lowers the dimension.

**Answer:** A. Because each coordinate becomes a single inner product.

**Why:** Both bases give unique coordinates and differ entirely in what it costs to get them: a general basis means solving a linear system for every vector, where an orthonormal one gives cᵢ = ⟨w, eᵢ⟩ because every other term is killed. That independence is what makes the idea far-reaching. Fourier series are this construction in a space of functions.

Page: https://tryals.app/practice/mathematics-i/inner-product-spaces-and-orthogonality/gram-schmidt-takes-a-perfectly-good-basis-and-replaces-it-with-an

### 4. Sort each pair of vectors by whether the two are orthogonal.

**Answer:**

- Orthogonal: (1, 2) and (2, -1), (1, 0, 1) and (0, 5, 0), (2, 1, -1) and (1, -1, 1), (1, -1, 2) and (2, 4, 1)
- Not orthogonal: (1, 1) and (1, 1), (3, 1) and (1, 3)

**Why:** The inner products are 0, 2, 0, 0, 6 and 0. Note the second pair: a nonzero vector is never orthogonal to itself, precisely because positive definiteness forces the inner product of a nonzero vector with itself to be positive.

Page: https://tryals.app/practice/mathematics-i/inner-product-spaces-and-orthogonality/sort-each-pair-of-vectors-by-whether-the-two-are-orthogonal

### 5. A search engine measures document similarity by the **angle** between vectors rather than the distance. Why is the angle the right choice?

A. Because the angle is always between 0 and 1, which distance is not
B. Because angles are faster to compute than distances
C. Because a long document has a larger vector on the same topic
D. Because distance is undefined in high-dimensional spaces

**Answer:** C. Because a long document has a larger vector on the same topic

**Why:** Doubling a document doubles its vector — same direction, twice the length — so distance says 'different' and the angle says 'identical', which is the answer anyone wants. Length here encodes verbosity rather than content, and cosine similarity is the decision to ignore it.

Page: https://tryals.app/practice/mathematics-i/inner-product-spaces-and-orthogonality/a-search-engine-measures-document-similarity-by-the-angle-between