# Multivariable Calculus and Gradients

Computer Science I · Calculus · https://tryals.app/learn/computer-science-i/multivariable-calculus-and-gradients

## Multivariable Calculus

Real problems depend on many variables; a machine-learning model's error depends on thousands of parameters. **Multivariable calculus** extends derivatives to functions of several inputs, and its central tool, the **gradient**, is the engine behind modern machine learning.

A **partial derivative** measures the rate of change with respect to _one_ variable while holding the others fixed. For $f(x, y)$, the partial $\frac{\partial f}{\partial x}$ is the ordinary derivative treating $y$ as a constant, and $\frac{\partial f}{\partial y}$ treats $x$ as constant. Each partial is a slope along one axis.

The **gradient** $\nabla f$ collects all partials into a vector: $\nabla f = (\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y})$. It **points in the direction of steepest increase** of $f$, and its magnitude is how steep that increase is. At a maximum or minimum, the gradient is the **zero vector**.

## The Gradient & Descent

| Concept | Definition & Rule |
| --- | --- |
| **Gradient** $\nabla f$ | Points uphill toward steepest increase |
| **Descent Step** | Moves **opposite** $\nabla f$ to minimise $f$ |
| **Zero Vector** | Reached at local extrema ($\nabla f = 0$) |
| **Learning Rate** | Size of the step taken each iteration |

This drives **gradient descent**, the workhorse optimisation algorithm: to minimise a function, repeatedly step in the direction _opposite_ the gradient by a **learning-rate** amount until the gradient is near zero.

> **Pitfall:** Differentiating with respect to the wrong variable, or forgetting that gradient _descent_ moves **opposite** the gradient vector because the gradient points uphill.

## Practice questions

5 of this lesson's 11 practice questions, with answers. The full set is in the app.

### 1. Select every statement that is TRUE of the gradient of a function of several variables.

A. At a smooth peak or trough, every partial derivative is zero at once.
B. The gradient points in the direction of steepest decrease.
C. The rate of change in a direction is the dot product of the gradient with it.
D. A partial derivative treats every other variable as a constant.
E. The gradient collects the partial derivatives into one vector.

**Answer:** A. At a smooth peak or trough, every partial derivative is zero at once.; B. The gradient points in the direction of steepest decrease.; D. A partial derivative treats every other variable as a constant.; E. The gradient collects the partial derivatives into one vector.

**Why:** The gradient collects the partials, points uphill, has a magnitude giving the steepness, and vanishes at an extremum. It points towards steepest increase, so a minimiser must step the opposite way, the reason the algorithm is called gradient descent.

Page: https://tryals.app/practice/computer-science-i/multivariable-calculus-and-gradients/select-every-statement-that-is-true-of-the-gradient-of-a-function-of

### 2. For a function of several variables the gradient is a vector of partial derivatives. Why does that vector point in the direction of steepest ascent rather than in some other direction?

A. Because the gradient always points toward the global maximum of the function.
B. Because each partial derivative points uphill, and their sum must therefore point uphill too.
C. Because the gradient is defined as the steepest direction, so the property is a definition.
D. Because a dot product is largest when the two vectors are aligned.

**Answer:** D. Because a dot product is largest when the two vectors are aligned.

**Why:** The rate of change in a unit direction u is ∇f · u, and a dot product is largest when the vectors align, so steepest ascent is along the gradient and steepest descent along its negative. The gradient is *defined* as the vector of partials; the steepest-ascent property is a theorem. It is purely local, which is why practitioners restart from several points.

Page: https://tryals.app/practice/computer-science-i/multivariable-calculus-and-gradients/for-a-function-of-several-variables-the-gradient-is-a-vector-of

### 3. A convergence study halves the step size and checks the error falls by the expected factor. Why is that a stronger check than comparing against a known answer?

A. Because it tests the method's order, not one output
B. Because known answers are usually wrong for realistic problems
C. Because halving the step eliminates the error entirely
D. Because it runs faster than comparing with an analytic solution

**Answer:** A. Because it tests the method's order, not one output

**Why:** The relationship between step size and error is predicted by theory independently of the answer, so a second-order method reducing error by 2 rather than 4 signals a bug even when every output looked plausible. Boundary conditions applied one cell off produce exactly that signature.

Page: https://tryals.app/practice/computer-science-i/multivariable-calculus-and-gradients/a-convergence-study-halves-the-step-size-and-checks-the-error-falls

### 4. Complete the update rule of gradient descent.

**Answer:** To minimise a function, each step moves from the current point **opposite the gradient by a learning-rate amount**.

**Why:** Descent negates the gradient and scales it by the learning rate. Moving perpendicular to the gradient would trace a contour and change nothing, and heading for the largest gradient would climb the steepest slope rather than descend it.

Page: https://tryals.app/practice/computer-science-i/multivariable-calculus-and-gradients/complete-the-update-rule-of-gradient-descent

### 5. Order one iteration of gradient descent.

**Answer:**

1. Evaluate the partial derivatives at the current point
2. Assemble them into the gradient vector
3. Negate the gradient to get the descent direction
4. Scale that direction by the learning rate
5. Add the result to the current point to get the next one

**Why:** Partials, gradient, negate, scale, move. Repeating this until the gradient is near zero is the whole of the algorithm, and because the gradient vanishes at a minimum, the algorithm's own stopping condition is the multivariable version of f'(x) = 0.

Page: https://tryals.app/practice/computer-science-i/multivariable-calculus-and-gradients/order-one-iteration-of-gradient-descent