Practice question · Put in order
Order one iteration of gradient descent.
- Negate the gradient to get the descent direction
- Evaluate the partial derivatives at the current point
- Scale that direction by the learning rate
- Add the result to the current point to get the next one
- Assemble them into the gradient vector
Hints
- Nothing can be negated or scaled before it has been computed.
- The learning rate is applied to a direction that has already been chosen.
Show the answer
- Evaluate the partial derivatives at the current point
- Assemble them into the gradient vector
- Negate the gradient to get the descent direction
- Scale that direction by the learning rate
- Add the result to the current point to get the next one
Why
Partials, gradient, negate, scale, move. Repeating this until the gradient is near zero is the whole of the algorithm, and because the gradient vanishes at a minimum, the algorithm's own stopping condition is the multivariable version of f'(x) = 0.
Practise Multivariable Calculus and Gradients
The app has 6 more questions on this lesson, and keeps your place in the course. Computer Science I is free to start.
More questions on Multivariable Calculus and Gradients
- Select every statement that is TRUE of the gradient of a function of several variables.
- For a function of several variables the gradient is a vector of partial derivatives. Why does that vector…
- A convergence study halves the step size and checks the error falls by the expected factor. Why is that a…
- Complete the update rule of gradient descent.