Practice question · Fill in the blanks
Complete the update rule of gradient descent.
To minimise a function, each step moves from the current point ______.
Word bank: along the gradient by a learning-rate amount · opposite the gradient by a learning-rate amount · perpendicular to the gradient · to wherever the gradient is largest
Hints
- The gradient is the steepest ascent direction, and you want the opposite of ascent.
- The learning rate scales the step, it does not choose its direction.
Show the answer
To minimise a function, each step moves from the current point opposite the gradient by a learning-rate amount.
Why
Descent negates the gradient and scales it by the learning rate. Moving perpendicular to the gradient would trace a contour and change nothing, and heading for the largest gradient would climb the steepest slope rather than descend it.
Practise Multivariable Calculus and Gradients
The app has 6 more questions on this lesson, and keeps your place in the course. Computer Science I is free to start.
More questions on Multivariable Calculus and Gradients
- Select every statement that is TRUE of the gradient of a function of several variables.
- For a function of several variables the gradient is a vector of partial derivatives. Why does that vector…
- A convergence study halves the step size and checks the error falls by the expected factor. Why is that a…
- Order one iteration of gradient descent.