Gradient descent steps in the direction of the derivative of the loss.

False

False. The gradient points uphill, toward increasing loss, so the step is taken in the opposite direction, which is where the minus sign in $\theta \leftarrow \theta - \eta \nabla L$ comes from. Stepping along the gradient is gradient ascent, which is what you want when maximising a likelihood.