minimum_gradient_descent#

named_arrays.optimize.minimum_gradient_descent(function, guess, step_size=None, momentum=0, gradient=None, min_gradient=None, max_iterations=1000, callback=None)#

Find the local minimum of the given function using the gradient descent method.

Parameters:

  • function (Callable[[InputT], OutputT]) – The function to minimize.

  • guess (InputT) – An initial guess for the local minimum.

  • step_size (None | InputT) – The learning rate for the gradient descent algorithm. This should have the same units as x / gradient(x). If None (the default), this takes the value 0.01 * na.unit(x / gradient(x)).

  • momentum (float | OutputT) – The momentum constant, \(\beta\) for the gradient descent algorithm. Should be a dimensionless number between zero and one. Defaults to zero, which equivalent to vanilla gradient descent with no momentum.

  • gradient (None | Callable[[InputT], InputT]) – The gradient of function. If None (the default), the gradient is computed using named_arrays.jacobian().

  • min_gradient (None | InputT) – The convergence threshold for the local minimum. If the gradient is smaller than this value, this function will stop. This should have the same units as gradient(x). If None (the default), this takes the value 1e-10 * na.unit(gradient(x)).

  • max_iterations (int) – The maximum number of steps to take before raising an error.

  • callback (None | Callable[[int, InputT, OutputT, AbstractArray], None]) – Optional callback function that is called on every iteration as callback(i, x, f, converged), where i is the current iteration, x is the current guess, f is the current function value, and converged is an array storing the convergence state for every minimum being computed.

Return type:

InputT

Notes

This function uses the update rules described in Goh [2017],

(1)#\[z_{k + 1} = \beta z_k + \nabla f(x_k)\]
(2)#\[x_{k + 1} = x_k - \alpha z_k,\]

where \(x_k\) is the current guess for iteration \(k\), \(f\) is the objective function, \(\alpha\) is the learning rate, and \(\beta\) is the momentum constant.