# Quasi-Newton method

One designed to capture the essence of the modified Newton's method, such as the Broyden family. The general idea is to build up an approximation to the inverse hessian to get an order of convergence between 1 and 2. Letting $LaTeX: H_k$ denote the inverse hessian estimate (where the direction is chosen by the deflection of the gradient, $LaTeX: d_k = \pm H_k \nabla f(x_k), H_{k+1}$ is generally obtained from $LaTeX: H_k$ as in the BFGS update. A quasi-Newton method is memoryless when $LaTeX: H_{k+1}$ is obtained by the same formula as a quasi-Newton method, but with $LaTeX: H_k$ replaced by some fixed matrix, like the identity. A partial quasi-Newton method follows the udate rules for some (specified) number of iterations, then restarts with $LaTeX: H_0$ (usually = identity matrix).