Quasi-Newton method

From Glossary

Jump to: navigation, search

One designed to capture the essence of the modified Newton's method, such as the Broyden family. The general idea is to build up an approximation to the inverse hessian to get an order of convergence between 1 and 2. Letting LaTeX: H_k denote the inverse hessian estimate (where the direction is chosen by the deflection of the gradient, LaTeX: d_k = \pm H_k \nabla f(x_k), H_{k+1} is generally obtained from LaTeX: H_k as in the BFGS update. A quasi-Newton method is memoryless when LaTeX: H_{k+1} is obtained by the same formula as a quasi-Newton method, but with LaTeX: H_k replaced by some fixed matrix, like the identity. A partial quasi-Newton method follows the udate rules for some (specified) number of iterations, then restarts with LaTeX: H_0 (usually = identity matrix).

Personal tools