Variable metric method

From Glossary

Jump to: navigation, search

Originally referred to the Davidon-Fletcher-Powell (DFP) method, this is a family of methods that choose the direction vector in unconstrained optimization by the subproblem: LaTeX: \textstyle d^* \in \arg\max\{\nabla f(x)*d: ||d|| = 1\}, where LaTeX: \textstyle ||d|| is the vector norm (or metric) defined by the quadratic form, LaTeX: \textstyle d'Hd. With LaTeX: H symmetric and positive definite, the constraint LaTeX: \textstyle d'Hd = 1 restricts d by being on a "circle" -- points that are "equidistant" from a stationary point, called the "center" (the origin in this case). By varying LaTeX: H, as in the DFP update, to capture the curvature of the objective function, LaTeX: f, we have a family of ascent algorithms. Besides DFP, if one chooses LaTeX: H=I, we have Cauchy's steepest ascent. If LaTeX: f is concave and one chooses LaTeX: H equal to the negative of the inverse hessian, we have the modified Newton's method.

Personal tools