Levenberg Marquardt algorithm

$LaTeX: (\nabla^2 f(x) + p I)d = -\nabla f(x),$
where $LaTeX: \nabla^2 f(x)$ is the Hessian. For unconstrained optimization, the solution $LaTeX: d$ serves as a direction vector for the iteration. This is also used in a trust region method. The parameter $LaTeX: p$ is set to give a balance between Newton's method $LaTeX: (p=0)$ and Cauchy's steepest descent $LaTeX: (p >> 0)$. A low value of $LaTeX: p$ helps get through difficult landscape curvature, and a high value yields some descent.