Is there any clear classification between different iterative methods?
What is the difference between Newton-type
and Newton-like
iterative methods?
Computational Science Stack Exchange is a question and answer site for scientists using computers to solve scientific problems. It only takes a minute to sign up.
Sign up to join this communityIs there any clear classification between different iterative methods?
What is the difference between Newton-type
and Newton-like
iterative methods?
They're equivalent. Both imply some variation of the root-finding method by linearization.
fixed point
iteration are all Newton type? e.g. Newton-Raphson, quasi Newton, Gauss-Newton, Levenberg-Marquardt, and their other variants with globalizing strategies (line search and trust region)?
$\endgroup$
– LCFactorization
Feb 6 '14 at 15:25
A few specific examples:
Quasi-Newton methods avoid computing second derivatives by using an approximation to the Hessian which is updated at each iteration by a low rank (rank one or rank two) update. This makes factoring the Hessian (or equivalently keeping the inverse in product form) easy to do computationally. There are also limited memory Quasi-Newton methods that keep track of only the most recent rank one updates.
The Gauss-Newton method for minimization of sums of squares takes advantage of the sum of squares structure of the problem to get an approximation to the Hessian that only involves first derivatives (the second order term is dropped.)
The Levenberg-Marquardt method is a particular approach to stabilizing the Gauss-Newton iteration by regularizing the linear system that is solved in each iteration (either by adding a regularizing term in the equations or by using a trust region method.)
Numerical methods for least square problems
(1996 by the Society for Industrial and Applied Mathematics, ISBN:0-89871-360-9), they classify Newton-type
and Gauss-Newton-type
as different iterative methods (Chapter 9)
$\endgroup$
– LCFactorization
Feb 8 '14 at 13:54
Newton-type
orNewton-like
... $\endgroup$ – LCFactorization Feb 6 '14 at 15:27Newton-type
andGauss-Newton-type
as different methods. So I want to make sure whether Newton-type can be used to cover all these.. orNewton-like
would be more suitable $\endgroup$ – LCFactorization Feb 8 '14 at 14:00