6
$\begingroup$

This is not a typical question with a deterministic answer. If this is not the right place, feel free to close it.

For the past one year I have been working on various kinds of inverse problem. Most of them are related to parameter estimation, ranging from small linear differential equation to equations of fluid dynamics. And consistently, I observed that it was easier to find a solution to the non linear problem(using same optimization method, let's say steepest descent). By easy I mean it requires lower number of iterations for the same decrease(very subjective). Although, it is nearly impossible to compare different cases exactly, this trend is surprising to me.

The question here is, did anyone else observe similar behaviour? Is there a qualitatively explanation for this behaviour?

This also raises a follow up question of whether we can quantify(from the initial jacobian may be?) the level of difficulty involve in finding a solution?(or estimate the order of magnitude of the number of optimizer steps)

Improve this question
$\endgroup$
3
  • 6
    $\begingroup$ In general, I would be surprised if a nonlinear inverse problem were easier than a linear inverse problem (for one thing, output least-squares for a linear problem is convex, which is usually not the case for nonlinear problems -- and convexity helps a lot). One thing to keep in mind is that for inverse problems, the functional decrease is a very poor measure for success -- what really counts is distance to the (unknown) true solution, and the functional value tells you very little about that. In a sense, you are comparing apples and oranges here. $\endgroup$
    – Christian Clason
    Aug 20, 2015 at 9:46
  • 3
    $\begingroup$ Also, for a nonlinear problem (without any global Lipschitz estimates), what really counts is the Jacobian at the minimizer, since you will leave a neighborhood of the starting point in a finite number of steps. (Actually, what you need to look at is the Hessian, since the Jacobian vanishes at a critical point.) $\endgroup$
    – Christian Clason
    Aug 20, 2015 at 9:50
  • $\begingroup$ If you use a non-linear optimizer on a linear inverse problem then you'll get the minimum in a single iteration so in that sense linear inverse problems are trivial compared to non-linear problems. $\endgroup$
    – stali
    Aug 20, 2015 at 18:11

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.