# scaling and preconditioning for trust region Newton methods

Geometrically, scaling and preconditioning seem to address similar challenges in optimization. However, these two concepts are implemented very differently. Take trust region Newton method, as an example. When a problem is poorly scaled, an elliptical trust region is recommended. Is it possible to formulate an equivalent preconditioner based approach such that one works with spherical trust regions?

update: Section 7.5 in Practical Optimization by Gill , Murray & Wright gives a clear connection between variable scaling and preconditioning the hessian.

The ideas are certainly related, at the very least if your preconditioner corresponds to a symmetric and positive definite matrix. This is because in that case, preconditioning simply means using a different inner product, i.e., a different metric. This can be interpreted as saying that what is a very elongated ellipsoid in the usual $l_2$ metric may turn out to be much closer to a sphere in the preconditioner metric. This picture is valid because the application of an SPD preconditioner can be interpreted as a rotation, axis-parallel scaling, and inverse rotation.
• Err, if the preconditioner varies from point to point I'm not sure it's so simple. Then you have a spatially varying inner product (ie., you are optimizing over a non-flat Riemannian manifold diffeomorphic to $\mathbb{R}^n$). Is there an easy transformation of the problem such that a black-box spherical trust region solver would generate the exact same sequence of iterates as a trust region solver with a spatially varying preconditioner? I've been thinking about this and haven't been able to figure it out. Jun 7, 2015 at 18:16