# Does the amount of correlation of model parameters matter for nonlinear optimizers?

I am using nonlinear optimizers such as BOBYQA to train a model with 10-20 parameters. It so happens that some of the parameters have high correlation. Roughly speaking, imagine that you are fitting parameters $a,b$ in a function like this:

$f(x)=a\frac{x}{1+|x|}-b\frac{x^2}{1+|x|^2}$

The response to the change in parameters $a,b$ is much larger than the response in to the change of $a-b$. Would performance of Powell's nonlinear optimization algorithms such as BOBYQA, NEWUOA, etc., improve if I would "rotate parameters" to isolate gross responses from fine ones?

That is, would nonlinear optimizers perform better if instead of optimizing $a,b$ in the above function I would optimize for parameters $\alpha,\beta$ a function like this:

$g(x)=\alpha(\frac{x}{1+|x|}+\frac{x^2}{1+|x|^2})+\beta(\frac{x}{1+|x|}-\frac{x^2}{1+|x|^2})$

The real model is more complicated, of course, and I'd like to find out whether changing parameters to minimize correlations has a chance to improve anything.

where $H$ is the Hessian, $f$ is the objective function, and $\nabla{f}$ is the gradient of the objective function, yielding an ill-conditioned linear system that will be solved inaccurately with numerical methods. (Here, I've assumed a Newton method, but quasi-Newton methods operate similarly.)