I am using nonlinear optimizers such as BOBYQA to train a model with 10-20 parameters. It so happens that some of the parameters have high correlation. Roughly speaking, imagine that you are fitting parameters $a,b$ in a function like this:
The response to the change in parameters $a,b$ is much larger than the response in to the change of $a-b$. Would performance of Powell's nonlinear optimization algorithms such as BOBYQA, NEWUOA, etc., improve if I would "rotate parameters" to isolate gross responses from fine ones?
That is, would nonlinear optimizers perform better if instead of optimizing $a,b$ in the above function I would optimize for parameters $\alpha,\beta$ a function like this:
The real model is more complicated, of course, and I'd like to find out whether changing parameters to minimize correlations has a chance to improve anything.