Suppose I am minimizing the following function:
where $y_i$ and $x_i$ are data, $f$ is a known non-linear function and $\alpha$ parameter (of dimension larger than 1) of interest. Is it better to minimise
instead in order to guard against round-off errors, etc? In my case I get conflicting results, for some problems normalisation leads to a vast improvement in proportion of achieved convergence (I do MC simulations so I know the true value $\alpha$) in other cases not so much. Maybe there is some kind of an algorithm or a general advice when to choose the scaling and when not to?
I use R's
optim function for optimisation. I tried
"BFGS". Results are different for
BFGS and the same for
Nelder-Mead. The differences are slight, but measurable especially for
BFGS. Naturally I use the same starting values and the same data for each optimisation run.