I plan on using a Quasi-Newton method (L-BFGS) to minimize a non-linear objective function.

$$ f: \mathbb{R}^n \rightarrow \mathbb{R}$$

The gradient is kind of interesting: as the values of the first set of parameters approach infinity, the gradient of the objective function with respect to the remaining parameters will approach zero. In other words,

$$ \lim_{x_i\to\infty} \frac{\partial f}{\partial x_j} = 0$$ $$ i < j$$

The important concept: this relationship has no bearing on whether the objective function is being minimized or not. I fear the L-BFGS algorithm will follow a path where the gradient is minimized by this relationship instead of being minimized appropriately.

Can I use this knowledge to aid in my optimization strategy?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.