5
$\begingroup$

I plan on using a Quasi-Newton method (L-BFGS) to minimize a non-linear objective function.

$$ f: \mathbb{R}^n \rightarrow \mathbb{R}$$

The gradient is kind of interesting: as the values of the first set of parameters approach infinity, the gradient of the objective function with respect to the remaining parameters will approach zero. In other words,

$$ \lim_{x_i\to\infty} \frac{\partial f}{\partial x_j} = 0$$ $$ i < j$$

The important concept: this relationship has no bearing on whether the objective function is being minimized or not. I fear the L-BFGS algorithm will follow a path where the gradient is minimized by this relationship instead of being minimized appropriately.

Can I use this knowledge to aid in my optimization strategy?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.