I'm using BFGS to optimize a smooth but non-convex function $f$ that is computed by a simulation. The simulation also gives me a semi-analytical gradient $g$, which is verified by the numerical gradient.
As $f$ is non-convex and I am using a penalty function to enforce bound constraints, I do not expect to find a global minimum, but rather a local one where the gradient also happens to vanish. However, when BFGS "converges" (in terms of very small $\Delta x$ and $\Delta f$), $\Vert g \Vert_\infty$ remains quite large. I can think of a few reasons for this:
The penalty function is large close to the bounds -- this is not the case here though, as the penalty function is sufficiently scaled down at this point
The downward slope of the gradient is intersecting with one of the boundaries -- BFGS can't move away from the boundary since that will increase the function, but also cannot approach it further, causing it to get stuck. This might be the case since the step size is virtually zero at "convergence".
Can anyone suggest any other reasons for this behavior, or a way to verify if (2) is indeed the cause?