I am trying to implement BFGS. The purpose is to approximate Hessian matrix only (not using the quasi-newton optimization steps), so i am using steepest ascent for optimization. What I observe is that the final Hessian approximate is very sensitive to the initial guess of Hessian. If I start with Identity matrix, most of the singular values of final hessian is closed to 1. Similarly if I start with any multiple of identity (let's say 5*I), the singular values of final Hessian is closed to 5.
It is a maximization problem, and also the solution is not unique so i expect a negative semi-definite Hessian matrix.
I hope my question is clear.
Thanks in advance.