In my optimization problem, the hessian has a structure such that it can be written as the sum of two matrices. Populating the first of the matrices is efficient. Populating the second one is computationally very heavy and practically makes methods like truncated Newton useless. However, the second matrix is block diagonal in nature. I am thinking I will use a Quasi-Newton approximation to the second matrix and compute the first matrix exactly. Hence, get an approximation for the hessian by adding these two matrices. Intuitively, this Hessian approximation should be better than having a Quasi-Newton approximation for the entire Hessian matrix. Is this thinking correct? Have people successfully tried such an approach before and got a better optimization approach than a direct Quasi-Newton based approach?

  • 1
    $\begingroup$ You're not likely to get a good theoretical answer to a question like this. Try it on your problem and see how well it works in practice. $\endgroup$ – Brian Borchers Sep 16 '16 at 14:05
  • $\begingroup$ ok. I will try and see. $\endgroup$ – Hari Sep 16 '16 at 14:12

You may find this paper relevant. They give some BFGS-inspired methods and convergence proofs for cases where the Hessian is the sum of a part you can compute easily and a part you can't ("structured secant" methods), and you want to leverage your knowledge of the computable part as much as possible.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.