I am struggling with the following exercise from the book of Nocedal, Numerical optimization, chapter 2, exercise 2.12:

Suppose that a function $f$ of two variables is poorly scaled at the solution $x^*$. Write two Taylor expansions of $f$ around $x^*$ - one along each coordinate direction - and use them to show that the Hessian $\nabla^2 f (x^*)$ is ill-conditioned.

  • 1
    $\begingroup$ Well, have you written down the Taylor expansions? Do you know the definition of ill-conditioned (and "poorly scaled")? $\endgroup$ May 9 '18 at 9:55
  • $\begingroup$ I have written Taylor expansions, however book does not provide exact definitions of ill-conditioned and poorly scaled. I am mostly struggling with cross-terms of hessian. $\endgroup$
    – curiosity
    May 9 '18 at 11:08
  • $\begingroup$ Then you should update your question with the details on exactly what you are struggling with; in its current form, it's unclear what a helpful answer would be. (The more detail you put in a question, the more inclined people are to spend the time trying to answer it.) $\endgroup$ May 9 '18 at 11:17
  • 1
    $\begingroup$ (For the purposes of this question, a matrix is ill-conditioned if small changes in the right-hand side would lead to very large changes in the solution of the corresponding linear equation system.) $\endgroup$ May 9 '18 at 11:19

It might help to start with a concrete example of a function, like $f(x,y)=10^9 x^2+y^2$ (the example they use on p.26). I'm trying to guess what went wrong with your analysis, because you haven't given much information.

I am mostly struggling with cross-terms of hessian.

Consider $$f_1(x,y) = 10^9 x^2+y^2$$ and its rotation by $\theta=\frac\pi4$ $$f_2(x,y) = 10^9 (x\cos\theta + y\sin\theta)^2 + (-x\sin\theta + y\cos\theta)^2.$$ These are equivalent, and both are poorly scaled, but in different directions. If you try to analyze the Hessian in a way that just assumes that the function is poorly scaled specifically in the coordinate directions (only the diagonal terms are important, and so you don't care about cross terms), it's not going to work for the second function. I think you just need a different way of proving ill-conditionedness of the Hessian.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.