The section 4.2 "Poor Conditioning" in the book Deep Learning defines the condition number of the function $f(x) = A^{-1}x$ as

\begin{align} \underset{i,j}{\max}~ \Bigg| \frac{\lambda_i}{ \lambda_j} \Bigg|. \end{align}

which is the ratio of the magnitude of the largest and smallest eigenvalue.

and says

When this number is large, matrix inversion is particularly sensitive to error in the input. This sensitivity is an intrinsic property of the matrix itself, not the result of rounding error during matrix inversion.

I guess "sensitivity" here means how rapidly a function changes with respect to small changes in its inputs.

Is "sensitivity" a term, something like the one in Medical statistics or plain English here?


Yes, its commonly used to describe the magnitude of an outputs change with respect to an input. In CFD optimization we provide a sensitivity vector to the optimizer (or the derivatives of each output to each input). In this case the matrix inversion is sensitive to (for example) small differences on the right hand side or floating point error.

  • $\begingroup$ thank you. Does "CFD" here refers to "Computational fluid dynamics"? $\endgroup$ – shi95 May 1 '20 at 3:33
  • $\begingroup$ @EMP Is there some difference with the "conditioning" of a problem? From what you described I immediately think about well/ill conditioned problems $\endgroup$ – VoB May 1 '20 at 8:17
  • $\begingroup$ @shi95, yes it does. $\endgroup$ – EMP May 1 '20 at 19:41
  • $\begingroup$ @VoB, I'm sorry I don't exactly understand your question. $\endgroup$ – EMP May 1 '20 at 19:41
  • $\begingroup$ @EMP I think Federico's post is the answer to my comment $\endgroup$ – VoB May 2 '20 at 16:17

It is "a term" that is used commonly (it appears many time, for instance, in Higham's classical book Accuracy and Stability of Numerical Algorithms), but it is used with its plain English meaning, in my experience. You can use it in a sentence like

The condition number measures the sensitivity of $A^{-1}$ with respect to changes in $A$.

but it does not have a precise mathematical definition like "condition number" or the one in statistics.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.