If a matrix $A_{n\times n}$ is invertible, then

$\left\|A^{-1}\right\|_2 = \dfrac{1}{\min\limits_{i} \sigma_i}$

where $\sigma_i$ is the $i$-th singular value of $A$

  • 11
    $\begingroup$ If this is a homework question (and it looks like something that appears verbatim in every linear algebra textbook ever written), you must tag it as such and show that you have put in reasonable effort. $\endgroup$ – Jed Brown Jan 2 '14 at 18:58

Singular value decomposition of $A$ can be written as $A=USV^T$ with $S$ being a diagonal matrix of singular values of $\sigma_i$.

Euclidean norm of a matrix can be written as : $||A||_2=\sigma_{max}(A)$, meaning that the norm is the maximum singular value.

If $A$ is invertible, then $A^{-1}=(USV^T)^{-1}$.

From here it follows:

$$A^{-1}=(USV^T)^{-1} =(V^T)^{-1}S^{-1}U^{-1} $$

If $A$ is a real valued matrix, then from orthogonality the expression further reduces to: $$A^{-1}=VS^{-1}U^T.$$

This new matrix $A^{-1}$ now has singular values $S^{-1}$, and its norm would be $\max(\text{diag}(S^{-1}))$, where $\text{diag}$ takes the diagonal of the matrix. Because $S$ is a diagonal matrix, its inverse is computed by simply inverting every single element. Therefore,

$||A^{-1}||_2=\sigma_{max}(A^{-1})=\max\limits_{i} \text{diag}(S^{-1})_i = \max\limits_{i}\frac{1}{\sigma_i}=\frac{1}{\min\limits_{i}{\sigma_i}}$ and this completes the proof.

| cite | improve this answer | |
  • 1
    $\begingroup$ Of course, you've started with $\| A \|_{2}=\sigma_{max}(A)$. It isn't clear whether the original poster is aware of that fact or is willing to assume it. $\endgroup$ – Brian Borchers Jan 2 '14 at 15:35
  • 3
    $\begingroup$ Well, considering the structure of the question, I guess this is a fair starting point as it is not explicitly stated. If desired it could also be interpreted either geometrically (cs.princeton.edu/courses/archive/spring12/cos598C/…) or algebraically using the induced norm $||A||_2=\max{\frac{||Ax||}{||x||}}$, where $x$ is real and non-zero. This is only a special case of $L_p$ norm. If $||x||_2=1$ is assumed, the maximizer is the largest singular value. $\endgroup$ – Tolga Birdal Jan 2 '14 at 15:55

I think I'd rather see it done this way. Since $A$ is invertible, then $\min_i\sigma_i>0$, $$ \min_i\sigma_i = \inf_{x\neq 0}\frac{\|Ax\|_2}{\|x\|_2} \quad\Longleftrightarrow \frac{1}{\min_i\sigma_i} = \sup_{x\neq 0}\frac{\|x\|_2}{\|Ax\|_2}. $$ Then we have $$ \frac{1}{\min_i\sigma_i} = \sup_{x\neq 0}\frac{\|x\|_2}{\|Ax\|_2} = \sup_{A^{-1}y\neq 0}\frac{\|A^{-1}y\|_2}{\|y\|_2} = \sup_{y\neq 0}\frac{\|A^{-1}y\|_2}{\|y\|_2} = \|A^{-1}\|_2. $$ where we have made the substitution $Ax=y$, and utilized the fact that $A^{-1}y=0\Longleftrightarrow y=0$, again since $A$ is invertible.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.