Bad scaling versus collinearity

Question

I was trying to solve a linear system:

$$ \mathbf{A}\mathbf{x} = \mathbf{y} $$

but the conditioning number was quite bad (around $10^{17}$). I thought that the system was singular, but after scaling the system using the diagonal matrix:

$$ \mathbf{D_{ii}} = 1/\sqrt{\mathbf{A_{ii}}} $$

and the fact:

$$ \mathbf{A}^{-1} = \mathbf{D} (\mathbf{D}\mathbf{A}\mathbf{D})^{-1}\mathbf{D}. $$

The conditioning number of $\mathbf{D}\mathbf{A}\mathbf{D}$ is around $10^3$! Does this imply that the system was just badly scaled and that there is no collinearity?

Thanks!

Answer 1

Yes, that's what it means. This is usually illustrated with $A$ a diagonal matrix having both large and small entries. Clearly such a matrix can be accurately inverted, but a simple measure of condition number being ratio of largest eigenvalue to smallest (in this case, largest diagonal entry to smallest) would easily give a high value.