I have a symmetric positive semidefinite covariance matrix $A$, which is approximately computed as the output of a quadratic regression. I then need to invert $A$, but often it is close to singular. I've reduced the problem by using scaling. That is, I create a diagonal matrix $D$, with elements $D_{ii} = 1/\sqrt{A_{ii}}$. Then
$A^{-1} = D(DAD)^{-1}D$
Where $DAD$ has a lower conditioning number then $A$. Unfortunately in some iterations this is not enough. The size of $A$ is quite small, say maximum $50 \times 50$.
I need the inverse of $A$ because I have to use it in a long calculation, where terms such as $x^TA^{-1}x$ and $A^{-1}B$ appear lots for time. Also: $A^{-1}$ represents a covariance matrix, so it has to be symmetric and positive definite.
Is there some better way to make $A$ invertible?