Given a near singular covariance matrix, the standard method of 'fixing' it seems to be to add a small damping coefficient $c>0$ to the diagonal, which serves to bump all the eigenvalues up by this amount.
Is there a way to determine the optimal value of this damping coefficient without resorting to some iterative procedure where I jump back and forth between inverting the matrix to look for infinities and NaNs and making $c$ increasingly small?
I could look at the minimum eigenvalue fairly cheaply, but it seems like the correct choice for $c$ would still also depend on both the entries of the matrix as well as its size.
Finally, it's more important that my algorithm always works than it is that the perturbation is as small as possible, so I'm willing to accept more error for the sake of a procedure with some sort of guarantee that it will always work.