Let $\Sigma$ be a covariance matrix (e.g. symmetric positive definite). For arbitrary vectors $\epsilon$, I need to compute $\chi^2 \equiv \epsilon^\top\Sigma^{-1}\epsilon$, which I do using a Cholesky decomposition.

If $\Sigma=I$, then $\chi^2$ would be the 2-norm of $\epsilon$. I would then say that each coefficient of $\epsilon$ had the same weight in $\chi^2$. Similarly, if $\Sigma$ is a diagonal matrix with positive diagonal elements $(\lambda_1,\ldots,\lambda_N)$, $\epsilon_i$ would be weighted by $\lambda_i^{-1}$ :

$$ \chi^2 = \sum_{i=1}^N \frac{\epsilon_i^2}{\lambda_i} $$

It is then trivial to conclude that, in general (i.e. irrespective of the value of $\epsilon$), the coefficients of $\epsilon$ that contribute most to $\chi^2$ are those where $\lambda_i$ is biggest.

Unfortunately, $\Sigma$ is not diagonal. We can of course diagonalize it: $PDP^\top \equiv \Sigma$ ($D$ is diagonal and $P$ is the matrix whose columns are the normalized eigenvectors of $\Sigma$; $P$ is unique if we order the eigenvalues from largest to smallest in $D$). Diagonalization allows us to isolate independent contributions to $\chi^2$, and we get

$$\chi^2 = \sum_{i=1}^N \frac{(p_i^\top\epsilon)^2}{\lambda_i} \qquad \text{where} \quad \Sigma p_i \equiv \lambda_i p_i$$

What previously was a scalar weight now becomes a vector weight, and is $p_i/\sqrt\lambda_i$

I am only interested in dominant contributions to $\chi^2$, i.e. where $\lambda_i$ is small and $p_{ij}$ is large.

  1. Analytically, what is a proper way to determine dominant contributions to $\chi^2 \equiv \epsilon^\top\Sigma^{-1}\epsilon$? Is it possible to tell that some region of $\epsilon$ will be weighted more or less than another one, accounting for the covariances?

  2. Numerically, the $1/\lambda$ dependence of the discussion suggests that the smaller the eigenvalue, the larger the contribution. That is very unfortunate: my $\Sigma$ matrix is very ill-conditioned, and computation of the smallest eigenvalues is probably just gibberish. Is there a way to do implement this discussion either robustly or by avoiding completely the calculation of eigenvalues?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.