3
$\begingroup$

Let $\Sigma$ be a covariance matrix (e.g. symmetric positive definite). For arbitrary vectors $\epsilon$, I need to compute $\chi^2 \equiv \epsilon^\top\Sigma^{-1}\epsilon$, which I do using a Cholesky decomposition.

If $\Sigma=I$, then $\chi^2$ would be the 2-norm of $\epsilon$. I would then say that each coefficient of $\epsilon$ had the same weight in $\chi^2$. Similarly, if $\Sigma$ is a diagonal matrix with positive diagonal elements $(\lambda_1,\ldots,\lambda_N)$, $\epsilon_i$ would be weighted by $\lambda_i^{-1}$ :

$$ \chi^2 = \sum_{i=1}^N \frac{\epsilon_i^2}{\lambda_i} $$

It is then trivial to conclude that, in general (i.e. irrespective of the value of $\epsilon$), the coefficients of $\epsilon$ that contribute most to $\chi^2$ are those where $\lambda_i$ is biggest.

Unfortunately, $\Sigma$ is not diagonal. We can of course diagonalize it: $PDP^\top \equiv \Sigma$ ($D$ is diagonal and $P$ is the matrix whose columns are the normalized eigenvectors of $\Sigma$; $P$ is unique if we order the eigenvalues from largest to smallest in $D$). Diagonalization allows us to isolate independent contributions to $\chi^2$, and we get

$$\chi^2 = \sum_{i=1}^N \frac{(p_i^\top\epsilon)^2}{\lambda_i} \qquad \text{where} \quad \Sigma p_i \equiv \lambda_i p_i$$

What previously was a scalar weight now becomes a vector weight, and is $p_i/\sqrt\lambda_i$

I am only interested in dominant contributions to $\chi^2$, i.e. where $\lambda_i$ is small and $p_{ij}$ is large.

  1. Analytically, what is a proper way to determine dominant contributions to $\chi^2 \equiv \epsilon^\top\Sigma^{-1}\epsilon$? Is it possible to tell that some region of $\epsilon$ will be weighted more or less than another one, accounting for the covariances?

  2. Numerically, the $1/\lambda$ dependence of the discussion suggests that the smaller the eigenvalue, the larger the contribution. That is very unfortunate: my $\Sigma$ matrix is very ill-conditioned, and computation of the smallest eigenvalues is probably just gibberish. Is there a way to do implement this discussion either robustly or by avoiding completely the calculation of eigenvalues?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.