2
$\begingroup$

I have a set of measurements $y_i$, $1 \leq i \leq N$, and I want to model these measurements with a linear model. I have two possible models I can use, $$ y \approx A c $$ and $$ y \approx B d $$ where $A$ is $N$-by-$P$ and $B$ is $N$-by-$Q$. The vectors $c,d$ are unknown coefficients that I would determine with a least-squares approach. The columns of $A$ and $B$ are vectors which approximately span my data space. I want to know if there is a way to measure how much $A$ and $B$ span the same space, as this may help guide my choice on selecting the best set of vectors to model my data.

I am looking for a quantity which allows me to say something like "The columns of $A$ and $B$ span X% of the same vector space". Or, "the columns of $A$ and $B$ span completely different vector spaces." Does such a thing exist?

$\endgroup$

1 Answer 1

5
$\begingroup$

The classical tool for this job is canonical angles.

The canonical angles between $\operatorname{Im} A$ and $\operatorname{Im} B$ can be computed as $\arccos \sigma_i$, where $\sigma_i$ are the singular values of $Q_A^*Q_B$, with $Q_A, Q_B$ being the orthogonal QR factors of $A$ and $B$.

If (and only if) the thinner matrix $A$ spans a subspace of $\operatorname{Im} B$, the canonical angles are all zero. In particular, for two matrices of the same size the canonical angles are all zero iff the two matrices span the same subspace.

Matlab has a ready-to-use subspace(A, B) that computes the largest angle (which is probably the one you want to use as a distance measure), and Scipy has scipy.linalg.subspace_angles.

$\endgroup$
2
  • $\begingroup$ Thank you. What is meant by the operator $\operatorname{Im}$? $\endgroup$
    – vibe
    Jul 2, 2023 at 11:46
  • $\begingroup$ @vibe The image (or range or column space) of a matrix. $\endgroup$ Jul 2, 2023 at 11:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.