# LSA, SVD and the Frobenius norm

In Latent Semantic Analysis one uses the SVD to perform a dimensional reduction of the term-document matrix, via the Eckart-Young theorem.

Now, the rank $k$ approximation obtained by E-Y is proven to be the best approximation in the Euclidean norm as well as in the Frobenius norm.

"For the purposes of LSA, we consider the best approximation in the Frobenius norm, provided by Eckart-Young."

But nobody says why. I mean, why not in the Euclidean norm?

Thanks.

• $\min_{rank(Y)=k}\|X-Y\|_F.$ $X=U\Sigma V^*,$ $\|X-Y\|_F=\|\Sigma-U^*YV\|_F$ since the factors $U,$ $V$ do not change F-norm. $\min_{rank(Z)=k}\|\Sigma-Z\|_F$ is attained only when $Z$ is diagonal and takes the largest $\sigma$'s ($\Sigma=diag(\sigma_i)$). Apr 22 '14 at 7:47

There is no such thing as the Euclidean norm of a matrix. It is only defined for vectors in ${\mathbb R}^n$. You need to decide whether you want to consider a matrix as an element of the vector space ${\mathbb R}^{n \times m}$, in which case the Frobenius norm is simply the easiest one to compute. Alternatively, you could consider a matrix as an operator ${\mathbb R}^m \rightarrow {\mathbb R}^n$ in which case the $l_2$ norm is very awkward to compute (it is the largest singular value) and one could go with the $l_1$ or $l_\infty$ norms which are the maximal sums of absolute values of matrix elements over rows or columns. In the end, the Frobenius norm is typically just the simplest norm to compute if you don't think of the matrix as an operator.
• Correct -- you can induce the operator norm that matches the Euclidean vector norm. But it's not called the "Euclidean norm" of a matrix -- it's called the spectral or $l_2$ norm of the matrix, and it's pretty complicated to compute in practice. The point I wanted to make is that if you think that the Euclidean norm of a matrix would just be the square root of the sum of squares of entries, then you are mistaken: That is the Frobenius norm, and if we want to consider the matrix norm that corresponds to the Euclidean vector norm, then that is something completely different. Apr 21 '14 at 19:58