How to compute the Frobenius norm of matrices whose entries are either too large or too small?

Question

While implementing in Matlab the Frobenius norm of a matrix

$$\| A\|_{\text F} := \sqrt{ \sum_{i=1}^m \sum_{j=1}^n a_{ij}^2 },$$

a problem arises when numbers are too big or too small:

• If a number in the matrix is too big (but within bounds of Matlab), taking its square will result in “infinity”. Adding anything else to it and taking the root will also result in infinity. However, the norm may actually be much smaller, since the root is taken at the end.

• Similarly, if all numbers in the matrix are sufficiently small, their squares and, thus, the norm will return $0$, while in reality it may be big enough to be represented by Matlab, since the root is taken.

How would we go about dealing with cases like this in the implementation of the function? I am aware that norm(A,'fro') exists in Matlab, but we have been asked to deal with this problem ourselves, and I’m baffled.

Answer 1

You can scale your matrix by, say, the inverse of the largest/smallest(nonzero) entry before you compute the norm and use that $\|aM\|=|a|\|M\|$.

And have a look into Higham's book Accuracy and Stability of Numerical Algorithms to understand how these scalings have an effect on the accuracy.