Least squares fitting

Question

I have the following equation I came across which was solved using least squares

$x = \sum_{n=1}^{N} A_{n} y_{n}$

Where $x$ is a $m \times p$ matrix and $y$ would be of size $m \times p$ as well ,where $p=N$.Both $x$ and $y$ are known. $A_n$ are the weights to be computed for every $y_{n}$.

How would one do a least squares approximation to solve for $A_n$ so that there would be only 1 value of $A_n$ for every column in $y$? Usually least squares (in MATLAB) requires a matrix and a column vector, but in this case there are 2 matrices. Is there any standard technique I am missing here?

Answer 1

The notation here is very confusing, and that's probably the main cause of your difficulty in formulating this as a linear least squares problem.

Introduce the notation $\mbox{vec}(x)$ for the $mp$ by $1$ vector formed by taking the columns of the $m$ by $p$ matrix $x$ consecutively. e.g.

$\mbox{vec}\left( \left[ \begin{array}{ccc} 1 & 2 & 3 \\ 4 & 5 & 6 \\ \end{array} \right] \right)= \left[ \begin{array}{c} 1 \\ 4 \\ 2 \\ 5 \\ 3 \\ 6 \\ \end{array} \right]. $

Clearly, two matrix $A$ and $B$ are equal if and only if $\mbox{vec}(A)=\mbox{vec}(B)$.

Now, your model can be written as

$\mbox{vec}(x) = \sum_{i=1}^{N} A_{i} \mbox{vec}(y_{i})$

Let $H$ be the $mp$ by $N$ matrix whose columns are $\mbox{vec}(y_{i})$, for $i=1, 2, \ldots, N$. Recognizing that this equation involves a matrix multiplication, we can write the model as

$\mbox{vec}(x)=HA$

where $H$ is a known matrix of size $mp$ by $N$, $\mbox{vec}(x)$ is a known vector of length $mp$, and $A$ is an unknown vector of length $N$. This is a conventional linear least squares problem that can be solved with standard software.