I'm building an application where I need to compare found data with the actual data it should be.

I have 5 sets of data, each with 3 variables a,b,c. Let matrix A be a 3x1 matrix with data a,b,c which represents the actual data. Let matrix B be a 3x3 matrix with unknown data that needs to be calculated. Let matrix C be a 3x1 matrix with found data a,b,c.

The equation A = B.C needs to be solved so that it gives the lowest difference for the 5 datasets.

How would I start with solving this? Do I just have to start with some numbers in B and try new ones each iteration?


1 Answer 1


Let me see if I can restate the problem to answer your question:

You have 5 sets of data: $y_{i}$, $x_{i}$ such that for each $i = 1, \ldots, 5$, $x_{i}$ and $y_{i}$ are both 3 by 1 vectors. You also have a 3 by 3 matrix $B$ that is supposed to relate $x_{i}$ to $y_{i}$ via:

$$y_{i} = Bx_{i} $$

Is this formulation correct?

If so, the proper methods for solving this sort of problem use linear regression.

One linear regression method is ordinary least squares(OLS). OLS will find a solution to the overdetermined set of equations:

$$[y_{1}, \ldots, y_{5}] = B \cdot [x_{1}, \ldots, x_{5}]$$

with minimum residual; that is, it minimizes the sum of squared distances between the data $y_{i}$ and the predictions $Bx_{i}$, hence "least squares". This sum of squared distances is $\sum_{i=1}^{5}\|y_{i} - Bx_{i}\|^{2}$ (the norm used is a 2-norm).

For convenience, define the 3 by 5 matrices $X$ and $Y$ by:

$$Y = [y_{1}, \ldots, y_{5}],$$ $$X = [x_{1}, \ldots, x_{5}],$$

so we are finding a solution to the overdetermined system of equations

$$Y = BX$$

with minimum sum-of-squared distances. This next part is important:

The notation I've used to describe your problem is the transpose of what the Wikipedia article on ordinary least squares uses. So if you try to compare everything, you're going to have to transpose every matrix.

Using some algebra, the value of $B$ that minimizes the sum of squared distances is

$$B = (XX^{T})^{-1}XY^{T};$$

you can check the dimensions of each matrix to see that $B$ is 3 by 3.

Do not calculate $B$ this way.

Calculating matrix inverses directly is usually bad for a variety of reasons; why it is bad is out of the scope of the question. If you need a matrix inverse, it's usually better to solve the system of equations related to that inverse.

A better way to calculate $B$ would be to solve the related linear system

$$XX^{T}B = XY^{T}.$$

This system of equations is called the normal equations (plural, because the system of equations is referred to as a group).

Do not calculate $B$ using the normal equations either.

Solving the normal equations is not usually a good idea because this system of equations can be ill-conditioned, which will degrade the accuracy of $B$.

An even better way to calculate $B$ would be to use QR decomposition.

Let $X^{T} = [Q_{1}, Q_{2}]([R_{1}, 0])^{T}$, obtained from a QR decomposition. Using the methods outlined here,

$$B = R_{1}^{-1}Q_{1}^{T}Y^{T},$$

so I would solve the linear system

$$R_{1}B = Q_{1}^{T}Y^{T}.$$

This last method should give you what you want (assuming I didn't make any mistakes during my lunch break).

  • $\begingroup$ Yes that is exactly what my problem is. I will look at the Wikipedia to see if I can make it work. Thanks for pointing me in the right direction. $\endgroup$
    – jHogen
    Mar 12, 2013 at 17:24
  • $\begingroup$ Thanks a lot! This will be of great help to me. I've read the Wikipedia page about lineair regression but this is a lot easier to understand. $\endgroup$
    – jHogen
    Mar 13, 2013 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.