I have a system of linear equations that is derived partially from experimental data. Theoretically, the system should have a single, exact solution; however, experimental error causes it to not have an exact solution. Is there a good method to find an approximate solution to this system? My idea for a naive implementation was to solve all pairs of two equations and average the value for the solutions to each variable, but this doesn't extend well as the number of equations increases. Is there any known algorithm to do this?

  • 4
    $\begingroup$ least squares might be a good candidate? $\endgroup$
    – James
    Nov 26, 2015 at 6:19
  • 3
    $\begingroup$ It's important to specify whether the linear system is in fact invertible (meaning you do get a single solution, although it might not be exactly the one you're looking for), underdetermined (you get multiple solutions, but might be able to pick a reasonable one), or overdetermined (there is no solution unless you hit the exact data). $\endgroup$ Nov 26, 2015 at 9:12
  • 3
    $\begingroup$ Also, do the experimental error enter into the matrix (i.e., coefficients of the linear system) or the right-hand side? $\endgroup$ Nov 26, 2015 at 9:13
  • $\begingroup$ And how many equations and variables are there? How do you solve all pairs of equations? Least squares (or are there only two variables?). $\endgroup$
    – Dirk
    Jan 26, 2016 at 5:23

2 Answers 2


It seems you are looking for an optimization (here, specifically a fit).

You have some experimental data and you want to have a simple model explaining it.

The simplest way is using the Ordinary Least Squares. The idea is to minimize the standard deviation or the error you make on the vertical axis.

Another one would be the PCA where you try to find an angle to view your variables to find the maximum information. It's based on the eigenvectors of your system. The advantage is that it only tries to find the best solution to minimize the orthogonal error.

Many other solutions are possibles but it would be preferable to have more details about what you're exactly looking for and the link between variables.

  • $\begingroup$ so this is essentially linear regression? $\endgroup$
    – Akababa
    Mar 28, 2019 at 2:54

You can use any of the numerical methods that is for linear equation systems. Some of them are;

  • Gauss-Jordan Elimination
  • Jacobi Method
  • Gauss-Seidel
  • Successive Over Relaxation(SOR)

For more info, you can look at this

  • 7
    $\begingroup$ I'm sorry, but this is bad advice. If the linear system does not have an exact solution, trying to compute it (no matter how) will necessarily blow up in your face. $\endgroup$ Nov 26, 2015 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.