# Questions tagged [least-squares]

For questions focused on implementing or applying least-squares regression.

113 questions
Filter by
Sorted by
Tagged with
3k views

### Newton-based methods in optimization vs. solving systems of nonlinear equations

I asked for clarification about a recent question about minpack, and got the following comment: Any system of equations is equivalent to an optimization problem, which is why Newton-based methods ...
6k views

### Solving a least squares problem with linear constraints in Python

I need to solve \begin{alignat}{1} & \min_{x}\|Ax - b\|^2_{2}, \\ \mathrm{s.t.} & \quad\sum_{i}x_{i} = 1, \\ & \quad x_{i} \geq 0, \quad \forall{i}. \end{alignat} I think it is a ...
2k views

### Purely rotational least squares match

Could anyone recommend a method for the following least-squares problem: find $R \in \mathbb{R}^{3 \times 3}$ that minimizes: $\sum\limits_{i=0}^N (Rx_i - b_i)^2 \rightarrow \min$, where $R$ is a ...
1k views

### Least squares approximation question

I am taking a course on scientific computation, and we just went over least squares approximation. My question is specifically about approximating using polynomials. I understand that if you have n+1 ...
1k views

The square root function for an old (pre-IEEE floating point, no hidden bit) mainframe (the Soviet BESM-6 works as follows: Making sure the argument $X$ is non-negative Splitting $X$ into the ...
6k views

### Fitting one set of points to another by a rigid motion

I'm not really sure how to explain this problem clearly, so please bear with me. I have a basis of 3 orthonormal unit vectors and a position, a standard 4x4 transform matrix in computer graphics. ...
9k views

### Least Squares and Fourier Series

I have a little bit of problem figuring out the relation between Fourier series and Least Squares. As far as I understand, LS is a way of minimizing the quadratic error between a measured value $y_i$ ...
302 views

234 views

### Pseudoinverse of perturbed matrix

How does the pseudo inverse of a full column rank matrix change if I rescale a single row? In more detail the problem is the following: We have a fixed matrix $V$ with linear independent columns and ...
2k views

### Least Squares: Numerically, is solving normal equations okay for nice matrices?

I have to solve a least squares problem: $$x=\arg \min\|Ax-b\|$$ where $A$ is a $m\times n$ matrix, $m>n$, and $b\in\mathbb{R}^m$. I always thought that doing this via QR factorization is ...
3k views

### How to determine whether two cylinders intersect or not?

Considering any two cylinders, defined as: the center of their bottoms $A_i$, the radius of their bottom $R_i$, the unit vector $W_i$ of their axis direction, and the length $L_i$ of the cylinders, ...
185 views

### Nonlinear least-squares solvers vs. generic minimization

A nonlinear least-squares problem with $F:\mathbb{R}^m\to\mathbb{R}^n$, $$F(x) \to \min_x \quad (\text{in the least-squares sense})$$ really means minimizing $$\frac{1}{2} \|F(x)\|^2 \to \min_x.$$ ...
135 views

### Large scale triangular least squares

I have to solve the following least squares problem: $$\| \left[ \begin{smallmatrix} \mathbf{L} \\ \mathbf{I} \end{smallmatrix} \right]\mathbf{x} - \mathbf{b} \|_2^2$$ ...
248 views

### Non-negative least squares with very small numbers

(I have asked this question on StackOverflow previously but it has been pointed to me that CSSE or MSE could be more appropriate) I have to solve a constrained optimization problem of the following ...
51 views

### Nonlinear least squares optimized Jacobian calculation

I have a nonlinear least squares problem, in which I am trying to minimize residuals which can be divided into four classes: $$\min_x ||\epsilon(x)||^2 + ||\xi(x)||^2 + ||\delta(x)||^2 + ||s(x)||^2$$...
150 views

### Solve ill-posed linear system without transposing matrices?

I am attempting to use an iterative solver to solve $p$ in $$Jp = -r$$ where $J$ is an $m\times m$ matrix ($m$ is in the order of $10^5$ and never explicitly stored). $J$ is a dense matrix ...
1k views

### Using MINPACK for curve fitting: implementation?

I need to implement a non-linear fitting algorithm in Fortran and chose to use MINPACK's flavor of the Levenberg-Marquardt algorithm as a basis for the least-squares stuff. However, I seem to ...
208 views

199 views

### Best solver/preconditioner for least-squares finite element method

I have seen a lot of literature, lecture videos, etc. on solvers/preconditioners for non-symmetric and/or indefinite systems. However, now I want to solve the mixed poisson/Darcy equation using the ...
595 views

### Complex least-squares problem

Having a matrix $\mathbf{A} \in \mathcal{C}^{m\times n}$ I solve following least-squares problem $$Re(\mathbf{A}^H \mathbf{A})x=Re(\mathbf{A}^H\mathbf{b}).$$ If the matrix $\mathbf{A}$ was a real ...
167 views

### Factorization for reweighted least squares

I am solving a problem using an iteratively-reweighted least squares method: http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares Essentially this requires solving a number of least-...
124 views

### Do computational scientists think about the statistical aspects of using least squares?

Correct me if I'm wrong, but when least squares is used in the computational science community, it's typically not in the context of regression. It could be used to solve for gradients in discretized ...
285 views

### Nonlinear least squares and regularization

Consider the nonlinear least-squares minimization of a vector of $n$ residuals $\mathbf{f}$ in $p$ parameters $\mathbf{x}$: $$\min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2$$ This can be done with ...
1k views

76 views

### Solving multiple least-square problems with the same constraints

The following least-square problem can be solved efficiently (e.g. using matlab's lsqlin): $$\vec{x}^*=\arg\min_\vec{x} ||C\vec{x}-\vec{t}||^2\,\ s.t.\ Ax \le \vec{b}$$ where the parameters of the ...
517 views

### indirect method for least-squares with inequality constraints

I aim to find $x \in \mathbb{R}^n$ that $\min_x |D \cdot F \cdot x|^2$ subject to $x_i = X_i$ and $x_j \geq X_j$ , $i \in I, j \in J$ and I and J partition ${1\cdots N}$ into two sets. it is ...
202 views

### How do you formulate the linear least-squares method for radiometric calibration?

In Debevec and Malik (mentioned similarly in Forsyth and Ponce's Computer Vision: A Modern Approach) they highlight a method of solving the camera response function using linear least-squares. We ...
55 views

### Least-squares fit of explicit parabolic sheet to data points

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
For a linear least squares problem, it is possible to define a resolution matrix, relating the estimated model parameters to the true model parameters. If we are solving a regularized problem, $$\... 0answers 40 views ### Reweighted least squares factorization This is a continuation of the question asked here. I want to solve numerous least squares systems of the form$$ D_i A x \approx D_i b $$where D_i are m \times m diagonal matrices with positive ... 0answers 118 views ### Backward stable algorithm to get orthogonal projection onto the column space of a matrix I have to find the orthogonal projection of a vector b onto the matrix A of size m \times n. In my application, I don't have the luxury of calculating the QR factorization. All I have are ... 0answers 1k views ### Polynomial Fitting with Least Squares using Numpy and Scipy I am trying to fit data to a polynomial using Python - Numpy. The points, with lines sketched above them are as in the picture. I am trying to fit those points to a polynomial of 4. or 5. degree. ... 2answers 144 views ### L1 least squares minimization with a sparse matrix I have the following problem:$$\min_{x\in \mathbb{R}^n}\|Ax-b\|_1$$where the matrix A is large and sparse. I am looking for methods/code that can minimize this efficiently. References are very ... 2answers 1k views ### Solving non-negative least squares in Matlab (by analogy with least squares) There is a least-squares problem. It can be solved using backslash in Matlab. If Ax = b, then x = A \ b. Let's assume that I ... 2answers 129 views ### efficient mean of solving constrained OLS problems? I was wondering whether there was a efficient procedure for solving constrained quadratic approximations of the form:$$\underset{k\in \mathbb{R}}{\min}\;||x_i-kx_0||_2$$for fixed values of x_0,... 1answer 98 views ### Description of algorithm for small scale linear least squares with box constraints I have small scale dense least squares problem with box constraints$$\mbox{argmin}||Ax - b||^2 \quad \mbox{subject to} \quad l_i \leq x_i \leq u_i, Number of variables is about 10-50, ...
Question I am performing least squares fitting using an objective function of the form $f(\mathbf{x})$ where $\mathbf{x}$ is a vector of parameters containing around 20 elements. The model function ...