Questions tagged [least-squares]

For questions focused on implementing or applying least-squares regression.

36 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
131 views

"Geometry of ill-conditioning" for least-squares problems

It is an idea that dates back to Demmel, 1987 that the condition number of a problem is often related to the distance to the closest ill-posed problems. In Section 3 of the above paper, the author ...
5
votes
0answers
54 views

Nonlinear least squares optimized Jacobian calculation

I have a nonlinear least squares problem, in which I am trying to minimize residuals which can be divided into four classes: $$ \min_x ||\epsilon(x)||^2 + ||\xi(x)||^2 + ||\delta(x)||^2 + ||s(x)||^2 $$...
5
votes
0answers
151 views

Solve ill-posed linear system without transposing matrices?

I am attempting to use an iterative solver to solve $p$ in $$ Jp = -r $$ where $J$ is an $m\times m$ matrix ($m$ is in the order of $10^5$ and never explicitly stored). $J$ is a dense matrix ...
5
votes
0answers
1k views

Using MINPACK for curve fitting: implementation?

I need to implement a non-linear fitting algorithm in Fortran and chose to use MINPACK's flavor of the Levenberg-Marquardt algorithm as a basis for the least-squares stuff. However, I seem to ...
4
votes
0answers
124 views

Do computational scientists think about the statistical aspects of using least squares?

Correct me if I'm wrong, but when least squares is used in the computational science community, it's typically not in the context of regression. It could be used to solve for gradients in discretized ...
4
votes
0answers
294 views

Nonlinear least squares and regularization

Consider the nonlinear least-squares minimization of a vector of $n$ residuals $\mathbf{f}$ in $p$ parameters $\mathbf{x}$: $$ \min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2 $$ This can be done with ...
3
votes
0answers
55 views

Least-squares fit of explicit parabolic sheet to data points

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
3
votes
0answers
40 views

Reweighted least squares factorization

This is a continuation of the question asked here. I want to solve numerous least squares systems of the form $$ D_i A x \approx D_i b $$ where $D_i$ are $m \times m$ diagonal matrices with positive ...
3
votes
0answers
120 views

Backward stable algorithm to get orthogonal projection onto the column space of a matrix

I have to find the orthogonal projection of a vector $b$ onto the matrix $A$ of size $m \times n$. In my application, I don't have the luxury of calculating the QR factorization. All I have are ...
3
votes
0answers
1k views

Polynomial Fitting with Least Squares using Numpy and Scipy

I am trying to fit data to a polynomial using Python - Numpy. The points, with lines sketched above them are as in the picture. I am trying to fit those points to a polynomial of 4. or 5. degree. ...
2
votes
0answers
56 views

LSMR: update $\|x\|$ cheaply

In Fong, Saunders: LSMR: An iterative algorithm for sparse least-squares problems, section 3.3 one reads Since only the last diagonal of $\tilde{R}_k$ and the bottom 2x2 part of $\hat{R}_k$ change ...
2
votes
0answers
76 views

Non-negative Least Squares to perform Inverse Laplace with weights

I'm trying to perform the inverse Laplace transform of a (noisy) dataset $y_i$ using Tikhonov regularization: $$\min \sum_{i=1}^{N} \left(\int_0^\infty e^{-s_i t} f(t) \, dt - y_i \right)^2 - \lambda^...
2
votes
0answers
52 views

Fitting a plane with the Prewitt gradient operator

Prewitt gradient operator Show that the Prewitt gradient operator can be obtained by fitting the least-squares plane through the 3 × 3 neighborhood of the intensity function. Hint: Fit a plane to ...
2
votes
0answers
262 views

Regularized least squares with QR factorization

Consider the regularized least squares problem $$ \min_x || b - A x ||^2 + \lambda^2 ||x||^2 $$ which is equivalent to $$ \min_x \left|\left| \pmatrix{b \\ 0} - \pmatrix{A \\ \lambda I} x \right|\...
2
votes
0answers
288 views

Convergence of a very large non-linear least squares optimization

(note: I also posted this question on stackoverflow before finding this community here, which seems a better place for it) I'm trying to solve the following problem: I have a lot (~80000) surface ...
2
votes
0answers
45 views

$(max(0, f(x)))^2$ or $(max(0, exp(f(x))))^2$ for soft constraints with Gauss-Newton

I need some kind of inequality constraint in my optimization problems (rude version of SVM for example or skeleton based mesh fitting). However hard constraints is not suitable for me because ...
2
votes
0answers
27 views

Choosing suitable polynomial degree based on information in advection stencil

I'm working on a finite volume advection scheme for unstructured meshes which uses a multidimensional polynomial weighted least squares fit for interpolating from cell centres onto faces. In 2D, the ...
2
votes
0answers
119 views

Calculating theoretical order of accuracy of least squares fit advection scheme

I'm familiar with finding the order of accuracy using von Neumann analysis for finite difference schemes formulated using Taylor series expansions. But is there a similar technique for finding the ...
2
votes
0answers
70 views

Fortran solver for the Sparse LSE problem

I was wondering if there is a Fortran library that contains a solver for the Sparse LSE(linear equality-constrained least squares) problem $$ min_{x}\|Cx-d\|^2 \text{ subject to } Ax=b $$ where $A$ ...
1
vote
0answers
53 views

Fitting data with a Voigt function

I have some data, (xrd data), that I would like peak fit with a pseudo-Voigt function, a combination of a Gaussian and a Lorentzian function. These are the functions $G(x) = I \exp\left( -\frac{4\ln(2)...
1
vote
0answers
36 views

Hessian-free preconditioner for non linear least squares

I am solving a nonlinear least squares problem using Gauss Newton method. Due to the large dimension of the problem, I use the Hessian-free approach. As a linear solver I use either MINRES or CG. To ...
1
vote
0answers
63 views

Spline interpolation for vector-valued data in 3D space

I have output from a 3D linear elasticity finite element simulation which uses linear tetrahedral elements, such that the displacement is continuous over the nodes but the gradient is not ($C_0$ ...
1
vote
0answers
67 views

How applying the gradient descent method for solving a least square problem can remove the blur from an image?

I got an assignment where it asked to implement (in MATLAB) the gradient descent algorithm in order to resolve an ill posed least square problem: $$ \min_u \Vert Gu - f \Vert $$ where $u$ is the ...
1
vote
0answers
35 views

What's the best way to implement a least-squares estimation of a motor system in MATLAB?

Basically, I'm trying to use Least-Squares to estimate the parameters of a DC motor. My system can be modeled by the following matrix equation: $$\begin{bmatrix}V_{input}(t)\\0\end{bmatrix}=\begin{...
1
vote
0answers
72 views

Numerically estimate the Jacobian from a scalar time-series

I'm trying to numerically estimate the Jacobian from a time-series. Following the paper, https://journals.aps.org/pra/abstract/10.1103/PhysRevA.43.2787. Consider that I have a scalar time series $x = (...
1
vote
0answers
45 views

Triangle on top of diagonal least squares

I need to solve many least squares problems with the following matrices: $$ \pmatrix{ R \\ D_i } $$ where $R$ is upper triangular and $D_i$ is diagonal. $R$ is the same for all the problems, while $...
1
vote
0answers
65 views

B-splines least squares with equality constraints

Can someone recommend the best way to solve a least squares fitting problem with B-splines, with additional equality constraints? I want to solve: $$ \min_x || b - A x ||^2, \textrm{subject to: } C x =...
1
vote
0answers
73 views

How to model a non-linear least-squares problem for triangles

I have a non-linear least-squares problem to solve and with my current modeling, the solver is either very slow or does not converge to a correct solution. For the problem, I need to minimize energy ...
1
vote
0answers
35 views

fitting exponential versus exponential w/ power

I have two models which I would like to investigate for my data. One form is: \begin{equation} \label{one} f(r) = A e^{-B r} \end{equation} and the second is: \begin{equation} \label{two} g(r)...
1
vote
0answers
79 views

Stable Method of orthogonal projection onto a subspace with the help of Moore-Penrose inverse,

Projection of a vector $v$ onto the column space of a matrix $A$ is given by $AA^\dagger v$. From the definition of Moore-Penrose Inverse we know that $AA^\dagger v = (A^T)^\dagger A^T v $. Below is ...
1
vote
0answers
61 views

linear solution of curve fitting on multiple linear functions differing by a multiplier

I am facing the following problem. I know nonlinear least squares can provide a solution but I am wondering if a linear way to solve this data fitting problem may exists. This is my input dataset: I'...
0
votes
0answers
46 views

Blown-up iterates in Gauss-Newton method

I am working on a non-linear least squares problem with standard form, in which I need to calibrate a parameter vector $\Theta$ to a set of inputs $\mathbf{x}$ and outputs $\mathbf{y}$: $$\begin{align}...
0
votes
0answers
44 views

Implementation method selection for sparse constrained linear least squares or quadratic programming

I need to slove one optimization problem of quadratic programming. The number of optimization variables is about 16,000. The constraints include equality constraints and inequality constraints. I have ...
0
votes
0answers
128 views

Parameter estimation using fmincon

This is a follow up to my previous question posted here. I am solving an optimization problem using fmincon in MATLAB. There are no equality constraints in my model....
0
votes
0answers
260 views

computation complexity of OLS in estimating a VAR model

Could someone tell me the computation cost of using Ordinary least squares in estimating a Vector autoregression model? I am thinking the cost is O(n) where n is the number of the training instance. ...
-1
votes
1answer
41 views

Trouble Estimating Motor Parameters with Least Squares in MATLAB

Basically, I'm trying to use Least-Squares to estimate the parameters of a DC motor. My system can be modeled by the following matrix equation: $$\begin{bmatrix}V_{input}(t)\\0\end{bmatrix}=\begin{...