# Questions tagged [least-squares]

For questions focused on implementing or applying least-squares regression.

116 questions
Filter by
Sorted by
Tagged with
52 views

151 views

### Solve ill-posed linear system without transposing matrices?

I am attempting to use an iterative solver to solve $p$ in $$Jp = -r$$ where $J$ is an $m\times m$ matrix ($m$ is in the order of $10^5$ and never explicitly stored). $J$ is a dense matrix ...
288 views

### Update for QR factorization least squares

I found after some research that the most numerically stable way to solve the least squares problem is through QR factorization. For $n$ number of observations and $p$ number of parameters it takes ...
66 views

### Augmented Dickey Fuller (ADF) test statistics GPU formulation

I have followed different sources of information and achieved the following formulation for the ADF $t$ test statistics. I implemented it to run several hundred thousands of ...
2k views

### Cholesky decomposition vs LDL decomposition

In different books and on Wikipedia, you can see mentions of Cholesky decomposition and only sometimes of LDL decomposition. As far as I understand, LDL decomposition can be applied to a broader ...
58 views

53 views

### Nonlinear least squares optimized Jacobian calculation

I have a nonlinear least squares problem, in which I am trying to minimize residuals which can be divided into four classes: $$\min_x ||\epsilon(x)||^2 + ||\xi(x)||^2 + ||\delta(x)||^2 + ||s(x)||^2$$...
132 views

### Spline regularization

I am fitting some B-splines to data, but the data has a "gap" region where the spline is less constrained by the data. I want to devise a regularization scheme to help prevent the spline from ...
131 views

### "Geometry of ill-conditioning" for least-squares problems

It is an idea that dates back to Demmel, 1987 that the condition number of a problem is often related to the distance to the closest ill-posed problems. In Section 3 of the above paper, the author ...
445 views

### Minimize cost with Levenberg-Marquart method

I want to minimize a cost function of the form, $$\min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right)$$ ...
594 views

### Fastest algorithm for pseudoinverse of skinny matrices

For a performance-sensitive problem, I need to compute the pseudoinverse of a skinny matrix (#rows = 1000–10000, #cols= 10–20). I already employ the traditional SVD econ method. For some problem ...
229 views

### What is a good library in Python for correlated fits in both the $x$ and $y$ data?

I have $x$ and $y$ data, both of which have their own covariance matrices. scipy.optimize.curve_fit will accept a covariance matrix for the $y$ data, called ...
396 views

### Preconditionning for solving a non-linear system of equations with least squares

I am trying to solve a large system of non-linear equations (about a few hundred equations and variable but with less variable than equations). Given that the system is really sparse and large I am ...
Consider the regularized least squares problem $$\min_x || b - A x ||^2 + \lambda^2 ||x||^2$$ which is equivalent to $$\min_x \left|\left| \pmatrix{b \\ 0} - \pmatrix{A \\ \lambda I} x \right|\... 1answer 344 views ### How to use LAPACK function (DGELSY) in Fortran I am trying to use Least Squares Minimization to solve a the matrix problem: b = A*x for x. The system is overdetermined, and A is a dense matrix. In the LAPACK library, I believe the routine DGELSY ... 1answer 297 views ### Pivoted Cholesky vs Modified Cholesky I am solving nonlinear least squares problems with the normal equations approach, so on each iteration, I need to solve:$$ J^T J \delta = -J^T f $$for the step \delta, where J is a large (... 1answer 282 views ### Formulation of the least-squares parameter estimation problem I have a system of 10 ordinary differential equations of the form,$$\frac{dy_1}{dt} = f1(V1,k1,y1,y2)\\ \vdots \\ \frac{dy_{10}}{dt} = f_{10}(V_{10},k_{10},y_{9},y_{10}) $$I want to estimate the ... 0answers 128 views ### Parameter estimation using fmincon This is a follow up to my previous question posted here. I am solving an optimization problem using fmincon in MATLAB. There are no equality constraints in my model.... 1answer 39 views ### Minimize squared error of linear function Let M be a m \times n matrix, x a n-vector, y a m-vector, and \|\cdot\|_2 represent the L_2 norm (i.e., Euclidean norm). Given M,y, the goal is to find x that minimizes the ... 1answer 70 views ### Correct weighting in least squares fitting I am trying to fit some data points d_i to a non-linear model function m_i, which depends on a number of fit parameters f_k (I want to determine these) and also on some known, constant values ... 3answers 1k views ### Least squares approximation question I am taking a course on scientific computation, and we just went over least squares approximation. My question is specifically about approximating using polynomials. I understand that if you have n+1 ... 0answers 292 views ### Nonlinear least squares and regularization Consider the nonlinear least-squares minimization of a vector of n residuals \mathbf{f} in p parameters \mathbf{x}:$$ \min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2 $$This can be done with ... 1answer 126 views ### Nonlinear least squares when some parameters are linear Consider the least squares problem,$$ \min_{\mathbf{a},\mathbf{b}} || \mathbf{f}(\mathbf{a},\mathbf{b})||^2 $$where \mathbf{a},\mathbf{b} represent the unknown parameters to be found. In my ... 0answers 65 views ### B-splines least squares with equality constraints Can someone recommend the best way to solve a least squares fitting problem with B-splines, with additional equality constraints? I want to solve:$$ \min_x || b - A x ||^2, \textrm{subject to: } C x =...
This is a continuation of the question asked here. I want to solve numerous least squares systems of the form $$D_i A x \approx D_i b$$ where $D_i$ are $m \times m$ diagonal matrices with positive ...