Questions tagged [least-squares]

For questions focused on implementing or applying least-squares regression.

116 questions
Filter by
Sorted by
Tagged with
52 views

151 views

Solve ill-posed linear system without transposing matrices?

I am attempting to use an iterative solver to solve $p$ in $$Jp = -r$$ where $J$ is an $m\times m$ matrix ($m$ is in the order of $10^5$ and never explicitly stored). $J$ is a dense matrix ...
288 views

Update for QR factorization least squares

I found after some research that the most numerically stable way to solve the least squares problem is through QR factorization. For $n$ number of observations and $p$ number of parameters it takes ...
66 views

Augmented Dickey Fuller (ADF) test statistics GPU formulation

I have followed different sources of information and achieved the following formulation for the ADF $t$ test statistics. I implemented it to run several hundred thousands of ...
2k views

Cholesky decomposition vs LDL decomposition

In different books and on Wikipedia, you can see mentions of Cholesky decomposition and only sometimes of LDL decomposition. As far as I understand, LDL decomposition can be applied to a broader ...
58 views

53 views

Nonlinear least squares optimized Jacobian calculation

I have a nonlinear least squares problem, in which I am trying to minimize residuals which can be divided into four classes: $$\min_x ||\epsilon(x)||^2 + ||\xi(x)||^2 + ||\delta(x)||^2 + ||s(x)||^2$$...
132 views

Spline regularization

I am fitting some B-splines to data, but the data has a "gap" region where the spline is less constrained by the data. I want to devise a regularization scheme to help prevent the spline from ...
131 views

"Geometry of ill-conditioning" for least-squares problems

It is an idea that dates back to Demmel, 1987 that the condition number of a problem is often related to the distance to the closest ill-posed problems. In Section 3 of the above paper, the author ...
445 views

Minimize cost with Levenberg-Marquart method

I want to minimize a cost function of the form, $$\min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right)$$ ...
594 views

Fastest algorithm for pseudoinverse of skinny matrices

For a performance-sensitive problem, I need to compute the pseudoinverse of a skinny matrix (#rows = 1000–10000, #cols= 10–20). I already employ the traditional SVD econ method. For some problem ...
229 views

What is a good library in Python for correlated fits in both the $x$ and $y$ data?

I have $x$ and $y$ data, both of which have their own covariance matrices. scipy.optimize.curve_fit will accept a covariance matrix for the $y$ data, called ...
396 views

Preconditionning for solving a non-linear system of equations with least squares

I am trying to solve a large system of non-linear equations (about a few hundred equations and variable but with less variable than equations). Given that the system is really sparse and large I am ...
Consider the regularized least squares problem $$\min_x || b - A x ||^2 + \lambda^2 ||x||^2$$ which is equivalent to $$\min_x \left|\left| \pmatrix{b \\ 0} - \pmatrix{A \\ \lambda I} x \right|\... 1answer 344 views How to use LAPACK function (DGELSY) in Fortran I am trying to use Least Squares Minimization to solve a the matrix problem: b = A*x for x. The system is overdetermined, and A is a dense matrix. In the LAPACK library, I believe the routine DGELSY ... 1answer 297 views Pivoted Cholesky vs Modified Cholesky I am solving nonlinear least squares problems with the normal equations approach, so on each iteration, I need to solve:$$ J^T J \delta = -J^T f $$for the step \delta, where J is a large (... 1answer 282 views Formulation of the least-squares parameter estimation problem I have a system of 10 ordinary differential equations of the form,$$\frac{dy_1}{dt} = f1(V1,k1,y1,y2)\\ \vdots \\ \frac{dy_{10}}{dt} = f_{10}(V_{10},k_{10},y_{9},y_{10}) $$I want to estimate the ... 0answers 128 views Parameter estimation using fmincon This is a follow up to my previous question posted here. I am solving an optimization problem using fmincon in MATLAB. There are no equality constraints in my model.... 1answer 39 views Minimize squared error of linear function Let M be a m \times n matrix, x a n-vector, y a m-vector, and \|\cdot\|_2 represent the L_2 norm (i.e., Euclidean norm). Given M,y, the goal is to find x that minimizes the ... 1answer 70 views Correct weighting in least squares fitting I am trying to fit some data points d_i to a non-linear model function m_i, which depends on a number of fit parameters f_k (I want to determine these) and also on some known, constant values ... 3answers 1k views Least squares approximation question I am taking a course on scientific computation, and we just went over least squares approximation. My question is specifically about approximating using polynomials. I understand that if you have n+1 ... 0answers 292 views Nonlinear least squares and regularization Consider the nonlinear least-squares minimization of a vector of n residuals \mathbf{f} in p parameters \mathbf{x}:$$ \min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2 $$This can be done with ... 1answer 126 views Nonlinear least squares when some parameters are linear Consider the least squares problem,$$ \min_{\mathbf{a},\mathbf{b}} || \mathbf{f}(\mathbf{a},\mathbf{b})||^2 $$where \mathbf{a},\mathbf{b} represent the unknown parameters to be found. In my ... 0answers 65 views B-splines least squares with equality constraints Can someone recommend the best way to solve a least squares fitting problem with B-splines, with additional equality constraints? I want to solve:$$ \min_x || b - A x ||^2, \textrm{subject to: } C x =...
This is a continuation of the question asked here. I want to solve numerous least squares systems of the form $$D_i A x \approx D_i b$$ where $D_i$ are $m \times m$ diagonal matrices with positive ...