Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

The tag has no usage guidance.

0
votes
1answer
16 views

Minimize squared error of linear function

Let $M$ be a $m \times n$ matrix, $x$ a $n$-vector, $y$ a $m$-vector, and $\|\cdot\|_2$ represent the $L_2$ norm (i.e., Euclidean norm). Given $M,y$, the goal is to find $x$ that minimizes the ...
1
vote
1answer
55 views

Correct weighting in least squares fitting

I am trying to fit some data points $d_i$ to a non-linear model function $m_i$, which depends on a number of fit parameters $f_k$ (I want to determine these) and also on some known, constant values $...
3
votes
0answers
71 views

Nonlinear least squares and regularization

Consider the nonlinear least-squares minimization of a vector of $n$ residuals $\mathbf{f}$ in $p$ parameters $\mathbf{x}$: $$ \min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2 $$ This can be done with ...
0
votes
1answer
65 views

Minimize cost with Levenberg-Marquart method

I want to minimize a cost function of the form, $$ \min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right) $$ ...
3
votes
1answer
70 views

Nonlinear least squares when some parameters are linear

Consider the least squares problem, $$ \min_{\mathbf{a},\mathbf{b}} || \mathbf{f}(\mathbf{a},\mathbf{b})||^2 $$ where $\mathbf{a},\mathbf{b}$ represent the unknown parameters to be found. In my ...
1
vote
0answers
30 views

B-splines least squares with equality constraints

Can someone recommend the best way to solve a least squares fitting problem with B-splines, with additional equality constraints? I want to solve: $$ \min_x || b - A x ||^2, \textrm{subject to: } C x =...
3
votes
0answers
34 views

Reweighted least squares factorization

This is a continuation of the question asked here. I want to solve numerous least squares systems of the form $$ D_i A x \approx D_i b $$ where $D_i$ are $m \times m$ diagonal matrices with positive ...
4
votes
1answer
69 views

Method to Efficiently Solve “Centered” Least Squares without centering “A”

Suppose I want to solve $$\text{arg min}_x \frac{1}{2}\|\tilde{A}x - b\|_2^2 + \frac{1}{2}\|x - c\|_2^2$$ where $A$ is a wide sparse matrix and $\tilde{A} = A C_n = A (I - \mathbf{1}\, \mathbf{1}^T/...
2
votes
0answers
197 views

Convergence of a very large non-linear least squares optimization

(note: I also posted this question on stackoverflow before finding this community here, which seems a better place for it) I'm trying to solve the following problem: I have a lot (~80000) surface ...
7
votes
1answer
165 views

Least Squares with Dense-Block Diagonal Structure

I need to solve a least squares problem that takes the following form: $$p = \arg \min_{x}\Vert J V x - y \Vert_2, $$ where $J \in \mathbb{R}^{N \times N}$ is a general dense matrix, and $V \in \...
4
votes
1answer
186 views

How to solve the following Frobenius norm-minimization problem?

Background We know how to solve the following minimization problem $$ \min_{X} \lVert AX - B \rVert_F^2 $$ But what about the extended version? $$ \min_{X} \lVert A \begin{bmatrix} X & X^2 \...
1
vote
0answers
24 views

How to model a non-linear least-squares problem for triangles

I have a non-linear least-squares problem to solve and with my current modeling the solver is either very slow or does not converge to a correct solution. For the problem I need to minimize an energy ...
3
votes
1answer
175 views

How to solve the inverse problem of least-squares?

Focusing on following least squares problem: $$\min\limits_{V} \lVert Z - WV \rVert _{_F}^2$$ $$Z∈{R}^{m\times n},\quad W∈{R}^{m\times k},\quad V∈{R}^{k\times n},\quad k\lt m\lt n $$ This problem ...
4
votes
2answers
173 views

treating “almost linear” nonlinear least-squares problems

As a model for a nonlinear least-squares problem with a large linear part problem, consider $$ \Delta u = 0 \quad\text{in } \Omega,\\ n\cdot\nabla u = 0 \quad \text{on } \Gamma,\\ (u(x_i) - u(y_i))^2 =...
4
votes
1answer
109 views

Nonlinear least-squares solvers vs. generic minimization

A nonlinear least-squares problem with $F:\mathbb{R}^m\to\mathbb{R}^n$, $$ F(x) \to \min_x \quad (\text{in the least-squares sense}) $$ really means minimizing $$ \frac{1}{2} \|F(x)\|^2 \to \min_x. $$ ...
0
votes
1answer
101 views

Using least squares for computing gradients

I am developing a code where I am using the least squares method to compute gradients. Generally, we use least squares to obtain some model based on a set of data (${q_1 \cdots q_N}$) at locations ($...
1
vote
0answers
33 views

fitting exponential versus exponential w/ power

I have two models which I would like to investigate for my data. One form is: \begin{equation} \label{one} f(r) = A e^{-B r} \end{equation} and the second is: \begin{equation} \label{two} g(r)...
4
votes
1answer
90 views

Obtaining a feasible solution for underdetermined system of linear equations satisfying inequality constraints

I would like to obtain a feasible solution for an under-determined system of linear equations, $$Ax=b$$ where, $A \in \mathbb{R}^{7\times9}, \, x \in \mathbb{R}^{9\times1}\text{and } b\in\mathbb{R}^...
1
vote
0answers
58 views

Stable Method of orthogonal projection onto a subspace with the help of Moore-Penrose inverse,

Projection of a vector $v$ onto the column space of a matrix $A$ is given by $AA^\dagger v$. From the definition of Moore-Penrose Inverse we know that $AA^\dagger v = (A^T)^\dagger A^T v $. Below is ...
3
votes
0answers
67 views

Backward stable algorithm to get orthogonal projection onto the column space of a matrix

I have to find the orthogonal projection of a vector $b$ onto the matrix $A$ of size $m \times n$. In my application, I don't have the luxury of calculating the QR factorization. All I have are ...
3
votes
1answer
113 views

Solving sparse least squares system with limited memory

This was a question on a past final that we can't figure out. Take the least squares system $$\min_x ||Ax-b||_2\, ,$$ where $A\in\mathbb{R}^{mxn}$, $m<n$, and A is full rank. A has $\mathcal{O}(n)...
0
votes
2answers
51 views

Finding parameters numerically

I suspect that a function $f(x,y)$ is of the form $f(x,y)=a(bx+c)^{dy+e}$. I have access to several values of $f(x,y)$. How do I proceed numerically to find the parameters $\{a,b,c,d,e\}$? By ...
0
votes
1answer
119 views

Linear Least-Squares Point-to-Plane ICP degenerative case

I'm trying to implement Linear Least-Squares Optimization for Point-to-Plane ICP Surface Registration Paper describes linear approximation for point to plane distance for rigid ICP. This approach is ...
1
vote
1answer
68 views

Solving for $C$ in $Q = YCZ$ using least squares in Matlab

I am trying to solve for the matrix $C$ in $Q = YCZ$ in matlab. I have preliminary results but they don't seem realistic. Here, $Q$ is $n \times m-1$, $Y$ is $n \times p$, $C$ is $p \times m$ and $Z$ ...
4
votes
0answers
116 views

Solve ill-posed linear system without transposing matrices?

I am attempting to use an iterative solver to solve $p$ in $$ Jp = -r $$ where $J$ is an $m\times m$ matrix. Unfortunately, this is an ill-posed problem and possibly infinite versions of $p$ exist. ...
3
votes
1answer
74 views

Description of algorithm for small scale linear least squares with box constraints

I have small scale dense least squares problem with box constraints $$\mbox{argmin}||Ax - b||^2 \quad $$ $$\mbox{subject to} \quad l_i \leq x_i \leq u_i,$$ Number of variables is about 10-50, ...
3
votes
1answer
71 views

Solving multiple least-square problems with the same constraints

The following least-square problem can be solved efficiently (e.g. using matlab's lsqlin): $$\vec{x}^*=\arg\min_\vec{x} ||C\vec{x}-\vec{t}||^2\,\ s.t.\ Ax \le \vec{b}$$ where the parameters of the ...
1
vote
2answers
41 views

Optimisation of purely integer quantity with bound-constraints for a 1D expensive function whose analytical form is not available

I have a computationally expensive objective function, whose analytical form is not available. The only input argument to the objective function is an integer variable. The goal is to compute the ...
1
vote
1answer
125 views

Least square with rectangular function

I have the function $c(t) = A \cdot \cos \left(\dfrac{2\pi}{\tau} \cdot t + \phi \right) $ which is used to define $ T(t) = \begin{cases} M + c(t), & c(t) > 0 \\ M, &c(t) \leq 0. \end{...
6
votes
3answers
720 views

Least Squares: Numerically, is solving normal equations okay for nice matrices?

I have to solve a least squares problem: $$ x=\arg \min\|Ax-b\| $$ where $A$ is a $m\times n$ matrix, $m>n$, and $b\in\mathbb{R}^m$. I always thought that doing this via QR factorization is ...
0
votes
1answer
197 views

How to code gradient descent-based Tikhonov denoising that exactly matches LSQ Tikhonov denoise?

(Note: Corrected code is posted below the original code.) For an exercise in optimization, I am interested in coding a simple example from scratch: a Tikhonov denoising routine using gradient descent,...
2
votes
0answers
37 views

$(max(0, f(x)))^2$ or $(max(0, exp(f(x))))^2$ for soft constraints with Gauss-Newton

I need some kind of inequality constraint in my optimization problems (rude version of SVM for example or skeleton based mesh fitting). However hard constraints is not suitable for me because ...
1
vote
2answers
183 views

Efficient methods to solve large dense singular least square problem (linear system)

I am trying to solve a singular linear least square problem: $$minimize: \phantom{2} ||Ax-b||^2 \\ subject \phantom{2} to: \phantom{2} x \ge 0$$ Here $ A \in R^{n \times m} $, and $ n\lt m$. here m ...
5
votes
1answer
124 views

Pseudoinverse of perturbed matrix

How does the pseudo inverse of a full column rank matrix change if I rescale a single row? In more detail the problem is the following: We have a fixed matrix $V$ with linear independent columns and ...
3
votes
0answers
26 views

Choosing suitable polynomial degree based on information in advection stencil

I'm working on a finite volume advection scheme for unstructured meshes which uses a multidimensional polynomial weighted least squares fit for interpolating from cell centres onto faces. In 2D, the ...
2
votes
1answer
90 views

Parameter reduction algorithm for least square model

Question I am performing least squares fitting using an objective function of the form $f(\mathbf{x})$ where $\mathbf{x}$ is a vector of parameters containing around 20 elements. The model function ...
5
votes
1answer
102 views

Large scale triangular least squares

I have to solve the following least squares problem: \begin{equation} \| \left[ \begin{smallmatrix} \mathbf{L} \\ \mathbf{I} \end{smallmatrix} \right]\mathbf{x} - \mathbf{b} \|_2^2 \end{equation} ...
4
votes
1answer
175 views

Best solver/preconditioner for least-squares finite element method

I have seen a lot of literature, lecture videos, etc. on solvers/preconditioners for non-symmetric and/or indefinite systems. However, now I want to solve the mixed poisson/Darcy equation using the ...
5
votes
2answers
1k views

How to determine whether two cylinders intersect or not?

Considering any two cylinders, defined as: the center of their bottoms $A_i$, the radius of their bottom $R_i$, the unit vector $W_i$ of their axis direction, and the length $L_i$ of the cylinders, ...
2
votes
1answer
169 views

Minimizing Cost Functions using Iterative Least Squares

I am currently trying to use iterative least squares to solve a system, $y = Hx + v$ where $y$ is a vector of observations, $H$ is the design matrix, and $v$ is the observation error. From my ...
1
vote
1answer
110 views

Matlab interface for minpack2

Is there a Matlab interface for minpack2? minpack2 http://ftp.mcs.anl.gov/pub/MINPACK-2 consists of Fortran programs , but i would like to call the test problems from matlab. What can i do ?
2
votes
0answers
101 views

Calculating theoretical order of accuracy of least squares fit advection scheme

I'm familiar with finding the order of accuracy using von Neumann analysis for finite difference schemes formulated using Taylor series expansions. But is there a similar technique for finding the ...
2
votes
0answers
59 views

Fortran solver for the Sparse LSE problem

I was wondering if there is a Fortran library that contains a solver for the Sparse LSE(linear equality-constrained least squares) problem $$ min_{x}\|Cx-d\|^2 \text{ subject to } Ax=b $$ where $A$ ...
1
vote
1answer
255 views

Least Square fit with Double Zernike Polynomials

I need to describe the optical aberrations of my set-up with Zernike polynomials. To do this, I want to fit the first 15 polynomials in the following way on my data: $$\chi^2 = \sum_k\left(\beta_k^x -...
2
votes
1answer
190 views

Projecting onto convex shapes - best fit convex polygon

I am interested in studying a problem of the form $\min F(\Omega)$ where $\Omega$ varies in the class of convex, open sets in the plane. An idea is to deform $\Omega$ at each step using a steepest ...
1
vote
0answers
55 views

linear solution of curve fitting on multiple linear functions differing by a multiplier

I am facing the following problem. I know nonlinear least squares can provide a solution but I am wondering if a linear way to solve this data fitting problem may exists. This is my input dataset: I'...
1
vote
3answers
272 views

rank-deficient NNLS

I want to find the minimum-norm solution to a rank-deficient least-squares problem, subject to positivity constraints, e.g. $$\min_x\ \|x\|^2 \quad s.t.\quad Ax = b,\ x \geq 0$$ where $A$ is large, ...
3
votes
0answers
908 views

Polynomial Fitting with Least Squares using Numpy and Scipy

I am trying to fit data to a polynomial using Python - Numpy. The points, with lines sketched above them are as in the picture. I am trying to fit those points to a polynomial of 4. or 5. degree. ...
7
votes
2answers
5k views

Least Squares and Fourier Series

I have a little bit of problem figuring out the relation between Fourier series and Least Squares. As far as I understand, LS is a way of minimizing the quadratic error between a measured value $y_i$ ...
3
votes
1answer
924 views

solving a linearly-constrained sparse linear least-squares problem

[ question reposted from https://math.stackexchange.com/questions/786612/solving-a-linearly-constrained-sparse-linear-least-squares-problem ] Given the system of equations $Ax=b$, subject to $Cx\le ...