Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
This site is currently in read-only mode; we'll return with full functionality soon. Follow @StackStatus or visit our status blog for more info.

Questions tagged [linear-solver]

Referring to methods for solving linear systems of equations.

49
votes
4answers
7k views

What guidelines should I follow when choosing a sparse linear system solver?

Sparse linear systems turn up with increasing frequency in applications. One has a lot of routines to choose from for solving these systems. At the highest level, there is a watershed between direct (...
30
votes
3answers
13k views

How to choose a method for solving linear equations

To my knowledge, there are 4 ways to solving a system of linear equations (correct me if there are more): If the system matrix is a full-rank square matrix, you can use Cramer’s Rule; Compute the ...
22
votes
3answers
734 views

Solving $(G^TA^{-1}G)x = b$ without inverting $A$

I have matrices $A$ and $G$. $A$ is sparse and is $n\times n$ with $n$ very large (can be on the order of several million.) $G$ is an $n\times m$ tall matrix with $m$ rather small ($1 \lt m \lt 1000$) ...
21
votes
2answers
9k views

Libraries for solving sparse linear systems

There are a number of different libraries out there that solve a sparse linear system of equations, however I'm finding it difficult to figure out what the differences are. As far as I can tell there ...
16
votes
3answers
3k views

Problems where Conjugate gradient works much better than GMRES

I am interested in cases where Conjugate gradient works much better than GMRES method. In general, CG is preferable choice in many cases of SPD (symmetric-positive-definite) because it requires less ...
15
votes
1answer
687 views

Are there any open source inverse-based multilevel ILU implementations?

I am very impressed with the serial performance of multilevel inverse-based ILU preconditioners, particularly for heterogeneous Helmholtz, but I am surprised to not be able to find any open source ...
14
votes
3answers
4k views

What are the symptoms of ill-conditioning when using direct methods?

Suppose we have a linear system and we know nothing about its conditioning and have no preliminary information about the solution. We blindly apply Gaussian elimination and obtain some solution $x$. ...
13
votes
3answers
3k views

Is the Thomas algorithm the fastest way to solve a symmetric diagonally dominant sparse tridiagonal linear system

I am wondering if the Thomas algorithm is the fastest way (provably?) to solve a symmetric diagonally dominate sparse tridiagonal system in terms of algorithmic complexity (not looking for ...
13
votes
1answer
727 views

How is Krylov-accelerated Multigrid (using MG as a preconditioner) motivated?

Multigrid (MG) may be used to solve a linear system $Ax=b$ by constructing an initial guess $x_0$ and repeating the following for $i=0,1..$ until convergence: Compute the residual $r_i = b-Ax_i$ ...
12
votes
5answers
2k views

Repeatedly solving $\mathbf{A} \mathbf{x} = \mathbf{b}$ with same $\mathbf{A}$, different $\mathbf{b}$

I am using MATLAB to solve a problem that involves solving $\mathbf{A} \mathbf{x}=\mathbf{b}$ at every timestep, where $\mathbf{b}$ changes with time. Right now, I am accomplishing this using MATLAB'...
12
votes
3answers
2k views

Sparse linear solver for many right-hand sides

I need to solve the same sparse linear system (300x300 to 1000x1000) with many right hand sides (300 to 1000). In addition to this first problem, I would also like to solve different systems, but with ...
12
votes
2answers
638 views

preconditioning a krylov method with another krylov method

In method like gmres or bicgstab it could be attractive to use another krylov method as a preconditioner. After all they are easy to implement in a matrix-free way and in a parallel environment. For ...
12
votes
3answers
4k views

Single versus double floating-point precision

Single precision floating point numbers take up half the memory and on modern machines (even on GPUs it seems) operations can be done with them at almost twice the speed compared to double precision. ...
12
votes
2answers
5k views

solve $xA=b$ for $x$ using LAPACK and BLAS

I am porting an existing code from MATLAB to C++ and have a linear system to solve $xA=b$ (rather than the more typical form $Ax=b$) The matrix $A$ is dense, and of general form, but is no larger ...
12
votes
2answers
299 views

Efficient preconditioner for Augmented Lagrangian

I want to solve a non-linear problem with non-linear equality constrains and I'm using a augmented Lagrangian with a penalty regularization term that, as well known, spoils the condition number of my ...
11
votes
4answers
4k views

What are the best Python packages/interfaces to sparse direct solvers?

Please list the Python package (petsc4py, etc...) and the sparse direct solvers it supports. One (community-wiki) answer per package, please.
11
votes
1answer
567 views

Solving huge dense linear system?

Is there any hope in solving the following linear system efficiently with an iterative method? $A \in \mathbb{R}^{n \times n}, x \in \mathbb{R}^n, b \in \mathbb{R}^n \text{, with } n > 10^6$ $Ax=...
11
votes
2answers
340 views

Do black-box preconditioners for matrix-free methods exist?

Jacobian-Free Newton-Krylov (JFNK) methods, and Krylov methods in general, can be very useful because they don't require explicit storage or construction of a matrix, only the results of matrix-vector ...
11
votes
1answer
654 views

Projecting out the null-space of $A$ from $b$ in $Ax=b$

Given the system $$Ax=b,$$ where $A\in\mathbb{R}^{n\times n}$, I read that, in case Jacobi iteration is used as a solver, the method will not converge if $b$ has a non-zero component in the null-space ...
10
votes
5answers
9k views

Best choice of solver for a large sparse symmetric (but not positive definite) system

I am presently working on solving very large symmetric (but not positive definite) systems, generated by some certain algorithms. These matrices have a nice block sparsity which can be used for ...
10
votes
1answer
13k views

in matlab, what differences are between linsolve and mldivide?

in matlab, both linsolve and mldivide are used for solving a system of linear equations, in all of determined, overdetermined and underdetermined cases. Reading their documents, I was wondering what ...
10
votes
2answers
956 views

Which iterative linear solvers converge for positive semidefinite matrices?

I want to know which of the classic linear solvers (e.g Gauss-Seidel, Jacobi, SOR) are guaranteed to converge for the problem $Ax=b$ where $A$ is positive semi definite and of course $b \in im(A)$ (...
10
votes
2answers
1k views

Selection of linear solver for GPGPU computation (OpenCL)

I have already developed a working solution of the Finite Element Method to solve heat transfer problems using GPU and OpenCL using the Conjugate Gradient method. The main disadvantage of this method ...
9
votes
4answers
334 views

Fast explicit solution for $\mathbf{A}\mathbf{x} = \mathbf{b}$, $ \mathbf{b} \in \mathbf{R}^3$, low condition number

I am looking for a fast (dare I say optimal?) explicit solution the 3x3 linear real problem, $\mathbf{A}\mathbf{x} = \mathbf{b}$, $\mathbf{A} \in \mathbf{R}^{3 \times 3}, \mathbf{b} \in \mathbf{R}^{3}$...
9
votes
3answers
4k views

Solving a sparse and highly ill-conditioned system

I intend to solve Ax = b where A is complex, sparse, unsymmetric and highly ill-conditioned (condition number ~ 1E+20) square or rectangular matrix. I have been able to solve the system with ZGELSS in ...
9
votes
3answers
1k views

Recommendations for a lightweight/no-install C or C++ based dense linear algebra solver

Most of my programming is one-off research codes in C for my own use. I have never distributed any code to other than close collaborators. I have developed an algorithm that I am publishing in a ...
9
votes
3answers
315 views

fastest linear system solve for small square matrices (10x10)

I am very interested in optimizing the hell out of linear system solving for small matrices (10x10), sometimes called tiny matrices. Is there a ready solution for this? The matrix can be assumed ...
9
votes
2answers
1k views

Safe application of iterative methods on diagonally dominant matrices

Suppose the following linear system is given $$Lx=c,\tag1$$ where $L$ is the weighted Laplacian known to be positive $semi-$definite with a one dimensional null space spanned by $1_n=(1,\dots,1)\in\...
8
votes
2answers
1k views

Does the matrix condition number affect accuracy of iterative linear solvers?

I have a rather specific question regarding the condition number. I run FEM simulations which have multiple length scales to them which results in a huge disparity between the largest entries and the ...
8
votes
2answers
1k views

Taxonomy of ILU preconditioners

I learned that for BiCGStab solver for sparse linear systems it's pretty much always necessary to use a preconditioner. I realized by now that choosing a good one is problem dependent. Surfing the ...
8
votes
2answers
127 views

Initial guesses for perturbed linear systems

Suppose you solve a linear system $Au = f$ by an iterative method, e.g. conjugate gradients or Richardson iteration. Then you try to solve a linear system that is slightly perturbed in the matrix and ...
8
votes
1answer
1k views

Solving system of linear equations with cyclic tridiagonal matrix

I have this problem in my textbook: Suggest efficient algorithm for solving system of linear equations with cyclic three-diagonal matrix, that is of the form: \begin{bmatrix} a_1&b_1&0&...
8
votes
3answers
3k views

Sort of problems where SOR is faster than Gauss-Seidel?

Is there any simple rule of thumb to say if it is worth to do SOR instead of Gauss-Seidel? ( and possible way how to estimate realxation parameter $\omega$) I mean just by looking on the matrix, or ...
8
votes
1answer
1k views

preconditioner for a matrix-free method to solve Ax=b

I need to solve Ax=b, but I realize that even if it is sparse, storing the matrix coefficients of my problem will take too much memory. So now I'm considering using a matrix-free method, because the ...
8
votes
1answer
528 views

Using algebraic multigrid for preconditioning convection-diffusion operators

I implemented a Navier Stokes based on FEM discretization and PETSc for solving the linear system of equations. To create an efficient solution procedure, I follow the paper "Efficient preconditioning ...
8
votes
0answers
178 views

Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Problem Solving a non-linear system of equations. The number of variables is the same as the number of equations. When I fix a set of variables (say $\vec{y}$) and keep another set free (say $\vec{...
8
votes
0answers
208 views

Wanted: sequences of linear systems for recycling Krylov solver analysis

In the solution of sequences of linear systems $$A_ix_i=b_i\quad\text{for}\quad i=1,2,\dots$$ with Krylov subspace methods, data can be recycled from already solved linear systems in order to speed up ...
7
votes
1answer
373 views

“Cookbook” about iterative linear solvers and preconditioners

I'm using a lot of linear solvers and preconditioners, but mostly, they are magical black boxes to me. Since I'll also have to implement some of them in future, I would like to learn a bit more, ...
7
votes
3answers
642 views

Solving shifted linear systems with LU factorization

I am interested in solving a sequence of shifted linear systems $(A+\sigma I)x = b$ for various values of $\sigma$. The matrix $A$ is sparse and not too large, so I have its LU factorization available....
7
votes
3answers
243 views

Wanting to learn about matrix solvers

Edit: I was advised to replace the question with a more specific one. Coming from a very theoretical background, I'm pretty ignorant about what practical matrix solvers exist. (I have been, and will ...
7
votes
1answer
2k views

full rank update to cholesky decomposition

Let $A$ be a real, symmetric, positive definite matrix. It has at least 500 rows, possibly much more. I compute its Cholesky decomposition, which allows me to calculate $det(A)$ $A^{-1}X$ for some ...
7
votes
1answer
2k views

Solving linear systems by fft

I read in a paper and also at wiki that we can solve the system $$Ax=B$$ by Fast Fourier Transform, where $A$ is a circulant matrix. The solution is $$x=\mathtt{ifft}(\mathtt{fft}(B)/\mathtt{fft}(a))$...
7
votes
1answer
1k views

Solving for null space of a matrix with mkl LAPACK

I want to find a solution for $xA=0$, where $A$ is a square matrix. I know that most of the LAPACK routines solve for $Ax=b$. So I take $A^T$ as a, and set $b=0$. I have an additional restriction of $\...
7
votes
2answers
2k views

Solving Linear Systems in Julia

To give you some context, I am currently implementing a simple finite element solver in Julia. I am getting run-times that are 70% of a Matlab code. (Both codes are essential equivalent in structure.)...
7
votes
1answer
77 views

Bad scaling versus collinearity

I was trying to solve a linear system: $$ \mathbf{A}\mathbf{x} = \mathbf{y} $$ but the conditioning number was quite bad (around $10^{17}$). I thought that the system was singular, but after scaling ...
7
votes
1answer
434 views

Which preconditioning for large linear elasticity problem?

The problem I want to solve is the displacement formulation of the linear elasticity : $$ \nabla \cdot \sigma = 0 \quad \text{in} \quad \Omega \\ \sigma = \lambda ( \nabla \cdot u ) I + \mu (\nabla \...
7
votes
1answer
121 views

What are some ideas to preprocess / precondition the following linear system?

Let $A\in \mathbb{R}^{n\times n}$ symmetric and positive semidefinite, and $\omega\in \mathbb{R}\setminus\{0\}$. I am interested in solving the following linear system for a range of values of $\...
7
votes
1answer
322 views

Least-squares for a diagonal matrix

This is a follow-up to a different question I asked with more detail. For $v\in\mathbb{R}^n$, denote $D_v\in\mathbb{R}^n$ as the diagonal matrix with elements in $v$. Given a "tall" matrix $B\in\...
6
votes
3answers
295 views

role of initial guess for iterative linear solver

Suppose we use a preconditioned iterative solver for a linear system. If the initial state for the solver can be chosen very close to the exact solution - does this reduce requirements for the ...
6
votes
3answers
4k views

How to find QR decomposition of a rectangular matrix in overdetermined linear system solution?

While trying to find cell-centered gradients in finite volume method computation of incompressible fluid flow I get over-determined linear system. This is a well known "cell based least-square" ...