Questions tagged [iterative-method]

A method which produces a sequence of numerical approximations which converges (provided technical conditions are satisfied) to the solution of a problem, generally through repeated applications of some procedure. Examples include Newton's method for root finding, and Jacobi iteration for matrix-vector solves.

Filter by
Sorted by
Tagged with
31
votes
3answers
14k views

How to choose a method for solving linear equations

To my knowledge, there are 4 ways to solving a system of linear equations (correct me if there are more): If the system matrix is a full-rank square matrix, you can use Cramer’s Rule; Compute the ...
26
votes
2answers
11k views

Why is my iterative linear solver not converging?

What can go wrong when using preconditoned Krylov methods from KSP (PETSc's linear solver package) to solve a sparse linear system such as those obtained by discretizing and linearizing partial ...
19
votes
3answers
2k views

What guidelines should I use when searching for good preconditioning methods for a specific problem?

For the solution of large linear systems $Ax=b$ using iterative methods, it is often of interest to introduce preconditioning, e.g. solve instead $M^{-1}(Ax=b)$, where $M$ is here used for left-...
15
votes
1answer
2k views

Can a Krylov subspace method be used as a smoother for multigrid?

As far as I am aware, multigrid solvers use iterative smoothers such as Jacobi, Gauss-Seidel, and SOR to dampen the error at various frequencies. Could a Krylov subspace method (like conjugate ...
15
votes
1answer
2k views

Intuitive motivation for BFGS update

I am teaching a numerical analysis survey class and am seeking motivation for the BFGS method for students with limited background/intuition in optimization! While I don't have time to prove ...
15
votes
1answer
568 views

What is the current state of polynomial preconditioners?

I wonder what has happened to polynomial preconditioners. I am interested in them, because they appear to be comparatively elegant from a mathematical perspective, but as far as I have read in surveys ...
13
votes
3answers
9k views

Understanding the “rate of convergence” for iterative methods

According to Wikipedia the rate of convergence is expressed as a specific ratio of vector norms. I'm trying to understand the difference between "linear" and "quadratic" rates, at different points of ...
13
votes
1answer
262 views

Non-monotonic convergence in fixed-point problem

Background I am solving a variant of the Ornstein-Zernike equation from liquid theory. Abstractly, the problem can be represented as solving the fixed point problem $A c(r)=c(r)$, where $A$ is an ...
12
votes
1answer
539 views

Using fixed point iteration to decouple a system of pde's

Suppose I had a boundary value problem: $$\frac{d^2u}{dx^2} + \frac{dv}{dx}=f \text{ in } \Omega$$ $$\frac{du}{dx} +\frac{d^2v}{dx^2} =g \text{ in } \Omega$$ $$u=h \text{ in } \partial\Omega$$ My ...
11
votes
2answers
307 views

What is the underlying structure of scientific code performance?

Consider two computers with different hardware and software configurations. When running the exact same serial Navier-Stokes code on each platform it takes x and y time to execute one iteration for ...
11
votes
1answer
708 views

Smallest eigenvalue without inverse

Suppose $A\in\mathbb{R}^{n\times n}$ is a symmetric, positive definite matrix. $A$ is big enough that it's expensive to solve $Ax=b$ directly. Is there an iterative algorithm for finding the ...
11
votes
1answer
687 views

Projecting out the null-space of $A$ from $b$ in $Ax=b$

Given the system $$Ax=b,$$ where $A\in\mathbb{R}^{n\times n}$, I read that, in case Jacobi iteration is used as a solver, the method will not converge if $b$ has a non-zero component in the null-space ...
11
votes
1answer
430 views

How to establish that an iterative method for large linear systems is convergent in practice?

In computational science we often encounter large linear systems which we are required to solve by some (efficient) means, e.g. by either direct or iterative methods. If we focus on the latter, how ...
10
votes
3answers
1k views

Basin of attraction for Newton's method

Newton's method for solving nonlinear equations is known to converge quadratically when the starting guess is "sufficiently close" to the solution. What is "sufficiently close"? Is there literature ...
10
votes
1answer
6k views

Are there any heuristics for optimizing the successive over-relaxation (SOR) method?

As I understand it, successive over relaxation works by choosing a parameter $0\leq\omega\leq2$ and using a linear combination of a (quasi) Gauss-Seidel iteration and the value at the previous ...
10
votes
3answers
914 views

Why does iteratively solving the Hartree-Fock equations result in convergence?

In the Hartree-Fock self-consistent field method of solving the time-independent electronic Schroedinger equation, we seek to minimize the ground state energy, $E_{0}$, of a system of electrons in an ...
10
votes
2answers
1k views

Which iterative linear solvers converge for positive semidefinite matrices?

I want to know which of the classic linear solvers (e.g Gauss-Seidel, Jacobi, SOR) are guaranteed to converge for the problem $Ax=b$ where $A$ is positive semi definite and of course $b \in im(A)$ (...
10
votes
1answer
148 views

What iterative method can effectively solve a linear system with this kind of spectrum

I have a linear system with matrix which eigenvalues are uniformly distributed on the unit circle like this: Is it possible to solve this kind of system effectively by iterative method, maybe with ...
9
votes
2answers
4k views

How to remove Rigid Body Motions in Linear Elasticity?

I want to solve $K u = b$ where $K$ is my stiffness matrix. However some constraints may be missing an therefore some rigid body motion may be still present in the system (due to eigenvalue zero). ...
9
votes
4answers
1k views

What is a robust, iterative solver for large 3-d linear-elastic problems?

I'm diving into the fascinating world of finite element analysis and would like to solve a large thermo-mechanical problem (only thermal $\rightarrow$ mechanical, no feedback). For the mechanical ...
9
votes
3answers
841 views

Iterative methods for indefinite systems without block structure

Indefinite systems of matrices appear for example in the discretization of saddle point problems by mixed finite elements. The system matrix can then be put in the form $$\begin{pmatrix} A & B^t \...
9
votes
2answers
1k views

Safe application of iterative methods on diagonally dominant matrices

Suppose the following linear system is given $$Lx=c,\tag1$$ where $L$ is the weighted Laplacian known to be positive $semi-$definite with a one dimensional null space spanned by $1_n=(1,\dots,1)\in\...
9
votes
3answers
4k views

Sort of problems where SOR is faster than Gauss-Seidel?

Is there any simple rule of thumb to say if it is worth to do SOR instead of Gauss-Seidel? ( and possible way how to estimate realxation parameter $\omega$) I mean just by looking on the matrix, or ...
9
votes
1answer
329 views

Solving a system with a small rank diagonal update

Suppose I have the original large, sparse linear system: $A\textbf{x}_0=\textbf{b}_0$. Now, I do not have $A^{-1}$ as A is too large to factor or any sort of decomposition of $A$, but assume that I ...
9
votes
1answer
250 views

Iterative “solver” for $x^t \Sigma^{-1} x$

I can't imagine I'm the first to think about the following problem, so I'll be satisfied with a reference (but a complete, detailed answer is always appreciated): Say you have a symmetric positive ...
8
votes
5answers
360 views

Iterative solution to a nonlinear equation

I appologize in advance if this question is silly. I need to compute the root of \begin{equation} u -f(u) =0 \end{equation} Where $u$ is a real vector and $f(u)$ is a real-vector valued function. ...
8
votes
3answers
508 views

How does the QR algorithm applied to a real matrix returns complex eigenvalues?

I'm a noob into eigenvalues algorithms, but something call my attention. QR algorithm works with real/complex matrices producing real/complex eigenvalues. However, it can not produce complex ...
8
votes
1answer
3k views

What is a good stop criterion when using an iterative method to find eigenvalues?

I read this answer, and realized I have been using the difference between sucessive iterates to define a stop criterion for an iterative method of finding eigenvalues/vectors. What are good stop ...
8
votes
1answer
339 views

Is the sparsity pattern of a linear system important for iterative (KSP) solvers?

Pretty much the question. Given a general sparse, non-symmetric (both numerically and structurally) matrix, how important is the sparsity pattern (i.e. row/column permutation of matrix/vector) for ...
8
votes
2answers
289 views

Looking for an algorithm that allocates climbing hold colors to wall sectors

I posted this question earlier on stackoverflow, where it was closed as off-topic. I hope it survives here. I our climbing gym, the routes need to be re-set from time to time. The following rules ...
7
votes
1answer
2k views

full rank update to cholesky decomposition

Let $A$ be a real, symmetric, positive definite matrix. It has at least 500 rows, possibly much more. I compute its Cholesky decomposition, which allows me to calculate $det(A)$ $A^{-1}X$ for some ...
7
votes
2answers
2k views

Markov (Chain) image generators?

Markov Chains can be used to generate, or auto-complete, text. https://en.wikipedia.org/wiki/Markov_chain#Markov_text_generators Training text is read, and some information about the text is ...
7
votes
2answers
1k views

Convergence/stagnation of BiCGStab(l)

I am solving 3D time-harmonic Maxwell FDFD problems (which result in huge sparse linear systems) using BiCGStab(l). I have tried out a bunch of different methods and for my specific use case, it seems ...
7
votes
2answers
172 views

Lanczos algorithms for Hermitian system with Toeplitz kernel

Basically, I am trying to compute the SVD of a large Hermitian matrix $H$ using Lanczos iteration, while $H$ consists if a Toeplitz kernel $K$, which should be able to help speed up the matrix-vector ...
7
votes
1answer
894 views

Integer vs float multiplication performance, modern CPUs

Are there benchmarks for how many multiplications of various integer types compared to floating point types can be achieved per second on modern CPUs? I'm trying to get some hint if it would be ...
7
votes
1answer
559 views

Decreasing spectral radius

I have a $4\times 4$ matrix and I want to use Jacobi iteration on it. Right now the spectral radius is higher than $1$. I know that the method is guaranteed to converge if the matrix is diagonally ...
7
votes
1answer
582 views

Jacobi iteration to reduce the quadratic function

Given certain function $f(X)$ which is quadratic in $X\in\mathbb{R}^{n\times d}$, $$\frac{1}{2}tr(X^TAX) - tr(Y^TBX)$$ for positive definite weighted Laplacian matrices $A, B\in\mathbb{R}^{n\times n}...
7
votes
2answers
533 views

Hartree Fock iteration problem

I am writing a program to compute the ground state energy for any closed shell atom using Hartree Fock Roothaan method, with GTO basis. The code works for the simplest case, the helium, but it fails ...
7
votes
0answers
463 views

DIIS method to accelerate SCF convergence for stretched geometries

I am implementing from scratch an Hartree-Fock calculation in the STO-3G basis set to perform Born-Oppenheimer molecular dynamics. I have a Restricted Hartree-Fock procedure that can reproduce very ...
6
votes
3answers
1k views

Eigenvectors with the Power Iteration

To compute the eigenvector corresponding to dominant eigenvalue of a symmetric matrix $A\in\mathbb{R}^{n\times n}$, one used Power Iteration, i.e., given some random initialization, $u_1\in\mathbb{R}^...
6
votes
3answers
1k views

Krylov Subspace Methods for Dense Systems

I am currently researching on the viability of using KS methods for solving large dense systems. What I wish to prove (or disprove) is that methods like CG, BiCG and QMR are as good (if not better) ...
6
votes
2answers
267 views

Krylov subspace iterative methods in floating point arithmetic

Is there any work that considers Krylov subspace iterative methods in floating point arithmetic? I'm especially interested in how rounding errors influence the convergence and the accuracy of the ...
6
votes
3answers
139 views

Newton's method for a given polynomial

Let $f(x)=\frac{1}{5}x^5+\frac{1}{3}x^3+x-1$ Show that $f$ has only one zero $r$ in interval $(0,1)$ To find approximation of $r$ we apply Newton's method $x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}...
6
votes
1answer
271 views

Question about the smoothing operators in multigrid methods for nonlinear PDEs

Suppose we are dealing with a nonlinear problem, say $$ A u := L u + G(u) = f $$ the nonlinearity of the operator $A$ is the polynomial type, ie, $L$ is a linear operator, and $G(u) = u^k$, or ...
6
votes
1answer
87 views

Numerical iterative method, estimating error

Given iterative method: $x_{n+1}=0.7\sin x_n +5 = \phi(x_n)$ for finding solution for $x=0.7\sin x +5$, I want to estimate $|e_6|=|x_6-r|$ as good as possible, with $x_0=5$, where $r$ is exact ...
6
votes
1answer
282 views

Iterative linear solver for “ugly” saddle point system

I am a graduate student majoring scientific computing. The numeric model I made caused a very ugly-looking saddle-point linear system. It is not symmetric at all and I will attach the sparsity pattern ...
5
votes
3answers
596 views

On Vanilla Preconditioners for solving dense $Ax=b$ iteratively

I am looking for preconditioners which don't assume anything about the matrix or its origins. I basically want to be able to type in the following in MATLAB and have quick solving time: ...
5
votes
4answers
3k views

Simulated Annealing proof of convergence

I implemented downhill simplex simulated annealing algorithm. Algorithm is very hard to tune, w.r.t. parameters including cooling schedule, starting temperature... My first question is about ...
5
votes
4answers
3k views

Fastest polynomial root finder for a given accuracy

I am looking for a very fast polynomial root finder, hopefully with a matlab code. I don't need very accurate results, 2-3 decimal places would be fine. Also the method should be able to optionally ...
5
votes
1answer
2k views

Quality of eigenvalue approximation in Lanczos method

I try to familiarize myself with iterative eigenvalue solvers such as Lanczos. So I tried rewrite it to python directly according to wiki. But it doesn't seem to work. The problem: it approximates ...