Questions tagged [iterative-method]

A method which produces a sequence of numerical approximations which converges (provided technical conditions are satisfied) to the solution of a problem, generally through repeated applications of some procedure. Examples include Newton's method for root finding, and Jacobi iteration for matrix-vector solves.

Filter by
Sorted by
Tagged with
2
votes
0answers
79 views

Numerical error in implementation of iterative algorithm

I am trying to implement (in Python for now) low thrust orbit propagation for spacecraft using universal variables. For a given central body with the gravitational parameter $\mu$ and an orbit with ...
1
vote
1answer
60 views

Appropriate algorithm for smallest several eigen(values|vectors) of a ~300 DoF system?

I'm no expert on the different types of algorithms to compute eigenvalues and vectors for a real, symmetric matrix (coming from linear mass and stiffness matrices for a frame FEA model). I am looking ...
0
votes
1answer
66 views

Exact distribution of random walk after 10 or 100 steps

(Variant of Stats Exercise to Protect the Innocent). Starting for $x = 0$ I can do one of two operations $\pm 1$ or $\pm 4$. Let $T$ be one iteration of this step. What is the mean and standard ...
1
vote
0answers
34 views

Difference between Chebyshev first and second degree iterative methods

Consider linear equation $Au = f$. We want to solve it with iterative method (assuming $A$ is good). First order iterative method is: $$ u^{k+1} = u^k - \alpha_{k+1}(Au^k - f), $$ The second degree ...
2
votes
0answers
67 views

Dynamic Successive Over/under Relaxation (SOR) with several variables

I am solving a partial differential algebraic equation (PDAE) system which has the following dependent variables: $f=f(X,T)$ and $g=g(T)$, along with a few others My current method for coupling is ...
3
votes
1answer
261 views

Convergence conditions of a stationary iteration method for linear systems

Recently, I obtain a linear system, $Ax = b$, where $A$ is a nonsingular, strictly diagonally dominant $M$-matrix. Then I also got a matrix splitting $A = S - T$, where $S$ is also a nonsingular, ...
0
votes
1answer
80 views

Linear stationary iteration method

Suppose you are attempting to solve $Ax = b$ using linear stationary iteration method defined by $$x_k = G x_{k-1} + f$$ that is consistent with $Ax = b$, i.e., for which $f = (I - G)A^{-1}b$. Suppose ...
7
votes
1answer
2k views

Integer vs float multiplication performance, modern CPUs

Are there benchmarks for how many multiplications of various integer types compared to floating point types can be achieved per second on modern CPUs? I'm trying to get some hint if it would be ...
9
votes
3answers
4k views

Sort of problems where SOR is faster than Gauss-Seidel?

Is there any simple rule of thumb to say if it is worth to do SOR instead of Gauss-Seidel? ( and possible way how to estimate realxation parameter $\omega$) I mean just by looking on the matrix, or ...
1
vote
1answer
1k views

How to solve the problem without using symbolic computation

I have the following simple nonlinear equations with two unknowns only: $$\left\{ \begin{array}{c} \int_1^2{\dfrac{ e^{{a_1} x+{a_2} x^3}}{1+x^2}} \, dx=1 \\[13pt] \int_1^2{ x^2 e^{{a_1} x+{a_2} x^...
0
votes
1answer
531 views

Variational Monte Carlo: Variational energy is lower than ground state energy

I'm writing a VMC simulation for hydrogen and helium atoms, but in both my codes my variational energy for certain wavefunctions is not only statistically different from my expectation value, but it ...
3
votes
1answer
204 views

Is this finite difference approach correct?

I am solving incompressible 2D Navier-Stokes equations with zero y-component velocity. The geometry is a simple 2D pipe of a length $L$ and diameter $W$ and there is only two boundary conditions: Non-...
2
votes
0answers
52 views

SPECT reconstrction using MLEM

In Single-Photon Emission Computerized Tomography (SPECT) parallel beam reconstruction using Maximum-Likelihood Expectation–Maximization(MLEM), is it sufficient to scan the object around 180 degree? ...
5
votes
1answer
265 views

Eigenvectors of slightly perturbed matrix

Assume I know eigen-pairs $(\epsilon_i,\vec\phi_i)$ for matrix operator $\hat H$ that $ \hat H \vec \phi_i = \epsilon_i \vec \phi_i $ Now I have slightly perturbed $\hat H' = \hat H + \hat R$ were ...
3
votes
1answer
1k views

Power Iteration over Rayleigh Quotient Iteration?

It is a commonly known fact that the Rayleigh Quotient converges cubically (1), while the Power Iteration may converge slowly if the difference between the dominant and second-dominant eigenvalue is ...
1
vote
1answer
637 views

Iterative Closest Point Algorithm

I am currently working on an iterative closest point algorithm (in C++, see here). I understand the basic premise of an ICP algorithm. You have two point clouds (a target and a reference) and you ...
5
votes
0answers
773 views

Can Gauss-Seidel/SOR (preconditioned?) be applied to a non-diagonally dominant matrix?

After applying finite difference method to a Laplace/Poisson problem always arises a diagonal dominant system of equations that can be solved with Gauss-Seidel or SOR methods. If the original PDE does ...
11
votes
1answer
884 views

Smallest eigenvalue without inverse

Suppose $A\in\mathbb{R}^{n\times n}$ is a symmetric, positive definite matrix. $A$ is big enough that it's expensive to solve $Ax=b$ directly. Is there an iterative algorithm for finding the ...
34
votes
3answers
15k views

How to choose a method for solving linear equations

To my knowledge, there are 4 ways to solving a system of linear equations (correct me if there are more): If the system matrix is a full-rank square matrix, you can use Cramer’s Rule; Compute the ...
8
votes
0answers
512 views

DIIS method to accelerate SCF convergence for stretched geometries

I am implementing from scratch an Hartree-Fock calculation in the STO-3G basis set to perform Born-Oppenheimer molecular dynamics. I have a Restricted Hartree-Fock procedure that can reproduce very ...
2
votes
0answers
240 views

Does Conjugate Residual really have convergence properties similar to that of Conjugate Gradient?

I have coded up a toy implementation of Conjugate Residual and have been testing it. Both wikipedia and the Saad claim that Conjugate Residual and Conjugate Gradient have similar convergence behavior....
9
votes
1answer
302 views

Iterative “solver” for $x^t \Sigma^{-1} x$

I can't imagine I'm the first to think about the following problem, so I'll be satisfied with a reference (but a complete, detailed answer is always appreciated): Say you have a symmetric positive ...
1
vote
1answer
145 views

Convergence problem in iterative method

I am trying to solve two non-linear equations self-consistently in a Gummel loop. Sometimes (every once in a while), I get to a situation when the loop repeats itself with wrong solutions and a ...
5
votes
0answers
174 views

What causes periodic humps in residual plots?

When using many iterative methods, whether for solving linear systems, looking for steady-state convergence in CFD, etc., the semilog plot of the residual often shows "humps" as the residual decays. ...
2
votes
1answer
319 views

Minimizing Cost Functions using Iterative Least Squares

I am currently trying to use iterative least squares to solve a system, $y = Hx + v$ where $y$ is a vector of observations, $H$ is the design matrix, and $v$ is the observation error. From my ...
10
votes
1answer
173 views

What iterative method can effectively solve a linear system with this kind of spectrum

I have a linear system with matrix which eigenvalues are uniformly distributed on the unit circle like this: Is it possible to solve this kind of system effectively by iterative method, maybe with ...
2
votes
1answer
329 views

Incomplete Cholesky

Is there an efficient way to perform an incomplete Cholesky factorization on a symmetric positive definite sparse matrix (CSR format), in order to use it as a preconditioner for a CG solver? Is there ...
6
votes
1answer
790 views

Inverse iteration to find the null singular vector of a rank-deficient matrix

I have an $n \times n$ unsymmetric matrix $A$ that results from the discretization of an ill-posed Poisson problem, and thus is rank-deficient with null space of dimension one. I want to compute just ...
2
votes
1answer
213 views

Best preconditioner for mixed-poisson problem (RT0 elements)

For a very large mixed-poisson problem with lowest order Raviart-Thomas elements (RT0), I plan on using an iterative solver. However, this kind of problem is not positive-definite (saddle point ...
3
votes
2answers
287 views

FLOPs of iterative vs direct solvers

In general, do iterative solvers require more floating point operations than the direct solver counterparts? I have some scientific code (written in both PETSc and FEniCS) for solving a mixed FE ...
0
votes
2answers
2k views

How to obtain a convergent solution iteratively for a linear system of equations? [closed]

I am working on a problem that requires an iterative procedure to solve a linear system of equations, the system of equations in matrix form is: $$\underbrace{\begin{bmatrix} r_{11} & r_{12} &...
0
votes
1answer
219 views

Strict Feasibility in Interior Point Methods

As we know, in the interior point methods, all the iterates have to be strictly feasible. I implemented an affine scaling interior point for nonlinear objective functions. For small examples (2D), it ...
4
votes
1answer
99 views

How to evaluate a series of derivatives?

Consider the function $$f(\mathbf{x}) = \sum_{n=0}^{N} a_n \left( (\mathbf{b}-\mathbf{x})\cdot \nabla \right)^n \frac{1}{r}$$ where $r = |\mathbf{x}| = \sqrt{(x-x_0)^2 + (y-y_0)^2}$ and $a_n$ and $\...
1
vote
1answer
324 views

Direct or iterative solver for ill-conditioned problems

I have to solve an ill-conditioned sparse matrix. Once I read that iterative solver are the better tool for such problems. Is that true? If yes, why?
1
vote
0answers
153 views

Search Direction in Conjugate Gradient

Could you help me with a Conjugate Gradient question? In using CG to solve Ax=b, why is the search direction $p_{k+1}$ in CG chosen as a linear combination of the residual $r_k$ and previous direction ...
4
votes
1answer
257 views

Methods for Constrained Optimization Problems with Box Constraints

Consider this problem: \begin{equation} \begin{array}{ll} \text{minimize } & f(x) \\ \text{subject to } & a \leq x \leq b \end{array} \end{equation} where $a,b,x \in \mathbb{...
9
votes
3answers
1k views

Basin of attraction for Newton's method

Newton's method for solving nonlinear equations is known to converge quadratically when the starting guess is "sufficiently close" to the solution. What is "sufficiently close"? Is there literature ...
0
votes
1answer
76 views

Iterative algorithms for sparsity using a function for operator A in Ax=b

I am going to solve an linear iterative inverse problem. I have two functions in matlab which one of them play the forward and ...
2
votes
0answers
267 views

Speeding up the classical Jacobi method using Scheduled Relaxation method? [closed]

There has been quite a flutter recently in the iterative world about an algorithm that speeds up the classical Jacobi method by as much as 200 times using a scheduled relaxation method where a ...
3
votes
1answer
315 views

Designing a preconditioner for a very Ill-conditionned matrix

I am a physicist with limited numerical methods knowledge and I am trying to speed up the inversion of a very ill-conditioned problem ($rcond>10^{30}$). The same sparse square matrix is used ...
3
votes
2answers
611 views

How many generations does it typically take for a differential evolution method to reach a global optimum?

For differential evolution methods in optimization, how many generations does it typically take to reach a global optimum? How do we know if the values are never going to converge?
2
votes
3answers
995 views

Test set for linear solvers

Lets assume I have a iterative linear system solver, e. g. this one. Whats the typical approach on verifying and testing this kind of solvers? Is there a standard test set of linear systems one ...
3
votes
2answers
7k views

What are some reasons that Conjugate Gradient iteration does not converge?

I would greatly appreciate it if you could share some reasons the Conjugate Gradient iteration for Ax = b does not converge? My matrix A is symmetric positive definite. Thank you so much! Edit with ...
2
votes
1answer
3k views

How to test convergence of an algorithm for constrained optimization

I am applying an iterative method (projected newton) to an optimization problem. Theoretically, the method should converge linearly. I would greatly appreciate it if you could share how should I test ...
4
votes
1answer
61 views

Iteratively refine bounds on exp for Metropolis criterion

In Monte Carlo simulations, using the Metropolis criterion, one often has to compare a random number $a$, $0 \leq a < 1$, to the Boltzmann distribution $exp(-\beta\Delta E)$, where $\Delta E$ is ...
2
votes
3answers
1k views

Eigenvalues of a sparse banded nonsymmetric matrix from an elliptic operator

I have a sparse matrix coming from the discretization of a 3D elliptic PDE. The matrix is banded with seven non-zeros diagonals. The sparsity pattern of the matrix looks like this (the actual matrix ...
1
vote
2answers
87 views

Performance metrics to compare initial-boundary value problem solutions

I am comparing the performance several finite difference methods of solving an initial-boundary value problem. There are several dimensions to this comparison: Number of cells Number of timesteps ...
4
votes
1answer
358 views

Linear equation system: Direct solver works, iterative solver does not

I have to solve for x in b = A*x, where a is sparse. This works fine with Matlab's mldivide: x = A \ b. Since I will have to use an iterative algorithm for very large A, I'm currently testing Matlab's ...
1
vote
2answers
2k views

What sparse linear programming solver it is better to use?

I have the following LP problem: $$ \min \limits_{\varepsilon, x_{1}, \ldots, x_{n}}f(\varepsilon, x_{1}, \ldots, x_{n}) = \varepsilon \;\;\;\;\; s.t. \;\;C x \geq 0, \;\; x_{i}^{0} - \varepsilon \...
16
votes
1answer
2k views

Intuitive motivation for BFGS update

I am teaching a numerical analysis survey class and am seeking motivation for the BFGS method for students with limited background/intuition in optimization! While I don't have time to prove ...