A popular krylov subspace method for solving linear systems of equations, particularly those that exhibit symmetric positive definiteness.

83 questions
Filter by
Sorted by
Tagged with
57 views

### Convergence of Conjugate Gradient Algorithm

I am trying to solve a linear elasticity model using finite element discretization in a rectangle domain [0,1]x[0,1]. For the solution of the the linear system $Ku=F$ I am using the CG algorithm. ...
63 views

In the standard formulation of Krylov subspace methods, you always have to divide by 0 somewhere in a solution, e.g., in CG, $$x_{k+1} = x_k + \frac{r_k^T r_k}{p_k^T A p_k} p_k\\ p_{k+1} = r_{k+1} + \... 0answers 51 views ### 2-norm of solution update suddenly becomes zero after a few iterations I am trying to solve the Poisson equation in 2D for heterostructure devices. I have linearized the equation and discretized it using FDM. I am using BiCGStab to iteratively solve for the solution as ... 1answer 68 views ### Why does smoothed aggregation multigrid method used as preconditioner in conjugate gradient slows down the solution time? I'm solving a system of linear equations obtained from the FEM discretization of a simple linear elasticity problem on a cube with zero displacements at one plane and a load on the opposite one. The ... 1answer 110 views ### What happens when I use a conjugate gradient solver with a symmetric positive semi-definite matrix? I have a symmetric positive semi-definite matrix, i.e., a laplacian and wonder what may happen when I use a CG solver, that is an algorithm for positive definite matrices. What happens when the ... 0answers 62 views ### Conjugate Gradient for nonlinear equation system Is it possible to apply adaptions of the conjugate gradient algorithm i.e. Fletcher-Reeves, Polak-Ribere or others to systems of nonlinear equations? How should the equation system be adjusted so one ... 0answers 132 views ### Explanation of subspace strategy regarding CG described in Golub's book I was wondering about the last paragraph in Matrix Computations (4th edition) by Golub, Chapter 11 (11.3.3), specifically his explanation of subspace strategy for Conjugate Gradient. Note that in ... 0answers 61 views ### Arnoldi Decomposition Algorithm I try to get into GMRES via Arnoldi-Decomposition. For my understanding, I Implemented the Arnoldi-Decomposition in python. ... 1answer 234 views ### How to find a good preconditioner to the system (A^T A + \lambda I) x = A^T b? The system in the title has a damper factor \lambda > 0 and the matrix A is sparse and rectangular, with a structure I can exploit to solve matrix vector products very fast. My current solver, ... 1answer 79 views ### In which cases does the nonlinear conjugate gradient method take more than n steps? I have programmed a couple of Matlab implementations of nonlinear Conjugate Gradient methods (Fletcher Reeves and Polak Ribeire). However, I am concerned with how many steps it's taking to optimise ... 1answer 189 views ### What's wrong with the **PCG and MINRES** in matlab? Last week, I have learned the details of the robust iterative methods of PCG, MINRES, GMRES, which will converges to the exact solution x^* of nonsingular system within N steps for A\in \mathbb{R}... 0answers 44 views ### Subspaces for Iterative methods In the original paper of Conjugate Gradients, the authors mention that if we pick the canonical basis \{e_1,e_2,\ldots,e_n\}, to obtain A-orthonormal vectors, we end up with the Gaussian elimination ... 1answer 40 views ### Accelerating Conjugate Gradients fitting for small localized kernel (like cubic B-spline) Question: Is there some pre-conditioner for Conjugate-Gradient (CG) cheap enough, that it is worth using even if my operator is very local (i.e. already has a low number of non-zero elements), as it ... 2answers 53 views ### What's the difference between the 2 ways of definitions of function handle? which is robust and better? Recently, I have been studying Krylov subspace iterative methods. I find the matlab robust command pcg and the new concept of the function handle to return a matrix-vector product. Then I use help pcg ... 1answer 37 views ### Why the iteration steps become twice if the step size reduces half for CG methods? For CG method for SPD matrices, (Ax = b arising from Poisson equation with homogeneous boundary condition) we know that the convergence theorem: After m steps of iteration, the error e^{(m)}=x-x_m ... 1answer 101 views ### Conjugate Gradient for singular 2D poisson finite element with Neumann Boundary Conditions Heavily edited question after I realised partly what the problem was I have programmed a simple 2D square finite element solution to the Poisson equation -\Delta u = f The source function ... 0answers 66 views ### Why not use the preconditioned residual as termination criterion for preconditioned CG? I have a Poisson equation with wildly varying material parameters (1 .. 1000), wildly varying element sizes (5 nm .. 100 um) and some quite anisotropic (tetrahedral) elements (100 nm x 100 um). I use (... 1answer 85 views ### Blowup of error in Conjugate Gradient method with periodic Dirichlet Poisson matrix My problem is that the L2-Norm of the residual for the periodic Poisson matrix P is initially decreasing but starts to blow up after a certain number of iterations. The blowup happens earlier the ... 1answer 83 views ### Nonlinear conjugate gradient with orthogonality constraint I have to solve a set of nonlinear optimization problems in the subspace defined as the orthogonal space to a given vector. More precisely,$$ \arg\min f(\vec x) \qquad \text{with} \qquad \vec x \...
132 views

Richardson's iteration introduce a scalar $\alpha$ to the update formula: $$\textbf{x}^{(k+1)} = \textbf{x}^{(k)} + \alpha \textbf{r}^{(k)}$$ And compute $\alpha$ by minimizing the spectral radius:...
114 views

### Conjugate gradient - ill-conditioning and numerical tolerance

I would like to solve system $Ax=b$, where $A$ is SPD, but very ill-conditioned ($\text{cond}(A)>10^{11}$). I am interested in using UNpreconditioned version of the conjugate gradient method. Is ...
111 views

### Relation between conjugate gradient method and finite elements method

What is difference beetwen this two method? Are these methods far from each other or are these methods complement each other? Could you take an example?
3k views

### Which C++ linear algebra library is probably the fastest on solving huge sparse [square matrix] linear system?

I am developing a 2D CFD solver for fluid-particle interaction. To solve Navier-Stokes equations on a grid of size $10000\times 10000$ cells (or >1 million cells), a large linear system $Ax=b$ with $A$...
609 views

### Estimate extreme eigenvalues with CG

CG may be used to estimate the extremal eigenvalues of a SPD matrix (by computing eigenvalues of tridiagonal matrix associated with the Lanczos algorithm). After a few iterations the largest ...
35 views

### Stochastic conjugate directions to improve convergence in narrow valleys

My question concerns a specific statement in this paper: N. N. Schraudolph and T. Graepel, "Conjugate Directions for Stochastic Gradient Descent," in Int. Conf. Artificial Neural Networks, Berlin, ...
173 views

### How can a CG solver solve a non positive definite sparse matrix

I am using the CUSP CG solver and I ran it on couple of sparse matrices from the University of Florida sparse matrix collection. The solver was able to solve non positive definite sparse matrices. My ...
331 views

### What is required of the objective function in order to use Gauss Newton method?

From what I understand, the Gauss-Newton method is used to find a search direction, then the step size, etc., can be determined by some other method. In addition to that, are the following ...
407 views

### Linear constraints for L-BFGS-B

I know L-BFGS-B only supports simple box constrains of the form: $l_i \leq x_i \leq u_i$, where $l_i$ and $u_i$ are constants. For my specific optimization problem, I need to specify some simple ...
66 views

I want to minimize some multivariable function $\Delta(\alpha, \beta)$. I know that this function has a zero point, $\Delta(5, 5) = 0$. Starting from some $(\alpha, \beta)$ close to $(5,5)$ (e.g. (4....
163 views

### Computing preconditioner for a non-linear conjugate gradient implementation

Consider the following steps for the $i$-th non-linear conjugate gradient iteration, in the context of 3D electromagnetic inversion, and as discussed in (Newman and Boggs, 2004): (1) set $i = 1$, ...
148 views

### How to verify solution to pre-conditioned linear systems solver?

I am solving Ax=b. A has a very large condition number (> O(10^10)) I am using the conjugate gradients method with point jacobi pre-conditioning. I obtained a solution 'x' that "looks" reasonable. ...
8k views

For a project, I have to implement these two methods and compare how they perform on different functions. It looks like the conjugate gradient method is meant to solve systems of linear equations of ...
1k views

103 views

### Optimization based integration for MPM

I'm considering implementing (just for simplicity) the unconstrained implicit optimization based integration for Material Point Method as described in Chenfanfu Jiang's thesis on MPM (the minimization ...
2k views

### Solving the Poisson equation with Neumann Boundary Conditions - Finite Difference, BiCGSTAB

I am trying to solve the Poisson equation in a rectangular domain using a finite difference scheme with a rectangular mesh. I have happily generated the matrix system of equations Ax = b which is ...
2k views

### Understanding MATLAB's fmincg optimization function

I'm researching numerical optimization. Recently I've come across a variant of a conjugate gradient method named fmincg. The function is written in MATLAB and is ...
158 views

### CG question: is symmetry always necessary?

Consider the 1D Poisson equation $$\nabla^2 u = f.$$ Using finite difference method on cell corner data and a uniform grid with ghost points, I think we can write the system of equations with Neumann ...