# Comparing Jacobi and Gauss-Seidel methods for nonlinear iterations

It is well known that for certain linear systems Jacobi and Gauss-Seidel iterative methods have the same convergence behavior, e.g. Stein-Rosenberg Theorem. I am wondering if similar results exist for nonlinear iterations, where at step $k$ the Jacobi iterations on, say $\mathbb{R}^n$, are defined as

$$x_i^{k+1}=F_i(x_1^{k+1},\cdots,x_{i-1}^{k+1},x_i^k,\cdots,x_n^k),$$

and the Gauss-Seidel iterations are defined as

$$x_i^{k+1}=F_i(x_1^k,\cdots,x_n^k)$$

for $i=1,\cdots,n$ and we have a set of nonlinear functions $F_i(\cdot): \mathbb{R}^n\to\mathbb{R}$.

• The statements of nonlinear Jacobi and Gauss-Seidel (the two displayed equations) are backwards. Are you expecting to assume that F is positive and perhaps some notion of coercivity? Jan 5 '14 at 4:37
• We need to start from the assumptions you need for convergence of each method. For example, what you are calling Jacobi (its usually called successive substitutions for nonlinear equations) is normally justified by requiring the operator F to be contractive. Jan 5 '14 at 19:12
• I guess the question is that since we have 2 iterative methods on the same functions ${F_i}$, the mappings from these 2 methods are definitely related. But in general it is not true that $(F_1,\cdots,F_n)$ being contractive, which makes Gauss-Seidel method converging, implies the Jacobi method converging. Jan 8 '14 at 17:31
• My first thought is that you could take a taylor series of the functions at a point, assume a small enough domain (with small-enough stepsize) that the higher order terms go to zero, and reduce this to a linear form with its answer, then look at the places where that breaks down such as discontinuities or such. Mar 8 '15 at 14:12

The part that is relevant to this question is Theorems 1 and 2. Theorem 1 states that if $$f$$ is twice differentiable in an open neighbourhood around the minimizer $$x^*$$ and the Hessian (of $$f$$) $$H(x^*)$$ is positive definite with Property A, then the nonlinear Jacobi iterations for $$f$$ converges to $$x^*$$ given that the initial guess $$x_0$$ is sufficiently good. Theorem 2 says that the non-linear SOR scheme will converge for any sufficiently close initial guess $$x_0$$ if $$f$$ is twice differentiable in an open neighborhood $$S_0$$ around the minimizer $$x^*$$.