Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [newton-method]

Method for finding successively better approximations to the roots (or zeroes) of a real-valued function

15
votes
2answers
2k views

Is it possible to solve nonlinear PDEs without using Newton-Raphson iteration?

I am trying to understand some results and would appreciate some general comments on tackling nonlinear problems. Fisher's equation (a nonlinear reaction-diffusion PDE), $$ u_t = du_{xx} + \beta u ...
13
votes
1answer
895 views

Can an approximated Jacobian with finite differences cause instability in the Newton method?

I have implemented a backward-Euler solver in python 3 (using numpy). For my own convenience and as an exercise, I also wrote a small function that computes a finite difference approximation of the ...
12
votes
2answers
3k views

Strategies for Newton's Method when the Jacobian at the solution is singular

I'm trying to solve the following system of equations for the variables $P,x_1$ and $x_2$ (all else are constants): $$\frac{A(1-P)}{2}-k_1x_1=0 \\ \frac{AP}{2}-k_2x_2=0 \\ \frac{(1-P)(r_1+x_1)^4}{L_1}...
9
votes
3answers
618 views

Methods of solving non-linear advection-diffusion systems beyond Newton-Raphson?

I'm working on a project where I have two adv-diff coupled domains through their respective source terms (one domain adds mass, the other subtracts mass). For brevity, I'm modeling them in steady ...
8
votes
1answer
551 views

Newton iteration applied to nonlinear PDE

I'm having difficulty understanding how to apply Newton iteration to nonlinear PDEs and then use a fully implicit scheme to time step. For example, I want to solve Burgers equation $$u_{t} + u u_{x} -...
8
votes
0answers
179 views

Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Problem Solving a non-linear system of equations. The number of variables is the same as the number of equations. When I fix a set of variables (say $\vec{y}$) and keep another set free (say $\vec{...
7
votes
1answer
196 views

Lack of quadratic convergence in Newton's method

It is well-known that Newton's method can converge quadratically, if initial guess is close enough and if the arising linear systems are solved accurately. I am applying Newton's method to highly ill-...
7
votes
3answers
360 views

Strong coupling of a non-linear multiphysic problem: failure with Newton Raphson method

I am trying to solve a multiphysic problem using finite elements and a Newton Raphson solution scheme. I have two non-linear subsystems that are coupled bi-directionally. The first subsystem includes ...
7
votes
1answer
286 views

Eigenvalues of $ab^T$

In deriving a Newton scheme, I end up with a Jacobian matrix of the form $J=I+ab^T$ where $a,b$ are vectors. For practical reasons, I want to approximate it by a symmetric positive definite matrix. ...
7
votes
1answer
102 views

Are there special methods for solving $f'(z)=0$ for analytic $f$?

I am trying to solve a bunch of equations for the zeros of the derivative of an analytic function, and I would like to know if there exist methods that exploit this structure to provide better ...
6
votes
2answers
115 views

Positive root of $x^q + bx - b$

Is there either a closed-form expression or fast/elegant algorithm for computing the positive root of the polynomial $$f(x)=x^q + \beta x - \beta,$$ where $\beta>0$ and $q\geq2$? How about the $q\...
6
votes
2answers
253 views

How to calculate/derive analytic FEM Newton Jacobian

I trying to wrap my head of derivation of the analytic FEM Jacobian for the Newton method. Say we have a nonlinear Poisson problem of the (weak) form $$ \int a(u)\nabla\ u\cdot \nabla v = \int f v $$ ...
6
votes
1answer
321 views

Caveats of Hessian free method

Hessian free iterative optimization techniques like Newton-CG, do not explicitly compute the Hessian but instead approximate the product of the Hessian with a vector through finite difference. The ...
6
votes
1answer
179 views

What would be a good approach to solving this large data non-linear least squares optimisation

Introduction to Problem I'm using a Truncated Signed Distance Function to perform 3D reconstruction from depth images. Essentially I have a large voxel grid where each voxel contains the signed ...
6
votes
1answer
130 views

Roots of a function for eigensystem

I want to find the roots for $\kappa$ for the equation $$\sqrt{\alpha - 1} \cos{\left (\frac{\sqrt{2} \sqrt{\alpha - 1}}{2 \sqrt{\epsilon}} \right )} \cosh{\left (\frac{\sqrt{2} \sqrt{\alpha + 1}}{2 \...
6
votes
1answer
156 views

Defining a condition number and termination criteria for Newton's method

The condition number of function evaluation $$ \mathrm{cond}(f,x) := \left| \frac{x f'(x)}{f(x)} \right| $$ is infinite at a root of $f$. Hence it is useless for rescaling a tolerance which defines an ...
5
votes
1answer
2k views

Difference between Gauss-Newton method and quasi-Newton method for optimization

Can anybody help me? I heard that Gauss-Newton method compute an aproximation of the Hessian instead of the true Hessian, but, quasi-Newton method too, don't it? what is the differences between them? ...
5
votes
2answers
955 views

Do I need to impose boundary conditions in the Jacobian matrix?

In the framework of Finite Element Method, when the Newton method is used, we solve $J(x^k) \delta x = -f(x^k)$, and the increment $\delta x$ would not change some entries from $x^k$ related to ...
5
votes
2answers
6k views

How to implement Newton's method for solving the algebraic equations in the backward Euler method

Can you explain me how does the backward Euler method works? I have seen the formula and try to understand the method, but what I can't understand is why and how to use the Newton-Rapson method. Do ...
5
votes
1answer
5k views

Newton-Raphson method for nonlinear partial differential equations

For the numerical solution of Reynolds equations (a non-linear partial differential equation), the Newton-Raphson method is generally proposed. After getting algebraic equations from a finite ...
5
votes
1answer
120 views

Can redundant variables be beneficial for root-finding convergence

Suppose I have $n$ generally nonlinear equations for $n$ variables, like e.g. for $n=2$ the system $F(x,y)=0$ $$ \begin{aligned} x^2+2y-4&=0\\ \sqrt{8}x+y^2-5&=0 \end{aligned} $$ By ...
5
votes
1answer
139 views

Matlab help related with the scaled Newton's iteration method

I need a little help with Matlab code for the method mentioned in this paper for computing the inverse of the matrix. In this paper, the author presented scaled Newton iteration given by $X_{k+1} = \...
5
votes
2answers
192 views

Solving a nonlinear algebraic system that includes a linear term

I am trying to solve a particular system of non linear equations written as $F(x) = 0$ in an efficient way. More specifically, $$F(x) = (I - \gamma A)x - g(x) + C$$ where $\gamma$ is a scalarconstant,...
5
votes
1answer
267 views

Solving a set of linear equations with block structure and weak coupling

I have a standard set of linear equations $Ax=b$ where the Hessian matrix $A$ has the special block structure as shown: $A= \begin{pmatrix} T & U\\ U^T & V \end{pmatrix}$, $x= \begin{...
5
votes
1answer
391 views

scaling and preconditioning for trust region Newton methods

Geometrically, scaling and preconditioning seem to address similar challenges in optimization. However, these two concepts are implemented very differently. Take trust region Newton method, as an ...
5
votes
1answer
800 views

Newton's method with box-constraints

I have to use an iterative method (Newton-Raphson, modified Newton and Broyden) to solve a system of nonlinear equations $f(x)=0$. Every unknown $x_i$ is bounded between $l_i$ and $u_i$, i.e., $l_i<...
4
votes
3answers
616 views

Best method to find the zero of a decreasing function numerically

I need to find the zero of a function $f(\lambda)$ which is of the form $\sum \frac{c_i^2}{(1+\lambda d_i)^2} -1 $. I tried using Newton's method, and it works sometimes, but it is higly dependent of ...
4
votes
3answers
868 views

Beale's function and newton iteration

I am trying to find the minimum of the so called Beale’s function given by $f(x_1,x_2) = (1.5-x_1+x_1x_2)^2 + (2.25-x_1+x_1x_2^2)^2 + (2.625-x_1+x_1x_2^3)^2$ Using Newton iteration $x^{(k+1)} = x^{...
4
votes
3answers
178 views

How can I use Projected Gradient Descent for this optimization problem with constraint?

Suppose $ q $ and $ A $ are given and that $ q, p \in R^{N} $ and $ A \in R^{NxN} $, then how can I find the vector $ p $ such that $$ (q - p)^{T}A(q - p) $$ is minimized constraint to $ \sum_{i=1}^{...
4
votes
1answer
265 views

Active Elements in Projected Newton's Method?

To those who are familiar with the projected Newton's method or projected gradient method... We consider a constrained optimization problem with simple bounds. Particularly, minimize f(x) subject to ...
4
votes
1answer
212 views

Efficient and stable computation of inverse CDF

What is the most efficient and numerically stable algorithm for computing the inverse CDF $F^{-1}(y)$ of a probability function, assuming that both the PDF $f(x)$ and the CDF $F(x)$ are known ...
4
votes
1answer
1k views

Newton's method goes to zero determinant Jacobian

I am using the Newton's method to solve $3\times3$ systems. For some particular cases, it turns out that at a given iteration, the Jacobian matrix cannot be inverted and that its determinant is very ...
4
votes
1answer
227 views

Methods for Constrained Optimization Problems with Box Constraints

Consider this problem: \begin{equation} \begin{array}{ll} \text{minimize } & f(x) \\ \text{subject to } & a \leq x \leq b \end{array} \end{equation} where $a,b,x \in \mathbb{...
4
votes
2answers
438 views

Poisson-Nernst-Planck equations with ill-conditioned sparse matrix

I am trying to solve Poisson-Nernst-Planck system of equations for ions diffusion problem using finite volume method. Nernst-Planck equation for mass transport and Poisson equation for electrostatic ...
4
votes
0answers
387 views

Best way to add a positivity constraint to Newton's Method

So given an objective function $f({\bf x})$, I would like to include a positivity constraint when I perform the fixed point iteration: $${\bf x}^{(t+1)}={\bf x}^{(t)} - \text{H}_f^{-1}\nabla f({\bf x}^...
3
votes
1answer
594 views

BFGS methods for constrained elasticity problems

My dear community, I am wondering why BFGS methods are not so widely used for simulating mechanical problems which heavily still relies on inverting the hessian matrix. I am essentially interested ...
3
votes
1answer
539 views

Solving a system of nonlinear equations with an ODE solver is faster than with the Newton method?

This is somehow unexpected, but my recent experience with solving a system of nonlinear equations is that treating them as the right hand side of a system of ordinary equations and then evolve the ...
3
votes
2answers
435 views

Low-rank updates in BFGS

I have read this and other threads on this site on BFGS, but I still don't have a clear understanding of what it's meant by low-rank updates. For example, I read the following in this book: The ...
3
votes
2answers
847 views

Slow convergence of Newton's method for finite elements

The application is a simple non-linear advection diffusion problem (steady state) using DGFEM. My error at each iteration is given by $$ e_{n+1} = ||\mathbf{J}^{-1}(\mathbf{u}_{n})\mathbf{F}(\mathbf{u}...
3
votes
1answer
586 views

Should the Jacobian of a system of PDEs be calculated from the main equations of the discretised equation?

I am solving a coupled system of non-linear PDEs in 1D. Something like, $$ u_t = F_1(u,v,w) \\ v_t = F_2(u,v,w) \\ w_t = F_3(u,v,w) $$ where each variable is a function of $x$ (the spatial dimension)...
3
votes
1answer
189 views

Computational complexity of Newton's method

the classical Newton's method for non-linear systems of equations is $x_{k+1} =x_k-J_F(x_n)^{-1} F(x_n)$. In pratice, rather than compute the inverse of the Jacobian matrix, one solves the systems $...
3
votes
1answer
146 views

Excluding roots from a system of nonlinear equations

I have a system of nonlinear equations of which I know it has a single root I am interested in, and has a continuum of roots I am not interested in. I am currently using Newton with line-searching in ...
3
votes
1answer
784 views

Non-linear root finding when the Jacobian is almost singular

I'm trying to solve a system non linear-equations: $$ \frac{\partial K(\mathbf{\lambda})}{\partial \lambda_i} - c_i = 0 $$ for $i = 1, \dots, 15$, using Newton's method: $$ \lambda^{k + 1} = \lambda^k ...
3
votes
2answers
247 views

System of quadratic algebraic equations

I have this problem $H_i(x_1,x_2,\dots, x_N) = a_{ijk} x_j x_k + b_{ij} x_j + c_i = 0 \quad 1\leq i \leq N$ And I need to show that applying Newton-Raphson can fail to find even one real solution ...
3
votes
0answers
110 views

Difference between Dishonest Newton method and Very Dishonest Newton method

What is the difference between the Dishonest Newton method and the Very Dishonest Newton method? Is there a difference or do they mean the same thing? I have tried searching for this on the internet ...
2
votes
1answer
71 views

GMRES vs Newton-GMRES for Solving nonlinear PDE's

Often when numerically solving nonlinear PDE's using method of lines approach with an implicit integrator a system of nonlinear equations have to be solved. To be more specific, let's say we have ...
2
votes
2answers
199 views

Why do Newton-Krylov iterations stagnate in this problem? [closed]

Consider this integro-differential heat equation taken from SciPy documentation page: $ \nabla^2 P = \alpha \left(\iint_\Omega \cosh(P)dx dy \right)^2 $ which was found in this question. In the ...
2
votes
1answer
421 views

Hessian-free and Truncated Newton methods

In this paper on Deep Learning for Machine Learning, the approach is referred to as Hessian-free method. That is because the Hessian is never computed explicitly. Instead, the product of the Hessian ...
2
votes
1answer
114 views

Newton's method stagnates at small error

I have a system of the form $$A(u)f(u)=b$$ where $A$ is basically a matrix originating from the Finite Element Method. I try to solve it using the Newton method: $$R = A(u_{i}) f(u_{i}) - b $$ $$...
2
votes
1answer
59 views

Doubt regarding principled approach towards approximating the Hessian

In my optimization problem, the hessian has a structure such that it can be written as the sum of two matrices. Populating the first of the matrices is efficient. Populating the second one is ...