# Questions tagged [quasi-newton]

Use for questions involving quasi-Newton method, which in contrast to [tag:newton-method] uses an approximation of the Hessian during the root-finding or optimization process.

21 questions
Filter by
Sorted by
Tagged with
1 vote
42 views

### Hessian-free preconditioner for non linear least squares

I am solving a nonlinear least squares problem using Gauss Newton method. Due to the large dimension of the problem, I use the Hessian-free approach. As a linear solver I use either MINRES or CG. To ...
398 views

### Difference between asymptotic and non-asymptotic convergence in optimization?

I am reading some optimization methods and I am facing some issues with two terms "asymptotic and non-asymptotic convergence". What is the difference between them?
38 views

### Interpretation of error between Hessian approximation and real Hessian - Quasi-Newton Method

$$||I- H_{k}^{BFGS}\nabla^{2}f(x_{k})||_{2}$$ , where $H_{k}$ is the inverse of hessian approximation at each iteration. I am given this expression to assess the error in Hessian approximation in ...
149 views

### General question related to BFGS

Based on my basic understanding of the BFGS method, the algorithm will iterate until the gradient norm is less than or equal to a set value called "gtol" in the case of Python. However, when ...
• 11
243 views

### When does L-BFGS outperform GD?

In practice, L-BFGS is frequently held comparably to other inexact QN methods, and it provides a middle ground of sorts between Hestenes–Stiefel CG and BFGS as memory goes from zero to infinity (...
• 201
1 vote
69 views

### Solution predictors for accelerating convergence in nonlinear FEM

I am looking for the details of commonly-used predictors for accelerating the convergence of iterations using Newton-Raphson scheme for nonlinear problems in FEM. I am looking specifically for static ...
• 685
102 views

### Does BFGS preserve the bandedness of the inverse hessian?

In the BFGS method we perform iterations by calculating an approximation $\boldsymbol{H}_k$ to the inverse Hessian $\boldsymbol{H}$ of the objective function. This method belongs to a family of ...
• 472
192 views

### Sensitivity of BFGS to the accuracy of the gradient

I am studying how to speed-up the BFGS method using quantum computing techniques. I have used a method of speeding up the gradient of the function, but it sacrifices the precision value of the ...
• 151
1 vote
290 views

### How to use Wolfe-Powell step-size control in quasi-Newton method?

I'm trying to find the minimum of a function using the quasi-Newton method with the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. But I want to change the following implementation, so that: 1) ...
• 131
1 vote
87 views

• 439
993 views

### Low-rank updates in BFGS

I have read this and other threads on this site on BFGS, but I still don't have a clear understanding of what it's meant by low-rank updates. For example, I read the following in this book: The ...
64 views

### Doubt regarding principled approach towards approximating the Hessian

In my optimization problem, the hessian has a structure such that it can be written as the sum of two matrices. Populating the first of the matrices is efficient. Populating the second one is ...
• 656
154 views

### Quasi-Newton Optimization with parallel function evaluation

I have a function of many variables (~200-2000) which I am optimizing with some success using L-BFGS. While the function is expensive to evaluate, the gradient can be computed with not much additional ...
944 views

### Weighted Frobenius norm in BFGS

In what sense is the weighted Frobenius norm "adimensional"/"scale-invariant" for any symmetric positive definite weight matrix $W$? If we plug in a positive diagonal matrix into $W$ wee see that \$||A|...
• 121
1 vote
171 views

### Quasi Newton taking very small steps

I have implemented a Quasi-Newton method based on the Hessian approximation. I am noticing that the algorithm takes too many iterations to converge, even though it does converge. What I am not able to ...
• 656
1 vote