Skip to main content
The 2024 Developer Survey results are live! See the results

Questions tagged [optimization]

This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
22 views

formula for the elliptical orbit of the magnetic field in a current carrying circular loop [closed]

In a circular current carrying loop the magnetic field lines form elliptical orbits if I have constant value for current and a point let's say at r distance from the center of th current carrying loop ...
user49758's user avatar
-1 votes
1 answer
69 views

optimal gradient algorithm to determine best $α_k$

Let's consider an optimal-step gradient algorithm and assume that: $g(α) := f(X_k - α∇f(X_k)) = 2α^2-4α+17$, how can we determine the optimal $α_k$? Here is my simple solution: $g(α) = 2α^2-4α+17$ $g'(...
V_head's user avatar
  • 15
0 votes
1 answer
45 views

step-fixed algorithm first iterates

let us have the fixed-step gradient algorithm, with $p = 2$ and we assume that for $X = (x, y)$, $∇ f(X) = \begin{pmatrix} x -1\\ y -2 \end{pmatrix}$ Let me assume we intialize with $X_0 = (0,0)$ what ...
V_head's user avatar
  • 15
0 votes
1 answer
66 views

step-fixed algorithm to minimize f, which step to ensure convergence?

If we want to apply the fixed-step gradient algorithm to the minimization of $f(x) = \frac{1}{2}(Ax, x)$ where $A$ is a symmetric 2x2 matrix with eigenvalues $\lambda_1 > \lambda_2 > 0$, for ...
V_head's user avatar
  • 15
0 votes
1 answer
39 views

Estimating the rate of convergence of Projected Gradient Descent on constrained polynomial objectives

I am estimating the order of convergence of Projected Gradient Descent (GD) on quadratic polynomials with random coefficients independently drawn from Uniform(-1,1) and bounded by a unit hypercube. I'...
ufghd34's user avatar
  • 23
1 vote
1 answer
90 views

How to run scipy.optimize.minimize with L-BFGS-B for maxiter (completely)

I want to run the below code for maxiter = 20001. I don't want it to stop by some default criteria. ...
Saif Ur Rehman's user avatar
1 vote
1 answer
191 views

Solving linear system of equations with constraints on unknowns

I would like to solve a system of linear equations $y=Uh$ for an unknown vector $h$, where I have a few constraints on some of the elements of $h$. The matrix $U$ is composed of a vector $u$ (length $...
Neuling's user avatar
  • 35
0 votes
0 answers
55 views

LBFGS-B initial gradients too high?

I'm optimizing a geometrical shape for electromagnetic performance. The shape is constrained with bounds, say between 0.2 and 0.8, whereas the parameters are all between 0.2 and 0.8. I am interested ...
James Li's user avatar
2 votes
0 answers
92 views

An alternative to Levenberg–Marquardt algorithm

When trying to solve for a (over)determined non-linear least square method: $$\underset{x}{\min}||f(x)||^2_2, f: \mathbb{R}^n \rightarrow \mathbb{R}^m, x\in \mathbb{R}^n, m\geq n$$ we use the Gauss-...
William Lin's user avatar
6 votes
1 answer
4k views

Are penalty functions still "necessary"?

In my constrained problems (box constraints) I simply set my cost function to INFINITY (the c99 macro) if an inequality constraint is violated. This prevents the point being used, seems to work very ...
m4r35n357's user avatar
  • 329
5 votes
1 answer
133 views

Algorithm to find local minima of function which is unbounded from below

I have a differentiable function $\mathbb{R}^n \to \mathbb{R} $ of several variables $f(x_1,\ldots,x_n)$, whose form I can write down and compute derivatives of. Typically $n = 8$. The function is ...
math_lover's user avatar
0 votes
0 answers
22 views

Unable to decipher the error in Anderson-Darling estimation code in R

I just ran the code in R, which estimates some parameters using Anderson-Darling estimation method. The code is given in the appendix of the Habib-Khalil paper. ...
vidyarthi's user avatar
  • 111
1 vote
1 answer
89 views

Is there existing code for solving a Lagrangian Dual problem using the subgradient method?

I know there is a generic code for solving the lagrangian relaxation of an LP. However, for an integer program, sometimes you want some constraints relaxed, but not all. For example, I want the ...
underdog987's user avatar
1 vote
0 answers
82 views

How to spot a "centre-bias operator"?

I recently asked this question after noticing abnormally good results from a popular "nature-based meta-heuristic algorithm". I established by "origin shifting" that the method ...
m4r35n357's user avatar
  • 329
15 votes
5 answers
2k views

Ensuring IEEE 754 Compliance and Numerical Precision in C++ HPC Projects

I'm currently engaged in a large-scale C++ HPC project focused on numerical simulation, particularly Finite Element Method (FEM) simulations. Our project spans various Linux-based platforms and ...
René Chenard's user avatar
6 votes
2 answers
310 views

Difficult because non-convex, or some other reason?

In my github project, I have come up with a very simple function that seems to cause problems for all the optimizers I have thrown it at, and I am wondering why. There is an obvious downhill ...
m4r35n357's user avatar
  • 329
1 vote
0 answers
53 views

How to solve ADMM Optimization Problem

We are trying to solve the following optimization problem using ADMM: $$ \begin{aligned} & \min _{\left\{y_{i j}^{m}\right\}} \sum_{i \in I_{m}} f_{i j}^{m}\left(y_{i j}^{m}\right)+\sum_{j \in J} \...
ANWESA ROY's user avatar
9 votes
2 answers
748 views

Poor test functions for optimization

I have been looking in detail into one of the many "meta-heuristic" optimization algorithms and became suspicious at how well it appeared to perform (compared to other methods like Nelder-...
m4r35n357's user avatar
  • 329
1 vote
1 answer
94 views

How does the slack variable work in the problem formulation?

Recently I am reading a paper. In it, after they achieve eq(15), which is $$ \operatorname{Tr}(\boldsymbol{Q})-\sqrt{2 \ln (1 / \rho)} \sqrt{\|\boldsymbol{Q}\|_F^2+2\|\boldsymbol{r}\|^2}+\ln (\rho) \...
tyrela's user avatar
  • 133
2 votes
2 answers
112 views

Gradient descent for solving polynomial equations while encouraging variables to be nonzero

I would like to use gradient descent to "randomly sample" solutions to a set of homogeneous polynomial equations. Because the equations are homogeneous, setting all variables to 0 is a valid ...
PPenguin's user avatar
  • 123
0 votes
1 answer
74 views

Why using large bound to supplement inifinity in interior point method can be bad

Here in the documentation of mosek (https://docs.mosek.com/latest/pythonfusion/debugging-numerical.html) we see: Never use a very large number as replacement for infinity . Instead define the ...
Taylor Fang's user avatar
0 votes
2 answers
135 views

BFGS Constrained Optimization Failure Due to Precision Loss

I am trying to optimize the following objective function according to some constraints. However, the optimization fails at the first iteration with the message that the desired error was not ...
user47212's user avatar
1 vote
2 answers
459 views

optimizing piecewise linear objective functions (perhaps non convex) with equality constraints

When I do my project, I need to optimize piecewise linear objective functions (perhaps non convex) with equality constraints. The piecewise linear objective function may be not convex like this in the ...
Yiyuan Chen's user avatar
2 votes
1 answer
197 views

Speeding up 3 body problem acceleration calculation

I want to find optimizations to my code for the 3BP, and more specifically computing accelerations. I'm using a data-driven approach, so I have a bodies structure ...
Remeraze's user avatar
0 votes
0 answers
24 views

Loop Bounds vs. Iteration Domain in Polyhedral optimization

Context: I was reading a tutorial on polyhedral optimization. But got confused while trying to translate the iteration domain (i.e. loop bound) to set builder notation. Problem Description: A code ...
F.C. Akhi's user avatar
  • 101
0 votes
1 answer
72 views

MIP - Large Piecewise Linear Constraints Over Continuous Intervals

I'm currently trying to run a MIP (have access to both Gurobi and CBC) with a piecewise linear function having ~200 intervals for each of the ~30 x values I have. I am using the standard decomposition ...
davidwashere's user avatar
1 vote
0 answers
29 views

1-dimensional nonlinear global minimization of kepler distance problem

I want to solve the problem to determine the next intersection of a Keplerian orbit with the Sphere of Influence of a celestial body to find the next intersection within one future period of the ...
lamont's user avatar
  • 143
0 votes
0 answers
24 views

dual svm square hinge loss

Let $x_1,\dots,x_n\in \mathbb{R}^n$, $y_1,\dots,y_n\in \{-1,1\}$, $\lambda \ge 0$ and $K$ be the invertible Gram matrix $K=(x_i\cdot x_j)_{ij}$. Consider $$ (P) \qquad \qquad \min_{a\in \mathbb{R}^n} \...
Smilia's user avatar
  • 478
0 votes
0 answers
62 views

Help with inferring Network topology from Spectral templates

I am trying to use matlab and YALMIP to solve a graph learning problem of recovering eigenvalues from the eigenvectors of the covariance of sampled graph signal data. This is to implement the ...
user86422's user avatar
0 votes
0 answers
39 views

Methods for delaying the "break" in non-linear least squares optimisation when the step size gets too small?

I am using the Levenberg-Marquardt method for calibration purposes. Typically, the RMSE of my calibration looks like: I want to break the algorithm when the algorithm step-updates start to slow down, ...
THATS MY QUANT MY QUANTITATIVE's user avatar
2 votes
1 answer
99 views

references for optimization in the context of parameter identification with finite elements

i am performing parameter identification for a non-linear partial differential equation (elasticity) that I solve with finite elements. My optimization problem is a non-linear least squares data-...
Simon's user avatar
  • 185
5 votes
0 answers
90 views

optimization scaling techniques

Consider a convex QP of the form $$ \min_x \bigl\{\tfrac12 x^\top Q x + q^\top x : Ax\leq b\bigr\}\tag{P} $$ with dual $$ \min_y \bigl\{\tfrac12 (A^\top y + q)^\top Q^{\dagger} (A^\top y+q) + b^\top y ...
jjjjjj's user avatar
  • 325
1 vote
1 answer
170 views

Why researchers use MATLAB based YALMIP or CasADi for MPC?

I was looking at various research papers and most of the researchers use CasADi, YALMIP, MPCTools to implement MPC. My question is "Why researchers use MATLAB based YALMIP or CasADi for MPC ...
Khalid Umer's user avatar
3 votes
1 answer
150 views

Role of rotation's pivot point in optimization?

In this paper, the authors describe how to use locally rigid transformations (sampled on nodes in space) to deform mesh vertices. In the paper, rotations are relative to the pivot point, which ...
jordi's user avatar
  • 31
0 votes
0 answers
64 views

PETSc non-linear solvers (SNES): specifying single Eval & Jacobian function

The PETSc documentation example of a non-linear solver call has the user provide separate functions for the Jacobian and function evaluations: ...
Sardine's user avatar
  • 378
1 vote
1 answer
121 views

Improvement to naive gradient descent implementation for the Thomson problem

I have a Python program (available on github) that uses naive gradient descent to find approximate solutions to the Thomson Problem. It works surprisingly well, but I've been wondering if there's a ...
Martin C.'s user avatar
  • 229
40 votes
11 answers
9k views

stupid + stupid = brilliant in scientific computing

I'm interested in examples of very effective methods in scientific computing that are the sum or naive combination of very ineffective or bad ones.
Daniel Shapero's user avatar
2 votes
2 answers
119 views

Automatic Differentiation In the Presence of Jump Points

I have a complex monte-carlo cashflow model that traditionally uses the finite difference (FD) method to calculate its derivative at any given point. To improve model performance, I coded forward-mode ...
Mild_Thornberry's user avatar
1 vote
2 answers
188 views

Are there good block sparse matrix solver libraries?

There are some great libraries with linear solvers for sparse matrices - SuiteSparse is the obvious one. The methods work on sparse matrices with scalar entries. However, often in optimization ...
user664303's user avatar
3 votes
0 answers
61 views

What 2nd-order optimization algorithms have convergence guarantees for strictly- but not strongly-convex problems?

A function $f$ is strictly convex if $$f((1 - \lambda)x + \lambda y) \le (1 - \lambda)f(x) + \lambda f(y)$$ with equality if and only if $x$ and $y$ are equal. This implies that the second derivative ...
Daniel Shapero's user avatar
1 vote
1 answer
60 views

Optimization: Find minimizer along linestring

Given some function f(x) and a set of points A representing a linestring (or polygonal chain), I am searching for the point on ...
Citizen3011's user avatar
1 vote
1 answer
48 views

Name this optimum-within-convex-hull algorithm: State is a convex combination of hull vertices; Nonnegativity ensured by reparameterization

I'm looking for the "official" name(s) for a procedure for optimizing a convex loss function over a convex subset. This seems to be a default/naïve algorithm that folks come up with before ...
MRule's user avatar
  • 153
1 vote
0 answers
58 views

Beyond the LP relaxation of binary least squares

I have a binary quadratic program with a convex objective function, of the form, \begin{align} \text{minimize}\;\;& x^tAx+b^tx\\ \text{subject to}\;\;& x_i\in\{0,1\} \end{align} where $A$ is ...
Set's user avatar
  • 503
5 votes
3 answers
432 views

Packages suitable for numerical optimization of functions with discontinuous gradient at the point of minimum

Are there packages for numerical optimization in julia or python, or in any other system for scientific computing, capable of taking into account the discontinuity of gradient at the minimum point? ...
Gec's user avatar
  • 153
1 vote
0 answers
66 views

min(f(x)) is convex or concave based on type of f(x)

i have f(x) that is concave function. My question is g=min(f(x)) is concave or convex? And max(g) is concave or convex? there is a theorem for this?
Maria's user avatar
  • 11
1 vote
1 answer
141 views

Possible bug with scipy.optimize SHGO sobol: TypeError: <lambda>() takes 1 positional argument but 3 were given

I have been trying to perform some global optimization with SciPy optimizer SHGO and I've had issues with the sampling method 'sobol'. Specifically, I get an error ...
Sasche's user avatar
  • 31
0 votes
0 answers
55 views

Solving a minimization problem without flattening inputs?

I am reading this paper on improving time steps for solving simulation problems: https://www.math.ucla.edu/~jteran/papers/GSSJT15.pdf The authors developed this energy function: $$E(x) = \frac{1}{2\...
Makogan's user avatar
  • 273
0 votes
1 answer
61 views

The row loss gradients

Suppose the original loss function is $$\min_{\mathbf{V}}\frac{1}{2}\|\mathbf{V} - Q(\mathbf{V})\odot\mathbf{U}\mathbf{E} - \beta Q(\mathbf{V})\mathbf{V}\|_2^2$$ where $\odot$ denotes the element-wise ...
Zuba Tupaki's user avatar
1 vote
1 answer
334 views

Python libraries for larges scale optimization/rootfinding

I have been dealing with the standard libraries of scipy.optimize for rootfinding and optimization problems, but the problems i want to solve are very large, which makes the standard solvers run out ...
Klaus3's user avatar
  • 133
1 vote
0 answers
65 views

Gradient based optimization involving a black box function where gradients (not the objective) are approximated using a surrogate model

Can we use gradients of surrogates of a black box function and the actual function evaluation for optimization involving an expensive black box simulation? Is there any merits to it?
almostKapil's user avatar

1
2 3 4 5
21