Questions tagged [optimization]

This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.

Filter by
Sorted by
Tagged with
6
votes
0answers
76 views

FEM : energy minimization VS PDE solving

Engineering FEM When I studied engineering, I learned the traditional approach for finite elements for elasticity. The point was to solve the PDE $-div(\sigma)=f$ as: Multiply your PDE with a test ...
2
votes
1answer
78 views

Difference between asymptotic and non-asymptotic convergence in optimization?

I am reading some optimization methods and I am facing some issues with two terms "asymptotic and non-asymptotic convergence". What is the difference between them?
3
votes
1answer
102 views

Parameters estimation with fewer variables than parameters

I am trying to estimate parameters, 4 of them, by fitting a system of 3 ordinary differential equations. I am using a model published that was using 3 parameters and gave a good fit to the data, and I ...
89
votes
19answers
113k views

Is there a high quality nonlinear programming solver for Python?

I have several challenging non-convex global optimization problems to solve. Currently I use MATLAB's Optimization Toolbox (specifically, fmincon() with algorithm=<...
-1
votes
0answers
27 views

Can constants and parameters in a Gekko model be simply Python variables?

I'm new to Gekko and I noticed that Gekko will still run even if I don't define fixed values in my model as Gekko constants and parameters. For example, this code still runs and gives an answer: ...
-1
votes
0answers
24 views

the convergence of the iterative algorithm has a major problem

In order to solve an optimization problem, I divided the main problem into two sub-problems. The two sub-problems require to be solved iteratively until the algorithm converges. I use the bi-section ...
-1
votes
0answers
27 views

Do we need smallest vectors to obtain the optimized solution in Gradient Descent?

I'm new to topics about optimization. I am currently reading about Steepest Descent Method in Gradient Descent optimization. I saw this equation where we need to find a new iterate with an initial ...
2
votes
1answer
83 views

Formulating this optimization problem

Suppose I want to minimize below objective function $\sum | g(x_i) \cdot I_{g(x_i)<0} |^2$ i.e, the latter penalty terms like $ |g(x_i)|^2 $ are only computed when $g(x_i)<0$. $|g(x_i)|^2$ are ...
0
votes
1answer
116 views

R function or package for carrying out maximum likelihood techniques in random effect models

I am applying optim() function in R to obtain maximum likelihood estimates of the fixed effects and random effects in a model with bivariate random effects. The ...
5
votes
2answers
102 views

Optimizing a quadratic form integral over unit sphere

I have an optimization problem, which is to maximize the following integral over the unit sphere: $$ \max_B \int d\Omega \mathbf{f}^{\dagger}(\theta,\phi) (B^{\dagger} + B) \mathbf{f}(\theta,\phi) $$ ...
0
votes
0answers
58 views

Comparing minimas of two different functions

The goal is to find vectors $x_u$ and $y_i$, both of the same length $f=64$, and to do this the following loss function is minimized: $$\sum_{u, i} (1 + \alpha \cdot r_{ui})(p_{ui} - x_{u}^{T}y_i)^2$$ ...
1
vote
0answers
25 views

Sample Average Approximation vs. Numerical Integration

In the sense of the calculation of the expected value of objective functions, we have two choices to evaluate the value; 1. Sample Average Approximation (SAA): $$ \frac{1}{N}\sum_{i=1}^N f(x,\xi^i). $$...
2
votes
0answers
38 views

Adding a "cost term" to a linear regression, so solution values are minimized

I'm using Python's optimize.lsq_linear method to run a linear regression with the bounds set between 0% and 100% power usage. ...
0
votes
0answers
83 views

Find coefficients in general second order differential equation

Suppose you have a system that can be described via the following equations of motion: $$\ddot{y}+\delta(t)\dot{y}+\alpha(t) y = \gamma\sin(\omega t)$$ The functions $\delta(t)$ and $\alpha(t)$ are ...
0
votes
1answer
60 views

Expressing a Constraint in an optimization problem

If I have a vector of M "continuous" decision variables (say it is called x) , and if I want a constraint to express that only one of them is allowed to have a nonzero value (i.e. no more ...
1
vote
0answers
65 views

Binarization for optimization problems

I have a nonlinear mixed-integer optimization problem, and because of very high complexity when solving it using methods like Branch and Bound, I resorted to solve it using alternating method and ...
22
votes
5answers
8k views

Why should non-convexity be a problem in optimization?

I was very surprised when I started to read something about non-convex optimization in general and I saw statements like this: Many practical problems of importance are non-convex, and most non-...
1
vote
0answers
49 views

Maximizing $l_1$-normalized entropy using CVXPY

Suppose that $x = (x_1, ..., x_n)$ is a vector of variables and I would like to maximize the Shannon entropy of $\frac{|x|}{||x||_1}$ (i.e. the vector of absolute values of $x_i$, normalized to have $...
2
votes
0answers
69 views

Efficient solver of a Integer programming

I am solving an Integer programming using MATLAB, yet the efficiency is low. Here is the problem: Suppose $v$ is a $N \times 1$ vector. For $v_i \in v$, $v_i \in \{0,1\}$. $D$ is a 0-1 matrix, which ...
0
votes
0answers
11 views

Optimal Time Series Weight for Quantile Estimation

Given a time series, $x_1$, $x_2$, ..., $x_t$, .... I want to solve for $$\mathrm{min}_{w_1, w_2, ..., w_m}\rho(x_t-\mathrm{Quantile}(q; x_{t-1}, x_{t-2}, ..., x_{t-m}; w_{t-1}, w_{t-2}, ..., w_{t-m}))...
-2
votes
1answer
63 views

How to get 10 in computer science, using the number 4 exactly four times, and two signs exactly and two operation + exactly? [closed]

How to get 10 in computer science, using the number 4 exactly four times, and two signs exactly and two operation + exactly ?
2
votes
1answer
109 views

Nonlinear least square optimization

Problem description Given data at many time instance $t$, $$\min _{\alpha, \Lambda, \beta} \lVert y(t) - \alpha e^{\Lambda t} \beta \rVert_F$$ with $$ \lVert \alpha \rVert_2^F = 1 $$ where $y(t) \...
0
votes
0answers
41 views

Can this volume intgral be expressed as a convex function?

This question is related to the following: https://math.stackexchange.com/q/4151405/685910 - the context is summarized below for clarity. In the setting of convex optimization, I am looking for a ...
28
votes
3answers
22k views

BFGS vs. Conjugate Gradient Method

What considerations should I be making when choosing between BFGS and conjugate gradient for optimization? The function I am trying to fit with these variables are exponential functions; however, the ...
1
vote
0answers
55 views

Constrained optimization for non-linear equations in octaveGNU

I have installed Optim1.6.1 package. I would like to solve a system of equations in non linear finite element analysis using constraints as u=1 at certain nodes. u=0 at certain nodes. Typically I find ...
0
votes
1answer
124 views

Interpreting multivariable root-finding results from Matlab's fsolve algorithm

Edit: So I was able to get the same value of r that's given, when coding up the sum of squares of function values directly in the script file, rather than on the Command Window. So, maybe there's a ...
0
votes
0answers
47 views

Equivalence between zero sum games and linear program

It is well known that you can use the algorithm for finding the equilibrium of a Zero-sum game to solve a linear program. In particular, you can take a LP and reduce it to a zero-sum game, and use the ...
14
votes
2answers
15k views

Confusion about Armijo rule

I have this confusion about Armijo rule used in line search. I was reading back tracking line search but didn't get what this Armijo rule is all about. Can anyone elaborate what Armijo rule is? The ...
0
votes
0answers
38 views

Relative interior requirement in Slater's condition

I'm reading Convex Optimization by Boyd and Vandenberghe. This is how they describe Slater's condition: What I don't understand is why it is necessary to enforce that $x$ be in the relative interior ...
4
votes
3answers
429 views

Which absolute and/or relative stopping criteria do use for Newton's method?

I saw many stopping criteria for Newton's method all around Web and books. Some are defined from the residuals: of either current iteration only: $$ \|f(\mathbf{x}^{(k)})\| \leq \epsilon $$ (https://...
6
votes
0answers
85 views

continuous analogues of Newton's method

Suppose we want to minimize some convex functional $J(u)$ where $u$ lives in some Banach space $V$. The classical Newton method $$\mathrm d^2J(u_n)(u_{n + 1} - u_n) = -\mathrm dJ(u_n)$$ can be viewed ...
1
vote
1answer
107 views

Optimization on the multinomial manifolds of stochastic non-square matrices

Thanks for note! So I have an optimization problem with simple form but the decision variable is a large-scale matrix. My problem is similar to a existing problem here about multinomial manifolds and ...
4
votes
4answers
485 views

Find representatives of vector-space in set of vectors?

Suppose I have a multi-dimensional vector space $X$, and a collection of $n$ vectors $\{x_i\}_{i=1}^n \subset X$, which are not evenly "spaced-out" in $X$. I am searching for $m<<n$ of these $...
5
votes
2answers
206 views

Arbitrary Precision Optimization Libraries?

Are there any well-known optimization libraries (ideally with Python bindings or even in Python) supporting (unconstrained) minimization (of $f:\mathbb{R}^n \to \mathbb{R}$ for $n$ for $n\sim 10^1,10^...
2
votes
1answer
66 views

Log-Determinant constraints in SDP

This is a belated follow up to my question here, because I didn't want to tack questions onto questions. According to the Mosek documentation here, one possibility for expressing $t \leq log(det(X))$, ...
0
votes
0answers
58 views

Optimization algorithm to find coinciding root and minimum

Are there any optimization algorithms aimed at finding a coinciding root and a (local) minimum of a multi-variable function f. Say it is known analytically that ...
1
vote
0answers
55 views

tiny typo in Numerical Recipes Eq. 9.4.6 [closed]

The Numerical Recipes Forum http://numerical.recipes/forum/ is closed, so I will record a tiny typo here for the benefit of others who may wonder about this. (This typo is not in the software, but ...
1
vote
1answer
58 views

What is the best cooling and flippling schedule in simulated annealing?

I've noticed that some heuristics for it on my problem which work surprisingly well. I guess it ought to be systematically studied although I cannot find guides or overviews for it.
6
votes
1answer
156 views

How does the number of function calls in BFGS scale with the dimensionality of space

Is there any estimate for the scaling of the number of function calls in BFGS-optimization with the dimensionality of the search space? Specifically I am assuming a (free) expression for the gradient ...
4
votes
3answers
169 views

Algorithms to generate spherical codes

A spherical code, specified by the parameters $(n,N,t)$, is a set of $N$ coordinates on the $n$-dimensional unit hypersphere such that the set of dot products between any two unit vectors from the ...
0
votes
0answers
46 views

How to efficiently perform this 2D integral in Quadpy?

I need to integrate a function defined in 2Dims (z and radius r), for which I don't have an expression. I can just query the ...
0
votes
0answers
43 views

Help with finding an objective function for optimization

Im working in comsol and needed help figuring out the objective function for the following situation (Comsol does have multiple objective function option as min max or sum of) I have 7 batteries ...
0
votes
2answers
85 views

Validating that a code is a good spherical code

Apologies if this is a trivial question. If that is the case I imagine I would benefit from someone explaining where my understanding is lacking. I am having some trouble interpreting the (putatively ...
0
votes
0answers
44 views

Blown-up iterates in Gauss-Newton method

I am working on a non-linear least squares problem with standard form, in which I need to calibrate a parameter vector $\Theta$ to a set of inputs $\mathbf{x}$ and outputs $\mathbf{y}$: $$\begin{align}...
5
votes
0answers
90 views

How to find a lot of (if not all) local minima / critical points of a function?

Briefly stated, I would like to find "all" local minima / critical points of a function. This function comes from the discretization of a continuous problem with infinitely many degrees of ...
-1
votes
1answer
34 views

Interpretation of error between Hessian approximation and real Hessian - Quasi-Newton Method

$$ ||I- H_{k}^{BFGS}\nabla^{2}f(x_{k})||_{2}$$ , where $H_{k}$ is the inverse of hessian approximation at each iteration. I am given this expression to assess the error in Hessian approximation in ...
-1
votes
1answer
56 views

Is there an overview of the runtime speed up of LP/MIP solvers throughout the years?

whenever I read papers on OR that use an LP/MIP approach, they include the time solver used, as well as the version and the year. I would like to know how much faster the same experiment would be ...
1
vote
0answers
35 views

Energy cannot decent during optimization despite non-zero gradient

Assume we have an (at least) 2nd-order differentiable energy $f(x), x\in R^n.$ And $n$ is very big. Mathematically, I think it is impossible to find a point $\bar{x}$ where the energy cannot be ...
3
votes
2answers
1k views

Cplex C++ Interface: How to add many constraints quickly?

I noticed that adding constraints to an IloModel one by one can be prohibitively slow. (I am referring to the construction of the model, not the optimization.) ...
1
vote
3answers
134 views

How can we solve the normal equations with limited memory?

I was asked this open ended question in an interview once: How would you find a solution to the normal equations with limited memory? Unlike Solving sparse least squares system with limited memory, ...

1
2 3 4 5
18