Questions tagged [optimization]
This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.
934
questions
0
votes
0
answers
3
views
Finding weighted sum of curves
This is related to my previous post here
I have a dataset with values of multiple curves. An example plot is shown below.
I want to scale the curves (move up/down) so that all curves overlap.
The ...
- 513
3
votes
3
answers
187
views
+100
Minimize distance between curves
I have a dataset with values of multiple curves. An example plot is shown below.
I want to shift the curves (up/down) so that all curves overlap. This would mean the data points in each curve is ...
- 513
14
votes
3
answers
1k
views
Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?
Historically, iterative methods for solving relatively simple-structured systems $Ax=b$ with $A$ being a $4\times 4$ matrix or to find the eigenvalues of that matrix assuming in both problems that $A$ ...
- 297
0
votes
0
answers
42
views
Notation for a time series with discrete time points
I want to mathematically describe a vector containing a series of time points. The time points must lie on an equidistant time grid of 15 minutes of a day. Thus, elements of the vector can contain ...
- 91
1
vote
1
answer
67
views
Optimization software for real-valued functions of complex arguments
I am interested in an optimization problem of the form
$$\min_{\boldsymbol z} \max_j \vert f_j(\boldsymbol z) \vert = \min_{\boldsymbol z} \Vert f_j(\boldsymbol z) \Vert_\infty. $$
Here, the ...
- 611
0
votes
0
answers
39
views
I would like to fit a data to integral type model but cannot figure it how?
I have cumulative data for concentration versus time. I would like to fit the following model to the data:
$$\frac{1}{\Gamma(n) \bar{t}}\int_{0}^{\frac{n t}{\bar{t}}}z^{n-1} e^{-z}dz$$
$n$ and $\bar{...
1
vote
0
answers
21
views
Unconstrained convex optimization: correlation between dimensionality and Lipschitz constant
The author of the SIAM News article "Optimization Theory and Perspectives on the Field of Machine Learning" mentions:
... For unconstrained convex optimization, GD (gradient descent) ...
- 8,382
3
votes
1
answer
74
views
Notation for an optimization function that receives a vector of pairs
In my optimization problem there are elements consisting of a time and a value, i.e. $(t_0, v_0)$.
These pairs are stored in a vector $v = [(t_0, v_0), (t_1, v_1), ... , (t_n, v_n)]$.
The vector $v$ ...
- 91
1
vote
0
answers
30
views
Does loss landscape provide a means to compare optimization algorithms?
I'm comparing two optimization algorithms for deep neural networks. To that end, I train a network with the same data and starting rom the same initial weights using two different optimization ...
- 111
1
vote
0
answers
56
views
Place points at maximum distance in a convex 2D set
I need to place a given finite set of points within maximum distance of each other on 2D, constrained by a convex boundary, on Python.
Honestly, I'm kind of lost. I have the explicit function to be ...
0
votes
0
answers
34
views
Reformulate a problem with concave objective function into a QP
I would like to convert this problem into a QP (Quadratic program).
$$\text{Maximize } \sum_{k=1}^{K}\sum_{n=1}^{N}log2(1+p_{kn}b_{kn})\\
\text{subject to } \sum_{k=1}^{K}\sum_{n=1}^{N}p_{kn}\leq P_{0}...
- 1
0
votes
1
answer
61
views
Efficiently solving SDP relaxation of an integer quadratic program
I have an integer quadratic program of the form,
\begin{align}
\underset{x}{\max}&\;\;\|Ax-b\|_2^2\\
\text{subject to}&\;\;x\in{\bf Z}\geq0
\end{align}
I'm currently using the (admittedly ...
- 443
2
votes
2
answers
121
views
How to ensure the numeric value is always positive in Optimization Python?
I am currently performing optimization onto a quadratic function by manually coding the algorithm:
$$\min f = x^T v x - r^T x\\
\text{subject to } x \geq 0\, .$$
Here, optimizing the function without ...
1
vote
2
answers
120
views
Secant Method for finding $\sup f^{-1}(0)$
Let $f \in C^0[0, 1]$, and suppose $f \ge 0$. How can I compute $\sup f^{-1}(0)$ efficiently?
- 119
1
vote
0
answers
102
views
scipy.optimize.minimize fails to converge but result is OK
I am trying to optimize a non-linear least squares problem with scipy.optimize.minimize. I have simplified my actual problem down to the case where I am just computing the top 'principal components' ...
- 119
3
votes
1
answer
106
views
Which optimization algorithm to max a single parameter by searching a landscape of five parameters?
Background: We're operating a small betatron which makes use of a vacuum tube where electrons are accelerated circularly. First, they get injected (like inserted) and contracted (like squeezed). After ...
- 133
1
vote
0
answers
84
views
Convergence of Evolutionary Algorithms
When it comes to Evolutionary Algorithms (e.g. Genetic Algorithm), I have often heard people make the following broad statement: "Evolutionary Algorithms Do Not Converge."
I was curious ...
- 129
1
vote
1
answer
101
views
Easy way to perform solver over pandas dataframe
I'm moving from Excel to Python and I'm trying to solve these equations:
$$\begin{align}
X_1&=\bigg[\big(3.47-\log(X_2)\big)^2+\big(\log(c)+1.22)^2\bigg]^{0.5}\\
X_2&=\frac{a}{101.32}\bigg(\...
- 113
1
vote
1
answer
70
views
About using SOCP solvers to solve QCQP
I have noticed that some commercial solvers transform QCQPs into SOCPs and use SOCP algorithms to solve the resulting problem. I am wondering if there is a benefit to this approach over using a pure ...
- 2,301
2
votes
1
answer
94
views
Quasi-Newton Method with a Transformed Hessian
I've recently came across an implementation of the BFGS algorithm but it has an additional step where the Hessian is transformed after the each update. This transformation is done so that certain ...
- 123
0
votes
0
answers
44
views
Examples of kinetic modeling with optimization techniques in Python
We are looking for studies or tools that implemented kinetic modeling with parameter estimation differential evolution or similar optimization techniques in Python.
We are trying to understand what ...
0
votes
1
answer
113
views
Optimal value of parameter/s from a set by scipy.optimize.minimize() method
I have this function $y = \exp(-x)$. I have a list of $x$ values and corresponding $y$ values.
...
0
votes
0
answers
58
views
Non-linear least squares with penalty term
I have a non-linear least squares function that I am trying to minimize, with the objective function:
$\underset{x}{\operatorname{argmin}} \sum\limits_{i}^{N} \frac{1}{2} f(x_i)^2$
I would like to add ...
- 21
0
votes
1
answer
56
views
Evolutionary Algorithms Vs. Gradient Optimization: Comparing Costs
When it comes to optimizing objective functions belonging to Statistical and Machine Learning Models, there seems to be a general rule:
High dimensional objective functions with many parameters are &...
- 129
3
votes
1
answer
158
views
Questions regarding the result of the CVXPY
I want to optimize the function
$$\min_{X \in \mathbb{S}^{n}_{+}} \mbox{tr} \left( C^T X \right) + \mbox{tr} \left( X^{-1} \right),$$
of which I optimize the equivalent problem
$$\min \mbox{tr}\left(C^...
- 133
2
votes
0
answers
73
views
How is ADMM Separable?
I'm learning about ADMM by reading Boyd's paper Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers.
The paper says that ADMM is an improvement over ...
- 21
1
vote
0
answers
22
views
Branch and Bound vs Evolutionary Algorithms
On an informal level, I have heard the following statements being made:
The "Branch and Bound" Algorithm has the ability to provide exact solutions to optimization algorithms, but can take ...
- 129
2
votes
1
answer
67
views
optimization problem with a positive definiteness constraint
Is there a name for the following optimization problem? Is it solvable?
\begin{align}
\min_{\pmb{u}} & \qquad \frac{1}{2}(\pmb{u}-\pmb{u}^m)^{T}(\pmb{u}-\pmb{u}^m)\\
\textrm{s.t.} & \qquad \...
- 428
2
votes
0
answers
71
views
"Unspoken Tradeoff" in Convergence Rates for "Quasi-Newton vs. Gradient Descent"
Is there an "Unspoken Tradeoff" in Convergence Rates for "Quasi-Newton Methods vs. Gradient Descent"?
As a quick summary:
Gradient Descent based algorithms try to find the minimum ...
- 129
2
votes
0
answers
61
views
Cyipopt fails to converge for NLP problem which fmincon() can solve
I'm currently trying to implement a python script for solving a constrained nonlinear optimization problem with ~800 variables and 2 constraints, one linear and one nonlinear. There already exists a ...
2
votes
0
answers
32
views
Calculating a submanifold of minimal values
I have a (numerical, noisy) function $f$ that I'd like to optimize (in 3 < dimension < 10).
The function $f$ has a fairly prominent valley/ravine structure due to physical symmetries present in ...
2
votes
1
answer
270
views
Python solvers for MINLP in Pyomo in Google Colab
I am looking for a MINLP solver that works with Pyomo models which can be used in the Google Colab environment. I have already found MindtPy but it doesn't work in google colab.
4
votes
0
answers
117
views
How amenable is this 2D Frenkel–Kontorova-like energy minimization problem in Python to the use of a modest PC + GPU? (Heavy reliance on indexing)
@Richard's answer to Going to try to move some of my scipy/numpy calculation to a new GPU, how to avoid disappointing results? is quite helpful, and as promised I've added a simple running example ...
- 852
9
votes
1
answer
480
views
How to find the smallest ellipse covering a given fraction of a set of points?
I have a set of points $P$ and want to find the ellipse with the smallest area that covers at least a fraction $f$ of these points. How can I do this?
These questions ask the same thing, but folks ...
- 3,111
3
votes
1
answer
204
views
Going to try to move some of my scipy/numpy calculation to a new GPU, how to avoid disappointing results?
update: I've refactored the question based on helpful advice in the linked meta.
I'm a heavy user of Python's NumPy and SciPy (and not much else) and for years I could run anything I need on my laptop....
- 852
12
votes
2
answers
1k
views
How do I find the minimum-area ellipse that encloses a set of points?
I have a set of points that resembles more of an ellipse than a circle. I implemented the optimization formulation below and the solution gives a circle. I tried with various initial values, still to ...
- 123
8
votes
2
answers
136
views
Approximating the boundary between two sets of points (in 2D): Fitting a region
Given two sets of points $p_{\text{in},i}$ and $p_{\text{out},j}$ inside and outside of what I intuitively call a "region", I would like to estimate and describe the boundary of this region. ...
1
vote
0
answers
109
views
How to solve the minization problem with $Z^{T}Z=I$
I have the following problem
$$\underset{Z}{\min}\left\|X-AZBH\right\|_F^2+\left\|H-B^{T}Z^{T}A^{T}X\right\|\\
\text{subject to } Z^{T}Z=I$$
How to solve the variable $Z$?
- 81
2
votes
1
answer
140
views
Is solving QP easier than a QCQP with linear objective?
Is solving a $QP$ (i.e.: quadratic program, hence a quadratic objective function with linear constraints) easier than solving a $QCQP$ (ie.: quadratic constrained quadratic problem) with linear ...
- 131
1
vote
1
answer
100
views
Finding the root of an equation (divergence at some points)
I am trying to solve this equation for $v_{Sl}$ in the range $[0.001,1]$ to obtain the values of $\delta_l$. I have tried Fsolve with python, which gives results ...
6
votes
0
answers
138
views
FEM : energy minimization VS PDE solving
Engineering FEM
When I studied engineering, I learned the traditional approach for finite elements for elasticity. The point was to solve the PDE $-div(\sigma)=f$ as:
Multiply your PDE with a test ...
- 61
2
votes
1
answer
304
views
Difference between asymptotic and non-asymptotic convergence in optimization?
I am reading some optimization methods and I am facing some issues with two terms "asymptotic and non-asymptotic convergence". What is the difference between them?
2
votes
1
answer
89
views
Formulating this optimization problem
Suppose I want to minimize below objective function
$\sum | g(x_i) \cdot I_{g(x_i)<0} |^2$
i.e, the latter penalty terms like $ |g(x_i)|^2 $ are only computed when $g(x_i)<0$. $|g(x_i)|^2$ are ...
- 121
5
votes
2
answers
115
views
Optimizing a quadratic form integral over unit sphere
I have an optimization problem, which is to maximize the following integral over the unit sphere:
$$
\max_B \int d\Omega \mathbf{f}^{\dagger}(\theta,\phi) (B^{\dagger} + B) \mathbf{f}(\theta,\phi)
$$
...
- 51
4
votes
1
answer
130
views
Parameters estimation with fewer variables than parameters
I am trying to estimate parameters, 4 of them, by fitting a system of 3 ordinary differential equations.
I am using a model published that was using 3 parameters and gave a good fit to the data, and I ...
- 141
0
votes
0
answers
61
views
Comparing minimas of two different functions
The goal is to find vectors $x_u$ and $y_i$, both of the same length $f=64$, and to do this the following loss function is minimized:
$$\sum_{u, i} (1 + \alpha \cdot r_{ui})(p_{ui} - x_{u}^{T}y_i)^2$$
...
- 41
4
votes
0
answers
64
views
Sample Average Approximation vs. Numerical Integration
To calculate the expected value of objective functions, we have two choices:
Sample Average Approximation (SAA):
$$
\frac{1}{N}\sum_{i=1}^N f(x,\xi^i).
$$
Numerical Integration (e.g., Monte Carlo ...
- 41
0
votes
1
answer
62
views
Expressing a Constraint in an optimization problem
If I have a vector of M "continuous" decision variables (say it is called x) , and if I want a constraint to express that only one of them is allowed to have a nonzero value (i.e. no more ...
- 11
1
vote
0
answers
80
views
Binarization for optimization problems
I have a nonlinear mixed-integer optimization problem, and because of very high complexity when solving it using methods like Branch and Bound, I resorted to solve it using alternating method and ...
- 11
2
votes
0
answers
90
views
Adding a "cost term" to a linear regression, so solution values are minimized
I'm using Python's optimize.lsq_linear method to run a linear regression with the bounds set between 0% and 100% power usage.
...
- 121