Questions tagged [optimization]

This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
3 views

Finding weighted sum of curves

This is related to my previous post here I have a dataset with values of multiple curves. An example plot is shown below. I want to scale the curves (move up/down) so that all curves overlap. The ...
user avatar
  • 513
3 votes
3 answers
187 views
+100

Minimize distance between curves

I have a dataset with values of multiple curves. An example plot is shown below. I want to shift the curves (up/down) so that all curves overlap. This would mean the data points in each curve is ...
user avatar
  • 513
14 votes
3 answers
1k views

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

Historically, iterative methods for solving relatively simple-structured systems $Ax=b$ with $A$ being a $4\times 4$ matrix or to find the eigenvalues of that matrix assuming in both problems that $A$ ...
user avatar
  • 297
0 votes
0 answers
42 views

Notation for a time series with discrete time points

I want to mathematically describe a vector containing a series of time points. The time points must lie on an equidistant time grid of 15 minutes of a day. Thus, elements of the vector can contain ...
user avatar
  • 91
1 vote
1 answer
67 views

Optimization software for real-valued functions of complex arguments

I am interested in an optimization problem of the form $$\min_{\boldsymbol z} \max_j \vert f_j(\boldsymbol z) \vert = \min_{\boldsymbol z} \Vert f_j(\boldsymbol z) \Vert_\infty. $$ Here, the ...
user avatar
  • 611
0 votes
0 answers
39 views

I would like to fit a data to integral type model but cannot figure it how?

I have cumulative data for concentration versus time. I would like to fit the following model to the data: $$\frac{1}{\Gamma(n) \bar{t}}\int_{0}^{\frac{n t}{\bar{t}}}z^{n-1} e^{-z}dz$$ $n$ and $\bar{...
user avatar
1 vote
0 answers
21 views

Unconstrained convex optimization: correlation between dimensionality and Lipschitz constant

The author of the SIAM News article "Optimization Theory and Perspectives on the Field of Machine Learning" mentions: ... For unconstrained convex optimization, GD (gradient descent) ...
user avatar
  • 8,382
3 votes
1 answer
74 views

Notation for an optimization function that receives a vector of pairs

In my optimization problem there are elements consisting of a time and a value, i.e. $(t_0, v_0)$. These pairs are stored in a vector $v = [(t_0, v_0), (t_1, v_1), ... , (t_n, v_n)]$. The vector $v$ ...
user avatar
  • 91
1 vote
0 answers
30 views

Does loss landscape provide a means to compare optimization algorithms?

I'm comparing two optimization algorithms for deep neural networks. To that end, I train a network with the same data and starting rom the same initial weights using two different optimization ...
user avatar
  • 111
1 vote
0 answers
56 views

Place points at maximum distance in a convex 2D set

I need to place a given finite set of points within maximum distance of each other on 2D, constrained by a convex boundary, on Python. Honestly, I'm kind of lost. I have the explicit function to be ...
user avatar
0 votes
0 answers
34 views

Reformulate a problem with concave objective function into a QP

I would like to convert this problem into a QP (Quadratic program). $$\text{Maximize } \sum_{k=1}^{K}\sum_{n=1}^{N}log2(1+p_{kn}b_{kn})\\ \text{subject to } \sum_{k=1}^{K}\sum_{n=1}^{N}p_{kn}\leq P_{0}...
user avatar
  • 1
0 votes
1 answer
61 views

Efficiently solving SDP relaxation of an integer quadratic program

I have an integer quadratic program of the form, \begin{align} \underset{x}{\max}&\;\;\|Ax-b\|_2^2\\ \text{subject to}&\;\;x\in{\bf Z}\geq0 \end{align} I'm currently using the (admittedly ...
user avatar
  • 443
2 votes
2 answers
121 views

How to ensure the numeric value is always positive in Optimization Python?

I am currently performing optimization onto a quadratic function by manually coding the algorithm: $$\min f = x^T v x - r^T x\\ \text{subject to } x \geq 0\, .$$ Here, optimizing the function without ...
user avatar
1 vote
2 answers
120 views

Secant Method for finding $\sup f^{-1}(0)$

Let $f \in C^0[0, 1]$, and suppose $f \ge 0$. How can I compute $\sup f^{-1}(0)$ efficiently?
user avatar
  • 119
1 vote
0 answers
102 views

scipy.optimize.minimize fails to converge but result is OK

I am trying to optimize a non-linear least squares problem with scipy.optimize.minimize. I have simplified my actual problem down to the case where I am just computing the top 'principal components' ...
user avatar
  • 119
3 votes
1 answer
106 views

Which optimization algorithm to max a single parameter by searching a landscape of five parameters?

Background: We're operating a small betatron which makes use of a vacuum tube where electrons are accelerated circularly. First, they get injected (like inserted) and contracted (like squeezed). After ...
user avatar
  • 133
1 vote
0 answers
84 views

Convergence of Evolutionary Algorithms

When it comes to Evolutionary Algorithms (e.g. Genetic Algorithm), I have often heard people make the following broad statement: "Evolutionary Algorithms Do Not Converge." I was curious ...
user avatar
1 vote
1 answer
101 views

Easy way to perform solver over pandas dataframe

I'm moving from Excel to Python and I'm trying to solve these equations: $$\begin{align} X_1&=\bigg[\big(3.47-\log(X_2)\big)^2+\big(\log(c)+1.22)^2\bigg]^{0.5}\\ X_2&=\frac{a}{101.32}\bigg(\...
user avatar
1 vote
1 answer
70 views

About using SOCP solvers to solve QCQP

I have noticed that some commercial solvers transform QCQPs into SOCPs and use SOCP algorithms to solve the resulting problem. I am wondering if there is a benefit to this approach over using a pure ...
user avatar
2 votes
1 answer
94 views

Quasi-Newton Method with a Transformed Hessian

I've recently came across an implementation of the BFGS algorithm but it has an additional step where the Hessian is transformed after the each update. This transformation is done so that certain ...
user avatar
  • 123
0 votes
0 answers
44 views

Examples of kinetic modeling with optimization techniques in Python

We are looking for studies or tools that implemented kinetic modeling with parameter estimation differential evolution or similar optimization techniques in Python. We are trying to understand what ...
user avatar
0 votes
1 answer
113 views

Optimal value of parameter/s from a set by scipy.optimize.minimize() method

I have this function $y = \exp(-x)$. I have a list of $x$ values and corresponding $y$ values. ...
user avatar
0 votes
0 answers
58 views

Non-linear least squares with penalty term

I have a non-linear least squares function that I am trying to minimize, with the objective function: $\underset{x}{\operatorname{argmin}} \sum\limits_{i}^{N} \frac{1}{2} f(x_i)^2$ I would like to add ...
user avatar
0 votes
1 answer
56 views

Evolutionary Algorithms Vs. Gradient Optimization: Comparing Costs

When it comes to optimizing objective functions belonging to Statistical and Machine Learning Models, there seems to be a general rule: High dimensional objective functions with many parameters are &...
user avatar
3 votes
1 answer
158 views

Questions regarding the result of the CVXPY

I want to optimize the function $$\min_{X \in \mathbb{S}^{n}_{+}} \mbox{tr} \left( C^T X \right) + \mbox{tr} \left( X^{-1} \right),$$ of which I optimize the equivalent problem $$\min \mbox{tr}\left(C^...
user avatar
  • 133
2 votes
0 answers
73 views

How is ADMM Separable?

I'm learning about ADMM by reading Boyd's paper Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. The paper says that ADMM is an improvement over ...
user avatar
1 vote
0 answers
22 views

Branch and Bound vs Evolutionary Algorithms

On an informal level, I have heard the following statements being made: The "Branch and Bound" Algorithm has the ability to provide exact solutions to optimization algorithms, but can take ...
user avatar
2 votes
1 answer
67 views

optimization problem with a positive definiteness constraint

Is there a name for the following optimization problem? Is it solvable? \begin{align} \min_{\pmb{u}} & \qquad \frac{1}{2}(\pmb{u}-\pmb{u}^m)^{T}(\pmb{u}-\pmb{u}^m)\\ \textrm{s.t.} & \qquad \...
user avatar
  • 428
2 votes
0 answers
71 views

"Unspoken Tradeoff" in Convergence Rates for "Quasi-Newton vs. Gradient Descent"

Is there an "Unspoken Tradeoff" in Convergence Rates for "Quasi-Newton Methods vs. Gradient Descent"? As a quick summary: Gradient Descent based algorithms try to find the minimum ...
user avatar
2 votes
0 answers
61 views

Cyipopt fails to converge for NLP problem which fmincon() can solve

I'm currently trying to implement a python script for solving a constrained nonlinear optimization problem with ~800 variables and 2 constraints, one linear and one nonlinear. There already exists a ...
user avatar
2 votes
0 answers
32 views

Calculating a submanifold of minimal values

I have a (numerical, noisy) function $f$ that I'd like to optimize (in 3 < dimension < 10). The function $f$ has a fairly prominent valley/ravine structure due to physical symmetries present in ...
user avatar
2 votes
1 answer
270 views

Python solvers for MINLP in Pyomo in Google Colab

I am looking for a MINLP solver that works with Pyomo models which can be used in the Google Colab environment. I have already found MindtPy but it doesn't work in google colab.
user avatar
4 votes
0 answers
117 views

How amenable is this 2D Frenkel–Kontorova-like energy minimization problem in Python to the use of a modest PC + GPU? (Heavy reliance on indexing)

@Richard's answer to Going to try to move some of my scipy/numpy calculation to a new GPU, how to avoid disappointing results? is quite helpful, and as promised I've added a simple running example ...
user avatar
  • 852
9 votes
1 answer
480 views

How to find the smallest ellipse covering a given fraction of a set of points?

I have a set of points $P$ and want to find the ellipse with the smallest area that covers at least a fraction $f$ of these points. How can I do this? These questions ask the same thing, but folks ...
user avatar
  • 3,111
3 votes
1 answer
204 views

Going to try to move some of my scipy/numpy calculation to a new GPU, how to avoid disappointing results?

update: I've refactored the question based on helpful advice in the linked meta. I'm a heavy user of Python's NumPy and SciPy (and not much else) and for years I could run anything I need on my laptop....
user avatar
  • 852
12 votes
2 answers
1k views

How do I find the minimum-area ellipse that encloses a set of points?

I have a set of points that resembles more of an ellipse than a circle. I implemented the optimization formulation below and the solution gives a circle. I tried with various initial values, still to ...
user avatar
8 votes
2 answers
136 views

Approximating the boundary between two sets of points (in 2D): Fitting a region

Given two sets of points $p_{\text{in},i}$ and $p_{\text{out},j}$ inside and outside of what I intuitively call a "region", I would like to estimate and describe the boundary of this region. ...
user avatar
1 vote
0 answers
109 views

How to solve the minization problem with $Z^{T}Z=I$

I have the following problem $$\underset{Z}{\min}\left\|X-AZBH\right\|_F^2+\left\|H-B^{T}Z^{T}A^{T}X\right\|\\ \text{subject to } Z^{T}Z=I$$ How to solve the variable $Z$?
user avatar
  • 81
2 votes
1 answer
140 views

Is solving QP easier than a QCQP with linear objective?

Is solving a $QP$ (i.e.: quadratic program, hence a quadratic objective function with linear constraints) easier than solving a $QCQP$ (ie.: quadratic constrained quadratic problem) with linear ...
user avatar
  • 131
1 vote
1 answer
100 views

Finding the root of an equation (divergence at some points)

I am trying to solve this equation for $v_{Sl}$ in the range $[0.001,1]$ to obtain the values of $\delta_l$. I have tried Fsolve with python, which gives results ...
user avatar
6 votes
0 answers
138 views

FEM : energy minimization VS PDE solving

Engineering FEM When I studied engineering, I learned the traditional approach for finite elements for elasticity. The point was to solve the PDE $-div(\sigma)=f$ as: Multiply your PDE with a test ...
user avatar
  • 61
2 votes
1 answer
304 views

Difference between asymptotic and non-asymptotic convergence in optimization?

I am reading some optimization methods and I am facing some issues with two terms "asymptotic and non-asymptotic convergence". What is the difference between them?
user avatar
2 votes
1 answer
89 views

Formulating this optimization problem

Suppose I want to minimize below objective function $\sum | g(x_i) \cdot I_{g(x_i)<0} |^2$ i.e, the latter penalty terms like $ |g(x_i)|^2 $ are only computed when $g(x_i)<0$. $|g(x_i)|^2$ are ...
user avatar
5 votes
2 answers
115 views

Optimizing a quadratic form integral over unit sphere

I have an optimization problem, which is to maximize the following integral over the unit sphere: $$ \max_B \int d\Omega \mathbf{f}^{\dagger}(\theta,\phi) (B^{\dagger} + B) \mathbf{f}(\theta,\phi) $$ ...
user avatar
4 votes
1 answer
130 views

Parameters estimation with fewer variables than parameters

I am trying to estimate parameters, 4 of them, by fitting a system of 3 ordinary differential equations. I am using a model published that was using 3 parameters and gave a good fit to the data, and I ...
user avatar
  • 141
0 votes
0 answers
61 views

Comparing minimas of two different functions

The goal is to find vectors $x_u$ and $y_i$, both of the same length $f=64$, and to do this the following loss function is minimized: $$\sum_{u, i} (1 + \alpha \cdot r_{ui})(p_{ui} - x_{u}^{T}y_i)^2$$ ...
user avatar
4 votes
0 answers
64 views

Sample Average Approximation vs. Numerical Integration

To calculate the expected value of objective functions, we have two choices: Sample Average Approximation (SAA): $$ \frac{1}{N}\sum_{i=1}^N f(x,\xi^i). $$ Numerical Integration (e.g., Monte Carlo ...
user avatar
  • 41
0 votes
1 answer
62 views

Expressing a Constraint in an optimization problem

If I have a vector of M "continuous" decision variables (say it is called x) , and if I want a constraint to express that only one of them is allowed to have a nonzero value (i.e. no more ...
user avatar
1 vote
0 answers
80 views

Binarization for optimization problems

I have a nonlinear mixed-integer optimization problem, and because of very high complexity when solving it using methods like Branch and Bound, I resorted to solve it using alternating method and ...
user avatar
2 votes
0 answers
90 views

Adding a "cost term" to a linear regression, so solution values are minimized

I'm using Python's optimize.lsq_linear method to run a linear regression with the bounds set between 0% and 100% power usage. ...
user avatar
  • 121

1
2 3 4 5
19