Questions tagged [optimization]
This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.
826
questions
0
votes
0answers
22 views
Estimating the dimension of a solution space in nonlinear least squares
Suppose I have a nonlinear least squares problem,
$$
\min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2
$$
with $n$ residuals and $m$ parameters, so that $\mathbf{x} \in \mathbb{R}^m$, and $\mathbf{f} \...
2
votes
0answers
30 views
Non-negative Least Squares to perform Inverse Laplace with weights
I'm trying to perform the inverse Laplace transform of a (noisy) dataset $y_i$ using Tikhonov regularization:
$$\min \sum_{i=1}^{N} \left(\int_0^\infty e^{-s_i t} f(t) \, dt - y_i \right)^2 - \lambda^...
4
votes
0answers
16 views
Optimization for sampling multiple points of maximized minimum distance
I'm trying to find a way to sample new points that have maximum minimum-distance (maximin distance).
The current situation is where there are ns number of pre-existing sample points. I want N number ...
0
votes
1answer
28 views
Norm constraint in CVXPY
I'm trying to implement the algorithm outlined in https://arxiv.org/abs/1211.5608 on a small scale. I have a linear operator $\mathcal{A}$ which is defined as $$\text{trace}(A^*_l(hm^*))$$ where $$A_l ...
0
votes
1answer
26 views
How to check curvature of a vector valued function
In terms of numerical optimization, the newton-rapson method requires a pos. definite Hessian $\nabla^2f$ respectively pos. curvature for computing the next step $p_k$ by solving $$\nabla^2 f p_k = -\...
1
vote
1answer
24 views
Genetic algorithm: fitness proportionate selection using RMSD as fitness function?
I'm implementing a genetic algorithm to optimise $x$ so as to minimise the RMSD error $r(x)$ between my model and experimental data.
During the selection stage of recombination, I wish to select '...
1
vote
1answer
47 views
Making Hessian positive semidefinite
I have a large problem that I'm optimizing with Newton method. This involves a large sparse Hessian matrix.
For better convergence and not to get stuck prematurely, I'd like to make the Hessian ...
1
vote
1answer
74 views
Machine Learning for Optimization
I have a function which takes 100+ coefficients and outputs $x$. I wish to optimise $x$.
Running the simulation 50 000 times will take around 15 minutes, however, this happens in parallel - and the ...
4
votes
1answer
110 views
How to define a dimensionless Objective function for determining how peaked a curve is?
I have attached 2 plots for FFT spectra. One is considered good and one is bad.
The good one is classified on the basis of how closely spaced the frequencies and the bad is based on how multiple ...
1
vote
1answer
36 views
How do I get scipy.minimize to terminate below a certain loss threshold?
I was looking at the scipy.minimize documentation to see if I could find a way to terminate optimization when the loss gets below some cut-off, and I couldn't see ...
2
votes
1answer
79 views
A concave maximization that is not supported on CVX
I try to solve a maximization problem using CVX. In its simplest form, I want to maximize
$$f(x,y)=y*h_b\left(\frac{x}{y}\right),$$
where $h_b(\cdot)$ is the binary entropy function. In the context ...
0
votes
1answer
19 views
Python: Getting second output variable from minimizing a computationally intensive function on first outputs
I have a function in python that is quite computationally expensive to evaluate, of the form:
...
1
vote
2answers
91 views
Using MILP to place a set of primers along a genome
Define variables $p_i,u_i\in\{0,1\}^G$, for $i=1,\ldots,8$ and $G=30000$.
Let $v$ be a constant vector also in $\{0,1\}^G$, with approximately 25% of its entries equal to $1$ (randomly located).
Let ...
2
votes
1answer
34 views
Plotting optimum as a function of parameter in the objective
I am trying to minimize a 2d function using scipy.optimize. Specifically I want to plot the minimum value of the function fun as a function of the parameter wjk. The problem is that I cannot pass wjk ...
0
votes
2answers
68 views
How to minimize $(x-a)^2+(y-b)^2$ subject to $ \sqrt{a}+\sqrt{b}=\sqrt{2}$?
I am not sure if this is on-topic here, but I am trying.
Let $x,y$ be positive real numbers. I am trying to find
$$ \min_{\sqrt{a}+\sqrt{b}=\sqrt{2}}(x-a)^2+(y-b)^2$$
I tried using Mathematica for ...
0
votes
0answers
21 views
Can Scipy.optimize take a user-defined objective function that contains an ML model?
I have an optimization task that requires me to choose the optimal combinations of parameters, according to the prediction of a random forest model. My main obstacle is that scipy.optimize always ...
0
votes
0answers
18 views
Convex performance measure of classification
In the context of binary classifcation methods, I am looking for a performance metric that can be optimized in MATLAB.
Since the data is not balanced, a good choice seems to be the so-called F1-...
0
votes
0answers
37 views
Optimizing vectors with equal elements
I am trying to distribute power across different devices, so that the sum is as equal as possible to the power setpoint. At the same time, the sum of power per phase must not exceed the power of the ...
0
votes
1answer
28 views
Avalability of SNOPT optimization solver
I'd like to know if SNOPT solver is available free of cost for academic research in any of the optimization software packages.
I came across a few softwares that have SNOPT, but those require a ...
2
votes
0answers
50 views
Does BFGS preserve the bandedness of the inverse hessian?
In the BFGS method we perform iterations by calculating an approximation $\boldsymbol{H}_k$ to the inverse Hessian $\boldsymbol{H}$ of the objective function. This method belongs to a family of ...
-1
votes
1answer
46 views
Detecting degenerate triangles with very thin structures
Between the two ears in the following bunny images, there are some degenerate triangles I want to detect. It looks like a volume-less thin slits.
If the question is not clear, please let me know.
1
vote
0answers
39 views
Inverse Newton Method for optimization: is this the correct algorithm?
I am trying to implement the algorithm in this article. I have already asked a question before about it here, and I am trying to figure out what I am doing wrong. This time, it's this section of the ...
4
votes
2answers
65 views
MInimizing cost function using iterative search for a minimum method
I want to estimated the parameters $\ \hat{\theta} $ of a model using an iterative search for the minimum of a cost function. The cost function is defined as follows:
$$ V_N(\hat{\theta}) = \frac{1}{...
4
votes
1answer
121 views
Which optimization method can be used to do the following?
I've the following system of equations for studying information flow in the below graph,
$$ \frac{d \phi}{dt} = -M^TDM\phi + \text{noise effects} \hspace{1cm} (1)$$
Here, M is the incidence ...
2
votes
2answers
95 views
What online optimisation algorithm can be used for a noisy cost function?
I am trying to optimise a function, but the function can be noisy and give varying results for the same parameters. Furthermore, it needs to be online, as the data from each new iteration happens ...
3
votes
2answers
188 views
log(det(X)) in Semidefinite Programming
I have been solving problems of the form $$max \ log(det(A)) \\ s.t. \ A = A^{T} \succeq 0, \\ p_{i}^{T}Ap_{i} \leq b_{i}$$ where $b_{i}$ and $p_{i}$ are input vectors (to be clear there is more than ...
4
votes
1answer
86 views
Arbitrary Precision Optimization Libraries?
Are there any well-known optimization libraries (ideally with Python bindings or even in Python) supporting (unconstrained) minimization (of $f:\mathbb{R}^n \to \mathbb{R}$ for $n$ for $n\sim 10^1,10^...
0
votes
0answers
34 views
exploding gradients in gradient descent procedure of multi-output ridge regression
Multi-output ridge regression:
$$W^{*}=\underset{W}{\arg \min } \frac{1}{\mathcal{N}}\|Y-WX\|_{F}^{2}+\lambda\|W\|_{F}^{2}$$
There are $Q$ outputs, $N$ samples, and $P$ covariates (features).
$\hat{...
4
votes
1answer
70 views
How to form the following constraint in cvx?
The optimization problem is
$$\min_{x\in K} \|h - x\|_2$$
where
$$K = \{v\in R^n : \exists \lambda \geq 0\ v_1=v_2=\ldots=v_k=\lambda \ \text{and} \ |v_i| \leq \lambda \ \text{for} \ i=k+1,\ldots,n \...
3
votes
1answer
107 views
What are the advantages of Level Set method in topology optimization?
I am studying the different topology optimization methods. There are numerous resources out there but when it comes to comparing different algorithms, in terms of strengths and weaknesses most of ...
0
votes
0answers
55 views
Are there unproblematic max constraints when modelling problems as Linear Programs?
Suppose we have a linear objective function that we want to maximize.
All variables are from the set of reals.
We have a constraint of the form:
$$\max(x_1,x_2) + \max(x_3,x_4)\leq c\,, \text{ with } ...
1
vote
0answers
18 views
SHREC 2010 Descriptors
I will appreciate if I may find someone how can clarify for me the part regarding the quality of feature descriptor, shown in the figure below:
and this screenshot is from the article: SHREC
All my ...
2
votes
0answers
88 views
Proving convexity of Frobenius norm and correlation function formulations of an optimization problem
I have been working on formulating my requirements in the form of an optimization problem in a multi-output regression setting.
Firstly, I would like to make the variables I used in the problem and ...
3
votes
1answer
66 views
Evaluate 3D Shape Descriptor
I'm trying to create my own 3d shape descriptor, the idea is that how I may evaluate how much my descriptor is well and good?
What I checked is that they evaluate descriptors through shape matching, ...
1
vote
0answers
59 views
Conjugacy in Non-linear Conjugate Gradient Descent
In linear conjugate gradient method, our goal is to solve the system of linear equations $$Ax = b$$ where A is a symmetric positive definite matrix, and that is equivalent to finding the minimizer of ...
0
votes
0answers
20 views
How to use the solution of a multistage stochastic program?
Given a multistage stochastic program, its solution (if it exists) consists of the first decision vector, as well as all the recourse decision vectors for all possible scenarios of an event tree. But ...
8
votes
1answer
288 views
When training a neural network, why choose Adam over L-BGFS for the optimizer?
More specifically, when training a neural network, what reasons are there for choosing an optimizer from the family consisting of stochastic gradient descent (SGD) and its extensions (RMSProp, Adam, ...
2
votes
1answer
54 views
Could the convex problem be tackled by CVX?
I want to solve the convex optimization as follows:
\begin{align}
\underset{X_1,X_2}{\min} &\ -\frac{1}{N}\sum_{i=1}^N\log\det\left(I+H_i^HX_2H_i\right)-\log\left[1+h^H(X_1+X_2)h\right]\\
&\...
2
votes
0answers
41 views
Fitting a plane with the Prewitt gradient operator
Prewitt gradient operator
Show that the Prewitt gradient operator can be obtained by fitting the least-squares plane through the 3 × 3 neighborhood of the intensity function.
Hint: Fit a plane to ...
4
votes
0answers
42 views
Nonlinear least squares optimized Jacobian calculation
I have a nonlinear least squares problem, in which I am trying to minimize residuals which can be divided into four classes:
$$
\min_x ||\epsilon(x)||^2 + ||\xi(x)||^2 + ||\delta(x)||^2 + ||s(x)||^2
$$...
1
vote
1answer
69 views
Prove that the set of maximizers are independent of parameter in the objective function
A maximization problem reads as
$$ J(y) = \sum_{k=1}^{K} \sigma_k(y)^q \mathop{\rightarrow}^{y} max$$
where $q \in [1,\infty]$ is a user-defined parameter and functions $\sigma_k, k=\{1,\dots,K\}$ ...
3
votes
0answers
51 views
Methods to approximate obective function gradients from point cloud
Problem statement:
Assume that I have an objective function $f(x)$ which takes as input a $D$-dimensional vector $x\in\mathbb{R}^D$, and that $f(x)$ is sufficiently smooth. Assume further that I ...
1
vote
0answers
67 views
How to approach geographic data interpolation by distance?
let's say I have a set of geographic locations (lat, lng) resulting from a query. Those locations have some kind of internal ranking, my set is sorted by this number in a descending order.
Now I'm ...
2
votes
1answer
63 views
How to obtain only the value of my variable using scipy.optimize.minimize
when I minimize a function using scipy.optimize.minimize I get a big list of things as a result, but I would like to only get the value of my variable, this is my code
:
...
2
votes
0answers
56 views
Multilevel minimization - boundary conditions
I am interested in minimizing
$$min_{x \in R^{n^l}} f^l(x),$$
where $f^l(x)$ is nonlinear objective function arising from discretization of PDE. I would like to use nonlinear multilevel minimization ...
3
votes
1answer
75 views
Calculate Transformation Matrix between two sensors
My question is if I can calculate the transformation matrix between two sensors.
Each sensor provides a $4\times 4$ matrix for every timestep recorded.
The sensors are moving and have some noise in ...
3
votes
2answers
54 views
Optimal line such that maximum points are between an upper and lower boundary
I have some 2D data and would like to find a line $y = mx + b$ such that a maximum number of points from the data is captured within the area between $y = mx + b + margin$ and $y = mx + b - margin$. ...
6
votes
2answers
198 views
How to solve calculus of variations problems numerically?
For example, how to solve the well-known isoperimetric problem (i.e., to enclose the largest area with a fixed-length curve)?
We can simplify things a bit and fix the two ends of the curve at $[a,0]$,...
0
votes
0answers
33 views
Method for implementing QP solver with matrix terms?
I am trying to implement (in code) a QP solver for the following equation:
$$\min_{u} u^{T} Wu$$
$$s.t. \; \beta u = \tau_{ref}$$
$$ Au \leq b $$
See this document, section 5.1 (Page 35)
$u$ is a ...
1
vote
0answers
69 views
Ramp least squares estimation
With some given $s$ value, let
\begin{equation}
\begin{aligned}
h(\beta)&=\min(\sum_{i=1}^n(Y_i - X_i\beta)^2, s)\\
&=\sum_{i=1}^n(Y_i - X_i\beta)^2-\max(0, \sum_{i=1}^n(Y_i - X_i\beta)...
