Questions tagged [optimization]
This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.
0
votes
0answers
44 views
Order of a principal term
In Yurii Nesterov's Introductory Lectures on Convex Optimization, there is a bound for the total number of iterations for some process. See page 109:
$$\left[\frac{1}{\ln(2(1-\kappa))} \ln\frac{t_0-t^...
1
vote
0answers
24 views
Logging vs outputs in iterative optimisation
I'm coding an iterative algorithm of constrained continuous optimisation. An augmented Lagrangian algorithm (outer) calls a bound-constrained L-BFGS-B algorithm (inner), which calls a line search ...
3
votes
0answers
46 views
Nonlinear functional optimization in radial coordinates
I am currently implementing classical density functional theory for a radially symmetric system.
In mathematical terms, I am searching for a function $f(r)$ that minimizes a functional $\Omega[f]$.
...
2
votes
0answers
68 views
How to numerically optimize affine transformations?
I need to optimize affine transformations for of a set of triangles using energy function based on the connectivity.
The energy of an edge $e_j$ between triangles $T_a, T_b$ is given by
$$
E_j = \...
0
votes
0answers
22 views
conditioning of complex objective function via change of coordinates
I want to minimize a function $f(z) = z^H A z + c_1^T \theta + c_2^T \log(r)$ where $z = r e^{i\theta}$, $z\in \mathbb{C}^n$, $A \in \mathbb{R}^{n\times n}$ is symmetric positive definite, and $c_1, ...
0
votes
0answers
12 views
R function or package for carrying out maximum likelihood techniques in random effect models
I am applying optim() function in R to obtain maximum likelihood estimates of the fixed effects and random effects in a model with bivariate random effects. The code works well but it is very slow in ...
0
votes
0answers
39 views
Euler-Lagrange Theorem and Conditions of Optimality
Can anyone explain me in simple language why applying Euler-Lagrange Theorem on Hamiltonian equations gives the optimal control law?
I am specifically talking about trajectory optimization problems ...
3
votes
0answers
91 views
Difference between Dishonest Newton method and Very Dishonest Newton method
What is the difference between the Dishonest Newton method and the Very Dishonest Newton method? Is there a difference or do they mean the same thing?
I have tried searching for this on the internet ...
1
vote
0answers
30 views
Efficient numerical optimization of an “almost separable” function
I have come across an optimization problem with the following objective function:
$$f(x_0,y_0,z_0,x_1,y_1,z_1,...,x_N,y_N,z_N) = \sum_{i=0}^N f_i(x_i,y_i,z_i, \alpha(x_{i+1}-x_i) + \beta(y_{i+1}-y_i))...
2
votes
1answer
45 views
Convergence rate and complexity for convex minimization problem
In Yurii Nesterov's Introductory Lectures on Convex Optimization, there is a description of the rate of convergence and corresponding upper bound for the analytical complexity of a minimization ...
2
votes
1answer
29 views
Log-transformation of decision variables in parameter estimation
I am trying to find the diffusion coefficient ($D$) and the partition coefficient ($KLP$) using experimental data of desorption of a pollutant from a film into a liquid. This process can be modelled, ...
3
votes
0answers
30 views
Stochastic conjugate directions to improve convergence in narrow valleys
My question concerns a specific statement in this paper:
N. N. Schraudolph and T. Graepel, "Conjugate Directions for Stochastic Gradient Descent," in Int. Conf. Artificial Neural Networks, Berlin, ...
3
votes
1answer
114 views
How does fmincon in MATLAB calculate gradients?
I am trying to solve numerically a constrained optimisation problem in MATLAB, and I am wondering how the fmincon function calculates gradients when one isn't ...
1
vote
1answer
35 views
What is a good way to select a small subset (say 50) of items from a large pool of items (say 5 million) while minimizing an objective function?
I have 5 million items that have 10 features (all continuous and not categorical) each and would like to select a small subset of these items. Ideally, I want to manually specify 10 features of my own ...
0
votes
1answer
67 views
Simulation-based Optimization vs PDE-constrained Optimization
What is the difference between Simulation-based Optimization and PDE-constrained Optimization?
Would studying a text on Simulation-based optimization be sufficient to understand and apply both?
1
vote
0answers
41 views
Artificial Neural Network for nonconvex optimization
I want to use artificial neural network(ANN) for solving non-convex optimisation problems. How do I map the constraints and the objective functions in the ANN architecture?
3
votes
1answer
60 views
What is a “good enough” method of assigning values to n variables subject to basic bounding constraints while maintaining relative weights?
Given triples of $n$ floating point values
$$(\min_1, \max_1, w_1), \dots, (\min_n, \max_n, w_n)$$
and a value $V$, what is a good algorithhm to assign values $v_i$ to each of the triples such that ...
2
votes
1answer
118 views
Can the Power Method be used here?
Given a set of $n$ points on which a triangulation is performed, it is possible to construct coefficients $\lambda_{ij}>0$ such that each point $x_i$ is a convex combination of the points connected ...
0
votes
0answers
45 views
Finding attribution of coefficient in a matrix
I have the following $d*n$ matrix in $\{0, 1\}$
\begin{bmatrix}
x_{11} & x_{12} & x_{13} & \dots & x_{1n} \\
x_{21} & x_{22} & x_{23} & \dots & x_{2n} \\
...
4
votes
0answers
76 views
What Derivative-free optimization method should I use when my initial guess is very good?
I am trying to minimize a function where my initial guess is quite close to the minimum.
I'm trying to minimize
$$f(q) = \text{angle}(qw_1q*, v_1) + \text{angle}(qw_2q*, v_2) + \text{angle}(qw_3q*, ...
5
votes
0answers
60 views
conjugate gradient for Newton's method with non positive definite Hessian matrix
I want to minimize a non-linear function $f(x)$ using Newton's method. At each optimization step, I compute a descent direction $d$ to update $x$ using a second-order approximation of $f(x)$:
$$ \...
3
votes
0answers
72 views
Inconsistency in optimize.minimize
I am trying to fit a time-dependent curve at each time step. I do so in minimizing along $x_c$ the quadratic error between the curve and a reference solution $ 1/(1 + \exp\left(\sqrt{S}(x-x_c)\right) $...
3
votes
1answer
32 views
Software for finding a minimum vertex cover for a hypergraph
A hypergraph $H = (V,E)$ consists of a finite set of vertices, say $V=\{1, \dots, n\}$ and a set of hyperedges $E \subseteq \mathcal{P}(V)$. We call $H$ a $k$-hypergraph if all $|e| = k$ for all $e\in ...
0
votes
0answers
40 views
Metric selection and scale-based preconditioning in quadratic optimization problem
I'm going to use scale-based preconditioning in a quadratic optimization problem: minimize $ x^T Q x + p^T x$ such that $ A x + b = 0$ and $D x + E \leq 0$, I want to speed up finding the optimal $x$ (...
0
votes
0answers
28 views
Converting logit coefficients into probabilities through the inverse multinomial logit
I am trying to come up with the transition intensities of a parameterized Markov model, i.e, the elements of the transition matrix weighted by the covariates through the inverse Logit relation:
$$\...
5
votes
1answer
117 views
Generate approximately semi-orthogonal tall matrix approximately satisfying constraints
I have a set of matrices $\{(A_i,D_i)\}$ for $i\in\{1,\ldots,n\}$, where:
Each $D_j\in\mathbb{R}^{S\times S}$ is diagonal, and every entry on the main diagonal is non-negative.
Each $A_j\in\mathbb{R}^...
5
votes
1answer
77 views
Solvers for Quadratically Constrained Quadratic Programs (QCQP) with complex variables
I'd like to know whether there are any publicly available tools for solving QCQP with complex variables (and constraints therefore expressed through Hermitian matrices). What I have found so far is ...
4
votes
1answer
181 views
How to solve the following Frobenius norm-minimization problem?
Background
We know how to solve the following minimization problem
$$
\min_{X} \lVert AX - B \rVert_F^2
$$
But what about the extended version?
$$
\min_{X} \lVert A
\begin{bmatrix}
X & X^2
\...
3
votes
1answer
168 views
How to solve the inverse problem of least-squares?
Focusing on following least squares problem:
$$\min\limits_{V} \lVert Z - WV \rVert _{_F}^2$$
$$Z∈{R}^{m\times n},\quad W∈{R}^{m\times k},\quad V∈{R}^{k\times n},\quad k\lt m\lt n $$
This problem ...
1
vote
1answer
174 views
How to use CSDP to express a semidefinite program?
I am trying to use CSDP and am struggling with it. Consider, for example, the following semidefinite program
$$\begin{array}{ll} \text{minimize} & 0\\ \text{subject to} & Q - A' Q A - \...
1
vote
1answer
106 views
FMINCON Step Size Tolerance
I get following error after implementing the attached code.
Error Message
"fmincon stopped because the size of the current step is less than
the default value of the step size tolerance but ...
1
vote
0answers
102 views
How to prevent BFGS from getting stuck on astronomically large gradient?
I have implemented BFGS myself from scratch in order to solve minimization problems. Part of BFGS, as I understand it, is that the approximation to the Hessian is supposed to be positive definite, ...
1
vote
0answers
32 views
Best Possible Convex bounds for optimization problems
Suppose we have a primal problem
$$
p^{*}=\min_x f(x), \\\text{s.t.}~~ h_i(x) \leq 0,
$$
where $f(.)$ and $h_i(.)$ are possibly non-convex.
Then its Lagrangian is
$$\mathcal{L}(x,z_i)= f(x) + \...
1
vote
0answers
59 views
Nonlinear least square optimization
Problem description
Given data at many time instance $t$,
$$\min _{\alpha, \Lambda, \beta} \lVert y(t) - \alpha e^{\Lambda t} \beta \rVert_F$$
with $$ \lVert \alpha \rVert_2^F = 1 $$
where $y(t) \...
1
vote
0answers
38 views
Space covering optimization
I have the following problem:
In the space $E=\{1, 2, \dots, N_x\} \times \{1, 2, \dots, N_y\}$, I want to define $N_R$ rectangles $R_k=\{x_k^0, \dots, x_k^1\}\times\{y_k^0, \dots, y_k^1\}$ which ...
0
votes
0answers
41 views
Optimization of a continuous function
This is more like an optimization problem but any solution is appreciated.
I have a data set with input specifying power(demand) to be generated for a particular time period(TP).
Input:
Time --- ...
1
vote
1answer
65 views
Some proof that linear translations and rotations of a bound-constrained function are equivalent
For example, I have a function to optimize:
$$f_1(x,y) = x^2+y^2, \quad x_{lb}\le x\le x_{ub},\quad y_{lb}\le y\le y_{ub}$$
Then I apply rotation by $\theta$ plus translation by $x_0$ and $y_0$:
$$f_2(...
0
votes
1answer
79 views
ill-conditioning
I am struggling with following exercise from book of Nocedal, Numerical optimization, chapter 2, excercise 2.12:
Suppose that a function $f$ of two variables is poorly scaled at the solution $x^*$. ...
3
votes
2answers
131 views
How do I check if a loss function can achieve its minimum?
For example, the convex function $f(t)=e^{-t}$ doesn't achieve its minimum 0 on the real line. In a linear regression with $p$ predictors $X$, the loss function $f(\beta)=||Y-X\beta||^2$ achieves its ...
0
votes
1answer
79 views
Can I convert CUDA core to CPU core and use it as cpu core while running any program?
I was using Metatrader5 and have designed a strategy for trading using MQL5 programming language.
While I was running a Strategy Optimization process, I saw the it will need 10,00= iterations or ...
0
votes
1answer
43 views
Conflicting definition of limit point
This question was raised at a different place without sufficient answers.
Definition 1:
We say that a vector $x \in R^n$ is a limit point of a sequence $\{x_k\}$ in $R^n$ if there exists a ...
0
votes
0answers
48 views
Methods for Parameter Scaling in Gradient-based Optimization
I am trying to minimize an objective function with 4 parameters, e.g., $a,b,c,d$ using gradient descent. $a < 0.1$, while $0 <b,c,d < 10$.
I'm using a learning rate for all parameters on the ...
6
votes
1answer
200 views
What is required of the objective function in order to use Gauss Newton method?
From what I understand, the Gauss-Newton method is used to find a search direction, then the step size, etc., can be determined by some other method.
In addition to that, are the following ...
1
vote
1answer
40 views
How to decide on what techniques to adopt in Genetic Algorithm optimization
Does it really matter which techniques we use in the process of GA optimization? For instance, if I use the Roulette Wheel Technique instead of Tournament Method for selection, two-point crossover ...
0
votes
1answer
266 views
I've developed a derivative-free optimization method, looking for comments
Here is the URL: https://github.com/avaneev/biteopt
I've tested it on numerous global optimization benchmarking functions (included), and on real-world hyperparameter optimization problems I have. ...
0
votes
0answers
50 views
Clarifying a dual problem solution
I’m referring to this question on MathSE about the sum of $k$ largest eigenvalues of a symmetric matrix as an SDP-problem.
I want to solve the maximization problem. As the dual problem is always ...
3
votes
1answer
112 views
How to numerically minimize a functional?
How to numerically minimize a functional, for example,
$$J[y]=\int_{x_1}^{x_2}L(x,y(x),y'(x))dx$$
An equivalent problem is to solve the Euler equation for this functional as a differential equation. ...
1
vote
1answer
40 views
Distribute sources among destinations
There are $n$ sources with the following positive volumes: $p_1, ..., p_n$ and there are $m$ destinations with the following positive volumes: $q_1, ..., q_m$. It is known that $p_1+ ...+ p_n=q_1+ ...+...
7
votes
3answers
455 views
Finding the first N roots of transcendental equation
I need to find the first $n$ roots of the transcendental equation
\begin{equation}
F(k) = J_m'(kr)Y_m'(k)-J'_m(k)Y'_m(kr)
\end{equation}
for integer values of $m$ and any $r \in [0,1)$ where $J'$ ...
0
votes
1answer
144 views
Defining a soft constraint in cvxpy
I am using cvxpy to do a simple portfolio optimization.
I implemented the following dummy code
...
