Questions tagged [optimization]
This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.
762
questions
1
vote
0answers
31 views
least squared optimization
I want to decompose a list of 3D vectors $X_j$ as linear combination of five 3D verctors $C_k$
$$X_j= \sum_{i=1}^{5}{w_{ji}C_i}$$ both $X_j$ and $C_i$ are 3 components vectors
$$C= \begin{bmatrix} ...
1
vote
0answers
71 views
How to perform local sensitivity analysis for partial differential equations
I am looking for a way to do local sensitivity analysis for PDEs, preferably in Python.
I get the impression that discretizing the equation then treating it as an ODE could work; however, would that ...
2
votes
1answer
112 views
Simultaneously maximize and minimize
I am virtually new to optimization (saw it years ago in a very shallow course) and now I came across a problem that I believe would require from it. The problem is I don't know exactly how to proceed.
...
1
vote
0answers
44 views
Kinetic preconditioning
Publication arXiv:0804.2583 describes a method for doing self-consistent iteration without having to diagonalize the Hamiltonian operator at every step.
IX. PRECONDITIONING
As already ...
0
votes
0answers
41 views
Need an example Legendre-Gauss-Radau pseudospectral differentiation matrix or Matlab code
I'm trying to implement various kinds of pseudospectral methods for direct optimization in Matlab using IPOPT. I've got some working Legendre-Gauss-Lobatto code, but would like to use the flipped ...
3
votes
1answer
73 views
Geometric Programming - symbolic version
I am interested in finding minimizers of functionals of the type $\sum x^ay^bz^c$ where the exponents are 1, 0 or -1. I have codes to find such minimizers when they exist up to machine precision, ...
1
vote
0answers
57 views
How to use Wolfe-Powell step-size control in quasi-Newton method?
I'm trying to find the minimum of a function using the quasi-Newton method with the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm.
But I want to change the following implementation, so that:
1) ...
3
votes
0answers
43 views
Constraint solver vs Bayesian optimizer for fast discontinuous processes
I have a complex domain-specific process that accepts inputs:
10-500 inputs, where each input is of type:
enum: choice between multiple string or numeric values
int: integers
float: floating point ...
1
vote
0answers
65 views
Best optimizer for unconnstrained non-convex nonlinear least-square optimization problem?
I am looking for a very good optimizer to the following problem:
$$\min_{P,\Theta}\lVert APD(\Theta)P^{-1} -B \rVert_F$$
where $A,B \in \mathbb{R}^{n\times m}$, $P \in \mathbb{R}^{m\times m}$, $D\in \...
1
vote
1answer
59 views
Pivoted Cholesky vs Modified Cholesky
I am solving nonlinear least squares problems with the normal equations approach, so on each iteration, I need to solve:
$$
J^T J \delta = -J^T f
$$
for the step $\delta$, where $J$ is a large (...
1
vote
0answers
41 views
Finite dimensional optimization problem over dynamical system
I am interested in solving numerically the following mathematical problem
Consider an ode of the form
$$
\dot q(t) = f(q(t),t_1,\ldots, t_N),\qquad t\in [0,T],
$$
where $q\in \mathbb{R}^n$ is the ...
5
votes
0answers
93 views
How to solve a 4th order nonnegative LASSO problem?
I need to solve the following 4th order nonnegative LASSO problem:
$$
\min_{x \geq 0} \quad || |Ax|^2 - b ||^2 + \lambda ||x||_1
$$
where $|\cdot|^2$ denotes element-wise squared. $A$ is small size (e....
4
votes
1answer
36 views
Single-variable multimodal derivative-free optimization (for a well-behaved function)
Are there well-established approaches to single-variable multimodal optimization?
Given $f:\mathbb{R}\rightarrow\mathbb{R}$ that:
has several local minima within a given range of interest $[a,b]$
is ...
4
votes
0answers
58 views
Fast approximate solver for vehicle routing problem
I need to solve capacitated asymmetrical vehicle routing problem with time windows on ~30k points. Time limits for calculations are 2 hours. I've tried using Clarke and Wright savings algorithm, it is ...
1
vote
0answers
51 views
User friendly scipy optimize wrapper package?
I'm creating too much throw away code for interfacing with the scipy optimize package in a user friendly way. (See code below for example of interruptible optimization that keeps last optimization ...
3
votes
1answer
85 views
Why would BFGS converge to a local minima of a non-convex function but maintain a large gradient?
I'm using BFGS to optimize a smooth but non-convex function $f$ that is computed by a simulation. The simulation also gives me a semi-analytical gradient $g$, which is verified by the numerical ...
2
votes
1answer
75 views
How to compute the determinant of Hessian of a multivariable function?
I have a function $F(\vec x)$ of many variables (let's say in the order of hundreds of thousands). I need to compute the determinant of the Hessian matrix at the point $x_0$.
Is there a way to ...
3
votes
2answers
65 views
Convexity of Sum of $k$-smallest Eigenvalue
If I have a real positive definite matrix $A\in\mathbb{R}^{n\times n}$, and denote its eigenvalues as $\lambda_1\leq \lambda_2 \leq ... \leq \lambda_n $.
Define the function as $f(A)=\sum_{i=1}^{k} \...
5
votes
2answers
109 views
Choose a subset of $m$ columns that maximize $|A^T A|$?
I have a set of $n$-dimensional vectors, and would like to choose $m$ of them to become the columns of an $n\times m$ matrix. I would like to choose the subset that maximizes $|A^T A|$, where $A^T$ is ...
0
votes
0answers
32 views
Inverse kinematics BFGS divergence
I am trying to implement inverse kinematics solver using BFGS as stated in the paper Xia2017.
In the test experiment, i created 4 objects in 3-dimensional space: Node, Node1, Node2, Node3. Each Node ...
0
votes
0answers
31 views
Parameter estimation using fmincon
This is a follow up to my previous question posted here.
I am solving an optimization problem using fmincon in MATLAB. There are no equality constraints in my model....
0
votes
0answers
43 views
Parameter estimation using shooting method
I want to do the following,
I have a set of 20 first order differential equations and I want to estimate some of the parameters.
I've got the following initial and boundary conditions.
The initial ...
3
votes
0answers
58 views
Solve ODE with non-negative and maximization constraints
My task is to solve $$\eta_k\frac{d^2C_k}{dz}(z)=-e_k, k = 1,2,3$$
$$C_k\ge0$$
$$C_1(0)=0, C_2(0)=A, C_3(0)=0$$
$$C_1(L)=B, \frac{dC_2}{dz}(L)=0, \frac{dC_3}{dz}(L)=0$$
with $$e_1 = -\beta_1-\beta_3$...
1
vote
1answer
79 views
Is there a name for this integer linear optimization problem?
I have an integer linear programming problem of the form:
$$\DeclareMathOperator{\tr}{tr} \min \tr WX$$ subject to:
$$\begin{align}
\sum_j X_{ij} < c_i && \forall i \\
\sum_i X_{ij} = 1 &...
1
vote
1answer
125 views
How do I solve the matrix equality constrained optimization problem using Lagrangian multipliers?
Solve the following minimization problem in $\mathbf{X} \in \mathbb{R}^{m \times n}$
$$\begin{array}{ll} \text{minimize} & \frac 12 \| \mathbf{X}\mathbf{X}^T -\mathbf{A} \|^2_\mathcal{F}\\ \text{...
2
votes
1answer
89 views
Formulation of the least-squares parameter estimation problem
I have a system of 10 ordinary differential equations of the form,
$$\frac{dy_1}{dt} = f1(V1,k1,y1,y2)\\
\vdots \\
\frac{dy_{10}}{dt} = f_{10}(V_{10},k_{10},y_{9},y_{10})
$$
I want to estimate the ...
0
votes
0answers
29 views
Scipy basinhopping custom step update and constrained looping
I am searching for the global minimum of a certain function and trying to use its gradient (here same as Jacobin) to guide the step counter. However, my x is fix ...
0
votes
0answers
13 views
Finding Maximum Value of CST Parameterization over an interval
I have a CST parameterization for a shape over an interval (0,1), so I have y as a function of x like so $$y = C(x)*s(x)$$ where $$C(x) = x^{n1}*(1-x)^{n2}$$ and $$S(x) = \sum_{i = 0}^{n} A_i(x)^i(1-x)...
0
votes
0answers
22 views
convex atomic function reformulation to meet concave dcp rule requirements
I have an atomic constraint of the form
abs(w - w_prev) >= some_threshold
It is supposed to get every value equal to or above my threshold.
I am working on a ...
2
votes
0answers
37 views
Biconvex problem whose objective function depends on only one variable
I am solving the following biconvex problem:
$$\min_{x,y} f(y)$$
$$s.t. ~~ g(x) \leq 0$$
$$~~~~~h(x,y) = 0$$
$$x \in X, y \in Y$$
where $X$ and $Y$ are compact convex sets, $g(x)$ and $f(y)$ are ...
4
votes
1answer
86 views
Examples of problems that cannot be formulated as optimization problems
An optimization problem has 3 main components: decision variables, constraints and an objective function. Such a problem can be mathematically modelled and solved using an optimization solver. For ...
0
votes
1answer
64 views
Solve multi-dimensional optimization problem using basinhopping
I am searching for an optimization solution, which is a 8d vector representing 4 complex elements, where each element is within the complex circle with maximal radius 1.2.
The objective function is:
...
3
votes
2answers
94 views
Optimization techniques for expensive multi-variable functions
I'm working with a finite element model in which I'm interested to minimize the average temperature at a surface. I have 15 independent variables in my model, including geometry, materials, flows, ...
2
votes
3answers
135 views
Find a solution of large system of inequalities
I have a large system of homogenous inequalities involving 33 real unknowns of the form
$$
\vec{F}(z_i)^T \cdot \vec{X}>0\,
$$
where $\vec{X} = \left(x_1,...,x_{24}\right)^T$ are the unknowns and ...
3
votes
1answer
58 views
Non-parametric models as solutions to Partial Differential Equations
In the realm of scientific computing, there are a plethora of techniques developed to solve Partial Differential Equations (PDEs). Many of the popular methods are variants of common techniques such as ...
5
votes
1answer
66 views
What is the name for this type of constraint?
I have what would be a straightforward mixed-integer linear programming problem, except for the fact that some of the constraints are of the form $f(x_1,x_2,x_3,\ldots,x_n) < c$, where $f$ is 'take ...
0
votes
0answers
13 views
Cost functions to judge time/memory/accuracy tradeoffs
I am working on an interesting algorithm: Its absolute error is exponential in a parameter $j \in \mathbb{N}$, and for a given $j$, I have complete freedom to choose between an $\mathcal{O}(1)$ time-...
0
votes
1answer
18 views
Minimize squared error of linear function
Let $M$ be a $m \times n$ matrix, $x$ a $n$-vector, $y$ a $m$-vector, and $\|\cdot\|_2$ represent the $L_2$ norm (i.e., Euclidean norm). Given $M,y$, the goal is to find $x$ that minimizes the ...
0
votes
0answers
31 views
Minimizing the ratio of two specific non negative quadratic convex functions
$F$ is $m\times m$ diagonal, with real non negative elements
$D$ is $n \times m$ complex
$P$ is $n \times 1$ complex
$A$ is $m \times 1$ complex.
Minimize $\Gamma(A)$, with respect to $A$.
$$\...
2
votes
1answer
85 views
Is there an optimization scheme/algorithm that converges, for this non-convex scenario but with some special properties
I have a smooth function $f(x) = \frac{g(x)}{h(x)}$ that is the ratio of two smooth convex functions $g(x)$ and $h(x)$. It is known that $f(x)$ has a global minimum, achieved at the unique point $x_0$....
0
votes
0answers
51 views
Simplification of an optimization objective
Let $G(V,E)$ is a weighted simple graph, where $V$ and $E$ are the set of vertices and Edges. The graph is undirected. Let $A \in \{0,1\}^{n\times n}$ and $W \in R_+^{n\times n}$ be the adjacency ...
0
votes
1answer
37 views
Vehicle Route assignment with capacity constraint
Problem Background
I'm trying to find a solution/model to the following problem:
Let's consider a cellular network (mobile network, ie., hexagonal cells) denoted $N$ composed of $|N|$ cells. Each ...
7
votes
1answer
197 views
Lack of quadratic convergence in Newton's method
It is well-known that Newton's method can converge quadratically, if initial guess is close enough and if the arising linear systems are solved accurately.
I am applying Newton's method to highly ill-...
3
votes
1answer
88 views
Minimize a function with sparse Hessian
The problem I am trying to solve involves minimising a function with respect to a large number (probably 10,000+) of parameters. I can cheaply compute both its Jacobian and its Hessian. The Hessian is ...
0
votes
0answers
34 views
In-exact line search
In my class notes, the author says:
"If $f:\mathbb{R}^n \to \mathbb{R}$ is bounded below and $p_k$ is a descent direction and the $\alpha-\beta$ also known as Armijo-Goldstein condition is met then ...
2
votes
0answers
39 views
Is this a form of stochastic gradient descent?
I want to minimize the following with respect to parameters $B$.
$$\sum_{k = 1}^{K} f(A_{k}, B)$$
where $A_k$ are $K$ different data-sets and $B$ is a matrix of parameters.
Can I do this by a ...
0
votes
1answer
23 views
Gradient ascent method with a constant step size?
I'm trying to use the gradient ascent method on a convex function like the multivariate-Normal density function with respect to its parameters (the original is a bit more complicated), something ...
-3
votes
1answer
78 views
4
votes
3answers
252 views
Best software to do big number calculations quickly
I am trying to do some work on some math conjecture. I am testing the conjecture numbers using very large math numbers (100+ digits ). I am currently using python to test these numbers.
In the ...
2
votes
1answer
56 views
Question about strange outputs from the CVXPY solver
I am familiarizing myself with CVXPY, and encountered a strange problem. I have the following simple toy optimization problem:
...
