Questions tagged [optimization]

This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.

Filter by
Sorted by
Tagged with
2
votes
0answers
134 views

How to prevent BFGS from getting stuck on astronomically large gradient?

I have implemented BFGS myself from scratch in order to solve minimization problems. Part of BFGS, as I understand it, is that the approximation to the Hessian is supposed to be positive definite, ...
1
vote
0answers
33 views

Best Possible Convex bounds for optimization problems

Suppose we have a primal problem $$ p^{*}=\min_x f(x), \\\text{s.t.}~~ h_i(x) \leq 0, $$ where $f(.)$ and $h_i(.)$ are possibly non-convex. Then its Lagrangian is $$\mathcal{L}(x,z_i)= f(x) + \...
1
vote
0answers
66 views

Nonlinear least square optimization

Problem description Given data at many time instance $t$, $$\min _{\alpha, \Lambda, \beta} \lVert y(t) - \alpha e^{\Lambda t} \beta \rVert_F$$ with $$ \lVert \alpha \rVert_2^F = 1 $$ where $y(t) \...
1
vote
0answers
43 views

Space covering optimization

I have the following problem: In the space $E=\{1, 2, \dots, N_x\} \times \{1, 2, \dots, N_y\}$, I want to define $N_R$ rectangles $R_k=\{x_k^0, \dots, x_k^1\}\times\{y_k^0, \dots, y_k^1\}$ which ...
2
votes
1answer
98 views

Some proof that linear translations and rotations of a bound-constrained function are equivalent

For example, I have a function to optimize: $$f_1(x,y) = x^2+y^2, \quad x_{lb}\le x\le x_{ub},\quad y_{lb}\le y\le y_{ub}$$ Then I apply rotation by $\theta$ plus translation by $x_0$ and $y_0$: $$f_2(...
0
votes
1answer
101 views

ill-conditioning

I am struggling with the following exercise from the book of Nocedal, Numerical optimization, chapter 2, exercise 2.12: Suppose that a function $f$ of two variables is poorly scaled at the solution $...
3
votes
2answers
155 views

How do I check if a loss function can achieve its minimum?

For example, the convex function $f(t)=e^{-t}$ doesn't achieve its minimum 0 on the real line. In a linear regression with $p$ predictors $X$, the loss function $f(\beta)=||Y-X\beta||^2$ achieves its ...
0
votes
1answer
106 views

Can I convert CUDA core to CPU core and use it as cpu core while running any program?

I was using Metatrader5 and have designed a strategy for trading using MQL5 programming language. While I was running a Strategy Optimization process, I saw the it will need 10,00= iterations or ...
0
votes
1answer
46 views

Conflicting definition of limit point

This question was raised at a different place without sufficient answers. Definition 1: We say that a vector $x \in R^n$ is a limit point of a sequence $\{x_k\}$ in $R^n$ if there exists a ...
6
votes
1answer
227 views

What is required of the objective function in order to use Gauss Newton method?

From what I understand, the Gauss-Newton method is used to find a search direction, then the step size, etc., can be determined by some other method. In addition to that, are the following ...
1
vote
1answer
41 views

How to decide on what techniques to adopt in Genetic Algorithm optimization

Does it really matter which techniques we use in the process of GA optimization? For instance, if I use the Roulette Wheel Technique instead of Tournament Method for selection, two-point crossover ...
0
votes
1answer
334 views

I've developed a derivative-free optimization method, looking for comments

Here is the URL: https://github.com/avaneev/biteopt I've tested it on numerous global optimization benchmarking functions (included), and on real-world hyperparameter optimization problems I have. ...
3
votes
1answer
162 views

How to numerically minimize a functional?

How to numerically minimize a functional, for example, $$J[y]=\int_{x_1}^{x_2}L(x,y(x),y'(x))dx$$ An equivalent problem is to solve the Euler equation for this functional as a differential equation. ...
1
vote
1answer
42 views

Distribute sources among destinations

There are $n$ sources with the following positive volumes: $p_1, ..., p_n$ and there are $m$ destinations with the following positive volumes: $q_1, ..., q_m$. It is known that $p_1+ ...+ p_n=q_1+ ...+...
7
votes
3answers
678 views

Finding the first N roots of transcendental equation

I need to find the first $n$ roots of the transcendental equation \begin{equation} F(k) = J_m'(kr)Y_m'(k)-J'_m(k)Y'_m(kr) \end{equation} for integer values of $m$ and any $r \in [0,1)$ where $J'$ ...
0
votes
1answer
451 views

Defining a soft constraint in cvxpy

I am using cvxpy to do a simple portfolio optimization. I implemented the following dummy code ...
4
votes
1answer
113 views

Nonlinear least-squares solvers vs. generic minimization

A nonlinear least-squares problem with $F:\mathbb{R}^m\to\mathbb{R}^n$, $$ F(x) \to \min_x \quad (\text{in the least-squares sense}) $$ really means minimizing $$ \frac{1}{2} \|F(x)\|^2 \to \min_x. $$ ...
2
votes
1answer
101 views

What is the fastest way to solve Ax=b (subject to constraints and an absolute term)

I am trying to solve/optimize $Ax=b$ in the least squares sense subject to box constraints; a few (less than 5) equality/inequality constraints; and an absolute function penalty (or some other ...
1
vote
0answers
16 views

How does the MADS algorithm work in practice

Mesh Adaptive Direct Search (MASH) is an algorithm for black box optimization I want to understand an implement this method to solve some 2D multivariate blackbox function $f(x,y)$, but am having ...
5
votes
2answers
753 views

What is the most appropriate derivative free optimization algorithm

We can use random optimization/ derivative free/ direct search to find the minimum of some black box function $f$. If I have some 2D black box function, $f(x,y)$ - which I know to be convex - what ...
3
votes
1answer
285 views

Linear constraints for L-BFGS-B

I know L-BFGS-B only supports simple box constrains of the form: $l_i \leq x_i \leq u_i$, where $l_i$ and $u_i$ are constants. For my specific optimization problem, I need to specify some simple ...
0
votes
1answer
663 views

How to define the derivative for Scipy.Optimize.Minimize

I am trying to use scipy.optimize.minimize to minimise a quadratic objective function: $f(x) =x^\top Q x$. As a start, I have successfully implemented this using the built-in Nelder-Mead Simplex ...
0
votes
1answer
161 views

CPU and GPU influence on task parallel execution performance

This question is mainly about hardware, but also about software. In my current work I have approximately 68 millions of combinations that I am iterating through, in parallel. For each of those ...
1
vote
0answers
41 views

Genetic Algorithm: Need some clarification on selection and what to do when crossover doesn't happen

I'm writing a genetic algorithm to minimize a function. I have two questions, one in regards to selection and the other with regards to crossover and what to do when it doesn't happen. Here's an ...
1
vote
0answers
168 views

Levenberg-Marquardt for root-finding: just square the function?

This question might be so obvious and trivial that I'm having a hard time googling it. I have a multivariate root finding problem that I'm trying to solve in C# and the library that I'm trying to use ...
0
votes
1answer
55 views

reduced system: primal-dual interior point method for nonconvex constrained problem

When solving a reduced KKT system of a nonlinear (and nonconvex) constrained program after eliminating slack and dual variables, how do we actually take the next step in a primal-dual method? For ...
1
vote
0answers
16 views

Procedure to identify characteristic properties of unknown functions in a DAE model

I have a system of 1st order odes given by $$ \dot{x_1}(t) = \alpha_1 f_1(x_1,t) + \beta_1 u(t) \\ \dot{x_2}(t) = \alpha_2 f_2(x_2,t) + \beta_2 u(t) $$ They are constrained by an algebraic equation ...
1
vote
0answers
62 views

overlapping additive schwarz [closed]

I am solving 1D laplace problem discretized with finite differences (3-point stencil). I would like to use additive Schwarz method in classical form: $U_{k+1}=U_{k}+M^{−1} r_k,$ where $r_k=F−A U_k$...
3
votes
1answer
103 views

Obtaining a feasible solution for underdetermined system of linear equations satisfying inequality constraints

I would like to obtain a feasible solution for an under-determined system of linear equations, $$Ax=b$$ where, $A \in \mathbb{R}^{7\times9}, \, x \in \mathbb{R}^{9\times1}\text{and } b\in\mathbb{R}^...
3
votes
1answer
343 views

Optimization of a blackbox function with an equality constraint?

I believe this would be an interesting problem. I have a blackbox function which can take 2-60 input variables $(X_1,X_2,...X_n)$ which are to be optimized. I'm calling this objective function as a ...
0
votes
1answer
61 views

Adaptive gradient descent

I want to minimize some multivariable function $\Delta(\alpha, \beta)$. I know that this function has a zero point, $\Delta(5, 5) = 0$. Starting from some $(\alpha, \beta)$ close to $(5,5)$ (e.g. (4....
5
votes
1answer
370 views

Ways to speed up solving an LP with Google's ortools

I'm having an issue solving an LP of the form: $$\min z = c^Tx$$ $$\text{s.t.}$$ $$Ax \geq b$$ $$x\geq p$$ $1 \leq a_{ij} \ll b_i$, $p \leq 0$, and $c \geq 0$ The specific problems I'm running into ...
3
votes
0answers
101 views

Are the No Free Lunch Theorems Useful for Anything?

I have been thinking about the No Free Lunch (NFL) theorems lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here,...
1
vote
3answers
164 views

Optimization of a blackbox function

Let's say that we have an objective function $f(\mathbf x,\mathbf y)$ which has the parameters $\mathbf x=[x_1\ldots x_n]$ and $\mathbf y=[y_1\ldots y_n]$. Here, $\mathbf y$ is a blackbox variable ...
1
vote
0answers
37 views

Meaning of “points where the first- and second-order optimality conditions fail”

I am reading a paper that summarizes a set of simulations. Basically, the authors are trying to minimize some function using different optimization algorithms. They conclude: "Our findings point to ...
4
votes
1answer
75 views

Large-scale almost-linear optimization

I'm currently trying to minimize the following function: $$\min_{\mathbf x,\mathbf y,s} \|F\mathbf x-\mathbf a\|_2^2 + \|SF\mathbf y-\mathbf b\|_2^2 + \lambda \|\max(\mathbf x, \mathbf y)\|_1,$$ $$\...
1
vote
0answers
70 views

Sequential Quadratic Programming for Quadratically Constrained Quadratic Programs

A standard Quadratically Constrained Quadratic Program (QCQP) is of the form: $$ \underset{x}{minimize} \frac{1}{2}x^TP_{0}x + q_{0}^{T}x $$ $$ subject \; to \quad \frac{1}{2}x^TP_{i}x + q_{i}^{...
0
votes
1answer
93 views

Free access to Xeon Phi clusters

Apologies if this is not the right forum to ask, but there has been a somewhat related question here. I am working on a piece of software (nonlinear constrained optimisation, coded in C++11 and ...
2
votes
2answers
71 views

Find combinations of variables with variable bounds to get summed up to a required value (using Python)

Let's say I have n variables. Each variable has lower and upper bounds. I want to find all suitable combinations of these variables to sum up to a required value. An example with two variables: $$ ...
2
votes
2answers
288 views

Knapsack problem with fixed number of elements?

I am looking at an optimization problem that looks like this: $$ \text{minimize}\;\; \mathbf{x}^TQ\mathbf x \;\;, \; \mathbf x \in \mathbb R^n, x_i \in \lbrace 0, 1 \rbrace\\ \text{subject to}\;\; ||...
1
vote
0answers
74 views

Find a vector B that minimizes |W-A*B|

I want to find a candidate vector $B$ that $$\min|(W - A_i * B_i)|$$ $$ a_i > 0,\ A_i=\{a_0,...,a_i\},\ B_i=\{-1,0,1\}^i$$ For example, given $$W = 0.6,\quad A_4 = [0.1, 0.2, 0.4, 0.7] $$ one ...
1
vote
0answers
53 views

Obtaining the lagrangian multipliers in an optimization problem

Suppose we have this simple optimization problem \begin{align*} \underset{x\in V}{\text{min}} &~ f(x) \\ \text{s.t.}& ~x \leq \beta \end{align*} Using slack variables \begin{align*} ...
1
vote
0answers
71 views

Nonlinear global optimization algorithm that can use dynamic programming

I've asked this question on stackoverflow 2 weeks ago, but, judging by zero response, that probably was the wrong forum. Therefore copying it here: Let F0,...,Fn ...
1
vote
0answers
56 views

BFGS convergence problem

I would like first to state that it is beyond my capability to identify whether this is a BFGS issue or a R package problem. I've been doing some mixed logit regression using the R package "mlogit". ...
1
vote
1answer
360 views

Trajectory optimization for smoothness

I want to achieve the following in 2D (and without obstacles): Given start position A and end position B, generate the path between the two points that optimizes a cost function that depends on total ...
0
votes
1answer
123 views

pde-constrained optimization

I'm trying to solve a problem where I have a initial and final distribution of tumor, and my goal is to find the best map of parameters (diffusion and reaction terms) for a reaction-diffusion equation,...
1
vote
1answer
70 views

Solving for $C$ in $Q = YCZ$ using least squares in Matlab

I am trying to solve for the matrix $C$ in $Q = YCZ$ in matlab. I have preliminary results but they don't seem realistic. Here, $Q$ is $n \times m-1$, $Y$ is $n \times p$, $C$ is $p \times m$ and $Z$ ...
1
vote
0answers
48 views

Finding minimum of Wronskian determinant

If I have a partial differentail equation such as $\frac{\partial^2 u}{\partial t^2} + c^2 \frac{\partial^2 u}{\partial x^2}$, with boundary conditions $u(0,t) = 0$ and $u(1,t)=0$, I can solve this ...
2
votes
0answers
331 views

Using C/C++ for Markov chain Monte Carlo (MCMC) methods

I'm working on optimizing the parameters of a mathematical model to fit experimental data, using an existing formula for the likelihood of observing the data given a set of parameter values. At the ...
4
votes
3answers
182 views

How can I use Projected Gradient Descent for this optimization problem with constraint?

Suppose $ q $ and $ A $ are given and that $ q, p \in R^{N} $ and $ A \in R^{NxN} $, then how can I find the vector $ p $ such that $$ (q - p)^{T}A(q - p) $$ is minimized constraint to $ \sum_{i=1}^{...