# Questions tagged [optimization]

This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.

725 questions
32 views

80 views

### ill-conditioning

I am struggling with following exercise from book of Nocedal, Numerical optimization, chapter 2, excercise 2.12: Suppose that a function $f$ of two variables is poorly scaled at the solution $x^*$. ...
136 views

### How do I check if a loss function can achieve its minimum?

For example, the convex function $f(t)=e^{-t}$ doesn't achieve its minimum 0 on the real line. In a linear regression with $p$ predictors $X$, the loss function $f(\beta)=||Y-X\beta||^2$ achieves its ...
86 views

### Can I convert CUDA core to CPU core and use it as cpu core while running any program?

I was using Metatrader5 and have designed a strategy for trading using MQL5 programming language. While I was running a Strategy Optimization process, I saw the it will need 10,00= iterations or ...
43 views

### Conflicting definition of limit point

This question was raised at a different place without sufficient answers. Definition 1: We say that a vector $x \in R^n$ is a limit point of a sequence $\{x_k\}$ in $R^n$ if there exists a ...
58 views

### Methods for Parameter Scaling in Gradient-based Optimization

I am trying to minimize an objective function with 4 parameters, e.g., $a,b,c,d$ using gradient descent. $a < 0.1$, while $0 <b,c,d < 10$. I'm using a learning rate for all parameters on the ...
205 views

### What is required of the objective function in order to use Gauss Newton method?

From what I understand, the Gauss-Newton method is used to find a search direction, then the step size, etc., can be determined by some other method. In addition to that, are the following ...
40 views

### How to decide on what techniques to adopt in Genetic Algorithm optimization

Does it really matter which techniques we use in the process of GA optimization? For instance, if I use the Roulette Wheel Technique instead of Tournament Method for selection, two-point crossover ...
280 views

### I've developed a derivative-free optimization method, looking for comments

Here is the URL: https://github.com/avaneev/biteopt I've tested it on numerous global optimization benchmarking functions (included), and on real-world hyperparameter optimization problems I have. ...
61 views

### Clarifying a dual problem solution

I’m referring to this question on MathSE about the sum of $k$ largest eigenvalues of a symmetric matrix as an SDP-problem. I want to solve the maximization problem. As the dual problem is always ...
116 views

### How to numerically minimize a functional?

How to numerically minimize a functional, for example, $$J[y]=\int_{x_1}^{x_2}L(x,y(x),y'(x))dx$$ An equivalent problem is to solve the Euler equation for this functional as a differential equation. ...
40 views

159 views

### Optimization of a blackbox function with an equality constraint?

I believe this would be an interesting problem. I have a blackbox function which can take 2-60 input variables $(X_1,X_2,...X_n)$ which are to be optimized. I'm calling this objective function as a ...
60 views

I want to minimize some multivariable function $\Delta(\alpha, \beta)$. I know that this function has a zero point, $\Delta(5, 5) = 0$. Starting from some $(\alpha, \beta)$ close to $(5,5)$ (e.g. (4....
172 views

### Ways to speed up solving an LP with Google's ortools

I'm having an issue solving an LP of the form: $$\min z = c^Tx$$ $$\text{s.t.}$$ $$Ax \geq b$$ $$x\geq p$$ $1 \leq a_{ij} \ll b_i$, $p \leq 0$, and $c \geq 0$ The specific problems I'm running into ...
96 views

### Are the No Free Lunch Theorems Useful for Anything?

I have been thinking about the No Free Lunch (NFL) theorems lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here,...
155 views

### Optimization of a blackbox function

Let's say that we have an objective function $f(\mathbf x,\mathbf y)$ which has the parameters $\mathbf x=[x_1\ldots x_n]$ and $\mathbf y=[y_1\ldots y_n]$. Here, $\mathbf y$ is a blackbox variable ...
35 views

### Meaning of “points where the first- and second-order optimality conditions fail”

I am reading a paper that summarizes a set of simulations. Basically, the authors are trying to minimize some function using different optimization algorithms. They conclude: "Our findings point to ...
71 views

I'm currently trying to minimize the following function: $$\min_{\mathbf x,\mathbf y,s} \|F\mathbf x-\mathbf a\|_2^2 + \|SF\mathbf y-\mathbf b\|_2^2 + \lambda \|\max(\mathbf x, \mathbf y)\|_1,$$ $$\... 0answers 54 views ### Sequential Quadratic Programming for Quadratically Constrained Quadratic Programs A standard Quadratically Constrained Quadratic Program (QCQP) is of the form:$$ \underset{x}{minimize} \frac{1}{2}x^TP_{0}x + q_{0}^{T}x  subject \; to \quad \frac{1}{2}x^TP_{i}x + q_{i}^{...
87 views

Apologies if this is not the right forum to ask, but there has been a somewhat related question here. I am working on a piece of software (nonlinear constrained optimisation, coded in C++11 and ...
66 views

47 views

### Obtaining the lagrangian multipliers in an optimization problem

Suppose we have this simple optimization problem \begin{align*} \underset{x\in V}{\text{min}} &~ f(x) \\ \text{s.t.}& ~x \leq \beta \end{align*} Using slack variables \begin{align*} ...
63 views

### Nonlinear global optimization algorithm that can use dynamic programming

I've asked this question on stackoverflow 2 weeks ago, but, judging by zero response, that probably was the wrong forum. Therefore copying it here: Let F0,...,Fn ...
52 views

### BFGS convergence problem

I would like first to state that it is beyond my capability to identify whether this is a BFGS issue or a R package problem. I've been doing some mixed logit regression using the R package "mlogit". ...
249 views

### Trajectory optimization for smoothness

I want to achieve the following in 2D (and without obstacles): Given start position A and end position B, generate the path between the two points that optimizes a cost function that depends on total ...
115 views

### pde-constrained optimization

I'm trying to solve a problem where I have a initial and final distribution of tumor, and my goal is to find the best map of parameters (diffusion and reaction terms) for a reaction-diffusion equation,...
67 views

### Solving for $C$ in $Q = YCZ$ using least squares in Matlab

I am trying to solve for the matrix $C$ in $Q = YCZ$ in matlab. I have preliminary results but they don't seem realistic. Here, $Q$ is $n \times m-1$, $Y$ is $n \times p$, $C$ is $p \times m$ and $Z$ ...
45 views

### Finding minimum of Wronskian determinant

If I have a partial differentail equation such as $\frac{\partial^2 u}{\partial t^2} + c^2 \frac{\partial^2 u}{\partial x^2}$, with boundary conditions $u(0,t) = 0$ and $u(1,t)=0$, I can solve this ...
Suppose $q$ and $A$ are given and that $q, p \in R^{N}$ and $A \in R^{NxN}$, then how can I find the vector $p$ such that $$(q - p)^{T}A(q - p)$$ is minimized constraint to \$ \sum_{i=1}^{...