Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [optimization]

This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.

2
votes
0answers
38 views

Nonlinear least squares and regularization

Consider the nonlinear least-squares minimization of a vector of $n$ residuals $\mathbf{f}$ in $p$ parameters $\mathbf{x}$: $$ \min_{\mathbf{x}} || \mathbf{f}(\mathbf{x}) ||^2 $$ This can be done with ...
0
votes
1answer
35 views

Is this a knapsack problem?

I have a set of $K$ keywords. Each of this keywords can have set of bids from $1\$,\dots,N\$$. For each bid for a keyword, it will get a specific amount of clicks and a specific cost. Clicks and Cost ...
0
votes
0answers
16 views

Given a data set of 3 item tuples (x, y, z), how to predict y to maximize z if given x?

I have a training set of a bunch of three element tuples in the form of (x, y, z) where x, y, and z are all continuous real valued scalars. Given a future x value (which will likely be close but not ...
0
votes
0answers
28 views

Minimize cost with Levenberg-Marquart method

I want to minimize a cost function of the form, $$ \min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right) $$ ...
0
votes
1answer
46 views

How to derive the optimal bayesian solution to a model of two normal distributed populations

In the "Introduction" section of the paper Support-Vector Networks, it mentioned Fisher's solution to a model of two normal distributed populations: My questions are: How to derive equation (1)? I ...
1
vote
1answer
42 views

sum of absolute difference constraint in optimization problem

I am writing a model for an optimization problem. I need to write the following constraint: $$\sum^{N - 1}_i \lvert (a_i - a_{i+1}) \rvert \leq 2\, .$$ How to write this constraint (or linearize)? ...
-1
votes
0answers
30 views

I am trying to compare the performance between lstsq and gradient-based method (e.g. 'L-BFGS-B')

I have a dataset X (NxD where N >> D), so it is an overdetermined system, we could use np.linalg.lstsq to find least square error. However, I computed the root mean square error, I found that L-BFGS-B ...
1
vote
2answers
112 views

Method to find PDE equation coefficient satisfying mean solution?

What is the best approach to go about solving a PDE problem of the type \begin{equation} k^3\Delta u - k(\mathbf{1}\cdot\nabla u) = 0\, ,\\ u=g\; \text{on}\; \Gamma_D\, ,\\ mean(u) = u_\text{...
1
vote
1answer
40 views

Nature of stationary points of a Lagrangian fuction

I would like to extremize a certain function $f$ with respect to a parameter $x$, under constraints $g_1(x) = 0, ..., g_m(x)=0$. In order to achieve this, I consider the Lagrangian function $L(x, \...
-1
votes
1answer
59 views

Compute affine transformation between two sets of points

Consider two sets of points $P = (P_1, ...,P_n), \ Q = (Q_1, ..., Q_m) $ included in $\mathbb{R}^3$. I'm looking to compute an optimal affine transformation that "maps" $Q$ to $P$, although the sets ...
3
votes
1answer
66 views

Nonlinear least squares when some parameters are linear

Consider the least squares problem, $$ \min_{\mathbf{a},\mathbf{b}} || \mathbf{f}(\mathbf{a},\mathbf{b})||^2 $$ where $\mathbf{a},\mathbf{b}$ represent the unknown parameters to be found. In my ...
1
vote
0answers
85 views

Good C++ optimization library for BFGS

To implement maximum likelihood estimators, I am looking for a good C++ optimization library that plays nicely with Eigen's matrix objects. Eigen has some capability of interfacing of its own but if ...
1
vote
1answer
33 views

Techniques to remove a function from Levenberg-Marquardt when it is against box constraints

I have a somewhat large (20+ dimensional) root finding problem that I'm solving with Levenberg-Marquardt. One of the functions has box constraints on [0, 2]. When it is against those bounds it will ...
2
votes
0answers
21 views

Where can I find sample data for large linear programming optimization problems?

I am doing a comparison of different algebraic modeling langues (AMPL, AIMMS, GAMS, Pyomo) in both theoretical and practical terms. As a practical experiment I am trying to measure problem model ...
1
vote
1answer
67 views

Why does Newton's method with Linear Equality Constraints use KKT condition?

Goal: Optimize convex function $f(\vec{x})$ subjected to constraint $A\vec{x} = \vec{b}$ starting at a point $\vec{x}_0$ that satisfies the constraint. The problem only has equality constraint. Why ...
1
vote
0answers
45 views

Order of a principal term

In Yurii Nesterov's Introductory Lectures on Convex Optimization, there is a bound for the total number of iterations for some process. See page 109: $$\left[\frac{1}{\ln(2(1-\kappa))} \ln\frac{t_0-t^...
1
vote
0answers
29 views

Logging vs outputs in iterative optimisation

I'm coding an iterative algorithm of constrained continuous optimisation. An augmented Lagrangian algorithm (outer) calls a bound-constrained L-BFGS-B algorithm (inner), which calls a line search ...
3
votes
0answers
47 views

Nonlinear functional optimization in radial coordinates

I am currently implementing classical density functional theory for a radially symmetric system. In mathematical terms, I am searching for a function $f(r)$ that minimizes a functional $\Omega[f]$. ...
2
votes
0answers
72 views

How to numerically optimize affine transformations?

I need to optimize affine transformations for of a set of triangles using energy function based on the connectivity. The energy of an edge $e_j$ between triangles $T_a, T_b$ is given by $$ E_j = \...
-1
votes
1answer
22 views

R function or package for carrying out maximum likelihood techniques in random effect models

I am applying optim() function in R to obtain maximum likelihood estimates of the fixed effects and random effects in a model with bivariate random effects. The ...
3
votes
0answers
95 views

Difference between Dishonest Newton method and Very Dishonest Newton method

What is the difference between the Dishonest Newton method and the Very Dishonest Newton method? Is there a difference or do they mean the same thing? I have tried searching for this on the internet ...
1
vote
0answers
30 views

Efficient numerical optimization of an “almost separable” function

I have come across an optimization problem with the following objective function: $$f(x_0,y_0,z_0,x_1,y_1,z_1,...,x_N,y_N,z_N) = \sum_{i=0}^N f_i(x_i,y_i,z_i, \alpha(x_{i+1}-x_i) + \beta(y_{i+1}-y_i))...
2
votes
1answer
48 views

Convergence rate and complexity for convex minimization problem

In Yurii Nesterov's Introductory Lectures on Convex Optimization, there is a description of the rate of convergence and corresponding upper bound for the analytical complexity of a minimization ...
2
votes
1answer
29 views

Log-transformation of decision variables in parameter estimation

I am trying to find the diffusion coefficient ($D$) and the partition coefficient ($KLP$) using experimental data of desorption of a pollutant from a film into a liquid. This process can be modelled, ...
3
votes
0answers
31 views

Stochastic conjugate directions to improve convergence in narrow valleys

My question concerns a specific statement in this paper: N. N. Schraudolph and T. Graepel, "Conjugate Directions for Stochastic Gradient Descent," in Int. Conf. Artificial Neural Networks, Berlin, ...
3
votes
1answer
187 views

How does fmincon in MATLAB calculate gradients?

I am trying to solve numerically a constrained optimisation problem in MATLAB, and I am wondering how the fmincon function calculates gradients when one isn't ...
1
vote
1answer
36 views

What is a good way to select a small subset (say 50) of items from a large pool of items (say 5 million) while minimizing an objective function?

I have 5 million items that have 10 features (all continuous and not categorical) each and would like to select a small subset of these items. Ideally, I want to manually specify 10 features of my own ...
-1
votes
1answer
70 views

Simulation-based Optimization vs PDE-constrained Optimization

What is the difference between Simulation-based Optimization and PDE-constrained Optimization? Would studying a text on Simulation-based optimization be sufficient to understand and apply both?
0
votes
0answers
54 views

Artificial Neural Network for nonconvex optimization

I want to use artificial neural network(ANN) for solving non-convex optimisation problems. How do I map the constraints and the objective functions in the ANN architecture?
3
votes
1answer
64 views

What is a “good enough” method of assigning values to n variables subject to basic bounding constraints while maintaining relative weights?

Given triples of $n$ floating point values $$(\min_1, \max_1, w_1), \dots, (\min_n, \max_n, w_n)$$ and a value $V$, what is a good algorithhm to assign values $v_i$ to each of the triples such that ...
2
votes
1answer
118 views

Can the Power Method be used here?

Given a set of $n$ points on which a triangulation is performed, it is possible to construct coefficients $\lambda_{ij}>0$ such that each point $x_i$ is a convex combination of the points connected ...
4
votes
0answers
76 views

What Derivative-free optimization method should I use when my initial guess is very good?

I am trying to minimize a function where my initial guess is quite close to the minimum. I'm trying to minimize $$f(q) = \text{angle}(qw_1q*, v_1) + \text{angle}(qw_2q*, v_2) + \text{angle}(qw_3q*, ...
5
votes
0answers
72 views

conjugate gradient for Newton's method with non positive definite Hessian matrix

I want to minimize a non-linear function $f(x)$ using Newton's method. At each optimization step, I compute a descent direction $d$ to update $x$ using a second-order approximation of $f(x)$: $$ \...
3
votes
0answers
73 views

Inconsistency in optimize.minimize

I am trying to fit a time-dependent curve at each time step. I do so in minimizing along $x_c$ the quadratic error between the curve and a reference solution $ 1/(1 + \exp\left(\sqrt{S}(x-x_c)\right) $...
3
votes
1answer
34 views

Software for finding a minimum vertex cover for a hypergraph

A hypergraph $H = (V,E)$ consists of a finite set of vertices, say $V=\{1, \dots, n\}$ and a set of hyperedges $E \subseteq \mathcal{P}(V)$. We call $H$ a $k$-hypergraph if all $|e| = k$ for all $e\in ...
0
votes
0answers
41 views

Metric selection and scale-based preconditioning in quadratic optimization problem

I'm going to use scale-based preconditioning in a quadratic optimization problem: minimize $ x^T Q x + p^T x$ such that $ A x + b = 0$ and $D x + E \leq 0$, I want to speed up finding the optimal $x$ (...
0
votes
0answers
28 views

Converting logit coefficients into probabilities through the inverse multinomial logit

I am trying to come up with the transition intensities of a parameterized Markov model, i.e, the elements of the transition matrix weighted by the covariates through the inverse Logit relation: $$\...
5
votes
1answer
119 views

Generate approximately semi-orthogonal tall matrix approximately satisfying constraints

I have a set of matrices $\{(A_i,D_i)\}$ for $i\in\{1,\ldots,n\}$, where: Each $D_j\in\mathbb{R}^{S\times S}$ is diagonal, and every entry on the main diagonal is non-negative. Each $A_j\in\mathbb{R}^...
5
votes
1answer
88 views

Solvers for Quadratically Constrained Quadratic Programs (QCQP) with complex variables

I'd like to know whether there are any publicly available tools for solving QCQP with complex variables (and constraints therefore expressed through Hermitian matrices). What I have found so far is ...
4
votes
1answer
183 views

How to solve the following Frobenius norm-minimization problem?

Background We know how to solve the following minimization problem $$ \min_{X} \lVert AX - B \rVert_F^2 $$ But what about the extended version? $$ \min_{X} \lVert A \begin{bmatrix} X & X^2 \...
3
votes
1answer
168 views

How to solve the inverse problem of least-squares?

Focusing on following least squares problem: $$\min\limits_{V} \lVert Z - WV \rVert _{_F}^2$$ $$Z∈{R}^{m\times n},\quad W∈{R}^{m\times k},\quad V∈{R}^{k\times n},\quad k\lt m\lt n $$ This problem ...
1
vote
1answer
181 views

How to use CSDP to express a semidefinite program?

I am trying to use CSDP and am struggling with it. Consider, for example, the following semidefinite program $$\begin{array}{ll} \text{minimize} & 0\\ \text{subject to} & Q - A' Q A - \...
1
vote
1answer
162 views

FMINCON Step Size Tolerance

I get following error after implementing the attached code. Error Message "fmincon stopped because the size of the current step is less than the default value of the step size tolerance but ...
1
vote
0answers
102 views

How to prevent BFGS from getting stuck on astronomically large gradient?

I have implemented BFGS myself from scratch in order to solve minimization problems. Part of BFGS, as I understand it, is that the approximation to the Hessian is supposed to be positive definite, ...
1
vote
0answers
32 views

Best Possible Convex bounds for optimization problems

Suppose we have a primal problem $$ p^{*}=\min_x f(x), \\\text{s.t.}~~ h_i(x) \leq 0, $$ where $f(.)$ and $h_i(.)$ are possibly non-convex. Then its Lagrangian is $$\mathcal{L}(x,z_i)= f(x) + \...
1
vote
0answers
61 views

Nonlinear least square optimization

Problem description Given data at many time instance $t$, $$\min _{\alpha, \Lambda, \beta} \lVert y(t) - \alpha e^{\Lambda t} \beta \rVert_F$$ with $$ \lVert \alpha \rVert_2^F = 1 $$ where $y(t) \...
1
vote
0answers
39 views

Space covering optimization

I have the following problem: In the space $E=\{1, 2, \dots, N_x\} \times \{1, 2, \dots, N_y\}$, I want to define $N_R$ rectangles $R_k=\{x_k^0, \dots, x_k^1\}\times\{y_k^0, \dots, y_k^1\}$ which ...
2
votes
1answer
76 views

Some proof that linear translations and rotations of a bound-constrained function are equivalent

For example, I have a function to optimize: $$f_1(x,y) = x^2+y^2, \quad x_{lb}\le x\le x_{ub},\quad y_{lb}\le y\le y_{ub}$$ Then I apply rotation by $\theta$ plus translation by $x_0$ and $y_0$: $$f_2(...
0
votes
1answer
79 views

ill-conditioning

I am struggling with following exercise from book of Nocedal, Numerical optimization, chapter 2, excercise 2.12: Suppose that a function $f$ of two variables is poorly scaled at the solution $x^*$. ...
3
votes
2answers
135 views

How do I check if a loss function can achieve its minimum?

For example, the convex function $f(t)=e^{-t}$ doesn't achieve its minimum 0 on the real line. In a linear regression with $p$ predictors $X$, the loss function $f(\beta)=||Y-X\beta||^2$ achieves its ...