Questions tagged [optimization]

This tag is intended for questions on methods for the (constrained or unconstrained) minimization or maximization of functions.

758 questions
110 views

Stochastic gradient descent for large deterministic optimization problems

The Wikipedia page for SGD describes optimizing a function $f = \sum f_i(\theta;x_i)$ by successively approximating gradients from random subsets of the data, while most literature poses the problem ...
171 views

First approximation to the TSP in a non-complete Graph

I'm trying to solve the Travelling Salesman Problem in a non-complete graph $(G,E)$ using genetic algorithms. My problem is that I can't find a good first approximation by the usual greedy algorithms,...
115 views

GAMS Optimization

I am writing a GAMS program where I am interested in using the value of a variable as a condition inside an another equation. Let's say I have two equations with two variables, $g_1(t)$ and $g_2(t)$, ...
147 views

How to test for convergence (smoothness of Pareto front) in DEAP

In the DEAP algorithms (see documentation here), I notice that we need to specify the the number of generations (NGEN). I was advised that convergence has been achieved if the Pareto curve is smooth. ...
147 views

Optimization of non-smooth, non-convex, locally Lipschitz functions of type exp(-abs(x))

What would be the numerical method of choice to find minima in a non-smooth, non-convex, locally Lipschitz function $f: \mathbb{R}^n\rightarrow \mathbb{R}$. The function $f$ is mostly smooth but ...
57 views

Comparing the solutions to a multi-objective optimization problem

Suppose I have a multi-objective optimization problem, and I wish to find solutions using two different methods/algorithms. The result of each algorithms is a Pareto front. Comparing two different ...
74 views

How does this Constrained Minimization algorithm work?

I don't fully understand the subsection 3.2 Constrained minimization of this paper. In particular, I don't understand the first step "Register active set" and the definition of projection $P(x)$. ...
94 views

Optimization based integration for MPM

I'm considering implementing (just for simplicity) the unconstrained implicit optimization based integration for Material Point Method as described in Chenfanfu Jiang's thesis on MPM (the minimization ...
199 views

What penalty function produces optimization-based Gaussian smoothing?

I have just read yet another introduction to image processing, which describes Gaussian smoothing from a convolution perspective and least-squares smoothing from an optimization perspective. It would ...
632 views

LU decomposition of large dense matrices

I wanted to generate LU decomposition of large size dense matrices ($N>10^7$), the LU decomposition I'm currently using is based on Adaptive Cross Approximation and is taking very long time to ...
179 views

Variable elimination in linear programming

I have a linear program of the form $$\underset{P,\;g}{\text{Minimize}}\hspace{3mm}c^Tg$$ \begin{align} \hspace{17mm}\text{Subject to}\hspace{3mm}AP_{\cdot,j}&=\begin{bmatrix} -g\\ d \end{...
93 views

A least square problem with a fixed mean constraint and a subspace constraint

Let $V_1,\ldots,V_n$ be $n$ vector subspaces of a Hilbert space, $y_i\in V_i$ for each $i$ and $\overline{x}$ be a fixed vector. I want to solve the optimization problem: \begin{equation*} \begin{...
60 views

Relaxing a variable in MIP

I have this MIP optimization problem, with couple of binary variables; however when I relax one of the binary variables the optimal solution of the objective does not change. But the solving time ...
1k views

Understanding MATLAB's fmincg optimization function

I'm researching numerical optimization. Recently I've come across a variant of a conjugate gradient method named fmincg. The function is written in MATLAB and is ...
62 views

Global optimization with known distributions of some variables

I'm solving simple single-objective multidimensional global optimization problem using various stochastic algorithms like Monte-Carlo, GA and other evolutionary approaches. The task is formulated as ...
183 views

422 views

214 views

179 views

What would be a good approach to solving this large data non-linear least squares optimisation

Introduction to Problem I'm using a Truncated Signed Distance Function to perform 3D reconstruction from depth images. Essentially I have a large voxel grid where each voxel contains the signed ...
435 views

I have read this and other threads on this site on BFGS, but I still don't have a clear understanding of what it's meant by low-rank updates. For example, I read the following in this book: The ...
924 views

Why are convex problems easy to optimize?

Motivated by this top answer to the question: Why is convexity more important than quasi-convexity in optimization?, I am now hoping to understand why convex problems are easy to optimize (or at least ...
68 views

Loooking for name of this geometrical optimization technique

From my knowledge if you fit geometrical objects into point clouds you want in general minimize the squared distances of the point cloud to your fitted objects. I do so with the downhill simplex ...
59 views

Doubt regarding principled approach towards approximating the Hessian

In my optimization problem, the hessian has a structure such that it can be written as the sum of two matrices. Populating the first of the matrices is efficient. Populating the second one is ...
210 views

How to code gradient descent-based Tikhonov denoising that exactly matches LSQ Tikhonov denoise?

(Note: Corrected code is posted below the original code.) For an exercise in optimization, I am interested in coding a simple example from scratch: a Tikhonov denoising routine using gradient descent,...
Consider random variables $X$ and $Y$, their distributions are given. $Z = f_a(X, Y)$ where $f(\cdot, \cdot)$ is a deterministic, not random function $f_a: \mathbb{R}^2 \to \mathbb{R}$ depending on a ...