2
$\begingroup$

I'm solving simple single-objective multidimensional global optimization problem using various stochastic algorithms like Monte-Carlo, GA and other evolutionary approaches. The task is formulated as follows:

  1. search space $X \subseteq R^n$
  2. objective function $f: R^n \rightarrow R$
  3. must find $\textbf{x}^*=\{x_1^*, ..., x_n^* \} \in X: f(\textbf{x}^*)=\min_{\textbf{x} \in X} f(\textbf{x})$

However, I also have some preferences of $x_i \in [a;b]$ which are presented in set of normalized values (from $0$ to $1$). This set can be interpolated (here no matter how) to probability distributions. So, for given $x_i$ exists function $g_i(x)$. Figure below shows typical distribution.

enter image description here

This doesn't mean that $x_i$ always possess preferable values and in peaks exists smaller value of objective. However, better values of $f$ tends to be near peaks.

The question is how to use this information about $x_i$ preferable values during optimization process and acheive better results (e.g. rate of convergence)? Any recommendations/books/articles with similar situations will be appreciated. I even don't know in what math area I should look for.

I have tried to use simulated annealing — just added $g_i$ (multiplied by $-1$) to objective function with a high weight at the beginning and then gradually reduce it to zero. With this simple scheme I didn't acheive much better results.

$\endgroup$
  • $\begingroup$ You've got prior distributions for $x_{1}$, $\ldots, $x_{n}$. Presumably your function $f$ is some sort of likelihood. Do you want to use optimization to find the maximum a posteriori (MAP) model? Are you interested in sampling from the posterior distribution? $\endgroup$ – Brian Borchers Dec 26 '16 at 1:58
  • $\begingroup$ @BrianBorchers, thanks for the reply. Actually the function $f$ is an energy function. It is the first time I hear about MAP model. Most likely I have to be interested in sampling from the posterior distribution. You definitely refer me in the right direction. Could you please share some links/algorithms/articles where optimization and these approaches is used together. $\endgroup$ – Serg Dec 26 '16 at 12:45
  • $\begingroup$ There are two different problems with very different solutions. Finding a MAP solution by global optimization techniques such as simulated annealing is generally easier than sampling from a posterior distribution using Markov Chain Monte Carlo methods. You'll need to provide more background for us to give you an answer to this question. $\endgroup$ – Brian Borchers Dec 26 '16 at 20:22
  • $\begingroup$ The optimization of function $f$ with the use of some population-based algorithm and density $g_i$ for some $x_i$ are the only information and there is no more background. I'm interested in a scheme of using such density during optimization process. However, it's not so easy as I thought and improving the optimization algorithm performance will take several steps including using MCMC method. Anyway, thank you for answers, they directed me correctly. $\endgroup$ – Serg Dec 27 '16 at 19:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.