Importance Sampling for multidimensional integrals and random numbers from multivariable pdf's

Question

I am aiming to get a numerical value for a five-dimensional integral using Monte Carlo Integration. I am getting good results using the Mean Value Method, but I would like to try to use Importance Sampling to get even better results.

My main source gives me an example of how to do this method using a one-dimensional integral, but it doesn't even mention how to replicate it for multivariable integrals. My main issue is generating a set of random numbers from a multivariable probability distribution function.

If I understand correctly the process of doing it for a single variable is setting up the equation:

$$\int_0^{x}dx^{\prime}\,p(x^\prime) = z$$

and then solving for $x$ ($z$ are the uniformly distributed random numbers generated).

How can I replicate this procedure for an arbitrary 5-dimensional pdf $p(x_1,x_2,x_3,x_4,x_5)$?

I could find no good sources on this, most of the ones that I found dealt specifically with n-dimensional Gaussian distributions.

Answer 1

Let's recap what you want to do: You have some set $V \subset \mathbb{R}^5$ and want to approximate the integral of some function $f\colon V \to \mathbb{R}$: $$ \int_V f(x) \,\mathrm{d}x $$ The "mean value method" sounds like a Monte Carlo-type approximation of the form $$ \int_V f(x) \,\mathrm{d}x \approx \frac{\operatorname{vol}(V)}{N} \sum_{i=1}^N f(X_i), $$ where $X_i$ are iid. random variables on $V$ with a uniform distribution and $\operatorname{vol}(V)$ is the volume of $V$.

The problem with this vanilla Monte Carlo approach is that, depending on the function $f$, most samples will not contribute to the integral since $f(X_i) \approx 0$. In importance sampling, the idea is to sample from a distribution that is adapted to the shape of $f$ instead of uniformly sampling from $V$.

To this end, let $p\colon V \to [0, \infty)$ be the density of a probability distribution $P$ on $V$. We approximate $$ \int_V f(x) \,\mathrm{d}x = \int_V f(x) \frac{p(x)}{p(x)} \,\mathrm{d}x = \int_V \frac{f(x)}{p(x)} \,\mathrm{d}P(x) \approx \frac{1}{N} \sum_{i=1}^N \frac{f(Y_i)}{p(Y_i)}, $$ where $Y_i$ are iid. random variables drawn from $P$.

As you've mentioned, we can exactly sample $Y_i$ from $P$ in one dimension by using inversion sampling. This only works if we can invert the cumulative distribution function of the distribution.

In multiple dimensions, there are several ways to draw from a given distribution:

• Rejection sampling,
• Gibbs sampling (if we can efficiently draw from the one-dimensional conditionals),
• Markov Chain Monte Carlo (only provides asymptotically correct samples in general, but could be applied to the integration problem directly without the importance sampling detour),
• some more.

For selecting a method, we need more information on the actual distribution. Is there some structure that can be exploited (e.g., mixture of Gaussians)?