# Marginalising over N-dimensional multivariate Gaussian

I have a multivariate Gaussian with mean $$\mu(x_1, x_2, \ldots)$$ and a covariance matrix $$C$$. In my case, the dimensions of the covariance matrix are $$200 \times 200$$. I have to marginalise over the $$x = \{x_1, x_2, \ldots\}$$ vector.

Is it possible to do that numerically? If so, can this numerical method be used to marginalise over any multivariate distribution (e.g. Student's - t) or products thereof?

Edit:

In particular, the integration I am interested in is

$$\int P(\mathbf{\hat{a}}|\boldsymbol{\alpha})P(\mathbf{\hat{b}}|\boldsymbol{\beta}(\boldsymbol{\alpha}))\boldsymbol{\alpha}^2 d\boldsymbol{\alpha}$$ which for the problem I'm attempting to solve is $$\propto \int \exp{\left(-\frac{1}{2}(\mathbf{\hat{a}}-f(\boldsymbol{\alpha}))^\mathrm{T}C_1^{-1}(\mathbf{\hat{a}}-f(\boldsymbol{\alpha}))\right)}\exp{\left(-\frac{1}{2}(\mathbf{\hat{b}}-g(\boldsymbol{\beta}(\boldsymbol{\alpha})))^\mathrm{T}C_2^{-1}(\mathbf{\hat{b}}-g(\boldsymbol{\beta}(\boldsymbol{\alpha})))\right)}\boldsymbol{\alpha}^2d\boldsymbol{\alpha},$$

where $$P(\mathbf{\hat{a}}|\boldsymbol{\alpha})$$ is a multivariate Gaussian with a diagonal covariance matrix $$C_1$$ and $$P(\mathbf{\hat{b}}|\boldsymbol{\beta})$$ is a multivariate Gaussian with a $$200 \times 200$$ covariance matrix $$C_2$$. So $$\boldsymbol{\alpha} = \{\alpha_1, \alpha_2, \ldots\}$$ is a vector of length $$200$$. $$f$$ and $$g$$ are functions of $$\boldsymbol{\alpha}$$ and $$\boldsymbol{\alpha}$$, $$\boldsymbol{\beta}$$, respectively.

• Marginalization requires integration, so as long as you can numerically integrate then yes. Commented Jul 27 at 14:57
• @lightxbulb I was asking how to do the integration over the vector μ={x1,x2,…} numerically. Commented Jul 27 at 16:07
• Let $p(x,y)$ be a joint probability density function. Let $u, v \in\mathbb{R}^2$ be a basis for $\mathbb{R}^2$. Define $q(s,t) = p(su + tv)|\det[u, v]|$, Then the marginal distribution where we have integrated along $v$ should afaict be $r(s) =\int_{\mathbb{R}} q(s,t) \,dt$. You can use your favourite 1D numerical integration method on this. Commented Jul 27 at 18:17
• Are you looking for the marginal distribution over some subset of the variables in the MVN vector or trying to compute some more complicated probability requiring integration over a region in n dimensions? Commented Jul 28 at 3:18
• @BrianBorchers, thank you, I mean the latter, please see edit. Commented Jul 28 at 14:06

$$\newcommand{\bm}[1]{\boldsymbol{#1}}$$From the clarifications it seems that this is not a marginalization or conditional probability problem, instead you are integrating over some parameter that affects the mean of two normal distributions. I am also not sure whether $$\bm{\alpha}^2$$ is really a vector, or you meant the squared norm. In any case I will assume it is the vector formed by the Hadamard product $$\bm{\alpha}\odot\bm{\alpha}$$. Define the function: \begin{align} \bm{q}(\bm{\alpha}) &= \exp{\left(-\frac{1}{2}(\mathbf{\hat{a}}-f(\boldsymbol{\alpha}))^\mathrm{T}C_1^{-1}(\mathbf{\hat{a}}-f(\boldsymbol{\alpha}))\right)}\\ &\cdot\exp{\left(-\frac{1}{2}(\mathbf{\hat{b}}-g(\boldsymbol{\beta}(\boldsymbol{\alpha})))^\mathrm{T}C_2^{-1}(\mathbf{\hat{b}}-g(\boldsymbol{\beta}(\boldsymbol{\alpha})))\right)}\boldsymbol{\alpha}^2. \end{align} Then the integration problem reads: $$$$\bm{I} = \int_{\prod_{i=1}^n [a_i,b_i]} \bm{q}(\bm{\alpha})\,d\bm{\alpha}.$$$$ To apply naive Monte Carlo you can then sample uniform random points $$\bm{\alpha}_1,\ldots,\bm{\alpha}_N\in \prod_{i=1}^n [a_i,b_i]$$ and compute: $$$$\bm{I} \approx \frac{\prod_{i=1}^n (b_i-a_i)}{N}\sum_{k=1}^N \bm{q}(\bm{\alpha}_k).$$$$
More generally, if we pick $$\bm{\alpha}_1,\ldots,\bm{\alpha}_N$$ independent and identically distributed according to a density $$r$$, then the estimator becomes: $$$$\bm{I} \approx \frac{1}{N} \sum_{k=1}^N \frac{\bm{q}(\bm{\alpha}_k)}{r(\bm{\alpha}_k)}.$$$$ In the uniform case $$r(\bm{\alpha})$$ is just equal to one divided by the volume of the domain.
• Thank you! Three questions: a) n is the number of dimensions, such that n=200 and N is the number of random samples drawn? b) I am actually not sure about $\alpha^2$. In the 1-D case, it would be $\int f(\alpha)\alpha^2d\alpha$, and in fact $\alpha^2$ is a prior, so I am not sure how I could write a prior on the vector $\alpha$ in higher dimensions. Any ideas? c) I am experiencing underflows in one of the multivariate Gaussians. Any thoughts on how to overcome this? Commented Jul 29 at 11:34
• $n$ is the number of dimensions yes, the integration domain doesn't have to be a hyperbox btw, $N$ is the number of samples. My only guess would be using $\|\alpha\|^2$ if you want a scalar-valued result, but this depends on your problem. I have no idea what you should do about the underflow issue. Commented Jul 29 at 16:29
• You can use a completely flat prior over $\mathbb{R}^{200}$ if you use Markov Chain Monte Carlo methods since they only need an unnormalized density. 200 dimensions is pretty high for any MC method though so you may need to use a more sophisticated implementation Commented Jul 29 at 19:58