# Gradient descent on the PDF of the multivariate normal distribution

I want to perform gradient descent optimization of the probability of a sample under a multivariate normal probability density function. For your convenience I state the PDF here:

$$\mathcal{N}(\boldsymbol\mu,\,\boldsymbol\Sigma) = (2\pi)^{-\frac{k}{2}}|\boldsymbol\Sigma|^{-\frac{1}{2}}\, e^{ -\frac{1}{2}(\mathbf{x}-\boldsymbol\mu)'\boldsymbol\Sigma^{-1}(\mathbf{x}-\boldsymbol\mu) }$$

The probability of a sample under this model for the special case of zero mean and unit covariance is given by

$$p(\vec{x}) = \frac{1}{(2\pi)^{-n/2}}\prod_{i=1}^n e^{-\frac{x_i^2}{2}}$$

where $n$ is the number of entries in $\vec{x}$. $\vec{x}$ is the projection of a sample onto a PCA model in each iteration and so should actually be seen as a function of a different set of variables $\vec{y}$ that are parametrizing the original problem. In addition, I would like to work with the log of the probability for numerical stability. All in all, I'm looking for the derivative

$$\frac{d}{d\vec{y}}\log p(\vec{x(\vec{y}})) = \frac{d}{d\vec{y}}\log \frac{1}{(2\pi)^{-n/2}}\prod_{i=1}^n e^{-\frac{\vec{x_i(\vec{y}})^2}{2}}$$

and this where I'm stuck. Help and pointers are really appreciated. So my questions are these:

(1) Is this approach to deriving a gradient descent cost function correct?

(2) What is the derivative given by?

Start by simplifying your expression by using the fact that the log of a product is the sum of the logarithms of the factors in the product. The resulting expression is a quadratic form that is easy to differentiate.

• Thanks! I guess the real source of my confusion was related to x being a function of y for which I don't have the analytical expression but I found a solution using the Jacobian. Your answer pointed me in the right direction so I'm accepting that as answer. – Snut Aug 14 '14 at 9:19