Return to Question

First post. I'm working on this problem using Bayesian methods. In desperation I'm considering using p-values (shock horror), specifically posterior predictive p-values. So I need to simulate from the posterior predictive. I don't see a lot of statistics stuff here - it's mostly applied math at the moment. I could take this to stats.sx I suppose, but this is definitely a computational problem, so I thought I'd give it a shot here. If there are any Bayesians lurking here, now would be a good time to delurk. :-)

In what follows, the hats denote predictive values. Consider

Let $p(\mathbf{p})$ be a Dirichlet prior.

\begin{align*} p(\mathbf{p}) = (k-1)! I(\sum p_i = 1) \prod_{i=1}^k p_i^{f_i} \end{align*}

Let the likelihood be

\begin{align*} p(y| \mathbf{p}) = \prod_{i=1}^k p_i^{f_i} \end{align*}

This is the likelihood for a sequence of categorical variables. Assume the prior predictive is (calculation omitted)

\begin{align*} p(y) = (k-1)! \frac{\prod_i f_i!}{(N + k - 1)!} \end{align*}

We obtain

\begin{align*} p(\mathbf{p}|y) = p(\mathbf{p}) p(y| \mathbf{p}) / p(y) = \frac{ (k-1)! I(\sum p_i = 1) \prod_{i=1}^k p_i^{f_i} } { (k-1)! \frac{\prod_i f_i!}{(N + k - 1)!} } \end{align*}

Dividing we obtain the posterior

\begin{align*} p(\mathbf{p}|y) = \frac{ I(\sum p_i = 1) \prod_{i=1}^k p_i^{f_i} (N + k - 1)! }{ \prod_i f_i! } \end{align*}

Define $f^{\prime}_i = f_i + \hat{f}_i$, and $N^{\prime} = N+\hat{N}$. Then the posterior predictive is (calculation omitted but can be added if anyone cares)

\begin{align*} p(\hat{y}|y) = \int p(\hat{y}|\mathbf{p}) p(\mathbf{p}|y) d\mathbf{p} = \frac{ \prod_i f^{\prime}_i! }{ \prod_i f_i! } \frac{ (N + k - 1)! }{(N^{\prime}+k-1)!} \end{align*}

So, can anyone suggest methods, preferably as efficient and simple as possible, to simulate from $p(\tilde{y}|y)$? This is a finite distribution, but too big to enumerate. At the moment, I'm not even sure what distribution it is. Thanks in advance.

PS. Can some kind person create "bayesian-computation" and "statistics" tags, and tag this message with them? "random" is not very useful. Thanks.

First post. I'm working on this problem using Bayesian methods. In desperation I'm considering using p-values (shock horror), specifically posterior predictive p-values. So I need to simulate from the posterior predictive. I don't see a lot of statistics stuff here - it's mostly applied math at the moment. I could take this to stats.sx I suppose, but this is definitely a computational problem, so I thought I'd give it a shot here. If there are any Bayesians lurking here, now would be a good time to delurk. :-)

In what follows, the hats denote predictive values. Consider

Let $p(\mathbf{p})$ be a Dirichlet prior.

\begin{align*} p(\mathbf{p}) = (k-1)! I(\sum p_i = 1) \end{align*}

Let the likelihood be

\begin{align*} p(y| \mathbf{p}) = \prod_{i=1}^k p_i^{f_i} \end{align*}

This is the likelihood for a sequence of categorical variables. Assume the prior predictive is (calculation omitted)

\begin{align*} p(y) = (k-1)! \frac{\prod_i f_i!}{(N + k - 1)!} \end{align*}

We obtain

\begin{align*} p(\mathbf{p}|y) = p(\mathbf{p}) p(y| \mathbf{p}) / p(y) = \frac{ (k-1)! I(\sum p_i = 1) \prod_{i=1}^k p_i^{f_i} } { (k-1)! \frac{\prod_i f_i!}{(N + k - 1)!} } \end{align*}

Dividing we obtain the posterior

\begin{align*} p(\mathbf{p}|y) = \frac{ I(\sum p_i = 1) \prod_{i=1}^k p_i^{f_i} (N + k - 1)! }{ \prod_i f_i! } \end{align*}

Define $f^{\prime}_i = f_i + \hat{f}_i$, and $N^{\prime} = N+\hat{N}$. Then the posterior predictive is (calculation omitted but can be added if anyone cares)

\begin{align*} p(\hat{y}|y) = \int p(\hat{y}|\mathbf{p}) p(\mathbf{p}|y) d\mathbf{p} = \frac{ \prod_i f^{\prime}_i! }{ \prod_i f_i! } \frac{ (N + k - 1)! }{(N^{\prime}+k-1)!} \end{align*}

So, can anyone suggest methods, preferably as efficient and simple as possible, to simulate from $p(\tilde{y}|y)$? This is a finite distribution, but too big to enumerate. At the moment, I'm not even sure what distribution it is. Thanks in advance.

PS. Can some kind person create "bayesian-computation" and "statistics" tags, and tag this message with them? "random" is not very useful. Thanks.