Which are the right configurations in the Markov chain of a Hamiltonian Monte Carlo algorithm?

Question

I have a question about the Markov Chain Hamiltonian Monte Carlo (MCHMC). Hamiltonian Monte Carlo is known as Hybrid Monte Carlo too. I'll describe the steps of the algorithm.

1. We have at the beginning a coordinates configuration {$q_i$} and generate momenta configuration {$p_i$} from a gaussian distribution at the desired temperature $T$.

   $i$ run over the degrees of freedom of the system. So we initially have the configuration {$q_i, p_i$}. This is the first configuration in our Markov chain

2. We make $n$ leapfrog steps and obtain the configuration {$q'_i, p'_i$}

3. We accept or reject this last configuration using a Metropolis test

Now we continue with an example. Suppose this last configuration is rejected. Then we keep the old configuration {$q_i,p_i$} (first question: this is the second configuration of our Markov chain, right?) and generate (for the second time) a new {$p''_i$} set of momenta gaussian distributed. So we now have the configuration {$q_i, p''_i$}.

Now we make other $n$ leapfrog steps and arrive at the configuration {$q''_i, p'''_i$}. Second question: if Metropolis rejects it, the third configuration of our Markov chain will be {$q_i, p_i$} or {$q_i, p''_i$}?.

Now, instead, we suppose Metropolis accepts {$q''_i, p'''_i$}, and we generate a new gaussian set {$p^{IV}_i$}. So we now have the configuration {$q'', p^{IV}_i$} and then continue from it with leapfrog. The last question is: which is the third configuration of our Markov chain, {$q''_i, p'''_i$} or {$q''_i, p^{IV}_i$}?

Answer 1

The algorithm for perfoming a single HMC step is as follows:

Input: Some initial configuration $\vec{y}_i$ and momentum $\vec{p}_i$.
Output: Next configuration $\vec{y}_{i+1}$ and momentum $\vec{p}_{i+1}$

1. Draw a random momentum $\vec{p}_*$ from a Gaussian distribution.
2. Numerically solve Hamilton's equations of motion for some time (i.e., perform some Leapfrog steps) with initial conditions $(\vec{y}_i, \vec{p}_*)$. The end point of the trajectory is denoted by $(\vec{y}_c, \vec{p}_c)$.
3. Compute the acceptance probability $\rho_i \in [0,1]$ of the proposal / candidate $(\vec{y}_c, \vec{p}_c)$: $$\rho_i = \min\{\exp\left( - \beta \left( \mathcal{H}(\vec{y}_c, \vec{p}_c) - \mathcal{H}(\vec{y}_i, \vec{p}_i) \right) \right), 1\}.$$ Here, $\mathcal{H}$ denotes the Hamiltonian and $\beta > 0$ is the inverse temperature.
4. Draw a random number $\alpha_i \sim U([0,1])$ from the uniform distribution on $[0,1]$ and set $$(\vec{y}_{i+1}, \vec{p}_{i+1}) = \begin{cases} (\vec{y}_c, \vec{p}_c) & \text{if } \alpha_i \leq \rho_i, \\ (\vec{y}_i, \vec{p}_i) & \text{otherwise}. \end{cases}$$

Now, to answer your questions (and to give some additional remarks):

• Usually, the degrees of freedom are not treated separately. This is especially problematic if the Hamiltonian $\mathcal{H}$ does not decompose into multiplicative factors for each DOF. In the algorithm above, the full state $\vec{y}$ is drawn (i.e., all DOFs are drawn simultaneously).
• If a proposal is rejected, the initial state is taken ($\vec{y}_{i+1} = \vec{y}_i$).
• If two proposals are rejected in a row, the chain is still in the original state ($\vec{y}_{i+1} = \vec{y}_i$ and $\vec{y}_{i+2} = \vec{y}_{i+1}$, which implies $\vec{y}_{i+2} = \vec{y}_i$).