In Hamiltonian Monte Carlo, Hamiltonian dynamics are used to generate new proposals from the current state. I understand why these dynamics are used as opposed to random walk behavior to generate proposals, but I don't understand why Hamiltonian dynamics are used as opposed to any other dynamics. Could someone please explain this to me? Also, this seems to be a difficult thing to search for as well, so any pointers to references would be appreciated too.
I'm a little late to reply but I found this review to be really informative. One particularly nice feature of Hamiltonian flows is that they preserve volume in phase space. If we take an infinitesimal box of size $(\delta q, \delta p)$ around a point $(q, p)$ in phase space, then propagate every point inside the box forward in time by $\delta t$ under the Hamiltonian flow, the volume of the transformed box will stay the same. If you were to generate new proposals using some arbitrary transformation $\Psi$ that does not necessarily preserve volume in phase space, you'd also have to keep track of Jacobian factors $|\det\nabla\Psi|$ that may or may not be feasible to compute. Hamiltonian flows are volume-conserving, so this Jacobian factor is always equal to 1.
Just because Hamiltonian systems are nice doesn't necessarily mean that whatever discretization scheme you come up with shares all of these properties though. If you use a naive ODE solver like forward or backward Euler, then you lose the phase volume invariance property altogether.
Fortunately, there's a very rich theory for how to construct time-stepping schemes for Hamiltonian systems that do preserve a lot of invariants. These are called symplectic integrators. Provided that you use a symplectic integrator, such as semi-implicit Euler, the numerical flow is an exact sampling from the trajectories of a slightly perturbed Hamiltonian. You can even use some fancy Lie algebra theory to figure out what the perturbed Hamiltonian is to arbitrary order in the timestep. This backwards stability in the Hamiltonian means that all of the nice structural properties of Hamiltonian systems are preserved when you discretize.