Is it possible to run a Metropolis Monte Carlo simulation in parallel?

Suppose I perform a Metropolis Monte Carlo simulation using four threads.

Suppose, the programming source code divides a specified number of iterations among four threads to compute parts of the simulation concurrently. Each thread generates random samples, accumulates a partial result, and writes its output to a file, ensuring thread-safe operations with locks.

Will this Monte Carlo simulation behave in the same way as a single-threaded Monte Carlo simulation?

Will the sequential and parallel monte carlo give us the same outcome?

  • 1
    $\begingroup$ You have to describe the problem you are applying Monte Carlo to, and the specific meyhod you use. For integration if your samples are independent are identically distributed you can run separate simulations, provided each gets a different random sequence (i.e. they need differemt seeds). Things are more tricky with quasi-Monte Carlo methods with low discrepancy sequences, where you have to potentially give different ranges of the sequence to the different threads, etc. But in general your question is very unspecific so it is hard to answer in a satisfactory manner. $\endgroup$
    – lightxbulb
    Apr 1 at 13:49
  • $\begingroup$ @lightxbulb, I am doing protein modeling and simulation. $\endgroup$
    – user366312
    Apr 1 at 13:58
  • 1
    $\begingroup$ What I mean is that you need to describe mathematically what you are doing. Naive Monte Carlo (MC) integration, vs quasi-MC integration, vs Metropolis-Hastings MC integration, vs MC optimisation, etc. are all different methods. Often the point is that you have an unbiased estimator which allows you to average different samples in order to get a solution with a lower variance, but that is not always the case. $\endgroup$
    – lightxbulb
    Apr 1 at 14:34
  • $\begingroup$ @lightxbulb, Metropolis-only, not Metropolis-Hastings. $\endgroup$
    – user366312
    Apr 1 at 14:47
  • $\begingroup$ Aymptotically, each thread is sampling from the same distribution, so you answer should be the same eventually. $\endgroup$
    – whpowell96
    Apr 1 at 16:53

1 Answer 1


In short, yes.

Mathematically you are free to separate any integral into parts over the integration domain. If you later add the results of these integrations, then the partial sums add up to the same value! In your case you may integrate these partial integrals via separate Monte Carlo simulations. If each thread uses the same sampling density per domain etc. then that works just fine. Keep in mind that CPU time is not everything. You may increase your memory footprint.

Edit: I did not read up on the Metropolis variant of MC. The classical use seems to be the sampling of 2D/3D paths via random offset vectors. This is slightly different than the classical integration-over-domain example. However, since each trajectory is independent of the previous run, you should also be able to seperate the problem accross CPU's and merge/add the results in the end.

  • 1
    $\begingroup$ This is correct, although it is important to note that you shouldn't expect any speed ups in convergence as each thread will still take the same time to reach the stationary distribution on average and there is no guarantee that these initial deviations from stationarity will cancel out $\endgroup$
    – whpowell96
    Apr 2 at 15:04
  • $\begingroup$ Correct. The convergence per sample is the same, but the walltime to calculate may be reduced. $\endgroup$
    – MPIchael
    Apr 3 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.