0
$\begingroup$

I am looking for MCMC codes with a GPU suport (like NVIDIA or OpenCL libraries) to make faster run chains.

I know there is a plenty of codes that does MCMC but which ones could allow to exploit GPU power with MCMC ?

In particular, it seems it is possible with the Gibbs sampler but I would like to know more in practice, i.e how to implement it ?

$\endgroup$
5
  • $\begingroup$ For simulating a single Markov chain, there's generally little parallelism available in a Gibbs sampler. However, if you want to simulate many chains in parallel, and you're willing to run these chains long enough to get through your burn-in period, then using a GPU may be worthwhile. $\endgroup$ Aug 11 at 16:27
  • $\begingroup$ @BrianBorchers .Thans for your quick answer. Could you provide some links which explains how to run multiple chains (I have well undertsood) in parallel. The main issue is that a Markov-Chain is intrisically serial, so are there ways to circumvent this intrinsic property ? $\endgroup$
    – youpilat13
    Aug 11 at 16:32
  • $\begingroup$ You didn't say what platform you'd prefer to use. I know Python has some different methods, but I've not yet exploited any of them. You could try reading more about it here or here. $\endgroup$
    – Kat
    Aug 12 at 16:27
  • $\begingroup$ @Kat . I have posted my MCMC code in my original question. Could you see at first sight the way to benefit from GPU to optimize this code ? Regards $\endgroup$
    – youpilat13
    Nov 5 at 17:37
  • $\begingroup$ @youpilat it looks like your original question was about libraries that could do this. Your recent edits seem like new question and should probably be posted separately (maybe with a link to this question for context). $\endgroup$
    – Tyberius
    Nov 6 at 19:18
2
$\begingroup$

The Julia probabilistic programming libraries like Turing.jl work on abstract array types and use the Julia automatic differentiation tooling, so if you write your model as something that uses CUDA.jl then your evaluations will take place on the GPU. I've used this for large stiff PDE discretizations.

$\endgroup$
1
  • $\begingroup$ @ChrisRackaukas . Thanks for your answer. Could you take a look at my MCMC code that I would like to use with GPU. $\endgroup$
    – youpilat13
    Nov 5 at 17:32
1
$\begingroup$

One like this one?

https://github.com/brandonckelly/CUDAHM

What exactly are you looking for, which language, how plug&play and for which kind of problem?

Depending on the case stuff like pyMC3 or tensorflow probability might already suit your needs.

$\endgroup$
5
  • $\begingroup$ Thanks for your answer. What does mean "Routines for using CUDA to accelerate Bayesian inference of Hierarchical Models using Markov Chain Monte Carlo with GPUs.", I mean especially, the Hierarchical Models using MCMC ? I don't know this notion ? Moreover, Can I use this code with 6 or 7 or more parameters for example with CUDAHM ? $\endgroup$
    – youpilat13
    Aug 30 at 4:23
  • $\begingroup$ Please add further details to expand on your answer, such as working code or documentation citations. $\endgroup$
    – Community Bot
    Aug 30 at 12:31
  • $\begingroup$ @HydroGuy . do pyMC3 or tensorflow support GPU for drawing or buid Markov chain) ? What is tensorflow exactly, I have heard it about don't understand its utility : is it a library like numpy ? Best regards $\endgroup$
    – youpilat13
    Sep 23 at 11:07
  • $\begingroup$ @youpilat13, Tensorflow is a library for machine learning, specially deep learning. The basic primitives of the library (tensors) are basically the same as numpy (ndarrays), but with a backend that supports GPU acceleration. On top of that the library allows you to define (GPU acceleration) operations on those tensors on the form of a graph. For example, regarding MCMC specifically: tensorflow.org/probability/api_docs/python/tfp/mcmc $\endgroup$
    – Hydro Guy
    Sep 23 at 15:09
  • $\begingroup$ @HydroGuy . Thanks for your quick answer. Could you take a look please at my MCMC code that I would like to use with GPU. How to include this code with CUDAHM or pyMC3 ? Regards $\endgroup$
    – youpilat13
    Nov 5 at 17:34
1
$\begingroup$

The library that pymc uses to do this is aesara.

https://github.com/aesara-devs/aesara

Stan uses OpenCL

https://mc-stan.org/cmdstanr/articles/opencl.html

$\endgroup$
1
  • $\begingroup$ Thanks for your suggestion. By the way, could you take a look at my MCMC code that I would like to use with GPU. $\endgroup$
    – youpilat13
    Nov 5 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.