1
$\begingroup$

I am looking for MCMC codes with a GPU suport (like NVIDIA or OpenCL libraries) to make faster run chains.

If someone could have a state of the art ...

$\endgroup$
5
  • $\begingroup$ For simulating a single Markov chain, there's generally little parallelism available in a Gibbs sampler. However, if you want to simulate many chains in parallel, and you're willing to run these chains long enough to get through your burn-in period, then using a GPU may be worthwhile. $\endgroup$ Aug 11, 2021 at 16:27
  • $\begingroup$ @BrianBorchers .Thans for your quick answer. Could you provide some links which explains how to run multiple chains (I have well undertsood) in parallel. The main issue is that a Markov-Chain is intrisically serial, so are there ways to circumvent this intrinsic property ? $\endgroup$
    – user14831
    Aug 11, 2021 at 16:32
  • $\begingroup$ You didn't say what platform you'd prefer to use. I know Python has some different methods, but I've not yet exploited any of them. You could try reading more about it here or here. $\endgroup$
    – Kat
    Aug 12, 2021 at 16:27
  • $\begingroup$ @Kat . I have posted my MCMC code in my original question. Could you see at first sight the way to benefit from GPU to optimize this code ? Regards $\endgroup$
    – user14831
    Nov 5, 2021 at 17:37
  • $\begingroup$ @youpilat it looks like your original question was about libraries that could do this. Your recent edits seem like new question and should probably be posted separately (maybe with a link to this question for context). $\endgroup$
    – Tyberius
    Nov 6, 2021 at 19:18

3 Answers 3

1
$\begingroup$

The Julia probabilistic programming libraries like Turing.jl work on abstract array types and use the Julia automatic differentiation tooling, so if you write your model as something that uses CUDA.jl then your evaluations will take place on the GPU. I've used this for large stiff PDE discretizations.

$\endgroup$
1
  • $\begingroup$ @ChrisRackaukas . Thanks for your answer. Could you take a look at my MCMC code that I would like to use with GPU. $\endgroup$
    – user14831
    Nov 5, 2021 at 17:32
0
$\begingroup$

One like this one?

https://github.com/brandonckelly/CUDAHM

What exactly are you looking for, which language, how plug&play and for which kind of problem?

Depending on the case stuff like pyMC3 or tensorflow probability might already suit your needs.

$\endgroup$
5
  • $\begingroup$ Thanks for your answer. What does mean "Routines for using CUDA to accelerate Bayesian inference of Hierarchical Models using Markov Chain Monte Carlo with GPUs.", I mean especially, the Hierarchical Models using MCMC ? I don't know this notion ? Moreover, Can I use this code with 6 or 7 or more parameters for example with CUDAHM ? $\endgroup$
    – user14831
    Aug 30, 2021 at 4:23
  • $\begingroup$ Please add further details to expand on your answer, such as working code or documentation citations. $\endgroup$
    – Community Bot
    Aug 30, 2021 at 12:31
  • $\begingroup$ @HydroGuy . do pyMC3 or tensorflow support GPU for drawing or buid Markov chain) ? What is tensorflow exactly, I have heard it about don't understand its utility : is it a library like numpy ? Best regards $\endgroup$
    – user14831
    Sep 23, 2021 at 11:07
  • $\begingroup$ @youpilat13, Tensorflow is a library for machine learning, specially deep learning. The basic primitives of the library (tensors) are basically the same as numpy (ndarrays), but with a backend that supports GPU acceleration. On top of that the library allows you to define (GPU acceleration) operations on those tensors on the form of a graph. For example, regarding MCMC specifically: tensorflow.org/probability/api_docs/python/tfp/mcmc $\endgroup$
    – Hydro Guy
    Sep 23, 2021 at 15:09
  • $\begingroup$ @HydroGuy . Thanks for your quick answer. Could you take a look please at my MCMC code that I would like to use with GPU. How to include this code with CUDAHM or pyMC3 ? Regards $\endgroup$
    – user14831
    Nov 5, 2021 at 17:34
0
$\begingroup$

The library that pymc uses to do this is aesara.

https://github.com/aesara-devs/aesara

Stan uses OpenCL

https://mc-stan.org/cmdstanr/articles/opencl.html

$\endgroup$
1
  • $\begingroup$ Thanks for your suggestion. By the way, could you take a look at my MCMC code that I would like to use with GPU. $\endgroup$
    – user14831
    Nov 5, 2021 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.