I am looking for MCMC codes with a GPU suport (like NVIDIA or OpenCL libraries) to make faster run chains.
If someone could have a state of the art ...
$\begingroup$
$\endgroup$
5
-
$\begingroup$ For simulating a single Markov chain, there's generally little parallelism available in a Gibbs sampler. However, if you want to simulate many chains in parallel, and you're willing to run these chains long enough to get through your burn-in period, then using a GPU may be worthwhile. $\endgroup$– Brian BorchersAug 11, 2021 at 16:27
-
$\begingroup$ @BrianBorchers .Thans for your quick answer. Could you provide some links which explains how to run multiple chains (I have well undertsood) in parallel. The main issue is that a Markov-Chain is intrisically serial, so are there ways to circumvent this intrinsic property ? $\endgroup$– user14831Aug 11, 2021 at 16:32
-
$\begingroup$ You didn't say what platform you'd prefer to use. I know Python has some different methods, but I've not yet exploited any of them. You could try reading more about it here or here. $\endgroup$– KatAug 12, 2021 at 16:27
-
$\begingroup$ @Kat . I have posted my MCMC code in my original question. Could you see at first sight the way to benefit from GPU to optimize this code ? Regards $\endgroup$– user14831Nov 5, 2021 at 17:37
-
$\begingroup$ @youpilat it looks like your original question was about libraries that could do this. Your recent edits seem like new question and should probably be posted separately (maybe with a link to this question for context). $\endgroup$– TyberiusNov 6, 2021 at 19:18
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
The Julia probabilistic programming libraries like Turing.jl work on abstract array types and use the Julia automatic differentiation tooling, so if you write your model as something that uses CUDA.jl then your evaluations will take place on the GPU. I've used this for large stiff PDE discretizations.
answered Nov 5, 2021 at 14:52
-
$\begingroup$ @ChrisRackaukas . Thanks for your answer. Could you take a look at my MCMC code that I would like to use with GPU. $\endgroup$– user14831Nov 5, 2021 at 17:32
$\begingroup$
$\endgroup$
5
One like this one?
https://github.com/brandonckelly/CUDAHM
What exactly are you looking for, which language, how plug&play and for which kind of problem?
Depending on the case stuff like pyMC3 or tensorflow probability might already suit your needs.
-
$\begingroup$ Thanks for your answer. What does mean "Routines for using CUDA to accelerate Bayesian inference of Hierarchical Models using Markov Chain Monte Carlo with GPUs.", I mean especially, the Hierarchical Models using MCMC ? I don't know this notion ? Moreover, Can I use this code with 6 or 7 or more parameters for example with CUDAHM ? $\endgroup$– user14831Aug 30, 2021 at 4:23
-
$\begingroup$ Please add further details to expand on your answer, such as working code or documentation citations. $\endgroup$– Community BotAug 30, 2021 at 12:31
-
$\begingroup$ @HydroGuy . do pyMC3 or tensorflow support GPU for drawing or buid Markov chain) ? What is tensorflow exactly, I have heard it about don't understand its utility : is it a library like numpy ? Best regards $\endgroup$– user14831Sep 23, 2021 at 11:07
-
$\begingroup$ @youpilat13, Tensorflow is a library for machine learning, specially deep learning. The basic primitives of the library (tensors) are basically the same as numpy (ndarrays), but with a backend that supports GPU acceleration. On top of that the library allows you to define (GPU acceleration) operations on those tensors on the form of a graph. For example, regarding MCMC specifically: tensorflow.org/probability/api_docs/python/tfp/mcmc $\endgroup$ Sep 23, 2021 at 15:09
-
$\begingroup$ @HydroGuy . Thanks for your quick answer. Could you take a look please at my MCMC code that I would like to use with GPU. How to include this code with CUDAHM or pyMC3 ? Regards $\endgroup$– user14831Nov 5, 2021 at 17:34
$\begingroup$
$\endgroup$
1
The library that pymc uses to do this is aesara.
https://github.com/aesara-devs/aesara
Stan uses OpenCL
-
$\begingroup$ Thanks for your suggestion. By the way, could you take a look at my MCMC code that I would like to use with GPU. $\endgroup$– user14831Nov 5, 2021 at 17:33