random number generation from cython

Question

I want to make my python program fast by using cython, but my inner loop is still making slow python calls to the random number generator! Several years ago this same issue was raised by someone on sage-support and there seemed to be no good solution at that time. It is not convenient for me to pre-generate a long list of random samples because I am actually sampling from various distributions in a way that is conditional on previous samples.

Here's a blog post explaining how this was kludged by connecting from cython to gsl:
http://pyinsci.blogspot.com/2010/12/efficient-mcmc-in-python-errata-and.html

And a stackoverflow post by someone trying to implement the gsl kludge:
https://stackoverflow.com/questions/8177446/random-number-generators-to-work-on-x86-64

Answer 1

Cython makes code faster by removing the type ambiguity. Since random.py is a pure python module, you can just copy it and add the types to the functions you need. Then cython can optimize the dynamic overhead away.

Answer 2

Following aterrel's suggestion, you could use pyximport to automatically compile the random module:

import pyximport
pyximport.install(pyimport=True)

import random

However, this still will not make it as fast as it would be if you declared static types for the variables in Cython.

Answer 3

I'm not sure if these were added recently but it seems like there are now easy ways to generate random numbers quickly without too much overhead. From this article about Monte Carlo simulations in cython we can do

from libc.stdlib cimport rand, RAND_MAX
r = 1 + int(rand()/(RAND_MAX*6.0)) # random integer 1,...,6

As far as I understand you don't need to do anything special when compiling.

For reproducibility during testing you can set a seed

# srand48(time(0)) # Do it this way in production
srand48(100) # For reproducibility in testing