-1
$\begingroup$

I am looking at a paper comparing performance of two pricers: same model (based on Monte Carlo simulation) but one implemented on CPU (C++) and one implemented on GPU (Cuda)

The paper mentions the time taken to execute N paths (let's say 100k paths) in both cases. Additional information:

  • CPU implementation relies on double precision, while GPU implementation relies on single precision.
  • The pricers use quasi-random generator (rate of convergence close to O(1/N))

To do a "fair" comparison (i.e achieving a similar precision), shouldn't we double the number of paths for GPU pricer (single precision)?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Can you also add a reference to the paper if it is published and/or available online? This question is hard to answer otherwise. My simple answer would be unless $N\geq 10^8$, it doesn't matter because you generally won't reach to the single precision machine epsilon levels of accuracy without using a large value for $N$. $\endgroup$
    – Abdullah Ali Sivas
    May 21, 2021 at 15:37
  • $\begingroup$ @AbdullahAliSivas the paper is unfortunately not available online, and lacks details. I tried to summarize it the best I could. $\endgroup$
    – Alex Dino
    May 21, 2021 at 23:33

1 Answer 1

Reset to default
2
$\begingroup$

First, single precision is not twice as fast as double precision. It will be quite difficult to determine theoretically what speed-up you get by moving from double to single precision; if you wanted to make this fair, you'd have to actually measure the speedup.

Second, for Monte Carlo methods the metric people generally use to determine how "good" an algorithm is is how fast they converge to an answer. Since the exponent (${\cal O}(N^{-1/2})$) is fixed, what matters most is the constant that sits in front of the $N^{-1/2}$, and that is (or should be) independent of the precision with which you do you calculation -- it depends on things such as the proposal distribution, whether you have a simple Metropolis-Hastings or a deferred rejection algorithm, etc.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Note that for most NVIDIA GPU's, double precision performance is limited to 1/32 of the single precision performance. This is a marketing move by NVIDIA to force customers who want to do double precision to purchase more expensive GPU's that are marketed for use in HPC instead of gaming GPU's. $\endgroup$
    – Brian Borchers
    May 21, 2021 at 16:38
  • $\begingroup$ @BrianBorchers: this is true for the maximum FLOPs for consumer GPUs (GeForce). In my experience with heavily memory bound algorithms, I observed a factor of roughly two only between single and double precision, because of doubling the amount of data. The reduced FLOPs (limited by the GPU driver btw) did not have an impact at all. $\endgroup$
    – dweber
    May 21, 2021 at 18:13
  • $\begingroup$ I didn't make that point very well. What I wanted to say is that I assume that the algorithm in question does not only do floating point calculations, but also other things, and that consequently the overall speed is not determined exclusively based on floating point speed. $\endgroup$
    – Wolfgang Bangerth
    May 24, 2021 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.