11
$\begingroup$

I'm very new to GPGPU programming so please forgive me if the question is not particularly appropriate. From what I understand GPU programming is a very intricate piece of engineering work when compared to usual CPU programming. One has to be very careful about divergence issues, tiling, pinned memory allocation, and host-device communication/device computation overlapping.

After doing a little bit of research I found the thrust library which seems to try to mimic C++ STL. This is quite nice. However, based on my very limited experience and having seen all the micro-managing required to get good performance, I'm a little bit skeptical about the performance. Can thrust efficiently handle all the intricate programming part internally? Some very well-known libraries, such as PETSc, seem to use this package which makes me believe it somehow should.

I was wondering if people with more experience on CUDA and thrust could say a word or two about the performance of the package when compared to the low level CUDA programming. When can I use thrust and when should I switch back to CUDA?

Improve this question
$\endgroup$
1
  • $\begingroup$ Have you considered ArrayFire? $\endgroup$
    – arrayfire
    Commented Jul 16, 2012 at 21:51

3 Answers 3

Reset to default
2
$\begingroup$

I don't have personal experienced with thrust, but I do use ViennaCL, which is another high level GPU library that hides almost all of the details. From my own personal benchmarking I can see speed-ups of 2x - 40x on the actual computation if you ignore the time it take to move around memory.

When you should use the CPU vs. thrust vs. CUDA all depends on the problem you are solving, your skill, and the time you have available. I would recommend starting by solving simple problems with all 3 methods to see their relative performance. Then you can write your actual software in a quick manner, benchmark it, and apply the appropriate gpu method in the areas that need the speed-up, rather than wasting your time writing CUDA software that will only gain you a couple minutes of execution time.

Improve this answer
$\endgroup$
2
  • $\begingroup$ That makes perfect sense to me. One always has to profile first. So in your example, the speedup you got was from using ViennaCL. Have you tried direct OpenCL to check for the difference? $\endgroup$
    – mmirzadeh
    Commented Jul 14, 2012 at 17:50
  • $\begingroup$ No, like you I am new to GPU computing. I plan over the next year or two to slowly expand my skills to include CUDA and OpenCL, but currently I only use the library. ViennaCL's documentation states that further speed up would be possible with a tuned openCL implementation which would likely be on the order of another 2x-10x, however I have learned that memory bandwidth is the 900 pound gorilla in the room that really defines your performance. $\endgroup$
    – Godric Seer
    Commented Jul 14, 2012 at 17:56
5
$\begingroup$

I have used Thrust in my linked cluster expansion project. Depending on the situation, Thrust can perform as well as or better than a low level implementation that you roll yourself (in particular, the reduce kernel has been working quite well for me). However Thrust's generic nature and flexibility means it sometimes has to do a lot of extra copying, array padding, etc. which can slow it down quite a bit in a few nasty edge cases. Last time I used sort it was quite slow compared to other libraries such as b40c or mgpu. However, NVIDIA has been working on improving Thrust's algorithmic performance so that may be less of an issue in the future.

You should try writing your code using both Thrust and CUDA and then using the Visual Profiler to determine which is better for the specific task you are interested in. If it's likely that memory transfer will take up the most running time of your program and you don't want to have to worry about optimizing your own kernels for bank conflicts, instruction count, etc. then I would use Thrust. It also has the side benefit of making your code much less verbose and easier for people who are not familiar with GPU programming to read.

Improve this answer
$\endgroup$
3
$\begingroup$

The purpose of thrust (as most template libraries) is to provide a high-level abstraction, while preserving good, or even excellent, performance.

I would suggest not to worry to much about performance, but to ask yourself if

  • your application can be described in terms of the algorithms implemented in thrust, and if

  • you like the possibility of writing "generic" parallel code, without the need of going into the gory details of finding an efficient mapping to the given hardware/software architecture.

If you respond positively to both questions you should be able to implement your program with less effort with respect to a CUDA only implementation. Then you can profile your application and decide if it is worthwhile to try improve performance.

This said, I have to confess that I do not like "generic" programming, because I'm willing to learn something new, when I write a program. I would follow another route: write a prototype implementation in python+numpy+scipy, then add CUDA kernels for those 1%--2% of the code that really needs optimization and is suitable to be run on a GPU. Of course by doing so you need some sort of pre-science, since a wrong decision in the prototyping phase (e.g a data structure unsuited for CUDA kernels) may have terrible results on performance. Usually more iterations are needed for obtaining a good code and there is no assurance of doing better than thrust.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.