34
$\begingroup$

How do CUDA and OpenCL compare to each other as of late 2013 from a programmer's perspective? My group is thinking about trying to make use of GPU computing. Would we be limiting ourself significantly by choosing hardware that only supports OpenCL but not CUDA?

To be a bit more specific, are the following assumptions correct?

  • Everything that's possible in CUDA is also possible in OpenCL

  • For as long as we're not using libraries, a given task is not significantly easier (or more difficult) to do in either of them

  • CUDA's main advantage is the availability of libraries

  • Both have good support for all three main platforms (Win/OSX/Linux)

$\endgroup$
  • 1
    $\begingroup$ I'll try to update the question if it's not specific enough ... we're new to this domain, and this is one of the basic questions that came up, especially because of the choice of hardware. One thing that grabbed my attention and made me cautious about OpenCL is this thread. If OpenCL is not good enough for raytracing (Blender) then maybe it's not good enough for HPC either ... $\endgroup$ – Szabolcs Dec 19 '13 at 20:20
  • $\begingroup$ I haven't been active in GPUs since 2011 so I'll let someone more up-to-date answer, but your statement "a given task is not significantly easier to do in either of them" was something I did not find to be true at all. There's personal bias here, but I felt CUDA was miles ahead of OpenCL in terms of development ease. I'd have to update myself, but if you asked me 2 years ago I'd have said I wouldn't touch OpenCL with a 10 foot pole. $\endgroup$ – Aurelius Dec 19 '13 at 20:25
  • $\begingroup$ I'm aware of this question: The Future of OpenCL, but it's not precisely the same question and it's 2 years old. I assume things may have changes in two years. $\endgroup$ – Szabolcs Dec 19 '13 at 20:27
  • $\begingroup$ One of the practical questions for us is if a 2013 Mac Pro with AMD FirePro cards is good for getting familiar and making use of GPU computing (no CUDA support). $\endgroup$ – Szabolcs Dec 19 '13 at 20:31
  • $\begingroup$ It might be worth it to keep C++ AMP on your radar as well. An llvm implementation is in the works. hsafoundation.com/bringing-camp-beyond-windows-via-clang-llvm. Microsoft has already created C++ AMP libraries for FFT, BLAS and LAPACK. $\endgroup$ – Roger Dahl Dec 19 '13 at 22:07
37
$\begingroup$

I'll try to summarize my experiences obtained in the course of developing ViennaCL, where we have CUDA and OpenCL backends with mostly 1:1 translations of a lot of compute kernels. From your question I'll also assume that we are mostly taking about GPUs here.

Performance Portability. First of all, there is no such thing as performance-portable kernels in the sense that you write a kernel once and it will run efficiently on every hardware. Not in OpenCL, where it is more apparent due to the broader range of hardware supported, but also not in CUDA. In CUDA it is less apparent because of the smaller range of hardware supported, but even here we have to distinguish at least three hardware architectures (pre-Fermi, Fermi, Kepler) already. These performance fluctuations can easily result in a 20 percent performance variation depending on how you orchestrate threads and which work group sizes you choose, even if the kernel is as simple as a buffer copy. It's probably also worth mentioning that on pre-Fermi and Fermi GPUs it was possible to write fast matrix-matrix multiplication kernels directly in CUDA, while for the latest Kepler GPUs it seems that one has to go down to the PTX pseudo-assembly language in order to get close to CUBLAS' performance. Thus, even a vendor-controlled language such as CUDA appears to have issues to keep the pace with hardware developments. Moreover, all CUDA code gets compiled statically when you run nvcc, which somewhat requires a balancing act via the -arch flag, while OpenCL kernels get compiled at run-time from the just-in-time compiler, so you can in principle tailor kernels down to the very specifics of a particular compute device. The latter is, however, quite involved and usually only becomes a very attractive option as your code matures and as your experience accumulates. The price to pay is the O(1) time required for just-in-time compilation, which can be an issue in certain situations. OpenCL 2.0 has some great improvements to address this.

Debugging and Profiling. The CUDA debugging and profiling tools are the best available for GPGPU. AMD's tools are not bad either, but they do not include gems like cuda-gdb or cuda-memcheck. Also, still today NVIDIA provides the most robust drivers and SDKs for GPGPU, system freezes due to buggy kernels are really the exception, not the rule, both with OpenCL and CUDA. For reasons I probably do not need to explain here, NVIDIA no longer offers debugging and profiling for OpenCL with CUDA 5.0 and above.

Accessibility and Convenience. It is a lot easier to get the first CUDA codes up and running, particularly since CUDA code integrates rather nicely with host code. (I'll discuss the price to pay later.) There are plenty of tutorials out there on the web as well as optimization guides and some libraries. With OpenCL you have to go through quite a bit of initialization code and write your kernels in strings, so you only find compilation errors during execution when feeding the sources to the jit-compiler. Thus, it takes longer to go through one code/compile/debug cycle with OpenCL, so your productivity is usually lower during this initial development stage.

Software Library Aspects. While the previous items were in favor of CUDA, the integration into other software is a big plus for OpenCL. You can use OpenCL by just linking with the shared OpenCL library and that's it, while with CUDA you are required to have the whole CUDA toolchain available. Even worse, you need to use the correct host compilers for nvcc to work. If you ever tried to use e.g. CUDA 4.2 with GCC 4.6 or newer, you'll have a hard time getting things to work. Generally, if you happen to have any compiler in use which is newer than the CUDA SDK, troubles are likely to occur. Integration into build systems like CMake is another source of headache (you can also find ample of evidence on e.g. the PETSc mailinglists). This may not be an issue on your own machine where you have full control, but as soon as you distribute your code you will run into situations where users are somewhat restricted in their software stack. In other words, with CUDA you are no longer free to choose your favourite host compiler, but NVIDIA dictates which compilers you are allowed to use.

Other Aspects. CUDA is a little closer to hardware (e.g. warps), but my experience with linear algebra is that you rarely get a significant benefit from it. There are a few more software libraries out there for CUDA, but more and more libraries use multiple compute backends. ViennaCL, VexCL, or Paralution all support OpenCL and CUDA backends in the meanwhile, a similar trend can be seen with libraries in other areas.

GPGPU is not a Silver Bullet. GPGPU has been shown to provide good performance for structured operations and compute-limited tasks. However, for algorithms with a non-negligible share of sequential processing, GPGPU cannot magically overcome Amdahl's Law. In such situations you are better off using a good CPU implementation of the best algorithm available rather than trying to throw a parallel, but less suitable algorithm at your problem. Also, PCI-Express is a serious bottleneck, so you need to check in advance whether the savings from GPUs can compensate the overhead of moving data back and forth.

My Recommendation. Please consider CUDA and OpenCL rather than CUDA or OpenCL. There is no need to unnecessarily restrict yourself to one platform, but instead take the best out of both worlds. What works well for me is to set up an initial implementation in CUDA, debug it, profile it, and then port it over to OpenCL by simple string substitutions.( You may even parametrize your OpenCL kernel string generation routines such that you have some flexibility in tuning to the target hardware.) This porting effort will usually consume less than 10 percent of your time, but gives you the ability to run on other hardware as well. You may be surprised about how well non-NVIDIA hardware can perform in certain situations. Most of all, consider the reuse of functionality in libraries to the largest extent possible. While a quick&dirty reimplementation of some functionality often works acceptable for single-threaded execution on a CPU, it will often give you poor performance on massively parallel hardware. Ideally you can even offload everything to libraries and don't ever have to care about whether they use CUDA, OpenCL, or both internally. Personally I would never dare to write vendor-locked code for something I want to rely on in several years from now, but this ideological aspect is should go into a separate discussion.

$\endgroup$
  • $\begingroup$ What do you think about the current performance and usability of standard 1D,2D,3D FFT in both platform? $\endgroup$ – unsym Dec 20 '13 at 11:44
  • $\begingroup$ Concerning JIT compilation, CUDA also offers that possibility, but with some constraints. $\endgroup$ – BenC Dec 20 '13 at 11:56
  • $\begingroup$ @hwlau: FFT is standard functionality for vendor libraries, so pretty independent of CUDA vs. OpenCL. $\endgroup$ – Karl Rupp Dec 20 '13 at 15:16
  • $\begingroup$ @BenC: The constraints are indeed very severe, it's merely a specialization of precompiled CUDA-kernels to the underlying hardware. $\endgroup$ – Karl Rupp Dec 20 '13 at 15:28
  • 1
    $\begingroup$ Do you have any comments on this potential problem? It's not clear to me if this is a problem with AMD hardware or with OpenCL itself (i.e. whether the problem is nonexistent with OpenCL on NVIDIA). Perhaps this is not a problem with most scientific computing applications because they tend to be smaller and less complex than an advanced raytracer? BTW thank you for the great answer! $\endgroup$ – Szabolcs Dec 20 '13 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.