3
$\begingroup$

Just speaking from my experience in c++ - Its really tricky to take advantage of the SIMD capabilities of modern CPU's for more complex algorithms. If I see an opportunity from a high level OO perspective, I would then have to break the whole thing down to a very non-OO kind of code, babysitting every register individually even if they have no interaction with each other and not only the part that benefits from vectorizing but often the entire flow of data along with data formats have to be redesigned into a more basic and harder to manage form.

The automatic vectorization tools(qvec/openmp) offer no benefit either since they can only deal with just the most basic data types and loops with no OO concepts supported at all and imo only succeed in obscuring the code and limiting your options over the manual way.

Meanwhile, when coding for GPGPU's, you essentially get to do it in a perfectly OO manner(more/less, depending on the language) while focusing on the journey of one piece of data through the pipeline which just gets duplicated to every other piece of data, brilliant!

I know that gpu "cores" aren't the same as a SIMD unit on a cpu but from what I've gathered they are much closer to that than actual individual cores since they have to move in a kind of a lockstep.

So basically, why is that?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ Can you refer to an example of programming a GPU in OOP style? In my experience with CUDA, you write tightly optimized, single-purpose kernels and then invoke them on one or several arrays of data -- much as you would with most SIMD optimizations. $\endgroup$ – cfh Mar 5 '15 at 9:11
  • $\begingroup$ All gpu languages support at least functions and custom data structures(as opposed to simd intrinsics). I don't know about others but c++ AMP also supports interactive objects and pointers, the only real difference from a fully OO environment is that you have to preallocate all data. $\endgroup$ – user14729 Mar 5 '15 at 9:15
  • $\begingroup$ If you use a modern compiler, it will typically try to auto-vectorize C/C++ code and use SIMD intrinsics automatically. Not to mention that SIMD is used in many library functions, say, memcpy. So you can work at a high OO level and get many benefits of SIMD instruction sets "for free". $\endgroup$ – cfh Mar 5 '15 at 9:22
  • $\begingroup$ Don't know about other compilers but MSVC is super picky about what it vectorizes, basically any function call, any data type other than basic numerics, any conditional will disqualify a loop from vectorization. Essentially you have to set up data retreival for the loop exactly like you would for a simd block so why not just use those. Also, vectorized library functions are nice and all but what I'm talking about is vectorizing logic. $\endgroup$ – user14729 Mar 5 '15 at 9:44
  • $\begingroup$ GPGPU methods are notoriously bad at vectorizing logic because any conditional statement which does not take the same branch on all stream processors in one thread block will cause some stream processors to sit idle while the remaining ones execute their branch. So I don't see the advantage of GPGPU that you claim. $\endgroup$ – cfh Mar 5 '15 at 9:53
4
$\begingroup$

Before I try to answer your question let me comment on the words "low level" in your question statement. In my opinion, I prefer not to say one programming model(one of SIMD and SIMT) is at a low level. As an example, during my undergraduage diploma project I had experienced many low level issues about GPU programming, which I have not encountered in CPU programming. Most of these issues involve the memory architecture of GPU, such as memory coalescing access, bank conflict, etc. You can search these topics to get a feeling of low-level-ness from GPU programming.

And now return to your question, by "low level" I assume you mean that you have to refactor your code or even redesign your algorithm to fit the SIMD model, while on GPU(e.g. using CUDA) it may not require too much work like that. In some cases your source code for CPU can even compiled without any errors using CUDA compiler. But to vectorize a serial program in SIMD you have to translate all branching code(if, while block).

(I am not quite sure of what you mean by "OOP" in your question statement. Please tell me if I made a wrong assumption.)

The reason is that GPU programming(CUDA) uses a different model from the one used in CPU programming. The model for GPU programming is called SIMT. T is for thread.

In SIMD model, all arithmetic operations have to performed in a synchronized way. So any branching execution is not allowed. While in CUDA, branching is allowed in term of CUDA syntax, and more importantly, it is supported by hardware architecture and CUDA runtime. When a branching execution happens in CUDA/SIMT, the thread manager will coordinate the execution: those threads with identical execution path will be executed, and the diverse threads will be queued for later execution. During this process, other independent threads may be brought in to keep a high throughput.

As you can see, the SIMT model helps you to handle the branching execution.

Finally, I would like to let you know that you can programming SIMD in a "high level" by introducing another syntax layer. You can try https://ispc.github.io/.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

I think you are comparing using high-level libraries built on GPGPU techniques, like C++ AMP, to programming SIMD at the lowest possible assembly language or intrinsics level. That's not a fair comparison.

Since you specifically mentioned C++ AMP, let me use a basic example to argue that your premise isn't true. The introductory AMP example shows how to parallelize the following simple vector addition using AMP:

#include <iostream>

void StandardMethod() {

    int aCPP[] = {1, 2, 3, 4, 5};
    int bCPP[] = {6, 7, 8, 9, 10};
    int sumCPP[5];

    for (int idx = 0; idx < 5; idx++)
    {
        sumCPP[idx] = aCPP[idx] + bCPP[idx];
    }

    for (int idx = 0; idx < 5; idx++)
    {
        std::cout << sumCPP[idx] << "\n";
    }
}

To do this, you have to use an AMP library function and write your loop in a specific way. What if we instead want to parallelize this loop using SIMD instructions on the CPU? We only have to compile it with a reasonably modern compiler and suitable flags!

Compiling the above code with gcc 4.8.3 on Linux and disassembling it, I obtain

$ gcc -O3 -c vectest.cpp && objdump -M intel -d vectest.o 

vectest.o:     file format elf64-x86-64


Disassembly of section .text:

0000000000000000 <_Z14StandardMethodv>:
   0:   55                      push   rbp
   1:   53                      push   rbx
   2:   48 83 ec 68             sub    rsp,0x68
   6:   c7 04 24 01 00 00 00    mov    DWORD PTR [rsp],0x1
   d:   c7 44 24 04 02 00 00    mov    DWORD PTR [rsp+0x4],0x2
  14:   00 
  15:   48 8d 5c 24 40          lea    rbx,[rsp+0x40]
  1a:   c7 44 24 08 03 00 00    mov    DWORD PTR [rsp+0x8],0x3
  21:   00 
  22:   c7 44 24 0c 04 00 00    mov    DWORD PTR [rsp+0xc],0x4
  29:   00 
  2a:   48 8d 6c 24 54          lea    rbp,[rsp+0x54]
  2f:   66 0f 6f 04 24          movdqa xmm0,XMMWORD PTR [rsp]
  34:   c7 44 24 20 06 00 00    mov    DWORD PTR [rsp+0x20],0x6
  3b:   00 
  3c:   c7 44 24 24 07 00 00    mov    DWORD PTR [rsp+0x24],0x7
  43:   00 
  44:   c7 44 24 28 08 00 00    mov    DWORD PTR [rsp+0x28],0x8
  4b:   00 
  4c:   c7 44 24 2c 09 00 00    mov    DWORD PTR [rsp+0x2c],0x9
  53:   00 
  54:   66 0f fe 44 24 20       paddd  xmm0,XMMWORD PTR [rsp+0x20]
  5a:   66 0f 7f 44 24 40       movdqa XMMWORD PTR [rsp+0x40],xmm0
  60:   c7 44 24 10 05 00 00    mov    DWORD PTR [rsp+0x10],0x5
  67:   00 
  68:   c7 44 24 30 0a 00 00    mov    DWORD PTR [rsp+0x30],0xa
  6f:   00 
  70:   c7 44 24 50 0f 00 00    mov    DWORD PTR [rsp+0x50],0xf
  77:   00 
  78:   8b 33                   mov    esi,DWORD PTR [rbx]
  7a:   bf 00 00 00 00          mov    edi,0x0
  7f:   48 83 c3 04             add    rbx,0x4
  83:   e8 00 00 00 00          call   88 <_Z14StandardMethodv+0x88>
  88:   ba 01 00 00 00          mov    edx,0x1
  8d:   be 00 00 00 00          mov    esi,0x0
  92:   48 89 c7                mov    rdi,rax
  95:   e8 00 00 00 00          call   9a <_Z14StandardMethodv+0x9a>
  9a:   48 39 eb                cmp    rbx,rbp
  9d:   75 d9                   jne    78 <_Z14StandardMethodv+0x78>
  9f:   48 83 c4 68             add    rsp,0x68
  a3:   5b                      pop    rbx
  a4:   5d                      pop    rbp
  a5:   c3                      ret    

Disassembly of section .text.startup:

0000000000000000 <_GLOBAL__sub_I__Z14StandardMethodv>:
   0:   48 83 ec 08             sub    rsp,0x8
   4:   bf 00 00 00 00          mov    edi,0x0
   9:   e8 00 00 00 00          call   e <_GLOBAL__sub_I__Z14StandardMethodv+0xe>
   e:   ba 00 00 00 00          mov    edx,0x0
  13:   be 00 00 00 00          mov    esi,0x0
  18:   bf 00 00 00 00          mov    edi,0x0
  1d:   48 83 c4 08             add    rsp,0x8
  21:   e9 00 00 00 00          jmp    26 <_GLOBAL__sub_I__Z14StandardMethodv+0x26>

As you can see, the compiler has automatically used MMX instructions to optimize your loop, and without having to add any CPU- or library-specific annotations. So I'd say that your claim that using SIMD is more difficult than using GPGPU techniques in high-level code is not true -- quite the contrary.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I'm not talking about as low as in assembly but rather the functions provided in libraries such as xmmintrin. Trying to utilize those leads to code thats most similar to languages like the original BASIC. Looping an integer array is one thing but what if you're looping data structures and performing series of operations on them in a perfectly parallel manner? no vectorization there. $\endgroup$ – user14729 Mar 5 '15 at 10:27
  • $\begingroup$ @user14729 "xmmintrin" is originally provided to C and C++ library writers so that they can write inline-assembly-like code that can compile into 32-bit and 64-bit libraries. (Normally, inline-assembly require different code for 32-bit and 64-bit; but code written using intrinsics can be compiled into both.) It is because progresses in automatic vectorization was delivered late (by a whole decade) that everyone had to play the role of library writers and learn to write in intrinsics. Nowadays, whenever auto-vec is available, you don't have to use intrinsics. If you use it then don't complain. $\endgroup$ – rwong May 21 '15 at 15:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.