CPU and GPU influence on task parallel execution performance

Question

This question is mainly about hardware, but also about software.

In my current work I have approximately 68 millions of combinations that I am iterating through, in parallel. For each of those combinations - there is processing involved on the same data set. The idea is very similar to how neural networks train, I guess.

My current problem is, that it takes 10 hours to process all combinations. I am using the C# and the code can be simplified to this:

long combinationCount = 68,211,200; // (68m)
List<Data> data = new List<Data>();
long dataCount = data.Count(); // 65,800 (65k)
// itterate every combination
Parallel.For(0, combinationCount, new ParallelOptions { MaxDegreeOfParallelism = 4}, (index, loopState) => {
    for(int i = 0; i < dataCount; i++) {
        // proccess each data record
    }
});

My current hardware is an HP laptop with processor i7-6500U (2 cores, 4 threads). Quite old perhaps not the best choice for these kinds of tasks.

So my question is - how do I optimize and improve the performance of my applications? To be more precise - I would like to hear out ideas on following topics:

1. CPU. Since I am planning on purchasing a stationary PC - would it be best to make a choice in flavor of:
   • AMD Ryzen 1700/1700x/1800x - 8 cores, 16 threads
   • Intel's i7 8700/8700k - 6 core, 12 threads, higher frequency per core

Please note! The question is not about AMD or Intel, but more about whether it is better to have 8 physical cores at a lower clock speed than, a CPU with 6 cores, but a higher clock speed. Specifically for programming.

2. GPU. It has come to my attention, that you can leverage calculations to GPU cores instead of CPU. This in theory will increase the performance X-fold. The issue is that you need to use a lower-level programming language such as C++ to make advantage of that. Since C# is the only toolset in my possession - I was wondering whether it is worth investing into a better Video card instead on an CPU?

I have googled a bit and found a project altimesh Hybridizer which seems to allow to code on C# and compile it in such a way that a GPU will run it. The idea is neat. If the above is true - then would that make More sense to purchase a higher-end GPU instead? Overall, how much of a performance increase can you expect from calculations being executed on GPU (CUDA cores?) as opposed to 6/8 core CPU? If it matters - I am looking at a GTX 1060 3GB/6GB price range video card.

And again. I am sorry if this is not the best place to ask such question. My main concern is building a system that would sustain nowadays requirements for computations, be it AI/neural networks/ or huge data processing, as in my case

Answer 1

Both CPUs you list are much more powerful than your existing laptop and assuming you're getting even reasonable parallelism out of the loop should contribute significant speedup.

Without knowing your specific application I [personal opinion] would likely choose the processor with fewer faster cores since on a wide variety of tasks single core performance is still important (is everything you do parallelized?), and it's easier to scale parallel applications to effectively use smaller numbers of cores.

At 68,000,000 elements you are well into the size of items that may be amenable to GPU programming. That said, there are many caveats without knowing the exact problem domain, and many problems fit poorly into GPU architectures. GPUs are somewhat harder to program effectively -- memory access patterns very important and conditionals/branching statements may be catastrophic to performance. If the problem turns out to be 68e6 independent mathematical formulas over 32 bit floating point numbers there is potential for many-fold performance over CPUs. Be mindful that gaming GPUs are awful at both integer and double precision.

Edit: Striking out comment about integer performance - see comments below from njuffa.