19 votes

Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

The simple example from electromagnetics (EM) would be performing a parallel frequency sweep for a frequency-domain simulation, say, full-wave extraction of network parameters (S, Y, Z, etc) for a ...
Anton Menshov's user avatar
  • 8,672
14 votes
Accepted

Performance of kd-tree vs brute-force nearest neighbor search on GPU?

Ultimately, naive brute-force KNN is an $O(n^2)$ algorithm, while kd-tree is $O(n \log n)$, so at least in theory, kd-tree will eventually win out for a large enough $n$. In practice, the leading ...
Richard Zhang's user avatar
14 votes

Options for solving ODE systems on GPUs?

DifferentialEquations.jl library is a library for a high level language (Julia) which has tools for automatically transforming the ODE system to an optimized version for parallel solution on GPUs. ...
Chris Rackauckas's user avatar
12 votes
Accepted

PETSc-like library for Julia

Julia is built in such a way that you will never see a full PETSc-like library, and that's on purpose. PETSc is not a single thing: it is an HPC library with some utility functions, linear solvers, ...
Chris Rackauckas's user avatar
11 votes

Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

High-quality video encoding is something like this. The search space is so huge that it requires branching to prune it rapidly, but GPUs are terrible at that. Modern CPU short-vector SIMD works well ...
Peter Cordes's user avatar
10 votes

How to properly calculate CPU and GPU FLOPS performance?

You can calculate GFLOP rates this way, but the numbers are pretty meaningless on today's hardware: Floating point operations require a variable number of clock cycles. An addition is generally ...
Wolfgang Bangerth's user avatar
9 votes
Accepted

Consumer hardware for scientific computing?

One issue that you should be aware of is that NVIDIA has a market segmentation strategy in which it sells relatively inexpensive GPU's to the gaming and graphics workstation markets (GeForce and ...
Brian Borchers's user avatar
9 votes

Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

GPUs work with the model SIMD (single instruction multiple data) i.e. they execute an instruction over multiple data. To give an idea: under CUDA technology when you have got an if-then-else ...
Mauro Vanzetto's user avatar
8 votes

Lost on Matrix Inversion

Several points I want to mention (with an encouragement to other CompSci users that are more familiar with Java specifics to give additional, more Java related answers): The solution of a system of ...
Anton Menshov's user avatar
  • 8,672
7 votes
Accepted

Lost on Matrix Inversion

Normally when you invert a sparse matrix the inverse is dense. This imply to have enough memory to store the inverse, in your case the matrix is not so big for nowdays computers. In double precision (...
Mauro Vanzetto's user avatar
7 votes

Going to try to move some of my scipy/numpy calculation to a new GPU, how to avoid disappointing results?

I buy the wrong CUDA GPU and the speed up is minimal or nonexistent. It is highly unlikely that your choice of GPU will have a significant impact on your speed-up unless your model is very big. To a ...
Richard's user avatar
  • 3,961
7 votes

Questions on the theory of distributed numerical algebraic computation

Building such a library from scratch is a serious undertaking. The SLATE library, which targets the functionality you're looking at, has taken about 6 years with 4-7 full-time developers, most of ...
Neil Lindquist's user avatar
6 votes

Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

When it comes to playing chess and other complex turn-based games using the MiniMax algorithm, then GPU acceleration is either not viable or only viable for a couple minor sub-problems. Chess engines ...
Philipp's user avatar
  • 161
5 votes

Is there a constrained nonlinear optimization library like IPOPT that runs on GPUs?

I'm a little late to the party, but the short answer is that yes it's possible to parallelize an interior point method for GPUs, but whether or not that is successful depends on the structure of the ...
wyer33's user avatar
  • 767
5 votes

Why am I not seeing faster neural network training after upgrading to a vastly better GPU?

I suppose, you right and your network is not that big to 100%-utilize the GPU. The bottle-neck here seems to be not the GPU itself, but the transfer rate between RAM and VRAM and here the difference ...
Vlad's user avatar
  • 114
4 votes
Accepted

Hyperscale Vs Strong-scale HPC

Those are both marketing words. Either ask NVIDIA what they mean by them, or ignore them. NVIDIA appears to be using them to classify different product lines, at either different cost levels or ...
Bill Barth's user avatar
  • 10.9k
4 votes

Can I convert CUDA core to CPU core and use it as cpu core while running any program?

I assume, that you have a code that works on a standard CPU. I am not particularly familiar with MQL and Metatrader, but I don't think the answer will be different. For compilable languages, the ...
Anton Menshov's user avatar
  • 8,672
4 votes

Implementation of sparse matrix SVD for GPU

I'm no expert on software and certainly not on GPU software, but I can hopefully give some advice of a mathematical nature that might be helpful to you. Given a matrix $W$, one can embed $W$ in the ...
eepperly16's user avatar
3 votes

GPGPU computing, software selection

OpenCL is runnable on multicore cpu, intel hd graphics and even DSP cards. It was pretty much the standard for cross platform gpu computing until compute shaders were introduced. There are various ...
iliar's user avatar
  • 253
3 votes
Accepted

Why is FLOP(Floating Point Operations Per Second) mentioned as a specification on every GPU?

This metric is pretty much as misleading (or useful, depending on your perspective) for GPUs as it is for CPUs. Currently, a lot of applications/algorithm's implementations are limited more by memory ...
Anton Menshov's user avatar
  • 8,672
3 votes

Sparse Matrix Library for GPU

Look at cupy. It supports basically all common sparse formats along with the common operations.
Uwe.Schneider's user avatar
3 votes

Is there a constrained nonlinear optimization library like IPOPT that runs on GPUs?

Generally nonlinear optimization is hard to parallelize because its stepping algorithm is very sequential. GPUs are slower than CPUs so you only get a speedup if you have your problem as something ...
Chris Rackauckas's user avatar
3 votes

GPU vs CPU FLOP counts

Dollars per FLOP isn’t really the question, it’s dollars per solution. Estimate or measure both if you can, and buy the best you can afford that gets you the best solutions in good time. Either use ...
Bill Barth's user avatar
  • 10.9k
3 votes
Accepted

GPU vs CPU FLOP counts

As others said in the comments, the RTX 3090 was designed for single-precision performance primarily for gaming or other single-precision uses. In which case, its TFLOP count is around 35 TFLOPS. ...
Slenderman's user avatar
3 votes

How to leverage the GPU for parallel 3-body problem computations

CUDA has an example repository with codes for different use cases. There are samples for dealing with matrix operations like the approach you took above. One of the advanced use cases listed is the ...
MPIchael's user avatar
  • 2,935
2 votes

GPU-accelerated libraries for solving sparse linear systems

Linear system solvers are generally limited by memory access, so in parallel systems communication becomes the bottleneck. Good scaling can be usually achieved only for very large systems (millions of ...
Jakub Klinkovský's user avatar
2 votes

HPC reading material

The Intel OpenMP Run Time library has been open sourced, so you might have look at a few functions in there. There are many open source MPI libraries (MPICH2, OpenMPI, MVAPICH2, etc.), and the MPI ...
Bill Barth's user avatar
  • 10.9k
2 votes

GPGPU language for AMD?

I can't speak to how you would best work with an AMD GPU. However, of the languages you list CUDA could be considered the lowest level, highest-performance language for general applications. This is ...
Richard's user avatar
  • 3,961
2 votes

Performance of kd-tree vs brute-force nearest neighbor search on GPU?

Additionally to what @richard-zang said, instead of a "naive brute-force" search, you can often use some refinement, e.g. a locality-based hashing or if you have fixed neighbor distance radius, a ...
pszilard's user avatar
  • 121
2 votes
Accepted

Use of GPU with respect to CPU

Unfortunately, GPUs will be of no help to you in this particular situation. Your problem is in the memory limitation; thus, you just do not have enough RAM resources to allocate/factorize/solve the ...
Anton Menshov's user avatar
  • 8,672

Only top scored, non community-wiki answers of a minimum length are eligible