59
votes
How mature is the "Julia" scientific computing language project?
Julia, at this point (May 2019, Julia v1.1 with v1.2 about to come out) is quite mature for scientific computing. The v1.0 release signified an end to yearly code breakage. With that, a lot of ...
- 12.2k
32
votes
Accepted
Why would you need frameworks like MPI when you can multi-task using threads?
There is one real and one practical reason.
First, MPI was developed at a time when machines had exactly one processor core and when we wanted to couple different machines. It is today used on ...
- 53.7k
24
votes
19
votes
Are there any embarrassingly parallel tasks that require a CPU rather than GPU?
The simple example from electromagnetics (EM) would be performing a parallel frequency sweep for a frequency-domain simulation, say, full-wave extraction of network parameters (S, Y, Z, etc) for a ...
- 8,592
16
votes
Parallelizing a for-loop in Python
What you're looking for is Numba, which can auto parallelize a for loop.
From their documentation
...
- 2,493
13
votes
How do I reliably generate random numbers in Python distributed across multiple nodes?
To the best of my knowledge, Numpy does not support independent streams. Indeed, getting independent streams from the Mersenne Twister (Pythons RNG) is notoriously difficult although it can be done.
...
- 2,493
13
votes
Accepted
Under what circumstances is parallel scaling of the finite element method not "solved"?
There are multiple questions in the post, so let me address these separately:
Scaling: Every parallel program is composed of sequential and parallel tasks, and Amdahl's law then guarantees that there ...
- 53.7k
12
votes
Accepted
What is "good" parallel scaling?
Good is a relative term, and it will depend on the nature of the problem, the nature of the algorithm, and properties of the hardware involved. The only absolute reference point is ideal scaling (100%...
- 16.4k
12
votes
Accepted
Solving linear system of the form $ABx=b$
Defining the auxiliary variable $y=Bx$ yields the following algebraically equivalent expanded system,
$$\underbrace{\begin{bmatrix}
0 & A \\ B & -I
\end{bmatrix}}_{K}
\underbrace{\begin{...
- 3,063
12
votes
Iteration counts of AMG solver changes in parallel
This is something that can happen in almost any numerical algorithm running in parallel.
It's important to know that floating-point addition is not associative due to round-off errors. Thus you can't ...
- 18.5k
11
votes
Parallelizing a for-loop in Python
Without assuming something special on my_function choosing multiprocessing.Pool().map() is a good guess for parallelizing such ...
- 211
11
votes
Are there any embarrassingly parallel tasks that require a CPU rather than GPU?
High-quality video encoding is something like this.
The search space is so huge that it requires branching to prune it rapidly, but GPUs are terrible at that. Modern CPU short-vector SIMD works well ...
- 310
10
votes
Accepted
Parallel Mersenne Twister for Monte Carlo
Like you say, using the Mersenne Twister for parallel computations is almost always done incorrectly, as the correct method is tricky to implement.
By far the easiest and best answer would be to move ...
- 2,493
9
votes
Accepted
C standard for computational science
In theory, as the original authors, you're free to pick and name a standard, then expect others to follow it. In practise, if you're supporting an HPC system, then your choice is likely to be ...
- 2,229
9
votes
Accepted
How to choose a python parallelization library?
Dask schedules tasks across processes and across nodes, so it is appropriate for use on a single computer, supercomputer, or cloud. Dask also provides specialized data structures to aid in this.
...
- 3,891
9
votes
Why would you need frameworks like MPI when you can multi-task using threads?
Wolfgang Bangerth's answer is totally correct, and I only want to add one practical aspect.
Portability across hardware
Let's say you write a research code from scratch. You have a powerful multi-...
- 2,621
9
votes
Are there any embarrassingly parallel tasks that require a CPU rather than GPU?
GPUs work with the model SIMD (single instruction multiple data) i.e. they execute an instruction over multiple data. To give an idea: under CUDA technology when you have got an if-then-else ...
- 1,320
9
votes
Accepted
Searching for recent code source for "Parallel scientific computing in C++ and MPI "
First of all, the book you mention is very old. In fact, it misses the last two MPI standards (3 and 4), and every C++ standard from C++11 on. Secondly, know that MPI has officially no C++ bindings, ...
- 1,305
8
votes
How mature is the "Julia" scientific computing language project?
From my experience Julia isn't ready for (scientific) everyday use yet (I'm talking about the stabilized version 0.4 of march 2016). The language itself is nice, feature rich and consistent; a ...
- 197
8
votes
Accepted
GPU vs CPU calculation
This may have gone unnoticed in the comments under the original question, but computing $10^9!$ yields a number with 8.5 billion digits, that is it is on the order of $10^{9\cdot 10^9}$. Given that $...
- 53.7k
8
votes
Accepted
Intel Knights Landing work loads vs NVIDIA GeForce
Like Brian said, the Xeon Phi cores are not at all comparable to the CUDA ones. The problem with the Phi is that it's somewhere between two horses.
If you are doing highly parallel floating point ...
- 2,493
7
votes
Intel Knights Landing work loads vs NVIDIA GeForce
CUDA cores aren't at all comparable to the separate processor cores in the Xeon Phi coprocessors. The Phi coprocessor cores are full fledged processors that can have their own loops, branching, etc. ...
- 18.5k
7
votes
Does OOP work easily and efficiently in parallel computations?
Empirically, your comment that "most numerical libraries 'do not like' OOP, that is why many software are written in C or Fortran" is not correct. Instead, I would say that almost all software ...
- 53.7k
7
votes
Linear Systems with Multiple Right Hand sides
It's possible that your performance is limited by the memory bandwidth of your system. It's not at all uncommon to have a situation where two or three of your cores can use all of the available ...
- 18.5k
7
votes
Speeding up a linear transform using Python
Assuming that your kernel is somewhat smooth, use low-rank approximation.
Here's a naive example:
...
- 586
7
votes
Accepted
A good, simple book/resource on Parallel Programming in C++ for scientific computing
One of the first things that you need to understand about parallel programming is the difference between shared memory multiprocessor computer systems and distributed memory clusters.
A shared ...
- 18.5k
6
votes
Accepted
What is the state of the art algorithm for diagonalizing real symmetric matrices?
The QR/Francis algorithm is the go-to choice for dense eigenproblems, but there are a few competitors around:
The Jacobi algorithm (like QR, another algorithm with an unfortunate name, which can be ...
- 10.6k
6
votes
MPI+OpenMP Scalability
No. You need to test it to that scale, especially if you have MPI calls within OpenMP regions.
- 10.9k
6
votes
Accepted
Efficiency of parallel direct linear solver
Your problem is too small. You have to consider that to get good efficiency, each processor has to have enough work to offset the cost of communication. In other words, there is a threshold how many ...
- 53.7k
6
votes
Parallel integration of dynamical systems
First of all, don't even consider "optimizing" before you're using the right integration method. Chucking more computers at the problem may sound like the easiest way to solve it, but in ...
- 12.2k
Only top scored, non community-wiki answers of a minimum length are eligible