58 votes

How mature is the "Julia" scientific computing language project?

Julia, at this point (May 2019, Julia v1.1 with v1.2 about to come out) is quite mature for scientific computing. The v1.0 release signified an end to yearly code breakage. With that, a lot of ...
user avatar
32 votes
Accepted

Why would you need frameworks like MPI when you can multi-task using threads?

There is one real and one practical reason. First, MPI was developed at a time when machines had exactly one processor core and when we wanted to couple different machines. It is today used on ...
user avatar
23 votes

How mature is the "Julia" scientific computing language project?

I believe Julia is worth learning. I have used it to produce a few research finite element codes, and produce them very quickly. I have been over all very pleased with my experience. Julia has ...
user avatar
23 votes

Parallelizing a for-loop in Python

Joblib does what you want. The basic usage pattern is: ...
user avatar
18 votes

Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

The simple example from electromagnetics (EM) would be performing a parallel frequency sweep for a frequency-domain simulation, say, full-wave extraction of network parameters (S, Y, Z, etc) for a ...
user avatar
  • 8,287
16 votes

What's the state of the art in parallel ODE methods?

Although this post is now two years old, in case someone stumbles across it, let me give a brief update: Martin Gander recently wrote a nice review article, that gives a historical perspective on the ...
user avatar
  • 1,198
16 votes
Accepted

Parallel optimization algorithms for a problem with very expensive objective function

As Paul states, without more information, it is hard to give advice without assumptions. With 10-20 variables and expensive function evaluations, the tendency is to recommend derivative-free ...
user avatar
15 votes

Parallelizing a for-loop in Python

What you're looking for is Numba, which can auto parallelize a for loop. From their documentation ...
user avatar
  • 2,463
13 votes

How do I reliably generate random numbers in Python distributed across multiple nodes?

To the best of my knowledge, Numpy does not support independent streams. Indeed, getting independent streams from the Mersenne Twister (Pythons RNG) is notoriously difficult although it can be done. ...
user avatar
  • 2,463
12 votes
Accepted

What is "good" parallel scaling?

Good is a relative term, and it will depend on the nature of the problem, the nature of the algorithm, and properties of the hardware involved. The only absolute reference point is ideal scaling (100%...
user avatar
12 votes
Accepted

Solving linear system of the form $ABx=b$

Defining the auxiliary variable $y=Bx$ yields the following algebraically equivalent expanded system, $$\underbrace{\begin{bmatrix} 0 & A \\ B & -I \end{bmatrix}}_{K} \underbrace{\begin{...
user avatar
  • 3,003
11 votes
Accepted

Removing non-determinism from molecular dynamics code

Both the standard cluster and custom supercomputer (Anton) versions of molecular dynamics at D. E. Shaw Research are both deterministic and parallel invariant. That is, a test run on a single core ...
user avatar
11 votes

Parallel vs Serial Thomas Algorithm

The Thomas algorithm is very efficient because its operation count is very low and because data accesses are very likely to be cache hits once data is initially read from memory. There are two ...
user avatar
11 votes

Parallelizing a for-loop in Python

Without assuming something special on my_function choosing multiprocessing.Pool().map() is a good guess for parallelizing such ...
user avatar
11 votes

Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

High-quality video encoding is something like this. The search space is so huge that it requires branching to prune it rapidly, but GPUs are terrible at that. Modern CPU short-vector SIMD works well ...
user avatar
10 votes
Accepted

Parallel Mersenne Twister for Monte Carlo

Like you say, using the Mersenne Twister for parallel computations is almost always done incorrectly, as the correct method is tricky to implement. By far the easiest and best answer would be to move ...
user avatar
  • 2,463
9 votes
Accepted

Is discontinuous Galerkin really any more parallelizable than continuous Galerkin?

One reason DG methods may receive more attention as a parallel method is that it is readily seen that the method is inherently local to an element. The coupling in DG methods is weak, as it only ...
user avatar
9 votes

Why would you need frameworks like MPI when you can multi-task using threads?

Wolfgang Bangerth's answer is totally correct, and I only want to add one practical aspect. Portability across hardware Let's say you write a research code from scratch. You have a powerful multi-...
user avatar
  • 1,936
9 votes

Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

GPUs work with the model SIMD (single instruction multiple data) i.e. they execute an instruction over multiple data. To give an idea: under CUDA technology when you have got an if-then-else ...
user avatar
8 votes

How mature is the "Julia" scientific computing language project?

From my experience Julia isn't ready for (scientific) everyday use yet (I'm talking about the stabilized version 0.4 of march 2016). The language itself is nice, feature rich and consistent; a ...
user avatar
8 votes

Is discontinuous Galerkin really any more parallelizable than continuous Galerkin?

From my many years writing FEM software, I believe that the statement that DG schemes are better suited to parallelization than CG schemes is apocryphal. It is frequently used in introductions of DG ...
user avatar
8 votes
Accepted

Why is computational cost measured in Floating Pt. Ops. in times of parallel computing?

In fact, the precise total number of operations is very rarely used as a measure of computational cost. Instead, you will most often see the computational order (i.e. $\mathcal{O}(n^3)$). This "big O" ...
user avatar
8 votes
Accepted

C - OpenMP, MPI, Serial Program

I think some of your issues are more important than others and some of your emphasis is misplaced. In pursuing overhead, you are in danger of making your program unmaintainable. It is easier to write ...
user avatar
  • 11.4k
8 votes

Comparing various implementations/software packages for large-scale finite element simulations

As one of the library's authors, I would of course love for deal.II to come out on top with this comparison. But I suspect it may not, and the answer lies in a factor you omit from your comparison: ...
user avatar
8 votes
Accepted

GPU vs CPU calculation

This may have gone unnoticed in the comments under the original question, but computing $10^9!$ yields a number with 8.5 billion digits, that is it is on the order of $10^{9\cdot 10^9}$. Given that $...
user avatar
8 votes
Accepted

Intel Knights Landing work loads vs NVIDIA GeForce

Like Brian said, the Xeon Phi cores are not at all comparable to the CUDA ones. The problem with the Phi is that it's somewhere between two horses. If you are doing highly parallel floating point ...
user avatar
  • 2,463
8 votes
Accepted

C standard for computational science

In theory, as the original authors, you're free to pick and name a standard, then expect others to follow it. In practise, if you're supporting an HPC system, then your choice is likely to be ...
user avatar
  • 2,189
8 votes
Accepted

How to choose a python parallelization library?

Dask schedules tasks across processes and across nodes, so it is appropriate for use on a single computer, supercomputer, or cloud. Dask also provides specialized data structures to aid in this. ...
user avatar
  • 3,091
7 votes

Does OOP work easily and efficiently in parallel computations?

Empirically, your comment that "most numerical libraries 'do not like' OOP, that is why many software are written in C or Fortran" is not correct. Instead, I would say that almost all software ...
user avatar
7 votes

Intel Knights Landing work loads vs NVIDIA GeForce

CUDA cores aren't at all comparable to the separate processor cores in the Xeon Phi coprocessors. The Phi coprocessor cores are full fledged processors that can have their own loops, branching, etc. ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible