# Tag Info

### How mature is the "Julia" scientific computing language project?

Julia, at this point (May 2019, Julia v1.1 with v1.2 about to come out) is quite mature for scientific computing. The v1.0 release signified an end to yearly code breakage. With that, a lot of ...
• 11.4k
Accepted

### Why would you need frameworks like MPI when you can multi-task using threads?

There is one real and one practical reason. First, MPI was developed at a time when machines had exactly one processor core and when we wanted to couple different machines. It is today used on ...
• 50.2k

### How mature is the "Julia" scientific computing language project?

I believe Julia is worth learning. I have used it to produce a few research finite element codes, and produce them very quickly. I have been over all very pleased with my experience. Julia has ...
• 3,203

### Parallelizing a for-loop in Python

Joblib does what you want. The basic usage pattern is: ...

### Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

The simple example from electromagnetics (EM) would be performing a parallel frequency sweep for a frequency-domain simulation, say, full-wave extraction of network parameters (S, Y, Z, etc) for a ...
• 8,287

### What's the state of the art in parallel ODE methods?

Although this post is now two years old, in case someone stumbles across it, let me give a brief update: Martin Gander recently wrote a nice review article, that gives a historical perspective on the ...
• 1,198
Accepted

### Parallel optimization algorithms for a problem with very expensive objective function

As Paul states, without more information, it is hard to give advice without assumptions. With 10-20 variables and expensive function evaluations, the tendency is to recommend derivative-free ...
• 29.8k

### Parallelizing a for-loop in Python

What you're looking for is Numba, which can auto parallelize a for loop. From their documentation ...
• 2,463

### How do I reliably generate random numbers in Python distributed across multiple nodes?

To the best of my knowledge, Numpy does not support independent streams. Indeed, getting independent streams from the Mersenne Twister (Pythons RNG) is notoriously difficult although it can be done. ...
• 2,463
Accepted

### What is "good" parallel scaling?

Good is a relative term, and it will depend on the nature of the problem, the nature of the algorithm, and properties of the hardware involved. The only absolute reference point is ideal scaling (100%...
• 16.2k
Accepted

### Solving linear system of the form $ABx=b$

Defining the auxiliary variable $y=Bx$ yields the following algebraically equivalent expanded system, \underbrace{\begin{bmatrix} 0 & A \\ B & -I \end{bmatrix}}_{K} \underbrace{\begin{...
• 3,003
Accepted

### Removing non-determinism from molecular dynamics code

Both the standard cluster and custom supercomputer (Anton) versions of molecular dynamics at D. E. Shaw Research are both deterministic and parallel invariant. That is, a test run on a single core ...
• 3,879

### Parallel vs Serial Thomas Algorithm

The Thomas algorithm is very efficient because its operation count is very low and because data accesses are very likely to be cache hits once data is initially read from memory. There are two ...

### Parallelizing a for-loop in Python

Without assuming something special on my_function choosing multiprocessing.Pool().map() is a good guess for parallelizing such ...

### Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

High-quality video encoding is something like this. The search space is so huge that it requires branching to prune it rapidly, but GPUs are terrible at that. Modern CPU short-vector SIMD works well ...
• 301
Accepted

### Parallel Mersenne Twister for Monte Carlo

Like you say, using the Mersenne Twister for parallel computations is almost always done incorrectly, as the correct method is tricky to implement. By far the easiest and best answer would be to move ...
• 2,463
Accepted

### Is discontinuous Galerkin really any more parallelizable than continuous Galerkin?

One reason DG methods may receive more attention as a parallel method is that it is readily seen that the method is inherently local to an element. The coupling in DG methods is weak, as it only ...

### Why would you need frameworks like MPI when you can multi-task using threads?

Wolfgang Bangerth's answer is totally correct, and I only want to add one practical aspect. Portability across hardware Let's say you write a research code from scratch. You have a powerful multi-...
• 1,936

### Are there any embarrassingly parallel tasks that require a CPU rather than GPU?

GPUs work with the model SIMD (single instruction multiple data) i.e. they execute an instruction over multiple data. To give an idea: under CUDA technology when you have got an if-then-else ...
• 1,300

### How mature is the "Julia" scientific computing language project?

From my experience Julia isn't ready for (scientific) everyday use yet (I'm talking about the stabilized version 0.4 of march 2016). The language itself is nice, feature rich and consistent; a ...

### Is discontinuous Galerkin really any more parallelizable than continuous Galerkin?

From my many years writing FEM software, I believe that the statement that DG schemes are better suited to parallelization than CG schemes is apocryphal. It is frequently used in introductions of DG ...
• 50.2k
Accepted

### Why is computational cost measured in Floating Pt. Ops. in times of parallel computing?

In fact, the precise total number of operations is very rarely used as a measure of computational cost. Instead, you will most often see the computational order (i.e. $\mathcal{O}(n^3)$). This "big O" ...
• 4,521
Accepted

### C - OpenMP, MPI, Serial Program

I think some of your issues are more important than others and some of your emphasis is misplaced. In pursuing overhead, you are in danger of making your program unmaintainable. It is easier to write ...
• 11.4k

### Comparing various implementations/software packages for large-scale finite element simulations

As one of the library's authors, I would of course love for deal.II to come out on top with this comparison. But I suspect it may not, and the answer lies in a factor you omit from your comparison: ...
• 50.2k
Accepted

### GPU vs CPU calculation

This may have gone unnoticed in the comments under the original question, but computing $10^9!$ yields a number with 8.5 billion digits, that is it is on the order of $10^{9\cdot 10^9}$. Given that \$...
• 50.2k
Accepted

### Intel Knights Landing work loads vs NVIDIA GeForce

Like Brian said, the Xeon Phi cores are not at all comparable to the CUDA ones. The problem with the Phi is that it's somewhere between two horses. If you are doing highly parallel floating point ...
• 2,463
Accepted

### C standard for computational science

In theory, as the original authors, you're free to pick and name a standard, then expect others to follow it. In practise, if you're supporting an HPC system, then your choice is likely to be ...
• 2,189
Accepted

### How to choose a python parallelization library?

Dask schedules tasks across processes and across nodes, so it is appropriate for use on a single computer, supercomputer, or cloud. Dask also provides specialized data structures to aid in this. ...
• 3,091