Questions tagged [benchmarking]
For questions about best/common practices for benchmarking the performance of computational code.
23 questions
0
votes
1
answer
57
views
Open-Source Benchmarks that Measure Compression Speed
I'm trying to do an experiment concerning various data-storage formats. I'd like to run an open-source compression benchmark that demonstrates storage format A allows for FASTER compression than ...
- 11
14
votes
3
answers
43k
views
How to properly calculate CPU and GPU FLOPS performance?
Problem
I'm trying to calculate CPU / GPU FLOPS performance but I'm not sure if I'm doing it correctly.
Let's say we have:
A Kaby Lake CPU (clock: 2.8 GHz, cores: 4, threads: 8)
A Pascal GPU (clock: ...
- 241
3
votes
1
answer
114
views
Why do people omit the lowest times when averaging timing results?
Most of the time, when I see someone reporting the average timing of a certain algorithm on a computer in a computational mathematics paper, they do something like this:
Run the operation $n$ times (...
- 12.6k
1
vote
0
answers
58
views
First-principles benchmark of CFD solver
A decade ago I saw a ~20 coupled state + random number problem that was used as a benchmark for meteorological CFD tools. It had a fractal dimension (of the chaotic attractor) around 2.3, I think. I ...
- 301
12
votes
4
answers
573
views
Should benchmarkings be done at all? What is the point?
I am reading a paper which compares algorithm A versus algorithm B.
It shows that algorithm A is faster than algorithm B via benchmarking that shows the CPU time.
What is the point of this? Any ...
- 121
13
votes
1
answer
1k
views
What are the guidelines for conducting computational experiments?
Physics, biology, chemistry etc. have different sets of rules for making experiments: what events are considered relevant, how to avoid contamination of samples, how to create and fix a reproducing ...
- 417
3
votes
2
answers
220
views
How to measure efficiency of the differential equations solver
I want to compare a few solvers of partial differential equations. I need to include the computational time and the solution accuracy (compared to an analytical solution or something similar).
What ...
- 475
1
vote
2
answers
337
views
Flow past square cylinder benchmark in 2D. Famous papers to compare
I'm looking for a benchmark for the flow past square cylinder problem in 2D to compare my results. I have found some papers, but it seems that there are different results in different papers. I want ...
- 23
3
votes
2
answers
247
views
Benchmark setup for incompressible Rayleigh-Taylor flow
I'm trying to benchmark a new numerical method for the incompressible Navier-Stokes equations. I'd like to use the most widely-accepted test cases possible. In particular, I'm looking for a ...
- 16.9k
1
vote
2
answers
424
views
Computational Science Hardware Benchmark Database
I am looking for a website/database of computer hardware (CPUs/GPUs) benchmarked with common scientific tools, e.g. *GEMM (see comment below for more info on this tool).
Currently, I am quite ...
- 111
8
votes
2
answers
386
views
How much does choice of OS matter for performance of scientific computing code?
It's common parlance to say that Linux is faster, and for good reasons. But as stated in the title, how much does choice of OS matter for performance of scientific computing code? For something things ...
- 12.5k
7
votes
1
answer
4k
views
Integer vs float multiplication performance, modern CPUs
Are there benchmarks for how many multiplications of various integer types compared to floating point types can be achieved per second on modern CPUs?
I'm trying to get some hint if it would be ...
- 380
4
votes
2
answers
735
views
Lid-driven Cavity benchmark in 3D. Classical paper to compare
I'm looking for a benchmark for the lid-driven cavity problem in 3D to compare the results of my code.
In 2D I used:
U. K. N. G. Ghia, K. N. Ghia and C. T. Shin (1982)
High-Re solutions for ...
- 527
4
votes
1
answer
4k
views
Is R or Matlab currently faster?
The most up-to-date performance benchmarks comparison between R and Matlab that I could find are several years out of date: 1 2
Is anyone aware of a more up-to-date comparison?
3
votes
1
answer
301
views
How does matrix-matrix product scale with multiple CPUs?
These days, one can have 64 cores in a single node. I wonder how well the dense matrix-matrix product (SGEMM and DGEMM) scales ...
- 155
5
votes
1
answer
217
views
What is the correct way of performing numerical experiments on desktops?
Suppose I want to set up an experiment to measure the performance of some numerical code on a desktop machine running Linux/Windows/MacOS. What kind of environment should I arrange in order to get ...
- 1,842
4
votes
1
answer
304
views
Useful well-known numeric performance benchmarks other than Linear Algebra?
I am currently testing some new (embedded) processors for their double-precision numeric performance.
The FPU capabilities vary wildly between platforms; For example, the ARM Cortex-A8 has a double ...
- 802
14
votes
3
answers
2k
views
Comparison of iteration methods: number of iterations vs. cpu time
I am comparing two iterative methods for inverting random square matrices. Since the matrices are random, every test case takes both different amounts of iterations and different elapsed times. My ...
- 365
9
votes
2
answers
494
views
How do Volkov and Demmel experimentally determine the latencies, line sizes, and page sizes of a GPU?
In "LU, QR and Cholesky Factorizations using Vector Capabilities of GPUs", by Vasily Volkov and James Demmel, there is an interesting way to interpret the latencies, line sizes, and page sizes of a ...
- 193
5
votes
3
answers
2k
views
CPU benchmarks for numerical kernels
CPU benchmarks available online mostly focus on desktop apps/games and rarely on serial/parallel numerical kernels, specially sparse ones (e.g., MatMult). Some benchmarks like NAS/SciMark exists but ...
- 1,759
5
votes
2
answers
476
views
Effects of memory speed/architecture on Pardiso scaling
I am using a program that utilizes the PARDISO solver as part of the Intel math kernel library. I am currently in the process of deciding on a new computer to run the simulations on. I have a ...
- 4,597
78
votes
5
answers
21k
views
How much better are Fortran compilers really?
This question is an extension of two discussions that came up recently in the replies to "C++ vs Fortran for HPC". And it is a bit more of a challenge than a question...
One of the most often-heard ...
- 9,563
6
votes
1
answer
679
views
Benchmarking performance in CFD: how to compare machines and codes?
Informally in our lab, we have developed 2 metrics to compare CFD solvers over
the range of machines we have access to. One is called COMP, which stands
for COde Machine Performance. This single ...
- 1,308