# Questions tagged [benchmarking]

For questions about best/common practices for benchmarking the performance of computational code.

23 questions
Filter by
Sorted by
Tagged with
54 views

### Open-Source Benchmarks that Measure Compression Speed

I'm trying to do an experiment concerning various data-storage formats. I'd like to run an open-source compression benchmark that demonstrates storage format A allows for FASTER compression than ...
• 11
38k views

### How to properly calculate CPU and GPU FLOPS performance?

Problem I'm trying to calculate CPU / GPU FLOPS performance but I'm not sure if I'm doing it correctly. Let's say we have: A Kaby Lake CPU (clock: 2.8 GHz, cores: 4, threads: 8) A Pascal GPU (clock: ...
113 views

### Why do people omit the lowest times when averaging timing results?

Most of the time, when I see someone reporting the average timing of a certain algorithm on a computer in a computational mathematics paper, they do something like this: Run the operation $n$ times (...
• 11.8k
1 vote
54 views

### First-principles benchmark of CFD solver

A decade ago I saw a ~20 coupled state + random number problem that was used as a benchmark for meteorological CFD tools. It had a fractal dimension (of the chaotic attractor) around 2.3, I think. I ...
• 301
557 views

### Should benchmarkings be done at all? What is the point?

I am reading a paper which compares algorithm A versus algorithm B. It shows that algorithm A is faster than algorithm B via benchmarking that shows the CPU time. What is the point of this? Any ...
• 121
1k views

### What are the guidelines for conducting computational experiments?

Physics, biology, chemistry etc. have different sets of rules for making experiments: what events are considered relevant, how to avoid contamination of samples, how to create and fix a reproducing ...
208 views

### How to measure efficiency of the differential equations solver

I want to compare a few solvers of partial differential equations. I need to include the computational time and the solution accuracy (compared to an analytical solution or something similar). What ...
• 475
1 vote
326 views

### Flow past square cylinder benchmark in 2D. Famous papers to compare

I'm looking for a benchmark for the flow past square cylinder problem in 2D to compare my results. I have found some papers, but it seems that there are different results in different papers. I want ...
• 23
246 views

### Benchmark setup for incompressible Rayleigh-Taylor flow

I'm trying to benchmark a new numerical method for the incompressible Navier-Stokes equations. I'd like to use the most widely-accepted test cases possible. In particular, I'm looking for a ...
• 16.6k
1 vote
421 views

### Computational Science Hardware Benchmark Database

I am looking for a website/database of computer hardware (CPUs/GPUs) benchmarked with common scientific tools, e.g. *GEMM (see comment below for more info on this tool). Currently, I am quite ...
• 111
374 views

### How much does choice of OS matter for performance of scientific computing code?

It's common parlance to say that Linux is faster, and for good reasons. But as stated in the title, how much does choice of OS matter for performance of scientific computing code? For something things ...
• 12.4k
4k views

### Integer vs float multiplication performance, modern CPUs

Are there benchmarks for how many multiplications of various integer types compared to floating point types can be achieved per second on modern CPUs? I'm trying to get some hint if it would be ...
• 380
723 views

### Lid-driven Cavity benchmark in 3D. Classical paper to compare

I'm looking for a benchmark for the lid-driven cavity problem in 3D to compare the results of my code. In 2D I used: U. K. N. G. Ghia, K. N. Ghia and C. T. Shin (1982) High-Re solutions for ...
• 515
4k views

### Is R or Matlab currently faster?

The most up-to-date performance benchmarks comparison between R and Matlab that I could find are several years out of date: 1 2 Is anyone aware of a more up-to-date comparison?
293 views

### How does matrix-matrix product scale with multiple CPUs?

These days, one can have 64 cores in a single node. I wonder how well the dense matrix-matrix product (SGEMM and DGEMM) scales ...
• 155
213 views

### What is the correct way of performing numerical experiments on desktops?

Suppose I want to set up an experiment to measure the performance of some numerical code on a desktop machine running Linux/Windows/MacOS. What kind of environment should I arrange in order to get ...
• 1,842
301 views

### Useful well-known numeric performance benchmarks other than Linear Algebra?

I am currently testing some new (embedded) processors for their double-precision numeric performance. The FPU capabilities vary wildly between platforms; For example, the ARM Cortex-A8 has a double ...
• 802
2k views

### Comparison of iteration methods: number of iterations vs. cpu time

I am comparing two iterative methods for inverting random square matrices. Since the matrices are random, every test case takes both different amounts of iterations and different elapsed times. My ...
• 365
480 views

### How do Volkov and Demmel experimentally determine the latencies, line sizes, and page sizes of a GPU?

In "LU, QR and Cholesky Factorizations using Vector Capabilities of GPUs", by Vasily Volkov and James Demmel, there is an interesting way to interpret the latencies, line sizes, and page sizes of a ...
• 193
2k views

### CPU benchmarks for numerical kernels

CPU benchmarks available online mostly focus on desktop apps/games and rarely on serial/parallel numerical kernels, specially sparse ones (e.g., MatMult). Some benchmarks like NAS/SciMark exists but ...
• 1,759
467 views

### Effects of memory speed/architecture on Pardiso scaling

I am using a program that utilizes the PARDISO solver as part of the Intel math kernel library. I am currently in the process of deciding on a new computer to run the simulations on. I have a ...
• 4,597
21k views

### How much better are Fortran compilers really?

This question is an extension of two discussions that came up recently in the replies to "C++ vs Fortran for HPC". And it is a bit more of a challenge than a question... One of the most often-heard ...
• 9,533