I'm not sure if I did something wrong or if I just didn't understand the concept of an optimized BLAS.
I'm a FEM engineer trying to optimize my setup on a small cluster computer (six nodes). I'm building everything on open source tools. My aim is to use a robust setup which allows me to manage large mechanical simulations. After finally compiling my programs using MPI I'm reading the using an optimized BLAS will help me further pushing down calculation time.
I installed OpenBlas and built Lapack 3.4.2. The results of the lapack benchmarks were surprising. They are fastest with the system (unoptimzed) BLAS and the run time increases with increasing number of threads specified (export OPENBLAS_NUM_THREADS=i
). The same is true for scalapack. Furthermore, i>16
doesn't seem to utilize more than 16
threads, although I have 32
available on this machine. That's ok, because I want some to run a couple of MPI processes (4?) on one node anyways, but I find it suspicious.
I'm a beginner and don't know where to start digging from here. Can someone give me a hint? I have a pastebin of the i=32
log here.
OpenBLAS build complete.
OS ... Linux
Architecture ... x86_64
BINARY ... 64bit
C compiler ... GCC (command line : gcc)
Fortran compiler ... GFORTRAN (command line : gfortran)
Library Name ... libopenblas_sandybridgep-r0.2.5.a (Multi threaded; Max num-threads is 32)
Lapack 3.4.2 make.inc (changes from default)
BLASLIB = /path/to/libopenblas.a -lpthread
-- EDIT: (more information because of @Aron Ahmadia's comments)
- system BLAS log
i=1
logi=32
log- CentOS release 6.3 (Final)
blas-devel-3.2.1-4.el6.x86_64
gcc
version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) (same forgfortran
).MemTotal: 198202172 kB
$ grep "Xeon(R) CPU E5-2690 0 @ 2.90GHz" /proc/cpuinfo | wc -l 32.
time
or/usr/bin/time
? machines are idle. This is the log for the system BLAS. $\endgroup$ – Sebastian Jan 29 '13 at 23:31