I'm not sure if I did something wrong or if I just didn't understand the concept of an optimized BLAS.
I'm a FEM engineer trying to optimize my setup on a small cluster computer (six nodes). I'm building everything on open source tools. My aim is to use a robust setup which allows me to manage large mechanical simulations. After finally compiling my programs using MPI I'm reading the using an optimized BLAS will help me further pushing down calculation time.
I installed OpenBlas and built Lapack 3.4.2. The results of the lapack benchmarks were surprising. They are fastest with the system (unoptimzed) BLAS and the run time increases with increasing number of threads specified (
export OPENBLAS_NUM_THREADS=i). The same is true for scalapack. Furthermore,
i>16 doesn't seem to utilize more than
16 threads, although I have
32 available on this machine. That's ok, because I want some to run a couple of MPI processes (4?) on one node anyways, but I find it suspicious.
I'm a beginner and don't know where to start digging from here. Can someone give me a hint? I have a pastebin of the
i=32 log here.
OpenBLAS build complete. OS ... Linux Architecture ... x86_64 BINARY ... 64bit C compiler ... GCC (command line : gcc) Fortran compiler ... GFORTRAN (command line : gfortran) Library Name ... libopenblas_sandybridgep-r0.2.5.a (Multi threaded; Max num-threads is 32)
Lapack 3.4.2 make.inc (changes from default)
BLASLIB = /path/to/libopenblas.a -lpthread
-- EDIT: (more information because of @Aron Ahmadia's comments)