I'm using ViennaCL's interface to Eigen as a way to leverage OpenCL. Specifically, I'm using the ::viennacl::linalg::bicgstab_tag with an Eigen sparse matrix. However, the performance isn't what I hoped that it would be.
What tools on Windows 7/Mac OS X/Linux should I use to understand the performance bottlenecks?
General statement: For small system sizes, there is little benefit of using OpenCL at all.
Alright, now for the justification: There is a certain amount of overhead associated with each OpenCL kernel launch. Exact timings depend on the underlying hardware, but as a rule of thumb one can use a pessimistic estimate of 10 microseconds for CPUs and 100 microseconds for GPUs as I've once reported in this thread at the Intel OpenCL forum. This is A LOT considering that modern hardware provides many GFLOPs of processing power. For example, adding two vectors with 100.000 entries each on a GPU with 100 GB/sec memory bandwidth requires 3 * 8 * 100.000 Bytes (approx. 2MB) of data to be transferred, taking 20 us. Thus, even adding up two vectors of size 100.000 can show significant kernel launch overhead. In practice, kernel launch overhead can be reduced if kernels are enqueued while another kernel is still active - this, however, requires again that kernel execution times are sufficiently large.
Since most operations inside Krylov solvers are comparable in complexity to vector additions (this often applies to sparse matrix-vector multiplications as well), benchmarks for system sizes below 10.000 by 10.000 essentially measure OpenCL kernel launch overheads only. I thus recommend to run benchmarks again with your upper limit 50k by 50k. For smaller system sizes, just use BiCGStab with the Eigen matrix directly (this is one of the reasons why ViennaCL offers generic implementations...) or give sparse direct solvers a try.
For dense systems (matrices), the notion of a 'small system' is certainly shifted to smaller values. Still, below system sizes of about 1000x1000 overheads become significant again.
A very simple means is to compute by hand the amount of useful work that are to be done (i.e. either amount of data to be transferred or the amount of useful floating point operations (flops) or both).
Then do some simple timings and average them to get an absolute prediction of performance for a given problem size (time vs. problem size) and from this you can assess overall performance characteristics. This is a simple and useful means for making comparisons with hardware specifications and/or performance of other software.
Furthermore, To do a complete performance breakdown you will need tools such as TotalView for profiling and identifying performance bottlenecks.
