Old question, but I think that [this answer from 2014][1] - related to statistical methods, but generalisable for anyone who knows what a loop is - is particularly illustrative and informative.


  [1]: https://scicomp.stackexchange.com/questions/5372/for-which-statistical-methods-are-gpus-faster-than-cpus/12891#12891