I'm currently evaluating how I could implement several algorithms on a parallel computer with a large number of nodes, each also equipped with a powerful GPU. It turns out that for many algorithms, it would be useful to "just" have a datastructure that is distributed over many nodes and on whose elements you can run a function in the style of a for-each kernel.
In the end, it could for example boil-down to a distributed implementation of C++'s STL. It would also mean, that you could just impelement the algorithm in a familiar mean and would (mostly) not need to bother about parallel details. It would also be cool to have Python bindings: Write some high-level python code and the low-level distributed stuff happens magically.
I did some research about existing projects in this direction:
- There's for example STAPL but I could not really find significantly more information, and did not get a response on my email.
- There are linear algebra packages such as Trilinos and PETSc, but they are mostly designed for linear algebra and they are also huge frameworks which need significant effort to be incorporated into own projects.
- GPU support is generally poor (I'm aware that you can not straight forward use a GPU for general purpose data structures.)
Are there any serious efforts on distributed general purpose data structures that could be used as a basis for parallelization of existing code? I would be happy to get some references, papers or search words.