update: I've refactored the question based on helpful advice in the linked meta.
I'm a heavy user of Python's NumPy and SciPy (and not much else) and for years I could run anything I need on my laptop.
Now I will start to write simulations with 100's to 10,000's of elements that go way beyond what a single CPU can do in a reasonable amount of time.
One potentially economical way to add cores is to use a PC and add a CUDA GPU, and CuPy is one way to move the heavy calculations to it:
CuPy's interface is highly compatible with NumPy and SciPy; in most cases it can be used as a drop-in replacement. All you need to do is just replace numpy and scipy with cupy and cupyx.scipy in your Python code. The Basics of CuPy tutorial is useful to learn first steps with CuPy. CuPy supports various methods, indexing, data types, broadcasting and more. This comparison table shows a list of NumPy / SciPy APIs and their corresponding CuPy implementations.
I will buy a modest CUDA GPU and start experimenting with how to do this.
I'm not a developer, and I see (at least) two potential problems that could cause me to flounder and ultimately fall flat on my face and fail:
- I buy the wrong CUDA GPU and the speed up is minimal or nonexistent.
- I configure my arrays wrong (e.g. fast vs slow axis) or rely heavily on indexed arrays, and so much time is spent moving data between CPU, memory and GPU that the speedup is minimal or nonexistent.
Question: I'm going to try to move some of my scipy/numpy calculation to a new GPU and would like some specific suggestions to help me avoid disappointing results due to the factors above.
While this is similar to a "best practices" question, I don't have any experience yet, and so it's easier for me to ask and support a question about what not to do wrong than one about how to do everything right.
There will be a separate but related question in the near future where I will include a specific example and some alternative implementations in NumPy, and hopefully at that time I will have benefitted from some help here and have a modest GPU up and running.
Background and related posts:
From this answer to CUDA & Python for numerical integration and solving differential equations:
- CuPy; and
From this answer to Writing code on the CPU while developing, running it on the GPU when live - which approach?:
ArrayFire has a C++ API as well as a Python API. You can switch between several backends including CPU, CUDA, and OpenCL. It will also handle memory movement and kernel fusion for you.