1
$\begingroup$

I'm planning on building a program that will carry out HPC. I'm aware that C/C++ is significantly better than Python in terms of the speed of computation. However, I'm not yet proficient in C/C++, so do you think it'll be wise to build a fully functioning prototype of sorts in Python and then get it rewritten in C/C++ by others/me in the future.

$\endgroup$
  • 2
    $\begingroup$ Short is yes. Something to think about as well is there are a growing number of examples where Python is used extensively in HPC environments. My co-authors and I did something along these lines in arxiv.org/abs/1111.6583 with the caveat that some of the underpinnings were in Fortran. Also check out github.com/mikaem/spectralDNS as another example. $\endgroup$ – Kyle Mandli Jan 18 '16 at 16:16
  • $\begingroup$ What is your definition of HPC? 36-core workstation, Linux cluster, custom supercomputer? $\endgroup$ – Jeff Jan 19 '16 at 6:54
  • $\begingroup$ It really all depends on what your skill level is in each language, what you plan on running it on, and what libraries you end up using. I think I used to align more with the PyClaw authors (I nearly worked with some of them), but the more HPC work I do, the more I tend to just write codes in C or C++. Plus, IIRC, Aron mentioned having to fiddle with loading the Python interpreter at scale for large-scale PyClaw runs on Shaheen. For small scale HPC work, I didn't have any trouble getting mpi4py running. $\endgroup$ – Geoff Oxberry Jan 19 '16 at 12:17
3
$\begingroup$

It is probably a good idea to stick with Python if you have expertise with that. I am assuming that this means you are using numpy/scipy. Depending on what you exactly want to do, well-written numpy/scipy code can be very fast and even comparable to C performance.

If you really need the extra speed of C, you can use Cython so that you keep the majority of the code in Python and write only the bottleneck in C.

$\endgroup$
0
$\begingroup$

Python can be used for HPC with no problem. Most computation intensive work will be done with libraries written in C/C++ such as Numpy/Scipy. You can write your key algorithm in C/C++ also. Even if you want to write algorithms in python, there are plenty of methods to gain comparable performance to C/C++. Cython compiles program written with python-like syntax to C; Numba compiles your key algorithms (a python function for example) to C when needed.

HPC is not only about performance on a single computer, you have the choice to use cluster or GPUs to further enhance the computation performance. Python is good at that also.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.