1
$\begingroup$

I'm planning on building a program that will carry out HPC. I'm aware that C/C++ is significantly better than Python in terms of the speed of computation. However, I'm not yet proficient in C/C++, so do you think it'll be wise to build a fully functioning prototype of sorts in Python and then get it rewritten in C/C++ by others/me in the future.

$\endgroup$

closed as primarily opinion-based by Brian Borchers, horchler, Christian Clason, nicoguaro, James Jan 20 '16 at 3:33

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    $\begingroup$ Short is yes. Something to think about as well is there are a growing number of examples where Python is used extensively in HPC environments. My co-authors and I did something along these lines in arxiv.org/abs/1111.6583 with the caveat that some of the underpinnings were in Fortran. Also check out github.com/mikaem/spectralDNS as another example. $\endgroup$ – Kyle Mandli Jan 18 '16 at 16:16
  • $\begingroup$ What is your definition of HPC? 36-core workstation, Linux cluster, custom supercomputer? $\endgroup$ – Jeff Jan 19 '16 at 6:54
  • $\begingroup$ It really all depends on what your skill level is in each language, what you plan on running it on, and what libraries you end up using. I think I used to align more with the PyClaw authors (I nearly worked with some of them), but the more HPC work I do, the more I tend to just write codes in C or C++. Plus, IIRC, Aron mentioned having to fiddle with loading the Python interpreter at scale for large-scale PyClaw runs on Shaheen. For small scale HPC work, I didn't have any trouble getting mpi4py running. $\endgroup$ – Geoff Oxberry Jan 19 '16 at 12:17
3
$\begingroup$

It is probably a good idea to stick with Python if you have expertise with that. I am assuming that this means you are using numpy/scipy. Depending on what you exactly want to do, well-written numpy/scipy code can be very fast and even comparable to C performance.

If you really need the extra speed of C, you can use Cython so that you keep the majority of the code in Python and write only the bottleneck in C.

$\endgroup$
0
$\begingroup$

Python can be used for HPC with no problem. Most computation intensive work will be done with libraries written in C/C++ such as Numpy/Scipy. You can write your key algorithm in C/C++ also. Even if you want to write algorithms in python, there are plenty of methods to gain comparable performance to C/C++. Cython compiles program written with python-like syntax to C; Numba compiles your key algorithms (a python function for example) to C when needed.

HPC is not only about performance on a single computer, you have the choice to use cluster or GPUs to further enhance the computation performance. Python is good at that also.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.