32
$\begingroup$

Background: I think I might want to port some code that calculates matrix exponential-vector products using a Krylov subspace method from MATLAB to Python. (Specifically, Jitse Niesen's expmvp function, which uses an algorithm described in this paper.) However, I know that unless I make heavy use of functions from modules derived from compiled libraries (i.e., I only use raw Python, and not many built-in functions), then it could be quite slow.

Question: What tools or approaches are available to help me speed up code I write in Python for performance? In particular, I'm interested in tools that automate the process as much as possible, though general approaches are also welcome.

Note: I have an older version of Jitse's algorithm, and haven't used it in a while. It could be very easy to make this code fast, but I felt like it would make a good concrete example, and it is related to my own research. Debating my approach for implementing this particular algorithm in Python is another question entirely.

Improve this question
$\endgroup$
3

4 Answers 4

Reset to default
41
$\begingroup$

I'm going to break up my answer into three parts. Profiling, speeding up the python code via c, and speeding up python via python. It is my view that Python has some of the best tools for looking at what your code's performance is then drilling down to the actual bottle necks. Speeding up code without profiling is about like trying to kill a deer with an uzi.

If you are really only interested in mat-vec products, I would recommend scipy.sparse.

Python tools for profiling

profile and cProfile modules: These modules will give you your standard run time analysis and function call stack. It is pretty nice to save their statistics and using the pstats module you can look at the data in a number of ways.

kernprof: this tool puts together many routines for doing things like line by line code timing

memory_profiler: this tool produces line by line memory foot print of your code.

IPython timers: The timeit function is quite nice for seeing the differences in functions in a quick interactive way.

Speeding up Python

Cython: cython is the quickest way to take a few functions in python and get faster code. You can decorate the function with the cython variant of python and it generates c code. This is very maintable and can also link to other hand written code in c/c++/fortran quite easily. It is by far the preferred tool today.

ctypes: ctypes will allow you to write your functions in c and then wrap them quickly with its simple decoration of the code. It handles all the pain of casting from PyObjects and managing the gil to call the c function.

Other approaches exist for writing your code in C but they are all somewhat more for taking a C/C++ library and wrapping it in Python.

Python-only approaches

If you want to stay inside Python mostly, my advice is to figure out what data you are using and picking correct data types for implementing your algorithms. It has been my experience that you will usually get much farther by optimizing your data structures then any low level c hack. For example:

numpy: a contingous array very fast for strided operations of arrays

numexpr: a numpy array expression optimizer. It allows for multithreading numpy array expressions and also gets rid of the numerous temporaries numpy makes because of restrictions of the Python interpreter.

blist: a b-tree implementation of a list, very fast for inserting, indexing, and moving the internal nodes of a list

pandas: data frames (or tables) very fast analytics on the arrays.

pytables: fast structured hierarchical tables (like hdf5), especially good for out of core calculations and queries to large data.

Improve this answer
$\endgroup$
10
  • 3
    $\begingroup$ You can use ctypes to call Fortran routines too. $\endgroup$
    – Matthew Emmett
    Jun 12, 2012 at 13:48
  • $\begingroup$ Yup, sagemath.org/doc/numerical_sage/ctypes.html $\endgroup$
    – aterrel
    Jun 12, 2012 at 13:51
  • $\begingroup$ Talking about wrapping code, what about f2py? $\endgroup$
    – astrojuanlu
    Jun 12, 2012 at 14:38
  • $\begingroup$ f2py is a great tool and used by many in the community. fwrap is a more recent replacement as f2py shows its age but its not really complete. $\endgroup$
    – aterrel
    Jun 12, 2012 at 18:27
  • $\begingroup$ Thanks! These are the types of resources I was looking for. I was only aware of some of them, and only in passing (or from looking at them on the Internet). Aron keeps mentioning numexpr. How does that work? Would that apply? $\endgroup$
    – Geoff Oxberry
    Jun 12, 2012 at 23:43
7
$\begingroup$

First of all, if there is a C or Fortran implementation available (MATLAB MEX function?), why don't you write a Python wrapper?

If you want your own implementation not only a wrapper, I would strongly suggest to use the numpy module for linear algebra stuff. Make sure it is linked to an optimized blas (like ATLAS, GOTOblas, uBLAS, Intel MKL, ...). And use Cython or weave. Read this Performance Python article for a good introduction and benchmark. The different implementations in this article are available for download here courtesy of Travis Oliphant (Numpy-guru).

Good luck.

Improve this answer
$\endgroup$
6
  • $\begingroup$ That Performance Python article seems a bit dated, it doesn't mention some of the newer tools available like numexpr. $\endgroup$
    – Aron Ahmadia
    Jun 12, 2012 at 7:28
  • $\begingroup$ I indeed overlooked numexpr. It would be nice to run the same laplace benchmark with numexpr... $\endgroup$
    – GertVdE
    Jun 12, 2012 at 8:46
  • $\begingroup$ Is scipy.weave still used and developed? It seems that the Performance Python article shows that it could be fast to use and gives a pretty good improvement on speed but I have rarely seen it mentioned outside of that article. $\endgroup$
    – Ken
    Jun 12, 2012 at 16:05
  • $\begingroup$ @Ken: scipy.weave is, as far as I know, no longer under active development. It is kept for backward compatibility but new projects are encouraged to use Cython. $\endgroup$
    – GertVdE
    Jun 12, 2012 at 18:13
  • $\begingroup$ For GotoBLAS and NumPy/SciPy, see der-schnorz.de/2012/06/optimized-linear-algebra-and-numpyscipy $\endgroup$
    – AlexE
    Jun 13, 2012 at 12:19
4
$\begingroup$

Basically I agree with the other answers. The best options for speedy numerical python-code are

  • Use specialized libraries like numpy
  • wrap your existing code so that your python-program can call it directly

But if you want to program the whole algorithm from scratch (I quote: "I only use raw Python") then you might want to consider http://pypy.org/ a JIT (Just In Time) implementation of python. I haven't been able to use it for my project (because that relies on numpyand the pypyguys are corrently working on supporting that) but the benchmarks are quite impressive (http://speed.pypy.org/)

Improve this answer
$\endgroup$
2
$\begingroup$

Some of the links above are outdated, hence look here:

http://wiki.scipy.org/PerformanceTips

http://wiki.scipy.org/PerformancePython

Some ideas:

Numpy, Numba, Cython, Numexpr, Theano, Tensorflow, f2py, CPython C API, pypy, cffi, Pythran, Nuitka, swig, boost.python

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.