I'm considering learning a new language to use for numerical/simulation modelling projects, as a (partial) replacement for the C++ and Python that I currently use. I came across Julia, which sounds kind of perfect. If it does everything it claims, I could use it to replace both C++ and Python in all my projects, since it can access high-level scientific computing library code (including PyPlot) as well as running for loops at a similar speed to C. I would also benefit from things like proper coroutines that don't exist in either of the other languages.

However, it's a relatively new project, currently at version 0.x, and I found various warnings (posted at various dates in the past) that it's not quite ready for day to day use. Consequently I would like some information about the status of the project right now (February 2014, or whenever an answer is posted), in order to help me assess whether I personally should consider investing the time to learn this language at this stage.

I would appreciate answers that focus on specific relevant facts about the Julia project; I'm less interested in opinions based on experience with other projects.

In particular, a comment by Geoff Oxberry suggests that the Julia API is still in a state of flux, requiring code to be updated when it changes. I would like to get an idea of the extent to which this is the case: which areas of the API are stable, and which are likely to change?

I guess typically I would mostly be doing linear algebra (e.g. soliving eigenproblems), numerical integration of ODEs with many variables, and plotting using PyPlot and/or OpenGL, as well as low level C-style number crunching (e.g. for Monte Carlo simulations). Is Julia's library system fully developed in these areas? In particular, is the API more or less stable for those types of activities, or would I find that my old code would tend to break after upgrading to a new version of Julia?

Finally, are there any other issues that would be worth considering in deciding whether to use Julia for serious work at the present time?

  • 2
    $\begingroup$ This question looks like it might not be a good fit for the Stack Exchange format because it is subjective. I know some people who develop actively in Julia, love it, and think that it's totally ready for prime-time (as long as you're willing to update your codebase in response to changes in the Julia API). There are other people who don't feel the need to use Julia right now (like me). $\endgroup$ – Geoff Oxberry Feb 27 '14 at 5:05
  • $\begingroup$ Being subjective does not in itself make a question unsuited for the Stack Exchange format - see blog.stackoverflow.com/2010/09/good-subjective-bad-subjective . If you have a site policy against subjective questions then I apologise. However, I think it's only a bit subjective: already your comment gives me a better idea of the situation than I had prior to asking the question. It can be very hard for an outsider to get even a rough idea of how mature a project is. $\endgroup$ – Nathaniel Feb 27 '14 at 7:31
  • $\begingroup$ Your point about API changes seems quite an important one for me, so I've added a paragraph to the question asking specifically for details about it. If updating Julia will tend to break my old code, that might be a bit of a deal-breaker for me at this stage. $\endgroup$ – Nathaniel Feb 27 '14 at 7:37
  • $\begingroup$ You're absolutely right about "good subjective versus bad subjective" (one of my favorite Stack Exchange blog posts); I posted the comment because I'm waiting to see the response. With these "what do you think about _____?" questions, answers can be limited to a couple very well-thought-out posts, or they can be sprawling, all-over-the-place, repetitive "me too!" posts. The former is great; the latter is not, so I'm giving you a courtesy heads up to say: let's see how this one plays out. $\endgroup$ – Geoff Oxberry Feb 27 '14 at 7:52
  • $\begingroup$ I've made a few changes to the post to try and make it more "good subjective". $\endgroup$ – Nathaniel Feb 27 '14 at 7:54

If not, is it possible to give a rough order-of-magnitude estimate for how long I should wait before considering it again?

My rough, order-of-magnitude estimate of how long it takes computational science languages to mature is around a decade.

Example 1: SciPy started in 2001 or so. In 2009, Scipy 0.7.0 was released, and the ODE integrator had an interface to VODE (which is the equivalent of ode15s, roughly; ode15s is NDF-based, VODE is BDF/Adams-Bashforth, depending). With SciPy 0.10.0, an interface to dopri5, which is roughly the equivalent of MATLAB's ode45, a Runge-Kutta 4th order method typically introduced as the first practical numerical integration method to undergraduates. SciPy 0.10.0 was released in December 2011, and it took about 10 years for them to include a feature of MATLAB introduced to every engineering undergrad I know.

Example 2: Mathworks was founded in 1984. In their first release, they used a LAPACK port to C named JACKPAC (after Jack Little, a MathWorks engineer who wrote it). They did not replace it with LAPACK until 2000.

Julia may take less time, but I'd estimate about 10 years from founding to become mainstream. (It's already been out a year or so; maybe 9-10 years, then?)

Is Julia's library system fully developed in these areas? In particular, is the API more or less stable for those types of activities, or would I find that my old code would tend to break after upgrading to a new version of Julia?

I don't use Julia, so take what I say with a grain of salt, since I've only seen Jeff Bezanson give presentations about Julia. They've bent over backwards to make it easy to link and use libraries from C, Python, and Fortran. If you can't find a Julia library that does what you want, write a Julia shim for a library in a more established language. Consequently, I don't think a lack of libraries will be a concern. I think a concern will be making sure that changes to core language features don't bite you in the ass. If you look at the milestones in the Julia Git repo, you'll see that the "breaking" tag is used quite a bit (12 times in the 0.2 release, 5 times in the 0.3 release). To me, that suggests that the core language is still evolving, which is why I hesitate to use the language right now. Julia looks very cool, so I anticipate that I may use it in the future, maybe a few years from now.

EDIT: Aurelius brings up a good point:

What makes you assume Julia will ever actually become mainstream, and not just die off in obscurity like so many other languages? SciPy/numpy had/has the backing of an ever-growing python community, which Julia doesn't have.

In the original answer, I decided to avoid the question of "Will Julia succeed in becoming mainstream?" as much as possible. Failing is easy; success is difficult. I think the best comparison of Julia is to technical computing languages like MATLAB, R, and Octave. HPC languages (Chapel, Fortress, UPC, etc.) have a narrower audience than technical computing languages, and general purpose languages (C, Python, C++, etc.) have a broader audience than technical computing languages.

Something I think that helps Julia is design for performance without sacrificing expressiveness. Julia is much more competitive with compiled languages like C than MATLAB, R, or even Python. This design goal is also a feature that can draw people from existing languages, like:

  • People who care a lot about performance and come from languages like C and Fortran, but are willing to sacrifice a tiny bit of performance (maybe a factor of 2ish) to go from compiled language to fewer lines of interpreted language (along with a REPL for more rapid development and testing).
  • People who care about high productivity and come from languages like Python, R, and MATLAB, but want more performance. When it comes to execution, pure Python, pure MATLAB, and pure R are slow. Developers in those languages have gone out of their way to wrap libraries in compiled languages, but you can't wrap everything, and at some point, the core language is going to slow you down. Core Julia is faster, which lets you do more science faster.
  • People who care about free software. Julia is interpreted and free (so is Python, Octave, etc.); MATLAB is not.

Julia is also attempting to facilitate parallelism; I don't feel terribly qualified to expand upon that point, and I don't think it's the main draw of the language, but I think it's a selling point that they are working on, and I hope others can shed light on it.

However, even with technical merit on their side, the language creators have to do the legwork to promote the language and evangelize. Jeff Bezanson, Alan Edelman, Stephen Karpinski, and Viral Shah are working very hard to make the language succeed. Alan Edelman has deep ties to the computational science community, and he's worked on language-level projects before (notably, the Star-P extension to MATLAB). Jeff Bezanson has been doing the conference circuit to promote Julia to computational scientists and engineers for a while. At MIT, they've been doing a good job of recruiting students and staff (notably, Steven G. Johnson) to contribute by adding libraries to Julia. They've got an article in Wired, and managed to get a Wikipedia article for themselves, all after only a year. Their Git repo has thousands of stars, hundreds of forks, and hundreds of watches, so by open-source standards, their project has been a success. I think they've done all the right things so far, so it's a matter of sustaining that effort and building community. They could still fail, but getting this far suggests to me that they have a reasonable chance at success.

  • $\begingroup$ How does one look at the milestones in a git repo? $\endgroup$ – Nathaniel Feb 27 '14 at 11:17
  • $\begingroup$ Ah, found it: github.com/JuliaLang/julia/issues/milestones?state=closed $\endgroup$ – Nathaniel Feb 27 '14 at 11:26
  • $\begingroup$ What makes you assume Julia will ever actually become mainstream, and not just die off in obscurity like so many other languages? SciPy/numpy had/has the backing of an ever-growing python community, which Julia doesn't have. Don't get me wrong, I'd love to have a better tool available than C++/Python/Fortran/Matlab (and I know nothing about Julia), but there have been many attempts at new HPC languages that have failed. $\endgroup$ – Aurelius Feb 27 '14 at 18:53
  • 3
    $\begingroup$ Regarding breaking changes, there have actually been very few breaking language changes (i.e. syntax, semantics) since before 0.1, over a year ago – in fact, I can't think of any – the core language is quite stable. Issues tagged as "breaking" are changes to standard library APIs. These are pretty easy to deal with by leaving deprecation methods around that allowing your old code to still work but print a warning. Packages are in much more flux, so I suspect at this point the real pain-point is that upgrading your packages may break your code even if the language itself doesn't break things. $\endgroup$ – StefanKarpinski Feb 28 '14 at 6:14
  • $\begingroup$ Thanks for the edit Geoff, good input. I do hope something better succeeds. It's a little perverse to think that on a weekly basis I'm using Matlab for rapid prototyping of algos, python for automation/scripting, and C++ and/or Fortran for "serious" work, but that's the world we live in. I'm sadly pessimistic though. The 5-10 year trend in HPC is heterogeneous programming and massive parallelism, and that's inherently difficult to craft a simple language for. Their uphill battle is a very steep gradient for a lot of reasons. $\endgroup$ – Aurelius Feb 28 '14 at 6:40

I believe Julia is worth learning. I have used it to produce a few research finite element codes, and produce them very quickly. I have been over all very pleased with my experience.

Julia has enabled a workflow for me that I have found difficult to achieve with other languages. You may use it as a prototyping language like MATLAB, but unlike MATLAB when you have a working code and are going through profiling iterations to speed it up, replacing hotspots with C code is painless. They have made interfacing with C (and Python) a priority in design.

Thus the language has enabled me to very rapidly move from my mathematical formulations to functional code and then from functional code to highly performant code. I think this feature of Julia is undersold, but it adds tremendous value.

In many instances I have obtained C performance (compared to my own C code) in a matter of hours after producing a functional finite element code. So far if I use only Julia features I usually get to within ~3X slower than C. After this I replace hotspots with C functions (Julia comes with a stack sampling profiler to help identify these). Often this requires only replacing the offending hotspot lines of code with a "ccall," with Julia handling any marshalling.

I think the main things Julia is missing right now though that makes me hesitate to consider it for bigger projects is the lack of a fully supported debugger, and poor support for plotting (right now your best bet in plotting is actually just an interface into matplotlib that I've had break more often than not). Immediate feedback on code is really important. I also don't know how to survive without interactive debugging, and in this respect I'm very spoiled by MATLAB and GDB.

  • $\begingroup$ Thanks, this is a very useful answer. I have some follow up questions. Firstly, do you use a released version or keep up with the source? If the latter, do you run into many problems with your code breaking due to updates? Secondly, how does the interface to matplotlib "break" exactly? I was actually quite excited to hear that plotting was through matplotlib, because it can render LaTeX in axis labels (actual LaTeX, using a TeX installation), and for me that's a killer feature. But if the interface is flaky then that's not so great... $\endgroup$ – Nathaniel Feb 28 '14 at 5:03
  • $\begingroup$ I keep up to date with the source because I also am trying to contribute to the project. So far I haven't had any breaks, but I just had a big span of writing and theory and am now coming back to my code and we'll see how that goes. Numpy arrays are 0-indexed and row-major by default. and Julia is 1-indexed and column-major by default, usually this doesn't make problems but unstructured data plotting requires index arrays to be passed around so I've had to do weird things like pass p'-1 to unstructured triangular mesh routines, where p is an index map. $\endgroup$ – Reid.Atcheson Feb 28 '14 at 5:26

From my experience Julia isn't ready for (scientific) everyday use yet (I'm talking about the stabilized version 0.4 of march 2016). The language itself is nice, feature rich and consistent; a refreshing step forward from both matlab and python. There are certainly uses cases where Julia is a good choice even at this early stage. But if you need reliable and mature scientific libraries I cannot recommend making the move now. The main problems I encountered are:

  • The packages outside of Julia's core aren't reliable. They break with updates, change, get replaced, sometimes are incomplete or simply broken.
  • Julia's parallelism features (imo the most promising potential matlab killer features) are are immature. You will encounter segmentation faults, memory leaks, crashes and disappointing performance. Sometimes they don't play well with other parts of julia, for example some of the solvers for linear systems or packages outside of the core. While these features sound promising, they often enough failed for me. Almost never was it enough to simply write @parallel in front of the for loop, where in theory it should.
  • Many little things, small bugs and issues one could live with: not so nice and sometimes erroneous call stack traces, a little buggy interpreter history, slow loading of packages, bad performance of one or another package/function, etc. What annoyed me the most is the PyPlot package, a wrapper for matplotlib, currently there's no alternative to it. It's really great that it's there and it mostly works. But if you need to plot data, be prepared for very slow performance and problems.

This will all get better. I am confident that some day julia will be superior to matlab and python in almost every aspect. Chances are great that innovative packages will be developed for it. It seems like there already is a big community. Even now there is a wealth of packages with use cases ranging from opencl to webservers. Using python or c libraries is very easy, so you probably won't hit a wall in terms of library availability. A great strength of Julia will be the effortlessness with which one can glue together high performance functionality from various sources in a modern and consistent high level language (see answers above). But as a whole I found it to be neither mature nor stable enough to be used productively.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.