Software is a fundamental part of computational science, and is increasingly recognized as an essential part of the scientific record. Given the value of using existing and well-tested code, it seems worthwhile to communicate the existence of useful codes as widely as possible and credit their creators. In an academic setting, this means publishing some papers that are primarily focused on software.

Where can one publish scholarly works whose primary focus is computational software? To be completely clear, I'm referring to works that may not include any new mathematics, algorithms, etc. -- they are really focused on software.

I would also be interested in hearing from those who have submitted such papers to these journals, what the experience was like and which venues they recommend.


Summary of answers given:

  • 1
    Since it's going to come up eventually for our community, and you are a mod, you have a very low accept rate. Also, this seems to be a question where there is no possible singularly right answer - perhaps its CW fodder? – Fomite Jan 13 '12 at 7:42

11 Answers 11

Advances in Engineering Software is a good one and the applications vary pretty widely.

The Journal of Statistical Software is another one that focuses on software, but is application-specific. A lot of R in this one.

I should add that I've never published in either of these, so I cannot speak to that experience, but I do follow them and find the articles to be good quality.

There are some other application-specific journals to list: such as Journal of Computational Physics or Computer Physics Communications, that accept articles both about algorithms as well as the software used to implement them. If you're in the chemistry field, Journal of Chemical Theory and Computation might be another journal to consider. All of these do allow packages to be published—I've seen codes I've used discussed in them. Computers and Chemical Engineering does allow software implementation papers, but they need to do something original—it can't be an "incremental advance" paper.

  • According to the aims and scope of Computers and Chemical Engineering, "Reports of software implementation must feature novel uses of state-of-the-art computing technologies." It's not exactly clear to me from that phrase alone how much of an advance would make something novel, but there are scores of examples in the journal itself. – Geoff Oxberry Jan 12 '12 at 20:09
  • Can you point me to a JCP paper whose primary emphasis is software? – David Ketcheson Jan 12 '12 at 20:52
  • Anderson et al. is the paper that basically launched HOOMD. I guess one can say that they're focusing more on "implementation" than code, but at a certain point, it's hard to treat the two as completely distinct. – aeismail Jan 13 '12 at 7:39
  • Hmmmm, yes, I wasn't thinking of performance-oriented papers when I wrote the question, but it is a grey area. – David Ketcheson Jan 13 '12 at 19:18

I've published a couple of software papers. One was in BMC Source Code in Biology and Medicine - The Multiscale Systems Immunology project: software for cell-based immunological simulation and PLoS ONE - SNPpy - Database Management for SNP Data from Genome Wide Association Studies.

These are two very different projects. The former is a simulation agent-based system written to model immunology systems, so belongs within computational biology. The latter is a database-based system to manage SNP data in a GWAS context, so belongs within bioinformatics.

As far as the publishing experience went - the wait time was low compared to mathematical papers, which was good. The reviewers did not give me a hard time, which was also good. On the cons side, publishing a software paper is insanely time consuming. You have a paper plus a codebase to deal with, not to mention user documentation.

One thing I did find disturbing was that I saw little evidence the reviewers had actually made an effort to run the software and zero evidence they had understood the code design or even looked at the code. Both the journals above had two reviewer reports each. In the former (BMC) case, one reviewer indicated he had tried to install/run the software and failed (an issue we addressed), but of the four reviews, that was the only one that specifically mentioned anything about the actual software. Most of the comments were more general, about scientific issues and addressing points I had made in the paper. Overall, I got the feelings the reviewers had mostly just read the paper. In an ideal world, a reviewer would have some interest in the software design and code details as well, including issues like performance, dependencies, portability, extensibility, and testing. The reviewers for SNPpy did ask about portability and extensibility. as I had claimed in the paper that the software was both portable and extensible, but I don't think they actually looked at the code. In fairness, getting an idea of what a non-trivial codebase is about is hard work, and perhaps more than can be reasonably expected of an unpaid reviewer.

I used LaTeX for both these papers. In the PLoS case, this involved quite a lot of jumping through hoops to make the manuscript look the way they wanted, since they don't actually use LaTeX and were running some conversion program. PLoS made a mess of converting my figures, which were done using TikZ, and were very nice, but you wouldn't know it looking at their version. It's also worth noting that PLoS doesn't provide manuscript proofs.

The Journal of Computational Chemistry (Special Articles Section) and International Journal of Quantum Chemistry report on software and algorithms.

  • I don't see anything like that in the aims & scope of the Journal of Computational Chemistry. Can you point to examples of software papers in the journal? – David Ketcheson Jan 13 '12 at 7:10
  • They have a "software news and updates" section specifically for software. See the author guidelines (onlinelibrary.wiley.com/journal/10.1002/(ISSN)1096-987X/…) under "special article types". – khinsen Jan 13 '12 at 16:01
  • Right, thanks for pointing it out! – David Ketcheson Jan 13 '12 at 19:14
  • JCompChem is a place to report updates to big software packages but is far from the best venue for algorithmic work. IJQC has dropped in standing quite a bit over the last 20 years. It's a third or fourth tier journal nowadays. Very little software or algorithm work is reported there and what is tends to be relatively uninteresting. – Jeff Apr 13 '13 at 20:27

Two that I haven't seen on this list yet, which may be of interest to someone besides me at some point, is the American Journal of Epidemiology and Epidemiology.

If you've written clever software to accomplish something Epidemiologists care about, they may very well publish it. I've seen recent at least short articles on SAS Macros to accomplish somewhat more sophisticated statistical analysis, a program that performs join-point regression, and one that does some field-specific things with directed acyclic graphs.

Turns out one of the mentioned publications was in Epidemiology: http://journals.lww.com/epidem/Fulltext/2011/09000/DAGitty__A_Graphical_Tool_for_Analyzing_Causal.22.aspx

Neither journal I suspect out and out says they're looking for articles on software - what with being field-specific journals in a discipline that isn't software, but analysis including software isn't uncommon, and the link is an example of an article focusing entirely on a software product.

AJE:

Papers dealing with the practical application of epidemiologic or statistical methods (such as the implementation of statistical techniques or the evaluation of interview strategies) are published in a section entitled Practice of Epidemiology.

Epidemiology: The article above was published as a letter. They also accept software reviews, if you've got a friendly colleague or someone in the field who might be interested.

I know for a fact they encourage electronic supplements with code, and I've had something at least reviewed there with a link to a Github repository.

  • Can you provide links to specific examples? I don't see anything about software in the journal description. – David Ketcheson Jan 13 '12 at 7:12
  • @DavidKetcheson As requested – Fomite Jan 13 '12 at 7:39

Geoscientific Model Development is an open access journal for this purpose. Like other journals of the European Geophysical Union, it has an open review process.

Journal of Machine Learning Research (JMLR) has a 'Machine Learning Open Source Software' track. It is fairly selective (which I think is a good thing). The journal is entirely free (for authors and readers) and ran by volunteers.

The mathematical Optimization Society recently launched the Journal

Mathematical Programming Computation

From their "Aims and Scope":

Mathematical Programming Computation (MPC) publishes original research articles covering computational issues in mathematical optimization. Articles report on innovative software, comparative tests, modeling environments, libraries of data, and/or applications. A main feature of the journal is the inclusion of accompanying software and data with submitted manuscripts.

Concerning the topics, they write:

Topics covered in MPC include linear programming, convex optimization, nonlinear optimization, stochastic optimization, robust optimization, integer programming, combinatorial optimization, global optimization, network algorithms, and modeling languages.

Computing in Science and Engineering is another option. It's a hybrid of a scientific journal (with refereed feature articles) and a magazine (with unrefereed regular columns). It publishes articles that its readers are likely to find interesting. An article about software should thus contain some interesting material, e.g. about specific techniques used, that are useful to a wider audience than the software itself.

I should perhaps add that I am a member of the editorial board of this magazine.

Scientific Programming is an option that allows for a focus on the programming techniques employed in writing scientific software. (Disclosure: I'm on the Editorial Advisory Board.)

  • 1
    Link for "Scientific Programming"? – Faheem Mitha Jan 20 '12 at 15:23

There are some application specific journals

Computers & Geoscience, Computers & Fluids, Computers & Structures,

etc.

  • 1
    Can you publish articles about software in those? It's not clear to me from their "aims & scope". It seems that articles are expected to include new algorithms or new applications. – David Ketcheson Jan 12 '12 at 15:43

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.