45

I think that documentation for scientific software can be divided into three categories, all of which are necessary for full understanding. The easiest and most common is individual method documentation. There are many systems for this. You mention doxygen, Python has pydoc, and in PETSc we have our own package sowing which generates the following. However, ...


20

What you are asking for is the Elsivier grand challenge of the "Executable Paper". While many approaches have been tried, none are as compelling as the authors might suggest. Here are a few examples of techniques used. Madagascar Project takes your approach, inside the make script have the simulations run that produce the figures and paper simultaneously. ...


19

In-code documentation The most important thing is to use the documentation facilities in your chosen development environment, so that means pydoc for python, javadoc in java or xml comments in C#. These make it easy to write the documentation at the same time as writing the code. If you rely on coming back and documenting things later, you may not get ...


17

In rough order of importance. Source Code Make the code that implements the key aspects of your algorithm available. Even if the user can't build or run it, they can read exactly what is done. I have several times noticed simple decisions that weren't documented in a paper, but which a couple minutes with the source code answered conclusively. Make it ...


15

I will take objection with almost every point Faheem makes. Specifically: 1/ "I think that it is unrealistic to expect scientific developers to spend a great deal of time documenting their software". This is a prescription for a failed project that soon nobody will be able to use who does not have access to the primary developers. It is for good reason that ...


14

As some comments have suggested, this approach has long been developed in the R community by building on Sweave and more recently, knitr. Obviously this approach has the disadvantage of being language-specific at the moment, but the advantage that its regularly used in academic papers. Use of Sweave in real publications The Journal of Biostatistics ...


13

I have not had a lot of success in using other people's solutions to this problem. I usually just want something simple that works for me and gets the job done. To this end, I generally try to write one python script which is in charge of running all the results, parsing the output, as well as building the figures/tables. I write my codes to generate ...


13

I don't think that the definition of "reproducible research" requires that the author provide for free all of the tools that are needed to reproduce the results obtained. If some of it is proprietary, then it is incumbent on the future user, not the author, to make the arrangements to acquire the needed software. (You wouldn't expect to have to build the ...


9

The following threads are tangentially related: This one on programmers.SE on leading the documentation effort for an undocumented project. This one on programmers.SE on learning undocumented APIs. This one on programmers.SE on documenting code. This one on Stack Overflow on bringing employees up to speed on undocumented APIs. If you're inherited the code, ...


9

This is a good question. To a first approximation, the code should attempt to be self documenting. So, for example, if the software is command line, you should be able to do executable --help or executable -h or even executable (if the executable does nothing with no arguments), and have a brief usage message return. Second, I think that it is unrealistic ...


7

Even more important, in my opinion, is making sure that you can figure out how to re-generate all your results from scratch in a month or a year (for instance, when referees ask you to add or modify something). For that purpose, what I do is include a text file with very detailed directions on how to reproduce all results. It's best if you test these out ...


6

Emacs's orgmode in combination with Babel achieves that. Babel can execute code snippets from various programming and scripting languages, e.g., it could open the file containing the simulation data and put it into a table in orgmode, which can be exported to LaTeX (and many other formats). It takes quite a while getting used to all the key combos in orgmode,...


6

Total running time (wall clock) is the only metric that matters in industry or real life applications: this figure should never be omitted, even if embarrassing. Of course this metric is very dependent on the test environment, so this should be described in detail. All other metrics can (or should be) reported if they provide insight into the problem, or ...


6

Most journals aren't set up for this in any formal way, but we've recently founded the Archive of Numerical Software that is specifically intended to have the source code and everything else that's necessary be part of the article. Check it out: http://journals.tdl.org/ans Submissions are welcome!


5

To address your question on how to document data and results, I'd recommend something like Python's doctest module. This allows you to write tutorials or tests in a way that can be automatically validated.


5

If you are interested in literate programming, have a look at org-babel. It is part of org-mode in Emacs and thus gives you a wide array of export options (LaTeX,PDF,HTML,ODT) for documentation. Emacs can display images inside the buffer and let you write mathematical equations in LaTeX syntax so you don't have to limit yourself to plain text documentation....


5

Running the command edit interp2 allows you to see the source code of this particular function and then you can read the piece of code that deals with bicubic interpolation. In MATLAB R2011, there is even a paper being cited: "Cubic Convolution Interpolation for Digital Image Processing", Robert G. Keys, IEEE Trans. on Acoustics, Speech, and Signal ...


4

It is often the case that one can only report the tip of the iceberg of all the work and compromises that went into a piece of software. Reporting performance is nice but the real deal is when the code is made freely accesible on internet, this way, anyone interested can evaluate and reproduce the results. Ideally, if you release the software, you can also ...


4

If you're using LaTeX, a relatively low-tech solution is to have your code spit out a file (or else use a script to filter from your code's output) containing a whole bunch of lines like this: \newcommand{\myresults1}{<value>} Then you can use the \input command to add that file into your document, and use the defined commands to place the values.


4

Normally the author of a work can license it under more than one of the open source licenses you mention (so called dual licensing). However there seems to be an issue with doing so under the referenced ACM editorial policy, which states that you would be obliged to transfer copyright ownership on published "algorithms": Authors of copyrightable ...


4

If running all your code is cheap then you could do something low-tech like the following: You could template your documents with formatted strings so that they look like this "we observed a %(fractional_improvement)s increase in ..." Have python scripts that look like this results = {"alpha" : run_alpha_computation(...), "...


4

I have never been of the opinion that "reproducible research" means that your fellow readers and scientists should only have to click "Run" (or compile or what have you) to entirely reproduce your results. If nothing else, that would kill the research of anyone who uses a proprietary statistical analysis package, or make things "unreproducable" if they were ...


3

I always tell my students to read a code from the bottom up: you start in main() and see what it calls. Typically this is only a small number of functions. Then you look into the functions called from main() which typically define the overall flow of the algorithm (time stepping loop, assembly, solver, output, etc). Go two or so levels deep to get an ...


3

In Stodden, V. 2009. “The Legal Framework for Reproducible Scientific Research.” CiSE. Victoria Stodden recommends publishing the full "research compendium", and lists the following components on p. 38: The research paper The data - including documentation and code for processing the data The experiment — all source code; documentation, parameters, ...


2

I work for Elsevier. My company has started using the Collage framework (developed in response to the Executable Paper Grand Challenge) in journal issues to enable authors to publish chunks of executable code with their article. This feature makes it easier for readers to reproduce results reported in the article and to reuse published material for their own ...


2

In your case, where you may not even have rights to distribute any of the code you used at all, a very in-depth description of the algorithm might be the best you can do. Depending on how complex your code is, someone might be able to reproduce whatever is of interest to them from whatever tools they have available. Otherwise, ask around and see how much ...


2

I work for Elsevier. My company has started using the Collage framework (developed in response to the Executable Paper Grand Challenge) in journal issues to enable authors to include all of the data and code needed to reproduce the results and figures in their papers. This feature makes it easier for readers to reproduce results reported in the article and ...


2

At least, the Source code and data you used to perform your experiments should be accesible somewhere. Add instructions to build your code if necessary. Really there are so few open access journals that there is no an open and established rule.


2

The Scalapack UG link brings up a misleading web page. It looks as if there isn't any Fortran documentation, when I suppose the page is instead intended to be a title page. Clicking on the "Next" button in the upper left-hand corner brings you to the first page of Fortran interface documentation for P_GEMR2D.


2

Currently there is a GMSH API in the works: https://gitlab.onelab.info/gmsh/gmsh/tree/master/api Also, there are rumors that there will be a fully documented API by version 4.0. In short, there is no documentation on the functions for GMSH (except for what is in the source all ready). But, they are planning on creating a documented version by GMSH 4.0. ...


Only top voted, non community-wiki answers of a minimum length are eligible