8
$\begingroup$

I am aiming at keeping my scientific studies and analyses reproducible: I am automating them as much as possible, I am sharing them, and I sharing them together with the execution environment(s) I've used to run the analyses. This is to make it possible, and easily possible, to rerun the analyses.

I create execution environments using conda with packages mainly from the conda-forge channel. As far as I understand, I have two possibilities of sharing them:

  1. Sharing the full environment, i.e. including all libraries.
  2. Sharing only a description of the environment, i.e. a conda environment file, and relying on the availability of the packages in the repository.

What are the limitations of these approaches with respect to the aim of reproducibility?

$\endgroup$

1 Answer 1

4
$\begingroup$

There are more possibilities, e.g.

  1. Sharing a Docker image containing the full conda environment, including all packages and associated code

  2. Sharing a virtual machine image containing either the Docker image from 3 or directly including the conda environment from 1.

Options 3 and 4 require only the ability to run Docker or VM images and no longer depend on any external repository of packages, which can be important when e.g. a package is marked as being unusable for some reason (causing conda to fail when constructing the environment, even if the package is available).

Personally, I'd do all four options for true reproducibility of archival-quality material, with option 2 hopefully working (as you can see for all our examples at examples.pyviz.org), then falling back to option 1, then option 3, and option 4 if need be! But for normal cases option 1 and 2 will typically be enough.

$\endgroup$
1
  • $\begingroup$ Also note that we are using anaconda-project to capture not just the environment, but also the commands used in that environment; again see examples.pyviz.org. We're also adding the ability to create a Docker image directly from a project .yml file (github.com/Anaconda-Platform/anaconda-project/pull/246), which would automate step 3 given step 2. $\endgroup$ Feb 11, 2020 at 4:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.