What programming strategies can I take for easily modifying algorithm parameters?

Question

Developing scientific algorithms is a highly iterative process often involving changing lots of parameters that I will want to vary either as part of my experimental design or as part of tweaking algorithm performance. What strategies can I take for structuring these parameters so that I can easily change them between iterations and so that I can easily add new ones?

Answer 1

It is cumbersome for the user to specify every aspect of an algorithm. If the algorithm allows nested components, then no finite number of options would be sufficient. Therefore, it is critical that options do not necessarily "bubble up" to the top level, as in the case of explicit arguments or template parameters. This is sometimes called the "configuration problem" in software engineering. I believe PETSc has a uniquely powerful system for configuration management. It is similar to the Service Locator pattern in Martin Fowler's essay on inversion of control.

PETSc's configuration system works through a combination of user-specified configuration managed by the solver objects (with get and set queries) and the Options Database. Any component of the simulation can declare a configuration option, a default value, and a place to put the result. Nested objects have prefixes which can be composed, such that every object that needs configuration can be addressed independently. The options themselves can be read from the command line, environment, configuration files, or from code. When an option is declared, a help string and man page are specified, so that the -help option is understandable and so that a properly linked GUI can be written.

The user calls a SetFromOptions method to make an object configure itself based on command line options. Calling this function is optional, and may not be called if the user (person writing code that calls PETSc) is exposing the options through some other interface. We highly recommend that the user expose the options database because it gives the end user (person running the application) a great deal of power, but it is not required.

A typical configuration, called via

PetscObjectOptionsBegin(object); /* object has prefix and descriptive string */
PetscOptionsReal("-ts_atol",                                      /* options database key */
                 "Absolute tolerance for local truncation error", /* long description */
                 "TSSetTolerances",                               /* function and man page on topic */
                  ts->atol,                                       /* current/default value *?
                  &ts->atol,                                      /* place to store value */
                  &option_set);                                   /* TRUE if the option was set */
PetscOptionsList("-ts_type","Time stepping method","TSSetType",TSList,
                 defaultType,typeName,sizeof typeName,&option_set);
TSAdaptSetFromOptions(ts->adapt);                                 /* configures adaptive controller method */
/* ... many others */
/* ... the following is only called from implicit implementations */
SNESSetFromOptions(ts->snes);                                     /* configure nonlinear solver. */
PetscOptionsEnd();

Notes:

• PetscOptionsList() presents the user with a choice from a dynamic list. There is a plugin architecture which new implementations can use to expose themselves as first-class to callers. (These implementations can be placed in shared libraries and used as first-class without recompiling programs.)
• SNESSetFromOptions() recursively configures the linear solvers, preconditioners, and any other components that need configuration.

Answer 2

I've faced this problem several times when developing my own simulation codes from scratch: which parameters should go in an input file, which should be taken from the command line, etc. After some experimenting, the following turned out to be efficient. (It is not as advanced as PETSc.)

Instead of writing an experimental simulation 'program', I'm more inclined to write a Python package that contains all the functions & classes needed to run the simulation. The traditional input file is then replaced by small Python script with 5 to 10 lines of code. Some lines are typically related to loading data files and specifying output. Others are instructions for the actual computation. Good default values for optional arguments in the Python package make it doable for beginners to use the library for simple simulations, while the advanced user still has access to all the bells and whistles.

A few examples:

• Atomistic Simulation Environment (also check out GPAW): https://wiki.fysik.dtu.dk/ase/
• TAMkin: http://molmod.ugent.be/code/wiki/TAMkin

Answer 3

As a first point, I would do the algorithm AND software as general as possible. I've learned this the hard way.

Let's say you start with a simple test case. You can do this faster. But then, if you made the software too specific (too few parameters) for this initial case, you'll loose more and more time adapting it every time you add a new degree of freedom. What I do now it's spend more time at the beginning making the thing pretty general, and increasing the variation of the parameters as I move forward.

This involves more testing from the beginning since you'll have more parameters from the starting point, but will mean that you can latter play a lot with the algorithm at zero or a very low cost.

Example: the algorithm involves calculating the surface integral the dot product of two vector functions. Don't assume from the beginning the size, geometry and discretization of the surface if in the future you may want to change that. Make a dot-product function, make the surface as general as possibly, calculate the integral in a nice formal way. You can test each function you make separately.

At the beginning, you can and start integrating over simple geometries and declaring may parameters at the start as constants. As time goes by, if you want to change the geometry, you can do it easily. Had you made assumptions at the beginning, you would have to change the whole code every time.