I'm currently working on a Finite Element Navier Stokes simulation and I would like to investigate the effects of a variety of parameters. Some parameters are specified in an input file or via a command line options; other parameters are provided as flags in a Makefile so my code has to be recompiled whenever I change those options. I would be interested to get some advice about a good way to systematically explore the parameter space.
- Are there useful C++/Python libraries/frameworks that can help with this sort of thing? For example discovering boost.Program_options was a big help since it's possible to overload input file options with command line arguments. I have also seen some people use a job file describing each case quite effectively and a collegue suggested that writing parameters into vtu files as comment blocks could work too.
- Perhaps it isn't worth investing much time in this at all? Is it just a distraction and a time-drain and it's best to just muscle through the testing process brute force and ad hoc?
I am currently doing things mostly by hand and I have encountered the following problems:
- Naming test cases. I tried storing results in folders named with the run parameters separated with underscores e.g.
Re100_dt02_BDF1.... These quickly become long or difficult to read/cryptic if they are abbreviated too much . Also, real number parameters include a
.which is awkward/ugly.
- Logging run data. Sometimes I would like to see the results written to the terminal and also saved to a text file. This answer from StackOverflow for instance is somewhat helpful but the solutions seem to be a bit intrusive.
- Plotting data according to parameter. It takes quite some time collect relevant data from a variety of log files into a single file which I can then plot, with a better system perhaps this would become easier.
- Recording comments on the data. After examining results I write some comments in a text file but keeping this is sync with the results folders is sometimes difficult.