Although I already did some work in the intersection of theory and simulation I'm still very new to this field and I need some guidance. If anybody can give some recommendations for introductory literature about computational science and how to plan scientific software I appreciate it. Below are some more concrete questions.
During the work for my thesis I did and still do numerical simulations of a quantum-mechanical system and I encountered the following problem. Basically, the three important steps of these simulations are: solving a system of ODEs, computation of physical observables using this solution, and plotting of the observables. My first question is, when it comes to writing the software what is the best practice for the separation of these steps? Should a program do all of these steps or is it better to divide the steps between different scripts and save the data in between?
My concern is that my current approach is chaotic and not efficient. My first scripts did all the steps in one go. When the ODE solver took a lot of time for the solution, I changed the script to save the raw solution for later purposes. My current approach is to calculate the solution and the quantities of interest in one go, and save the data to text files. The plotting is done by separate scripts. Currently it takes me about 0.5-3h to revise a figure and imho this is very slow.
Since there are a lot of parameters, there is a lot of room for choosing which variables are fixed, which are varied, what the x-axis is, and so forth. This led me to write a lot of small scripts each tailored to a specific region of the parameter space and a specific choice of observables which subsequently led to a complicated and fragmented folder and file structure. Is this a good style or should one choose to write programs that cover all possible cases and observables?