Our plasma dynamics simulations often produce too much information. During the simulations we record various physical properties on a grid (x,y,z,t) that is as large as (8192x1024x1024x1500), for at least 10 properties. This information is processed after the simulation is completed. With it we
- make movies of properties,
- perform a Fourier analysis,
- calculate average properties.
This simple dumping of as much information as possible worked fine when we studied smaller systems. This gave us the flexibility to interact with the results and decide later what we wanted to do with it. It also allowed us to allot our computational resources (CPU time) to simply running the simulations.
We've begun the process of doing the Fourier analysis on the fly, and filtering for only a select range of length scales. For numerical reasons, we sometimes need to resolve length scales that are smaller than we are actually interested in, so in those cases, this filter helps greatly. We are also exploring various parallel IO libraries, e.g. Parallel I/O options, in particular parallel HDF5.
What strategies are available to maximize the efficiency of data processing?
Is there any benefit to performing all analysis (not including post processing, e.g. movies and plots) on the fly?
I can imagine this issue coming up in other areas of research. For example, you might have a molecular dynamics simulation that needs to evolve for a long time, but you are interested in the brief moment when something interesting is happening. Or in CFD, the early time development might be slow, but once turbulence sets in, you might need a higher time resolution to monitor dynamics.
Are there freely available examples of sophisticated results collection from simulations?