Our workgroup produces a desktop application that simulates building energy performance. It is a .NET application and when the user is running a lot of simulations, they can be quite time consuming. The simulations are totally parallelizable, and we have some very significant HPC computing resources at "the office".
One idea we've had is to allow the users to offload simulations that we know will be very long running (while each individual simulation runs about 30-120 seconds, running large numbers of simulations could take several days). Has anyone done this before? If so, did you use any libraries to make the job easier? Was it worth the effort?
edited to add:
Individual tasks for offloading the simulation would be
- packaging a file (about 5Mb),
- uploading it to our servers,
- decomposing the package into individual simulations (each takes about 30-120 seconds and is totally parallelizable), the number of simulations is a function of the number of options selected by the user (insulation, building orientation, etc) and the worst case of selecting every possible option would result in about 1E50 simulations. Running 100 to ~1E5 simulations isn't unknown, but the majority of users will run less than 10.
- reassembling the completed simulations and downloading the now much larger file.
We aren't sure what interface to use, as our group is new to this, and with budget cuts, it might get completed in time, but needs to be easy for the next folks (if any) to maintain.
This app already uses .NET 4 and can expand to use all the cores that the user has (our dev machines have 8 cores).