# Scripting for High Performance Computing

I would like links to resources which provide codes for scripting (preferably in python) for high performance computing. I'm looking for sample codes and examples which can,

(1) automatically create various input files based on the ranges provided for input values or by performing combinatorics of input values (ex: atomic positions that can be switched in a quantum mechanical simulation)

(2) create batch submission files (ex:PBS) based on the provided input values of no. of nodes/processors provided.

(3) Compare the output files of standard and non-standard (just column) formats and create plots as needed.

• The scripts for preparing inputs are usually so short and simple that I don't think anyone bothers to publish theirs. I've certainly never seen any. For the outputs, they should be straightforward as well. – Bill Barth May 12 '14 at 18:37
• As @BillBarth suggested, rolling your own Python scripts to do this shouldn't be too hard. I use Fabric (fabfile.org) and find it pretty helpful. – Matthew Emmett May 12 '14 at 23:21
• If you're looking to generate input files, you could create a template of the parts of these files that don't change from run to run using Jinja2, and then use the Jinja2 library to take the parameters that do change as inputs and generate the input files you need from your templates. – Geoff Oxberry May 13 '14 at 2:37
• Checkout iPython cluster tools. Its pretty easy to make a PBS script that runs it, from there just setup a server and farm the work out :) – meawoppl May 16 '14 at 23:01
• Also, I just remembered Starcluster uses SGE under the hood, and likely has this step kicking around somewhere. – meawoppl May 16 '14 at 23:22

Personally I would break the problem into to separate parts, the Python part (if that is your weapon of choice) doing file processing etc and the PBS job launch script part. I would definitely use bash scripting for the PBS part because it sounds like you want to make (easy) use of PBS variables and leverage what PBS gives you to launch in a variety of ways. Your PBS script then calls your Python code from within the bash script with these options.

Things I have used are job arrays, which allow the PBS variable to give each job a different value in a $PBS_ARRAYID environment variable. I have also used the -v option to pass my own user defined variable names. This has the advantage that you define your variables at launch time. You can also use the job name itself to launch your code with a particular parameter. This has an advantage in that the result log file will also has the job name. The quickest way to see what you are working with is to launch a job with "env" and you can see what you get to work with in the output. It is worth doing this as I have found what you get depends on the batch system and how the job is launched. Also be warned that what the documentation (man qsub) says an option "should" do and what it "actually" does are often different (so trial and error is called for). Here's an example of how I launch OpenFOAM to adapt to the number of cores/nodes provided. It creates a config file that OpenFOAM interprets for splitting up the calculation. I used the jobname to configure the speed of the simulation. #!/bin/bash #PBS -N vespa_LES #PBS -l walltime=48:00:00 #PBS -l nodes=2:ppn=10,pmem=4000MB #PBS -m abe cd$PBS_O_WORKDIR
echo Here are the Vespa variables used
cat ./vespa_variables
echo Running with $PBS_NP processors across$PBS_NUM_NODES nodes. $PBS_NUM_PPN procs per node rm ./vespa_env echo "vespa_env_procs$PBS_NP;" >> ./vespa_env
echo "vespa_env_num_nodes         $PBS_NUM_NODES;" >> ./vespa_env echo "vespa_env_num_ppn$PBS_NUM_PPN;" >> ./vespa_env
echo "vespa_env_jobname          $PBS_JOBNAME;" >> ./vespa_env cat ./vespa_env module purge module load openfoam/2.2.2 .$foamDotFile
./Allclean
./Allrun


The short version of the above is that PBS provides some helper variables and I would use those to simplify launching a variety of jobs with the same code.

Might UNICORE fit the bill? It is more than a scripting language, as it attempts to provide a uniform computing interface to the user, so one needs not worry about the kind of clusters accepting the job etc (i.e. no need to tune PBS or other queuing formats by hand). It also provides a graphical workflow, which can help the replicability of numerical experiments.

This will require the cluster admin to install the software on the server side, but I think the people from UNICORE are usually happy to help.

That said, I tend to fall back on my old habit of hacking together one-off solutions combining awk and bash scripts very specific to the given task.

• Thank you for suggesting unicore. I will definitely have a look at it. Can you point me to awk or bash scripts available online which I can hack and modify to change it to my requirement. – WanderingMind May 12 '14 at 17:21
• I am not aware of any online repositories of bash/awk hacks for supercomputing. The problem is that these things are very platform/application specific. The codes I use are modified from those inherited from collaborators. My typical setup is to have basic input files set up, then to run a bash script which starts off by making a directory and copying the input files there. Then it uses some simple awk calculations (eg combinatorics) to determine proper input values. Next, the script modifies the input files using sed, before finally queueing the job. Ugly and not very general but works for me – alarge May 12 '14 at 17:34

A friend of mine is just about to finish this book which covers many ways of optimising python scripts to achieve high performance speeds:

http://shop.oreilly.com/product/0636920028963.do