2
$\begingroup$

I have just finished a new version of a parallel code that I have been working on. I would like to do a strong scaling test, but I am always bothered by how much monotonous work scaling tests seem to require.

Currently, I make folders for each processor count (np1, np2, np4, ...), copy my code into each folder, assign the correct parameters to the PBS file for submission to the cluster, and then submit the jobs one-by-one.

After job completion, I copy the data from each folder onto my machine and peice-by-piece collect the timing data from each folder. Then it goes into MATLAB.

I am curious if any of you have developed more efficient ways to run scaling tests. Do you write scripts to do most of this for you? If so, how do you accomplish that? Bash scripting? Perl, perhaps?

Thanks!

$\endgroup$
  • $\begingroup$ I, for one, write a lot of scripts that include creating the PBS files, parsing the screen output of my programs that contains timing information, collating this into data files (all of this typically in the form of shell scripts), and then plotting with gnuplot. It is all scripted. $\endgroup$ – Wolfgang Bangerth May 30 '17 at 20:54
  • $\begingroup$ Thanks for the response. I'm still relatively new to shell scripting. Is there a good place you would recommend starting? Or should I just Google "how to shell scripting"? $\endgroup$ – EternusVia May 30 '17 at 21:00
  • 1
    $\begingroup$ Yes, or read an intro to linux book. $\endgroup$ – Wolfgang Bangerth May 30 '17 at 23:42
  • 1
    $\begingroup$ Alternatively, lecture 2.9 here: math.colostate.edu/~bangerth/videos.html $\endgroup$ – Wolfgang Bangerth May 30 '17 at 23:42
  • 2
    $\begingroup$ @EternusVia You should just google stuff about shell scripting.. or, if possible, use Python instead of the typical shell scripting languages(bash, cshell) if you expect to do a lot with the scripting. Python is more flexible and has more you can benefit from. $\endgroup$ – spektr May 31 '17 at 0:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.