I am used to running calculations on a server/cluster, and we have used both PBS and SLURM. However, I am looking to run some calculations on my local machine. I would like to be able to schedule my jobs so that they run more efficiently. Does such software exist for macOS? Ideally it would be free. Please let me know if you need any additional information, or if this question is more appropriate on another stack exchange.
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
Task spooler has worked great for me in the past. It does everything you requested.
-
$\begingroup$ Thanks! This is what I was looking for. Although, I need to figure out how to stop Task Spooler from overwriting my output file with the jobid. $\endgroup$ Commented Jul 30, 2020 at 20:37
-
1$\begingroup$ manpages.ubuntu.com/manpages/xenial/man1/tsp.1.html there are a surprising number of options and environment variables. I think they can take care of this. $\endgroup$– user20857Commented Jul 30, 2020 at 20:38
$\begingroup$
$\endgroup$
I would guess that it is possible to deploy PBS and SLURM on your local machine, but I'd personally say that's a bit overkill.
I tend to use simple bash scripts. You can then list the execution commands like a to-do list, it is possible to run multiple jobs in parallel, and place a wait signal for them to finish, before continuing down the list. You can also easily pipe the individual program outputs to output files. Chances are that your local machine has 4/8/16 cores, so the decision of how many jobs you want to start at the same time is not that hard.
example:
#!/bin/sh
echo "starting first batch of jobs"
job1 &> job1_output.txt &&
job2 &> job2_output.txt &&
wait
echo "first two jobs finished"
job3 &> job3_output.txt &&
job4 &> job4_output.txt &&
wait
echo "all jobs finished"
$\begingroup$
$\endgroup$
In the same style of task-spooler there is also pueue which probably fits the needs of anyone stumbling in this question as I did.
Works really great for my uses with computational chemistry and some jobs in almost the same style as PBS, even though it's not meant to be as scriptable and customizable as those schedulers.
