On a slurm cluster, is there ever a time when it’s **appropriate** to use
sbatch
*inside* an sbatch
script? Or is it always a bad pattern? I’ve seen this in use, and it looks iffy:
#SBATCH -J RecursiveSbatchInvocation
things=(apples bananas carrots)
for thing in "${things[@]}"; do
sbatch --wrap ./myscript.sh "$thing"
done
Compared to the two alternatives below,
- Is there a danger to running sbatch
inside sbatch
? Will it potentially hog a lot of resources uselessly, or run very slowly for me, or be impolite to other users on the cluster, or have unpredictable/bad side effects?
- Is there a realistic scenario where something *can only* be done with sbatch
inside sbatch
?
**Alternative 1:** job array:
#SBATCH -J JobArrayInvocation
#SBATCH -a 0-2
things=(apples bananas carrots)
./myscript.sh "${things["$SLURM_ARRAY_TASK_ID"]}"
**Alternative 2:** srun
inside sbatch
with multiple tasks:
#SBATCH -J SrunInvocations
#SBATCH --ntasks=3
things=(apples bananas carrots)
for thing in "${things[@]}"; do
srun --ntasks=1 ./myscript.sh "$thing" &
done
Somebody asked a similar question over on StackOverflow: “[**Can** I call sbatch recursively?](https://stackoverflow.com/questions/72747837/can-i-call-sbatch-recursively)” The two points in the discussion were (1) the cluster probably won’t allow it and (2) you can do the same thing with srun
inside sbatch
. But in my case the cluster _is_ allowing it. So I know it’s possible and I know “more canonical” ways to do this kind of thing, but I wonder if there’s a specific thing I can show people to say “please never use sbatch
recursively.”
Asked by wobtax
(1135 rep)
Mar 19, 2025, 04:18 PM
Last activity: Mar 20, 2025, 03:00 PM
Last activity: Mar 20, 2025, 03:00 PM