Sample Header Ad - 728x90

Is sbatch-inside-sbatch a bad idea?

0 votes
1 answer
210 views
On a slurm cluster, is there ever a time when it’s **appropriate** to use sbatch *inside* an sbatch script? Or is it always a bad pattern? I’ve seen this in use, and it looks iffy:
#SBATCH -J RecursiveSbatchInvocation
things=(apples bananas carrots)
for thing in "${things[@]}"; do
  sbatch --wrap ./myscript.sh "$thing"
done
Compared to the two alternatives below, - Is there a danger to running sbatch inside sbatch? Will it potentially hog a lot of resources uselessly, or run very slowly for me, or be impolite to other users on the cluster, or have unpredictable/bad side effects? - Is there a realistic scenario where something *can only* be done with sbatch inside sbatch? **Alternative 1:** job array:
#SBATCH -J JobArrayInvocation
#SBATCH -a 0-2
things=(apples bananas carrots)
./myscript.sh "${things["$SLURM_ARRAY_TASK_ID"]}"
**Alternative 2:** srun inside sbatch with multiple tasks:
#SBATCH -J SrunInvocations
#SBATCH --ntasks=3
things=(apples bananas carrots)
for thing in "${things[@]}"; do
  srun --ntasks=1 ./myscript.sh "$thing" &
done
Somebody asked a similar question over on StackOverflow: “[**Can** I call sbatch recursively?](https://stackoverflow.com/questions/72747837/can-i-call-sbatch-recursively)” The two points in the discussion were (1) the cluster probably won’t allow it and (2) you can do the same thing with srun inside sbatch. But in my case the cluster _is_ allowing it. So I know it’s possible and I know “more canonical” ways to do this kind of thing, but I wonder if there’s a specific thing I can show people to say “please never use sbatch recursively.”
Asked by wobtax (1135 rep)
Mar 19, 2025, 04:18 PM
Last activity: Mar 20, 2025, 03:00 PM