Converting Moab/Torque scripts to Slurm
Moab/Torque commands and flags and their Slurm equivalents. For more detailed information, see https://slurm.schedmd.com/sbatch.html
Job Submission Options
| Option | Moab/Torque (msub) | Slurm (sbatch) |
|---|---|---|
| Script directive | #MSUB | #SBATCH |
| Job name | -N <name> | --job-name=<name> -J <name> |
| Account | -A <account> | --account=<account> -A <account> |
| Queue | -q <queue> | --partition=<queue> |
| Wall time limit | -l walltime=<hh:mm:ss> | --time=<hh:mm:ss> -t <hh:mm:ss> |
| Node count | -l nodes=<count> | --nodes=<count> -N <count> |
| Core count | -l procs=<count> | -n <count> |
| Process count per node | -l ppn=<count> | --ntasks-per-node=<count> |
| Core count (per process) | |
--cpus-per-task=<cores> |
| Memory limit | -l mem=<limit> | --mem=<limit> (Memory per node in MB) |
| Minimum memory per processor | -l pmem=<limit> | --mem-per-cpu=<memory> |
| Request GPUs | -l gpus=<count> | --gres=gpu:<count> |
| Request specific nodes | -l nodes=<node>[,node2[,...]]> | -w, --nodelist=<node>[,node2[,...]]> -F, --nodefile=<node file> |
| Job array | -t <array indices> | -a <array indices> |
| Standard output file | -o <file path> | --output=<file path> (path must exist) to replicate Moab outfile formatting: #SBATCH -o "%x.o%j" |
| Standard error file | -e <file path> | --error=<file path> (path must exist) to replicate Moab error file formatting: #SBATCH -e "%x.e%j" |
| Combine stdout/stderr to stdout | -j oe | --output=<combined out and err file path> |
| Architecture constraint | -l partition=<architecture> | --constraint=<architecture> -C <architecture> |
| Copy environment | -V | --export=ALL (default) --export=NONE to not export environment |
| Copy environment variable | -v <variable[=value][,variable2=value2[,...]]> | --export=<variable[=value][,variable2=value2[,...]]> |
| Job dependency | -W depend=after:jobID[:jobID...] -W depend=afterok:jobID[:jobID...] -W depend=afternotok:jobID[:jobID...] -W depend=afterany:jobID[:jobID...] |
--dependency=after:jobID[:jobID...] --dependency=afterok:jobID[:jobID...] --dependency=afternotok:jobID[:jobID...] --dependency=afterany:jobID[:jobID...] |
| Request event notification | -m <events> | --mail-type=<events> Note: multiple mail-type requests may be specified in a comma separated list: --mail-type=BEGIN,END,FAIL |
| Email address | -M <email address> | --mail-user=<email address> |
| Defer job until the specified time | -a <date/time> | --begin=<date/time> |
| Node exclusive job | -l naccesspolicy=singlejob | --exclusive |
Common Job Commands
| Option | Moab/Torque (msub) | Slurm (sbatch) |
|---|---|---|
| Submit a job | msub <job script> | sbatch <job script> |
| Delete a job | qdel <job ID> canceljob <job ID> | scancel <job ID> |
| Job status (all) | qstat showq |
squeue |
| Job status (by job) | qstat <job ID> | squeue -j <job ID> |
| Job status (by user) | qstat -u <netID> showq -u <netID> | squeue -u <netID> |
| Job status (detailed) | qstat -f <job ID> checkjob <job ID> |
scontrol show job -dd <job ID> checkjob <job ID> |
| Show expected start time | showstart <job ID> | squeue -j <job ID> --start |
| Queue list / info | qstat -q [queue] | scontrol show partition [queue] |
| Hold a job | qhold <job ID> | scontrol hold <job ID> |
| Release a job | qrls <job ID> | scontrol release <job ID> |
| Start an interactive job | msub -I <args> | salloc <args> srun --pty <args> |
| X forwarding | msub -l -X <args> | srun --pty <args> --x11 |
| Monitor or review a job's resource usage | |
sacct -j <job_num> --format JobID,jobname,NTasks,nodelist,CPUTime,ReqMem,Elapsed (see sacct for all format options) |
| View job batch script | |
scontrol write batch_script <jobID> [filename] |
Script Variables
| Info | Torque | Slurm | Notes |
|---|---|---|---|
| Version | $PBS_VERSION | – | Can extract from sbatch --version |
| Job name | $PBS_JOBNAME | $SLURM_JOB_NAME | |
| Job ID | $PBS_JOBID | $SLURM_JOB_ID | |
| Batch or interactive | $PBS_ENVIRONMENT | – | |
| Submit directory | $PBS_O_WORKDIR | $SLURM_SUBMIT_DIR | Slurm jobs starts from the submit directory by default |
| Node file | $PBS_NODEFILE | |
A filename and path that lists the nodes a job has been allocated |
| Node list | cat $PBS_NODEFILE | $SLURM_JOB_NODELIST | The Slurm variable has a different format to the PBS one. To get a list of nodes use: scontrol show hostnames $SLURM_JOB_NODELIST |
| Job array index | $PBS_ARRAY_INDEX | $SLURM_ARRAY_TASK_ID | |
| Queue name | $PBS_QUEUE | $SLURM_JOB_PARTITION | |
| Number of nodes allocated | $PBS_NUM_NODES | $SLURM_JOB_NUM_NODES $SLURM_NNODES |
|
| Number of processes | $PBS_NP | $SLURM_NTASKS | |
| Number of processes per node | $PBS_NUM_PPN | $SLURM_TASKS_PER_NODE | |
| Requested tasks per node | – | $SLURM_NTASKS_PER_NODE | |
| Requested CPUs per task | – | $SLURM_CPUS_PER_TASK | |
| Scheduling priority | – | $SLURM_PRIO_PROCESS | |
| Job user | – | $SLURM_JOB_USER | |
| Hostname | $HOSTNAME | $HOSTNAME == $SLURM_SUBMIT_HOST | Unless a shell is invoked on an allocated resource, the $HOSTNAME variable is propagated (copied) from the submit machine environment to all allocated nodes. |
Adapted with permission from https://arc-ts.umich.edu/migrating-from-torque-to-slurm/