Converting Moab/Torque scripts to Slurm

On May 1st, 2019 the Quest scheduler changed from Moab to Slurm. Researchers using Quest will have to update their job submission scripts and use different commands to run under the new scheduler.

Moab/Torque commands and flags and their Slurm equivalents. For more detailed information, see

Job Submission Options

Option Moab/Torque (msub) Slurm (sbatch)
Script directive #MSUB #SBATCH
Job name -N <name> --job-name=<name>
-J <name>
Account -A <account> --account=<account>
-A <account>
Queue -q <queue> --partition=<queue>
Wall time limit -l walltime=<hh:mm:ss> --time=<hh:mm:ss>
Node count -l nodes=<count> --nodes=<count>
-N <count>
Core count -l procs=<count> -n <count>
Process count per node -l ppn=<count> --ntasks-per-node=<count>
Core count (per process)
Memory limit -l mem=<limit> --mem=<limit> (Memory per node in MB)
Minimum memory per processor -l pmem=<limit> --mem-per-cpu=<memory>
Request GPUs -l gpus=<count> --gres=gpu:<count>
Request specific nodes -l nodes=<node>[,node2[,...]]> -w, --nodelist=<node>[,node2[,...]]>
-F, --nodefile=<node file>
Job array -t <array indices> -a <array indices>
Standard output file -o <file path> --output=<file path> (path must exist)
to replicate Moab outfile formatting:
#SBATCH -o "%x.o%j"
Standard error file -e <file path> --error=<file path> (path must exist)
to replicate Moab error file formatting:
#SBATCH -e "%x.e%j"
Combine stdout/stderr to stdout -j oe --output=<combined out and err file path>
Architecture constraint -l partition=<architecture> --constraint=<architecture>
-C <architecture>
Copy environment -V --export=ALL (default)
--export=NONE to not export environment
Copy environment variable -v <variable[=value][,variable2=value2[,...]]> --export=<variable[=value][,variable2=value2[,...]]>
Job dependency -W depend=after:jobID[:jobID...]
-W depend=afterok:jobID[:jobID...]
-W depend=afternotok:jobID[:jobID...]
-W depend=afterany:jobID[:jobID...]
Request event notification -m <events> --mail-type=<events>
Note: multiple mail-type requests may be specified in a comma separated list:
Email address -M <email address> --mail-user=<email address>
Defer job until the specified time -a <date/time> --begin=<date/time>
Node exclusive job -l naccesspolicy=singlejob --exclusive

Common Job Commands

Option Moab/Torque (msub) Slurm (sbatch)
Submit a job msub <job script> sbatch <job script>
Delete a job qdel <job ID>
canceljob <job ID>
scancel <job ID>
Job status (all) qstat
Job status (by job) qstat <job ID> squeue -j <job ID>
Job status (by user) qstat -u <netID>
showq -u <netID>
squeue -u <netID>
Job status (detailed) qstat -f <job ID>
checkjob <job ID>
scontrol show job -dd <job ID>
checkjob <job ID>
Show expected start time showstart <job ID> squeue -j <job ID> --start
Queue list / info qstat -q [queue] scontrol show partition [queue]
Hold a job qhold <job ID> scontrol hold <job ID>
Release a job qrls <job ID> scontrol release <job ID>
Start an interactive job msub -I <args> salloc <args>
srun --pty <args>
X forwarding msub -l -X <args> srun --pty <args> --x11
Monitor or review a job's resource usage
sacct -j <job_num> --format JobID,jobname,NTasks,nodelist,CPUTime,ReqMem,Elapsed
(see sacct for all format options)
View job batch script
scontrol write batch_script <jobID> [filename]

Script Variables

Info Torque Slurm Notes
Version $PBS_VERSION Can extract from sbatch --version
Batch or interactive $PBS_ENVIRONMENT
Submit directory $PBS_O_WORKDIR $SLURM_SUBMIT_DIR Slurm jobs starts from the submit directory by default
A filename and path that lists the nodes a job has been allocated
Node list cat $PBS_NODEFILE $SLURM_JOB_NODELIST The Slurm variable has a different format to the PBS one.
To get a list of nodes use:
scontrol show hostnames $SLURM_JOB_NODELIST
Number of nodes allocated $PBS_NUM_NODES $SLURM_JOB_NUM_NODES

Number of processes $PBS_NP $SLURM_NTASKS
Number of processes per node $PBS_NUM_PPN $SLURM_TASKS_PER_NODE
Requested tasks per node $SLURM_NTASKS_PER_NODE
Requested CPUs per task $SLURM_CPUS_PER_TASK
Scheduling priority $SLURM_PRIO_PROCESS
Hostname $HOSTNAME $HOSTNAME == $SLURM_SUBMIT_HOST Unless a shell is invoked on an allocated resource, the $HOSTNAME variable is propagated (copied) from the submit machine environment to all allocated nodes.

Adapted with permission from

See Also:

Keywords:quest,slurm,moab,torque,hpc,rosetta   Doc ID:89454
Owner:Research Computing .Group:Northwestern
Created:2019-02-01 17:22 CDTUpdated:2019-05-06 11:35 CDT
Feedback:  0   0