GPU job in GPU queue
The Anvil cluster nodes contain GPUs that support CUDA and OpenCL. See the detailed hardware overview for the specifics on the GPUs in Anvil or use sfeatures command to see the detailed hardware overview..
Link to section 'How to use Slurm to submit a SINGLE-node GPU program:' of 'GPU job in GPU queue' How to use Slurm to submit a SINGLE-node GPU program:
#!/bin/bash
# FILENAME: myjobsubmissionfile
#SBATCH -A myGPUallocation # allocation name
#SBATCH --nodes=1 # Total # of nodes
#SBATCH --ntasks-per-node=1 # Number of MPI ranks per node (one rank per GPU)
#SBATCH --gpus-per-node=1 # Number of GPUs per node
#SBATCH --time=1:30:00 # Total run time limit (hh:mm:ss)
#SBATCH -J myjobname # Job name
#SBATCH -o myjob.o%j # Name of stdout output file
#SBATCH -e myjob.e%j # Name of stderr error file
#SBATCH -p gpu # Queue (partition) name
#SBATCH --mail-user=useremailaddress
#SBATCH --mail-type=all # Send email to above address at begin and end of job
# Manage processing environment, load compilers, and applications.
module purge
module load modtree/gpu
module load applicationname
module list
# Launch GPU code
./myexecutablefiles
Link to section 'How to use Slurm to submit a MULTI-node GPU program:' of 'GPU job in GPU queue' How to use Slurm to submit a MULTI-node GPU program:
#!/bin/bash
# FILENAME: myjobsubmissionfile
#SBATCH -A myGPUallocation # allocation name
#SBATCH --nodes=2 # Total # of nodes
#SBATCH --ntasks-per-node=4 # Number of MPI ranks per node (one rank per GPU)
#SBATCH --gpus-per-node=4 # Number of GPUs per node
#SBATCH --time=1:30:00 # Total run time limit (hh:mm:ss)
#SBATCH -J myjobname # Job name
#SBATCH -o myjob.o%j # Name of stdout output file
#SBATCH -e myjob.e%j # Name of stderr error file
#SBATCH -p gpu # Queue (partition) name
#SBATCH --mail-user=useremailaddress
#SBATCH --mail-type=all # Send email to above address at begin and end of job
# Manage processing environment, load compilers, and applications.
module purge
module load modtree/gpu
module load applicationname
module list
# Launch GPU code
mpirun -np $SLURM_NTASKS ./myexecutablefiles
Make sure to use --gpus-per-node command, otherwise, your job may not run properly.