Skip to main content

Slurm Partitions (Queues)

Anvil provides different queues with varying job sizes and walltimes. There are also limits on the number of jobs queued and running on a per-user and queue basis. Queues and limits are subject to change based on the evaluation from the Early User Program.

Anvil Production Queues
Queue Name Node Type Max Nodes per Job Max Cores per Job Max Duration Max running Jobs in Queue Max running + submitted Jobs in Queue Charging factor
debug regular 2 nodes 256 cores 2 hrs 1 2 1
gpu-debug gpu 1 node 2 gpus 0.5 hrs 1 2 1
wholenode regular 16 nodes 2,048 cores 96 hrs 64 128 1 (node-exclusive)
wide regular 56 nodes 7,168 cores 12 hrs 5 10 1 (node-exclusive)
shared regular 1 node 128 cores 96 hrs 6400 cores - 1
highmem large-memory 1 node 128 cores 48 hrs 2 4 4
gpu gpu - - 48 hrs - - 1

For gpu queue: max of 12 GPU in use per user and max of 32 GPU in use per allocation.

Make sure to specify the desired partition when submitting your jobs (e.g. -p shared). If you do not specify one, the job will be directed into the default partition (wholenode) which may lead to unexpected SU usage - see next note below.

If the partition is node-exclusive (e.g. the wholenode and wide queues), even if you ask for 1 core in your job submission script, your job will get allocated an entire node and would not share this node with any other jobs. Hence, it will be charged for 128 cores' worth and squeue command would show it as 128 cores, too. See SU accounting for more details.

Link to section 'Useful tools' of 'Slurm Partitions (Queues)' Useful tools

  1. To display all Slurm partitions and their current usage, type showpartitions at the command line.
    x-anvilusername@login03.anvil:[~] $ showpartitions
    Partition statistics for cluster anvil at CURRENTTIME
          Partition     #Nodes     #CPU_cores  Cores_pending   Job_Nodes MaxJobTime Cores Mem/Node
          Name State Total  Idle  Total   Idle Resorc  Other   Min   Max  Day-hr:mn /node     (GB)
    wholenode:*    up   750   684  96000  92160      0   1408     1 infin   infinite   128     257 
        shared    up   250   224  32000  30208      0      0     1 infin   infinite   128     257 
          wide    up   750   684  96000  92160      0      0     1 infin   infinite   128     257 
       highmem    up    32    32   4096   4096      0      0     1 infin   infinite   128    1031 
         debug    up    17     5   2176   2176      0      0     1 infin   infinite   128     257 
           gpu    up    16    10   2048   1308      0    263     1 infin   infinite   128     515 
     gpu-debug    up    16    10   2048   1308      0      0     1 infin   infinite   128     515

  2. To show the list of available constraint feature names for different node types, type sfeatures at the command line.
    x-anvilusername@login03.anvil:[~] $ sfeatures
    NODELIST     CPUS   MEMORY    AVAIL_FEATURES   GRES
    a[000-999]   128    257526    A,a              (null)
    b[000-031]   128    1031669   B,b              (null)
    g[000-015]   128    515545    G,g,A100         gpu:4

Helpful?

Thanks for letting us know.

Please don’t include any personal information in your comment. Maximum character limit is 250.
Characters left: 250
Thanks for your feedback.