Rosen Center for Advanced Computing
Collapse Topics

Applications

AMD

Audio/Visualization

Bioinformatics

Climate

Computational chemistry

Container

Electrical engineering

Fluid dynamics

Geospatial tools

Libraries

Material scienc

Mathematical/Statistics

ML toolkit

NVIDIA

Physics

Programming languages

System

Text Editors

Tools/Utilities

Workflow automation

abaqus

Description

Abaqus is a software suite for Finite Element Analysis (FEA) developed by Dassault Systèmes.

Versions

  • Bell: 2019, 2020, 2021, 2022
  • Brown: 6.14-6, 2017, 2018, 2019, 2020, 2021, 2022
  • Scholar: 6.14-6, 2017, 2018, 2019, 2020, 2021, 2022
  • Gilbreth: 2017, 2018, 2019, 2020, 2021, 2022
  • Workbench: 6.14-6, 2017, 2018, 2019, 2020, 2021, 2022
  • Negishi: 2022

Module

You can load the modules by:

module load abaqus

amber

Description

AMBER (Assisted Model Building with Energy Refinement) is a package of molecular simulation programs.

Versions

  • Bell: 16
  • Brown: 16
  • Scholar: 16
  • Gilbreth: 16
  • Negishi: 20
  • Anvil: 20
  • Workbench: 16

Module

You can load the modules by:

module load amber

amdblis

Description

AMD Optimized BLIS. BLIS is a portable software framework for instantiating high-performance BLAS-like dense linear algebra libraries.

Versions

  • Anvil: 3.0

Module

You can load the modules by:

module load amdblis

amdfftw

Description

FFTW AMD Optimized version is a comprehensive collection of fast C routines for computing the Discrete Fourier Transform DFT and various special cases thereof.

Versions

  • Anvil: 3.0

Module

You can load the modules by:

module load amdfftw

amdlibflame

Description

libFLAME AMD Optimized version is a portable library for dense matrix computations, providing much of the functionality present in Linear Algebra Package LAPACK. It includes a compatibility layer, FLAPACK, which includes complete LAPACK implementation.

Versions

  • Anvil: 3.0

Module

You can load the modules by:

module load amdlibflame

amdlibm

Description

AMD LibM is a software library containing a collection of basic math functions optimized for x86-64 processor-based machines. It provides many routines from the list of standard C99 math functions. Applications can link into AMD LibM library and invoke math functions instead of compilers math functions for better accuracy and performance.

Versions

  • Anvil: 3.0

Module

You can load the modules by:

module load amdlibm

amdscalapack

Description

ScaLAPACK is a library of high-performance linear algebra routines for parallel distributed memory machines. It depends on external libraries including BLAS and LAPACK for Linear Algebra computations.

Versions

  • Anvil: 3.0

Module

You can load the modules by:

module load amdscalapack

anaconda

Description

Anaconda is a free and open-source distribution of the Python and R programming languages for scientific computing, that aims to simplify package management and deployment.

Versions

  • Bell: 2019.10-py27, 2020.02-py37, 2020.11-py38
  • Brown: 5.1.0-py27, 5.1.0-py36, 5.3.1-py27, 5.3.1-py37, 2019.10-py27, 2020.02-py37, 2020.11-py38
  • Scholar: 5.1.0-py27, 5.1.0-py36, 5.3.1-py27, 5.3.1-py37, 2019.10-py27, 2020.02-py37, 2020.11-py38
  • Gilbreth: 5.1.0-py27, 5.1.0-py36, 5.3.1-py27, 5.3.1-py37, 2019.10-py27, 2020.02-py37, 2020.11-py38
  • Negishi: 2021.05-py38, 2022.10-py39
  • Anvil: 2021.05-py38
  • Workbench: 5.1.0-py27, 5.1.0-py36, 5.3.1-py27, 5.3.1-py37, 2019.10-py27, 2020.02-py37, 2020.11-py38

Module

You can load the modules by:

module load anaconda

ansys

Description

Ansys is a CAE/multiphysics engineering simulation software that utilizes finite element analysis for numerically solving a wide variety of mechanical problems. The software contains a list of packages and can simulate many structural properties such as strength, toughness, elasticity, thermal expansion, fluid dynamics as well as acoustic and electromagnetic attributes.

Versions

  • Bell: 2019R3, 2020R1, 2021R2, 2022R1, 2022R2, 2023R1
  • Brown: 17.1, 18.2, 19.2, 2019R3, 2020R1, 2021R2, 2022R1
  • Scholar: 18.2, 2019R3, 2020R1, 2021R2, 2022R1
  • Gilbreth: 19.2, 2019R3, 2020R1, 2021R2, 2022R1
  • Workbench: 18.2, 2019R3, 2020R1, 2021R2, 2022R1
  • Negishi: 2022R2

Ansys Licensing

The Ansys licensing on our community clusters is maintained by Purdue ECN group. There are two types of licenses: teaching and research. For more information, please refer to ECN Ansys licensing page. If you are interested in purchasing your own research license, please send email to software@ecn.purdue.edu.

Ansys Workflow

Ansys software consists of several sub-packages such as Workbench and Fluent. Most simulations are performed using the Ansys Workbench console, a GUI interface to manage and edit the simulation workflow. It requires X11 forwarding for remote display so a SSH client software with X11 support or a remote desktop portal is required. Please see Logging In section for more details. To ensure preferred performance, ThinLinc remote desktop connection is highly recommended.

Typically users break down larger structures into small components in geometry with each of them modeled and tested individually. A user may start by defining the dimensions of an object, adding weight, pressure, temperature, and other physical properties.

Ansys Fluent is a computational fluid dynamics (CFD) simulation software known for its advanced physics modeling capabilities and accuracy. Fluent offers unparalleled analysis capabilities and provides all the tools needed to design and optimize new equipment and to troubleshoot existing installations.

In the following sections, we provide step-by-step instructions to lead you through the process of using Fluent. We will create a classical elbow pipe model and simulate the fluid dynamics when water flows through the pipe. The project files have been generated and can be downloaded via fluent_tutorial.zip.

Loading Ansys Module

Different versions of Ansys are installed on the clusters and can be listed with module spider or module avail command in the terminal.

$ module avail ansys/
---------------------- Core Applications -----------------------------
   ansys/2019R3    ansys/2020R1    ansys/2021R2    ansys/2022R1 (D)

Before launching Ansys Workbench, a specific version of Ansys module needs to be loaded. For example, you can module load ansys/2021R2 to use the latest Ansys 2021R2. If no version is specified, the default module -> (D) (ansys/2022R1 in this case) will be loaded. You can also check the loaded modules with module list command.

Launching Ansys Workbench

Open a terminal, enter rcac-runwb2 to launch Ansys Workbench.

You can also use runwb2 to launch Ansys Workbench. The main difference between runwb2and rcac-runwb2 is that the latter sets the project folder to be in your scratch space. Ansys has an known bug that it might crash when the project folder is set to $HOME on our systems.

ansysem

Description

This module enables the use of ANSYSEM, a popular Electromechanical application.

Versions

  • Bell: 2020r1, 2021r2
  • Brown: 19.2, 2020r1, 2021r2
  • Scholar: 2021r2
  • Workbench: 19.2, 2020r1, 2021r2

Module

You can load the modules by:

module load ansysem

aocl

Description

AOCL are a set of numerical libraries tuned specifically for AMD EPYCTM processor family. They have a simple interface to take advantage of the latest hardware innovations.

Versions

  • Bell: 2.1

Module

You can load the modules by:

module load aocl

arpack-ng

Description

ARPACK-NG is a collection of Fortran77 subroutines designed to solve large scale eigenvalue problems.

Versions

  • Negishi: 3.8.0
  • Anvil: 3.8.0

Module

You can load the modules by:

module load arpack-ng

aws-cli

Description

The AWS Command Line Interface CLI is a unified tool to manage your AWS services from command line.

Versions

  • Bell: 2.4.15
  • Brown: 2.4.15
  • Scholar: 2.4.15
  • Gilbreth: 2.4.15
  • Negishi: 2.9.7
  • Anvil: 2.4.15
  • Workbench: 2.4.15

Module

You can load the modules by:

module load aws-cli

bamtools

Description

C++ API & command-line toolkit for working with BAM data.

Versions

  • Anvil: 2.5.2

Commands

  • bamtools

Module

You can load the modules by:

module load bamtools

Example job

To run bamtools our our clusters:

#!/bin/bash
#SBATCH -A Allocation      # Allocation name 
#SBATCH -t 20:00:00
#SBATCH -N 1
#SBATCH -n 24
#SBATCH -p PartitionName 
#SBATCH --job-name=bamtools
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load bamtools

bamtools convert -format fastq -in in.bam -out out.fastq

bbftp

Description

bbFTP is a file transfer software. It implements its own transfer protocol, which is optimized for large files (larger than 2GB) and secure as it does not read the password in a file and encrypts the connection information.

Versions

  • Bell: 3.2.1

Module

You can load the modules by:

module load bbftp

beagle

Description

Beagle is a software package for phasing genotypes and for imputing ungenotyped markers.

Versions

  • Anvil: 5.1

Commands

  • beagle

Module

You can load the modules by:

module load beagle

Example job

To run Beagle on our clusters:

#!/bin/bash
#SBATCH -A myAllocation 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=beagle
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load beagle

beagle gt=test.vcf.gz out=test.out

beast2

Description

BEAST is a cross-platform program for Bayesian inference using MCMC of molecular sequences. It is entirely orientated towards rooted, time-measured phylogenies inferred using strict or relaxed molecular clock models. It can be used as a method of reconstructing phylogenies but is also a framework for testing evolutionary hypotheses without conditioning on a single tree topology.

Versions

  • Anvil: 2.6.4

Commands

  • applauncher
  • beast
  • beauti
  • densitree
  • loganalyser
  • logcombiner
  • packagemanager
  • treeannotator

Module

You can load the modules by:

module load beast2

Example job

To run BEAST 2 on our clusters:

#!/bin/bash
#SBATCH -A myQueue     
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 4
#SBATCH --job-name=beast2
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load beast2

beast -threads 4 -prefix input input.xml

bismark

Description

A tool to map bisulfite converted sequence reads and determine cytosine methylation states

Versions

  • Anvil: 0.23.0

Commands

  • bam2nuc
  • bismark
  • bismark2bedGraph
  • bismark2report
  • bismark2summary
  • bismark_genome_preparation
  • bismark_methylation_extractor
  • coverage2cytosine
  • deduplicate_bismark
  • filter_non_conversion
  • NOMe_filtering

Module

You can load the modules by:

module load bismark

Example job

To run Bismark on our clusters:

#!/bin/bash
#SBATCH -A myAllocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 12
#SBATCH --job-name=bismark
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load bismark

bismark_genome_preparation --bowtie2 data/ref_genome

bismark --multicore 12 --genome data/ref_genome seq.fastq

blast-plus

Description

Basic Local Alignment Search Tool. BLAST finds regions of similarity between biological sequences. The program compares nucleotide or protein sequences to sequence databases and calculates the statistical significance.

Versions

  • Anvil: 2.12.0

Module

You can load the modules by:

module load blast-plus

BLAST Databases

Local copies of the blast dabase can be found in the directory /anvil/datasets/ncbi/blast/latest. The environment varialbe BLASTDB was also set as /anvil/datasets/ncbi/blast/latest. If users want to use cdd_delta, env_nr, env_nt, nr, nt, pataa, patnt, pdbnt, refseq_protein, refseq_rna, swissprot, or tsa_nt databases, do not need to provide the database path. Instead, just use the format like this -db nr.

Example job

To run bamtools our our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 20:00:00
#SBATCH -N 1
#SBATCH -n 24
#SBATCH -p PartitionName 
#SBATCH --job-name=blast
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load blast-plus

blastp -query protein.fasta -db nr -out test_out -num_threads 4

blis

Description

BLIS is a portable software framework for instantiating high-performance BLAS-like dense linear algebra libraries.

Versions

  • Negishi: 0.9.0
  • Anvil: 0.8.1

Module

You can load the modules by:

module load blis

boost

Description

Boost provides free peer-reviewed portable C++ source libraries, emphasizing libraries that work well with the C++ Standard Library.

Versions

  • Bell: 1.68.0, 1.70.0
  • Brown: 1.64.0, 1.66.0, 1.70.0
  • Scholar: 1.64.0, 1.66.0, 1.70.0
  • Gilbreth: 1.66.0, 1.70.0
  • Negishi: 1.80.0
  • Anvil: 1.74.0
  • Workbench: 1.64.0, 1.66.0, 1.70.0

Module

You can load the modules by:

module load boost

bowtie2

Description

Bowtie 2 is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences

Versions

  • Anvil: 2.4.2

Module

You can load the modules by:

module load bowtie2

Example job

To run Bowtie 2 on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=bowtie2
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load bowtie2

bowtie2-build ref.fasta ref
bowtie2 -p 4 -x ref -1 input_1.fq -2 input_2.fq -S test.sam

bwa

Description

Burrow-Wheeler Aligner for pairwise alignment between DNA sequences.

Versions

  • Anvil: 0.7.17

Module

You can load the modules by:

module load bwa

Example job

To run BWA on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=bwa
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load bwa

bwa index ref.fasta
bwa mem ref.fasta input.fq > test.sam

bzip2

Description

bzip2 is a freely available, patent free high-quality data compressor. It typically compresses files to within 10% to 15% of the best available techniques the PPM family of statistical compressors, whilst being around twice as fast at compression and six times faster at decompression.

Versions

  • Negishi: 1.0.8

Module

You can load the modules by:

module load bzip2

caffe

Description

Caffe is a deep learning framework made with expression, speed, and modularity in mind.

Versions

  • Bell: 1.0
  • Gilbreth: 1.0.0

Module

You can load the modules by:

module load learning
module load caffe

cdo

Description

CDO is a collection of command line Operators to manipulate and analyse Climate and NWP model Data.

Versions

  • Bell: 1.9.5
  • Brown: 1.9.5
  • Scholar: 1.9.5
  • Gilbreth: 1.9.5
  • Negishi: 1.9.9
  • Anvil: 1.9.9
  • Workbench: 1.9.5

Module

You can load the modules by:

module load cdo

cmake

Description

A cross-platform, open-source build system. CMake is a family of tools designed to build, test and package software.

Versions

  • Bell: 3.18.2, 3.20.6
  • Brown: 3.15.4, 3.20.6
  • Scholar: 3.15.4, 3.20.6
  • Gilbreth: 3.15.4, 3.20.6
  • Negishi: 3.23.1, 3.24.3
  • Anvil: 3.20.0
  • Workbench: 3.15.4, 3.20.6

Module

You can load the modules by:

module load cmake

cntk

Description

The Microsoft Cognitive Toolkit is a unified deep-learning toolkit that describes neural networks as a series of computational steps via a directed graph.

Versions

  • Gilbreth: 2.6

Module

You can load the modules by:

module load learning
module load cntk

comsol

Description

Comsol Multiphysics (previously named Femlab) is a modeling package for the simulation of any physical process you can describe with partial differential equations (PDEs).

Versions

  • Bell: 5.3a, 5.4, 5.5_b359, 5.6, 6.0, 6.1
  • Brown: 5.3a, 5.4, 5.5_b359, 5.6, 6.0, 6.1
  • Scholar: 5.3a
  • Negishi: 6.1
  • Workbench: 5.3a, 5.4, 6.0, 6.1

Module

You can load the modules by:

module load comsol

cp2k

Description

CP2K is a quantum chemistry and solid state physics software package that can perform atomistic simulations of solid state, liquid, molecular, periodic, material, crystal, and biological systems

Versions

  • Negishi: 2022.1
  • Anvil: 8.2

Module

You can load the modules by:

module load cp2k

cplex

Description

IBM ILOG CPLEX Optimizer's mathematical programming technology enables decision optimization for improving efficiency, reducing costs, and increasing profitability.

Versions

  • Bell: 12.8.0
  • Brown: 12.8.0

Module

You can load the modules by:

module load cplex

cuda

Description

CUDA is a parallel computing platform and programming model invented by NVIDIA. It enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU).

Versions

  • Scholar: 9.0.176, 10.2.89, 11.2.2, 11.8.0
  • Gilbreth: 8.0.61, 9.0.176, 10.0.130, 10.2.89, 11.0.3, 11.2.0, 11.7.0
  • Anvil: 11.0.3, 11.2.2, 11.4.2

Module

You can load the modules by:

module load cuda

Monitor Activity and Drivers

Users can check the available GPUs, their current usage, installed version of the nvidia drivers, and running processes with the command nvidia-smi. The output should look something like this:
 

User@gilbreth-fe00:~/cuda $ nvidia-smi
Sat May 27 23:26:14 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.48.07    Driver Version: 515.48.07    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A30          Off  | 00000000:21:00.0 Off |                    0 |
| N/A   29C    P0    29W / 165W |  19802MiB / 24576MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     29152      C   python                           9107MiB |
|    0   N/A  N/A     53947      C   ...020.11-py38/GP/bin/python     2611MiB |
|    0   N/A  N/A     71769      C   ...020.11-py38/GP/bin/python     1241MiB |
|    0   N/A  N/A     72821      C   ...8/TorchGPU_env/bin/python     2657MiB |
|    0   N/A  N/A     91986      C   ...2-4/internal/bin/gdesmond      931MiB |
+-----------------------------------------------------------------------------+

We can see that the node gilbreth-fe00 is running driver version 515.48.07 and is compatible with CUDA version 11.7. We do not recommend users to run jobs on front end nodes, but here we can see there are three python processes and one gdesmond process. 

Compile a CUDA code

The below vectorAdd.cu is modified from the textbook Learn CUDA Programming. 

#include<stdio.h>
#include<stdlib.h>

#define N 512

void host_add(int *a, int *b, int *c) {
	for(int idx=0;idx<N;idx++)
		c[idx] = a[idx] + b[idx];
}

//basically just fills the array with index.
void fill_array(int *data) {
	for(int idx=0;idx<N;idx++)
		data[idx] = idx;
}

void print_output(int *a, int *b, int*c) {
	for(int idx=0;idx<N;idx++)
		printf("\n %d + %d  = %d",  a[idx] , b[idx], c[idx]);
}
int main(void) {
	int *a, *b, *c;
	int size = N * sizeof(int);

	// Alloc space for host copies of a, b, c and setup input values
	a = (int *)malloc(size); fill_array(a);
	b = (int *)malloc(size); fill_array(b);
	c = (int *)malloc(size);

	host_add(a,b,c);

	print_output(a,b,c);

	free(a); free(b); free(c);


	return 0;
}

We can compile the CUDA code by the CUDA nvcc compiler:

nvcc -o vector_addition vector_addition.cu

Example job script

#!/bin/bash

#SBATCH -A XXX
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=20
#SBATCH --cpus-per-task=1
#SBATCH --gpus-per-node=1
#SBATCH --time 1:00:00

module purge
module load gcc/XXX
module load cuda/XXX

#compile the vector_addition.cu file
nvcc -o vector_addition vector_addition.cu

#runs the vector_addition program
./vector_addition

 

cudnn

Description

cuDNN is a deep neural network library from Nvidia that provides a highly tuned implementation of many functions commonly used in deep machine learning applications.

Versions

  • Scholar: cuda-9.0_7.4, cuda-10.2_8.0, cuda-11.2_8.1.1, cuda-11.8_8.6.0
  • Gilbreth: cuda-8.0_6.0, cuda-8.0_7.1, cuda-9.0_7.3, cuda-9.0_7.4, cuda-10.0_7.5, cuda-10.2_8.0, cuda-11.0_8.0, cuda-11.2_8.1, cuda-11.7_8.6
  • Anvil: cuda-11.0_8.0, cuda-11.2_8.1, cuda-11.4_8.2

Module

You can load the modules by:

module load cudnn

cue-login-env

Description

XSEDE Common User Environment Variables for Anvil. Load this module to have XSEDE Common User Environment variables defined for your shell session or job on Anvil. See detailed description at https://www.ideals.illinois.edu/bitstream/handle/2142/75910/XSEDE-CUE-Variable-Definitions-v1.1.pdf

Versions

  • Anvil: 1.1

Module

You can load the modules by:

module load cue-login-env

curl

Description

cURL is an open source command line tool and library for transferring data with URL syntax

Versions

  • Bell: 7.63.0, 7.79.0
  • Brown: 7.63.0, 7.79.0
  • Scholar: 7.63.0, 7.79.0
  • Gilbreth: 7.79.0
  • Negishi: 7.78.0, 7.85.0
  • Anvil: 7.76.1
  • Workbench: 7.63.0, 7.79.0

Module

You can load the modules by:

module load curl

cutadapt

Description

Cutadapt finds and removes adapter sequences, primers, poly-A tails and other types of unwanted sequence from your high-throughput sequencing reads.

Versions

  • Anvil: 2.10

Module

You can load the modules by:

module load cutadapt

Example job

To run Cutadapt on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=cutadapt
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load cutadapt

cutadapt -a AACCGGTT -o output.fastq input.fastq

eigen

Description

Eigen is a C++ template library for linear algebra matrices, vectors, numerical solvers, and related algorithms.

Versions

  • Negishi: 3.3.9
  • Anvil: 3.3.9

Module

You can load the modules by:

module load eigen

emacs

Description

The Emacs programmable text editor.

Versions

  • Negishi: 28.2
  • Anvil: 27.2

Module

You can load the modules by:

module load emacs

envi

Description

ENVI is the premier software solution for processing and analyzing geospatial imagery used by scientists, researchers, image analysts, and GIS professionals around the world.

Versions

  • Bell: 5.5.2
  • Brown: 5.5.2
  • Scholar: 5.5.2
  • Gilbreth: 5.5.2
  • Workbench: 5.5.2

Module

You can load the modules by:

module load envi

fastqc

Description

A quality control tool for high throughput sequence data.

Versions

  • Anvil: 0.11.9

Module

You can load the modules by:

module load fastqc

Example job

To run Fastqc on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 4
#SBATCH --job-name=fastqc
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load fastqc

fastqc -o fastqc_out -t 4 FASTQ1 FASTQ2

fasttree

Description

FastTree infers approximately-maximum-likelihood phylogenetic trees from alignments of nucleotide or protein sequences. FastTree can handle alignments with up to a million of sequences in a reasonable amount of time and memory.

Versions

  • Anvil: 2.1.10

Module

You can load the modules by:

module load fasttree

Example job using single CPU

To run FastTree on our our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 20:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=fasttree
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load fasttree

FastTree alignmentfile > treefile

Example job using multiple CPUs

To run FastTree on our our clusters using multiple CPUs:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 20:00:00
#SBATCH -N 1
#SBATCH -n 24
#SBATCH --job-name=FastTreeMP
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load fasttree

export OMP_NUM_THREADS=24

FastTreeMP alignmentfile > treefile

fastx-toolkit

fastx-toolkit

Description

The FASTX-Toolkit is a collection of command line tools for Short-Reads FASTA/FASTQ files preprocessing.

Versions

  • Anvil: 0.0.14

Commands

  • fasta_clipping_histogram.pl
  • fasta_formatter
  • fasta_nucleotide_changer
  • fastq_masker
  • fastq_quality_boxplot_graph.sh
  • fastq_quality_converter
  • fastq_quality_filter
  • fastq_quality_trimmer
  • fastq_to_fasta
  • fastx_artifacts_filter
  • fastx_barcode_splitter.pl
  • fastx_clipper
  • fastx_collapser
  • fastx_nucleotide_distribution_graph.sh
  • fastx_nucleotide_distribution_line_graph.sh
  • fastx_quality_stats
  • fastx_renamer
  • fastx_reverse_complement
  • fastx_trimmer
  • fastx_uncollapser

Module

You can load the modules by:

module load fastx-toolkit

Example job

To run FASTX-Toolkit on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=fastx_toolkit
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load fastx_toolkit

ffmpeg

Description

FFmpeg is a complete, cross-platform solution to record, convert and stream audio and video.

Versions

  • Bell: 4.2.2
  • Brown: 4.2.1
  • Scholar: 4.2.1
  • Gilbreth: 4.2.1
  • Negishi: 4.4.1
  • Anvil: 4.2.2
  • Workbench: 4.2.1

Module

You can load the modules by:

module load ffmpeg

fftw

Description

FFTW is a C subroutine library for computing the discrete Fourier transform DFT in one or more dimensions, of arbitrary input size, and of both real and complex data as well as of even/odd data, i.e. the discrete cosine/sine transforms or DCT/DST. We believe that FFTW, which is free software, should become the FFT library of choice for most applications.

Versions

  • Bell: 3.3.8
  • Brown: 3.3.4, 3.3.7
  • Scholar: 3.3.4, 3.3.7
  • Gilbreth: 3.3.7
  • Negishi: 2.1.5, 3.3.10
  • Anvil: 2.1.5, 3.3.8
  • Workbench: 3.3.4, 3.3.7

Module

You can load the modules by:

module load fftw

gamess

Description

The General Atomic and Molecular Electronic Structure System (GAMESS) is a general ab initio quantum chemistry package.

Versions

  • Bell: 18.Aug.2016.R1, 30.Jun.2019.R1
  • Brown: 18.Aug.2016.R1, 30.Jun.2019.R1
  • Scholar: 18.Aug.2016.R1
  • Gilbreth: 30.Jun.2019
  • Workbench: 18.Aug.2016.R1

Module

You can load the modules by:

module load gamess

gams

Description

The General Algebraic Modeling System is a high-level modeling system for mathematical optimization. GAMS is designed for modeling and solving linear, nonlinear, and mixed-integer optimization problems.

Versions

  • Workbench: 25.1.1

Module

You can load the modules by:

module load gams

gatk

Description

Genome Analysis Toolkit Variant Discovery in High-Throughput Sequencing Data

Versions

  • Anvil: 4.1.8.1

Commands

  • gatk

Module

You can load the modules by:

module load gatk

Example job

To run gatk our our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 20:00:00
#SBATCH -N 1
#SBATCH -n 24
#SBATCH -p PartitionName 
#SBATCH --job-name=gatk
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load gatk

gatk  --java-options "-Xmx12G -XX:ParallelGCThreads=24" HaplotypeCaller -R hg38.fa -I 19P0126636WES.sorted.bam  -O 19P0126636WES.HC.vcf --sample-name 19P0126636

gaussian09

Description

Gaussian is a general purpose computational chemistry software package initially released in 1970. It utilizes fundamental laws of quantum mechanics to predict energies, molecular structures, spectroscopic data (NMR, IR, UV) and much more advanced calculations. It provides state-of-the-art capabilities for electronic structure modeling.

Versions

  • Bell: E.01
  • Brown: E.01
  • Scholar: E.01
  • Workbench: E.01

Module

You can load the modules by:

module load gaussian09

gaussian16

Description

Gaussian is a general purpose computational chemistry software package initially released in 1970. It utilizes fundamental laws of quantum mechanics to predict energies, molecular structures, spectroscopic data (NMR, IR, UV) and much more advanced calculations. It provides state-of-the-art capabilities for electronic structure modeling.

Versions

  • Bell: B.01
  • Brown: A.03, B.01
  • Scholar: A.03, B.01
  • Gilbreth: A.03, B.01-gpu
  • Negishi: B.01
  • Workbench: A.03, B.01

Module

You can load the modules by:

module load gaussian16

gaussview

Description

GaussView is a graphical interface used with Gaussian. It aids in the creation of Gaussian input files, enables the user to run Gaussian calculations from a graphical interface without the need for using a command line instruction, and helps in the interpretation of Gaussian output (e.g., you can use it to plot properties, animate vibrations, visualize computed spectra, etc.).

Versions

  • Bell: 5.0.8, 6.0.16
  • Brown: 5.0.8, 6.0.16
  • Scholar: 5.0.8, 6.0.16
  • Gilbreth: 6.0.16
  • Negishi: 6.0.16
  • Workbench: 5.0.8, 6.0.16

Module

You can load the modules by:

module load gaussview

gdal

Description

GDAL Geospatial Data Abstraction Library is a translator library for raster and vector geospatial data formats that is released under an X/MIT style Open Source license by the Open Source Geospatial Foundation. As a library, it presents a single raster abstract data model and vector abstract data model to the calling application for all supported formats. It also comes with a variety of useful command line utilities for data translation and processing.

Versions

  • Bell: 2.4.2, 3.4.2, 3.5.3, 3.5.3_sqlite3
  • Brown: 2.4.2, 3.4.2, 3.5.3
  • Scholar: 2.4.2, 3.4.2, 3.5.3
  • Gilbreth: 2.4.2, 3.5.3, 3.5.3-grib
  • Negishi: 2.4.4, 3.5.3
  • Anvil: 2.4.4, 3.2.0
  • Workbench: 2.4.2, 3.4.2, 3.5.3

Module

You can load the modules by:

module load gdal

gdb

Description

GDB, the GNU Project debugger, allows you to see what is going on inside another program while it executes -- or what another program was doing at the moment it crashed.

Versions

  • Bell: 11.1
  • Negishi: 11.1, 12.1
  • Anvil: 11.1

Module

You can load the modules by:

module load gdb

geos

Description

GEOS Geometry Engine - Open Source is a C++ port of the Java Topology Suite JTS. As such, it aims to contain the complete functionality of JTS in C++. This includes all the OpenGIS Simple Features for SQL spatial predicate functions and spatial operators, as well as specific JTS enhanced topology functions.

Versions

  • Bell: 3.8.1, 3.9.4
  • Brown: 3.7.2, 3.9.4
  • Scholar: 3.7.2, 3.9.4
  • Gilbreth: 3.7.2, 3.9.4
  • Negishi: 3.9.1
  • Anvil: 3.8.1, 3.9.1
  • Workbench: 3.7.2, 3.9.4

Module

You can load the modules by:

module load geos

gmp

Description

GMP is a free library for arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating-point numbers.

Versions

  • Bell: 6.1.2
  • Brown: 6.1.2
  • Scholar: 6.1.2
  • Gilbreth: 6.1.2
  • Negishi: 6.2.1
  • Anvil: 6.2.1
  • Workbench: 6.1.2

Module

You can load the modules by:

module load gmp

gmt

Description

GMT Generic Mapping Tools is an open source collection of about 80 command-line tools for manipulating geographic and Cartesian data sets including filtering, trend fitting, gridding, projecting, etc. and producing PostScript illustrations ranging from simple x-y plots via contour maps to artificially illuminated surfaces and 3D perspective views.

Versions

  • Bell: 5.4.4
  • Brown: 5.4.4
  • Scholar: 5.4.4
  • Gilbreth: 5.4.4
  • Anvil: 6.1.0
  • Workbench: 5.4.4
  • Negishi: 6.2.0

Module

You can load the modules by:

module load gmt

gnuplot

Description

Gnuplot is a portable command-line driven graphing utility for Linux, OS/2, MS Windows, OSX, VMS, and many other platforms. The source code is copyrighted but freely distributed i.e., you don't have to pay for it. It was originally created to allow scientists and students to visualize mathematical functions and data interactively, but has grown to support many non-interactive uses such as web scripting. It is also used as a plotting engine by third-party applications like Octave. Gnuplot has been supported and under active development since 1986

Versions

  • Bell: 5.2.8
  • Brown: 5.2.7
  • Scholar: 5.2.7
  • Gilbreth: 5.2.7
  • Negishi: 5.4.2
  • Anvil: 5.4.2
  • Workbench: 5.2.7

Module

You can load the modules by:

module load gnuplot

gpaw

Description

GPAW is a density-functional theory DFT Python code based on the projector-augmented wave PAW method and the atomic simulation environment ASE.

Versions

  • Anvil: 21.1.0

Module

You can load the modules by:

module load gpaw

grads

Description

The Grid Analysis and Display System (GrADS) is an interactive desktop tool that is used for easy access, manipulation, and visualization of earth science data. GrADS has two data models for handling gridded and station data. GrADS supports many data file formats, including binary (stream or sequential), GRIB (version 1 and 2), NetCDF, HDF (version 4 and 5), and BUFR (for station data).

Versions

  • Bell: 2.2.1
  • Brown: 2.2.1
  • Scholar: 2.2.1
  • Gilbreth: 2.2.1
  • Negishi: 2.2.1
  • Anvil: 2.2.1
  • Workbench: 2.2.1

Module

You can load the modules by:

module load grads

gromacs

Description

GROMACS GROningen MAchine for Chemical Simulations is a molecular dynamics package primarily designed for simulations of proteins, lipids and nucleic acids. It was originally developed in the Biophysical Chemistry department of University of Groningen, and is now maintained by contributors in universities and research centers across the world.

Versions

  • Bell: 2018.4, 2019.2
  • Brown: 2018.4, 2019.2
  • Scholar: 2018.4, 2019.2
  • Gilbreth: 2018.4
  • Negishi: 2022.3
  • Anvil: 2021.2

Module

You can check available gromacs version by:

module spider gromacs

You can check how to load the gromacs module by the module's full name:

module spider gromacs/XXXX

Note: RCAC also installed some containerized gromacs modules.
To use these containerized modules, please following the instructions in the output of "module spider gromacs/XXXX"

You can load the modules by:

module load gromacs # for default version
module load gromacs/XXXX # for specific version

Usage

The GROMACS executable is gmx_mpi and you can use gmx help commands for help on a command.

For more details about how to run GROMACS, please check GROMACS.

Example job

#!/bin/bash
# FILENAME:  myjobsubmissionfile

#SBATCH --nodes=2       # Total # of nodes 
#SBATCH --ntasks=256    # Total # of MPI tasks
#SBATCH --time=1:30:00  # Total run time limit (hh:mm:ss)
#SBATCH -J myjobname    # Job name
#SBATCH -o myjob.o%j    # Name of stdout output file
#SBATCH -e myjob.e%j    # Name of stderr error file

# Manage processing environment, load compilers and applications.
module purge
module load gcc/XXXX openmpi/XXXX # or module load intel/XXXX impi/XXXX | depends on the output of "module spider gromacs/XXXX"
module load gromacs/XXXX
module list

# Launch MPI code
gmx_mpi pdb2gmx -f my.pdb -o my_processed.gro -water spce
gmx_mpi grompp -f my.mdp -c my_processed.gro -p topol.top -o topol.tpr
srun -n $SLURM_NTASKS gmx_mpi mdrun -s topol.tpr

Note

Using mpirun -np $SLURM_NTASKS gmx_mpi or mpiexex -np $SLURM_NTASKS gmx_mpi may not work for non-exclusive jobs on some clusters. Use srun -n $SLURM_NTASKS gmx_mpi or mpirun gmx_mpi instead. mpirun gmx_mpi without specifying the number of ranks will automatically pick up the number of SLURM_NTASKS and works fine.

gsl

Description

The GNU Scientific Library GSL is a numerical library for C and C++ programmers. It is free software under the GNU General Public License. The library provides a wide range of mathematical routines such as random number generators, special functions and least-squares fitting. There are over 1000 functions in total with an extensive test suite.

Versions

  • Bell: 2.4
  • Brown: 2.4
  • Scholar: 2.4
  • Gilbreth: 2.4
  • Negishi: 2.4
  • Anvil: 2.4
  • Workbench: 2.4

Module

You can load the modules by:

module load gsl

gurobi

Description

The Gurobi Optimizer was designed from the ground up to be the fastest, most powerful solver available for your LP, QP, QCP, and MIP MILP, MIQP, and MIQCP problems.

Versions

  • Bell: 9.0.1, 9.5.1, 10.0.1
  • Brown: 9.0.1, 9.5.1, 10.0.1
  • Scholar: 9.0.1
  • Anvil: 9.5.1
  • Workbench: 7.5.2, 9.0.1
  • Negishi: 10.0.1

Module

You can load the modules by:

module load gurobi

gym

Description

The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents.

Versions

  • Bell: 0.17.3
  • Gilbreth: 0.18.0

Module

You can load the modules by:

module load learning
module load gym

hadoop

Description

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models.

Versions

  • Bell: 2.7.7
  • Brown: 2.7.7
  • Scholar: 2.7.7
  • Gilbreth: 2.7.7
  • Negishi: 3.3.2
  • Anvil: 3.3.0
  • Workbench: 2.7.7

Module

You can load the modules by:

module load hadoop

hdf

Description

HDF4 also known as HDF is a library and multi-object file format for storing and managing data between machines.

Versions

  • Bell: 4.2.15
  • Brown: 4.2.14
  • Scholar: 4.2.14
  • Gilbreth: 4.2.14
  • Negishi: 4.2.15
  • Anvil: 4.2.15
  • Workbench: 4.2.14

Module

You can load the modules by:

module load hdf

hdf5

Description

HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data.

Versions

  • Bell: 1.8.21, 1.10.6
  • Brown: 1.8.16, 1.10.5
  • Scholar: 1.8.16, 1.10.5
  • Gilbreth: 1.8.16, 1.10.5
  • Negishi: 1.13.2
  • Anvil: 1.10.7
  • Workbench: 1.8.16, 1.10.5

Module

You can load the modules by:

module load hdf5

hpctoolkit

Description

HPCToolkit is an integrated suite of tools for measurement and analysis of program performance on computers ranging from multicore desktop systems to the nations largest supercomputers. By using statistical sampling of timers and hardware performance counters, HPCToolkit collects accurate measurements of a programs work, resource consumption, and inefficiency and attributes them to the full calling context in which they occur.

Versions

  • Negishi: 2022.05.15
  • Anvil: 2021.03.01

Module

You can load the modules by:

module load hpctoolkit

hspice

Description

Hspice is a device level circuit simulator. Hspice takes a spice file as input and produces output describing the requested simulation of the circuit. It can also produce output files to be used by the AWAVES post processor.

Versions

  • Bell: 2017.12, 2019.06, 2020.12
  • Brown: 2017.12, 2019.06, 2020.12
  • Workbench: 2017.12, 2019.06, 2020.12

Module

You can load the modules by:

module load hspice

htseq

Description

HTSeq is a Python package that provides infrastructure to process data from high-throughput sequencing assays.

Versions

  • Anvil: 0.11.2

Commands

  • htseq-count
  • htseq-qa

Module

You can load the modules by:

module load htseq

Example job

To run HTSeq on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=htseq
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load htseq

htseq-count input.bam ref.gtf > test.out

hwloc

Description

The Hardware Locality hwloc software project.

Versions

  • Anvil: 1.11.13

Module

You can load the modules by:

module load hwloc

hyper-shell

Description

Process shell commands over a distributed, asynchronous queue.

Documentation: https://hyper-shell.readthedocs.io/en/latest/ 

Versions

  • Bell: 1.8.3, 2.0.2
  • Brown: 1.8.3, 2.0.2
  • Scholar: 1.8.3, 2.0.2
  • Gilbreth: 1.8.3, 2.0.2
  • Negishi: 2.0.2, 2.1.0
  • Anvil: 2.0.2
  • Workbench: 1.8.3, 2.0.2

Module

You can load the modules by:

module load hyper-shell

hypre

Description

Hypre is a library of high performance preconditioners that features parallel multigrid methods for both structured and unstructured grid problems.

Versions

  • Bell: 2.18.1

Module

You can load the modules by:

module load hypre

idl

Description

IDL is a data analysis language that provides powerful, core visualization and analysis functionality, and capabilities that allow data analysts and developers to leverage IDL's power in multiple software environments.

Versions

  • Bell: 8.7
  • Brown: 8.7
  • Scholar: 8.7
  • Gilbreth: 8.7
  • Workbench: 8.7

Module

You can load the modules by:

module load idl

intel-mkl

Description

Intel Math Kernel Library (MKL) contains ScaLAPACK, LAPACK, Sparse Solver, BLAS, Sparse BLAS, CBLAS, GMP, FFTs, DFTs, VSL, VML, and Interval Arithmetic routines. MKL resides in the directory stored in the environment variable MKL_HOME, after loading a version of the Intel compiler with module.

Versions

  • Bell: 2017.1.132, 2019.5.281
  • Negishi: 2019.9.304
  • Anvil: 2019.5.281

Module

You can load the modules by:

module load intel-mkl

By using module load to load an Intel compiler your environment will have several variables set up to help link applications with MKL. Here are some example combinations of simplified linking options:

$ module load intel
$ echo $LINK_LAPACK
-L${MKL_HOME}/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread

$ echo $LINK_LAPACK95
-L${MKL_HOME}/lib/intel64 -lmkl_lapack95_lp64 -lmkl_blas95_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread

RCAC recommends you use the provided variables to define MKL linking options in your compiling procedures. The Intel compiler modules also provide two other environment variables, LINK_LAPACK_STATIC and LINK_LAPACK95_STATIC that you may use if you need to link MKL statically.

RCAC recommends that you use dynamic linking of libguide. If so, define LD_LIBRARY_PATH such that you are using the correct version of libguide at run time. If you use static linking of libguide, then:

  • If you use the Intel compilers, link in the libguide version that comes with the compiler (use the -openmp option).
  • If you do not use the Intel compilers, link in the libguide version that comes with the Intel MKL above.

Here are some more documentation from other sources on the Intel MKL:

intel-oneapi-mkl

Description

Intel oneAPI Math Kernel Library Intel oneMKL; formerly Intel Math Kernel Library or Intel MKL, is a library of optimized math routines for science, engineering, and financial applications. Core math functions include BLAS, LAPACK, ScaLAPACK, sparse solvers, fast Fourier transforms, and vector math.

Versions

  • Negishi: 2023.0.0

Module

You can load the modules by:

module load intel-oneapi-mkl

intel-oneapi-tbb

Description

Intel oneAPI Threading Building Blocks oneTBB is a flexible performance library that simplifies the work of adding parallelism to complex applications across accelerated architectures, even if you are not a threading expert.

Versions

  • Negishi: 2021.8.0

Module

You can load the modules by:

module load intel-oneapi-tbb

julia

Description

Julia is a flexible dynamic language, appropriate for scientific and numerical computing, with performance comparable to traditional statically-typed languages. One can write code in Julia that is nearly as fast as C. Julia features optional typing, multiple dispatch, and good performance, achieved using type inference and just-in-time (JIT) compilation, implemented using LLVM. It is multi-paradigm, combining features of imperative, functional, and object-oriented programming.

Versions

  • Bell: 1.7.1, 1.8.1
  • Brown: 1.7.1
  • Gilbreth: 1.7.1
  • Negishi: 1.8.5
  • Anvil: 1.6.2

Module

You can load the modules by:

module load julia

Package installation

Users do not have write permission to the default julia package installation destination. However, users can install packages into home directory under ~/.julia.

Users can side step this by explicitly defining where to put julia packages:

$ export JULIA_DEPOT_PATH=$HOME/.julia
$ julia -e 'using Pkg; Pkg.add("PackageName")'

jupyter

Description

Complete Jupyter Hub/Lab/Notebook environment.

Versions

  • Anvil: 2.0.0
  • Negishi: 3.1.1

Module

You can load the modules by:

module load jupyter

jupyterhub

Description

Complete Jupyter Hub/Lab/Notebook environment.

Versions

  • Bell: 2.0.0
  • Brown: 2.0.0
  • Scholar: 2.0.0
  • Gilbreth: 2.0.0

Module

You can load the modules by:

module load jupyterhub

keras

Description

Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow. It was developed with a focus on enabling fast experimentation.

Versions

  • Bell: 2.4.3
  • Gilbreth: 2.4.3

Module

You can load the modules by:

module load learning
module load keras

lammps

Description

LAMMPS is a classical molecular dynamics code with a focus on materials modelling. It’s an acronym for Large-scale Atomic/Molecular Massively Parallel Simulator.

LAMMPS has potentials for solid-state materials (metals, semiconductors) and soft matter (biomolecules, polymers) and coarse-grained or mesoscopic systems. It can be used to model atoms or, more generically, as a parallel particle simulator at the atomic, meso, or continuum scale.

Versions

  • Bell: 20200721, 20201029
  • Brown: 7Aug19, 31Mar17
  • Scholar: 31Mar17
  • Gilbreth: 20190807
  • Negishi: 20220623
  • Anvil: 20210310, 20210310-kokkos

Module

You can check available lammps version by:

module spider lammps

You can check how to load the lammps module by the module's full name:

module spider lammps/XXXX

You can load the modules by:

module load lammps # for default version
module load lammps/XXXX # for specific version

Usage

LAMMPS reads command lines from an input file like "in.file". The LAMMPS executable is lmp, to run the lammps input file, use the -in command:

lmp -in in.file

For more details about how to run LAMMPS, please check LAMMPS.

Example job

#!/bin/bash
# FILENAME:  myjobsubmissionfile

#SBATCH --nodes=2       # Total # of nodes 
#SBATCH --ntasks=256    # Total # of MPI tasks
#SBATCH --time=1:30:00  # Total run time limit (hh:mm:ss)
#SBATCH -J myjobname    # Job name
#SBATCH -o myjob.o%j    # Name of stdout output file
#SBATCH -e myjob.e%j    # Name of stderr error file

# Manage processing environment, load compilers and applications.
module purge
module load gcc/XXXX openmpi/XXXX # or module load intel/XXXX impi/XXXX | depends on the output of "module spider lammps/XXXX"
module load lammps/XXXX
module list

# Launch MPI code
srun -n $SLURM_NTASKS lmp

Note

Using mpirun -np $SLURM_NTASKS lmp or mpiexex -np $SLURM_NTASKS lmp may not work for non-exclusive jobs on some clusters. Use srun -n $SLURM_NTASKS lmp or mpirun lmp instead. mpirun lmp without specifying the number of ranks will automatically pick up the number of SLURM_NTASKS and works fine.

launcher

Description

Framework for running large collections of serial or multi-threaded applications

Versions

  • Negishi: 3.9

Module

You can load the modules by:

module load launcher

learning

Description

The learning module loads the prerequisites (such as anaconda and cudnn ) and makes ML applications visible to the user

Versions

  • Bell: conda-2020.11-py38-cpu
  • Brown: conda-5.1.0-py27-cpu, conda-5.1.0-py36-cpu
  • Scholar: conda-5.1.0-py27-cpu, conda-5.1.0-py27-gpu, conda-5.1.0-py36-cpu, conda-5.1.0-py36-gpu
  • Gilbreth: conda-5.1.0-py27-cpu, conda-5.1.0-py27-gpu, conda-5.1.0-py36-cpu, conda-5.1.0-py36-gpu, conda-2020.11-py38-cpu, conda-2020.11-py38-gpu
  • Anvil: conda-2021.05-py38-gpu
  • Workbench: conda-5.1.0-py27-cpu, conda-5.1.0-py36-cpu

Module

You can load the modules by:

module load learning

Example job

This is the example jobscript for our cluster `Gilbreth`:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 20:00:00
#SBATCH -N 1
#SBATCH -n 10
#SBATCH --gpus-per-node=1 
#SBATCH -p PartitionName 
#SBATCH --job-name=learning
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out


module --force purge
module load learning/conda-2020.11-py38-gpu
module load ml-toolkit-gpu/pytorch/1.7.1

python torch.py

libfabric

Description

The Open Fabrics Interfaces OFI is a framework focused on exporting fabric communication services to applications.

Versions

  • Negishi: 1.12.0
  • Anvil: 1.12.0

Module

You can load the modules by:

module load libfabric

libflame

Description

libflame is a portable library for dense matrix computations, providing much of the functionality present in LAPACK, developed by current and former members of the Science of High-Performance Computing SHPC group in the Institute for Computational Engineering and Sciences at The University of Texas at Austin. libflame includes a compatibility layer, lapack2flame, which includes a complete LAPACK implementation.

Versions

  • Negishi: 5.2.0
  • Anvil: 5.2.0

Module

You can load the modules by:

module load libflame

libiconv

Description

GNU libiconv provides an implementation of the iconv function and the iconv program for character set conversion.

Versions

  • Bell: 1.16
  • Brown: 1.16
  • Scholar: 1.16
  • Gilbreth: 1.16
  • Negishi: 1.16
  • Anvil: 1.16

Module

You can load the modules by:

module load libiconv

libmesh

Description

The libMesh library provides a framework for the numerical simulation of partial differential equations using arbitrary unstructured discretizations on serial and parallel platforms.

Versions

  • Negishi: 1.7.1
  • Anvil: 1.6.2

Module

You can load the modules by:

module load libmesh

libszip

Description

Szip is an implementation of the extended-Rice lossless compression algorithm.

Versions

  • Bell: 2.1.1
  • Negishi: 2.1.1
  • Anvil: 2.1.1

Module

You can load the modules by:

module load libszip

libtiff

Description

LibTIFF - Tag Image File Format TIFF Library and Utilities.

Versions

  • Bell: 4.0.10
  • Brown: 4.0.10
  • Scholar: 4.0.10
  • Gilbreth: 4.0.10
  • Negishi: 4.4.0
  • Anvil: 4.1.0
  • Workbench: 4.0.10

Module

You can load the modules by:

module load libtiff

libv8

Description

Distributes the V8 JavaScript engine in binary and source forms in order to support fast builds of The Ruby Racer

Versions

  • Bell: 3.14
  • Brown: 3.14
  • Scholar: 3.14
  • Anvil: 6.7.17
  • Workbench: 3.14

Module

You can load the modules by:

module load libv8

libx11

Description

Xlib − C Language X Interface is a reference guide to the low-level C language interface to the X Window System protocol. It is neither a tutorial nor a user’s guide to programming the X Window System. Rather, it provides a detailed description of each function in the library as well as a discussion of the related background information.

Versions

  • Anvil: 1.7.0

Module

You can load the modules by:

module load libx11

libxml2

Description

Libxml2 is the XML C parser and toolkit developed for the Gnome project but usable outside of the Gnome platform, it is free software available under the MIT License.

Versions

  • Bell: 2.9.9
  • Brown: 2.9.9
  • Scholar: 2.9.9
  • Gilbreth: 2.9.9
  • Negishi: 2.10.1
  • Anvil: 2.9.10

Module

You can load the modules by:

module load libxml2

mathematica

Description

Mathematica is a technical computing environment and programming language with strong symbolic and numerical abilities.

Versions

  • Bell: 11.3, 12.1, 12.3, 13.1
  • Brown: 9.0, 11.3, 12.1, 12.3, 13.1
  • Scholar: 12.3, 13.1
  • Gilbreth: 11.3, 12.1, 12.3, 13.1
  • Negishi: 13.1
  • Workbench: 11.3, 12.1, 12.3, 13.1

Module

You can load the modules by:

module load mathematica

Running Mathematica

Users can run Mathematica GUI in interactive jobs or run it as batch jobs.

Interactive jobs

sinteractive -N1 -n24 -t4:00:00 -A standby
module load mathematica
Mathematica

Batch job

To submit a sbatch job on our clusters:

#!/bin/bash
#SBATCH -A XXX
#SBATCH -t 10:00:00
#SBATCH -N 1
#SBATCH -n 24
#SBATCH --job-name=mathematica
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load mathematica

math -noprompt < input.m

matlab

Description

MATLAB MATrix LABoratory is a multi-paradigm numerical computing environment and fourth-generation programming language. A proprietary programming language developed by MathWorks, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, C#, Java, Fortran and Python.

Versions

  • Bell: R2019a, R2020a, R2020b, R2021b, R2022a
  • Brown: R2017a, R2018a, R2019a, R2020a, R2020b, R2021b, R2022a
  • Scholar: R2017a, R2018a, R2019a, R2020a, R2022a
  • Gilbreth: R2017a, R2018a, R2019a, R2020a, R2022a
  • Negishi: R2021b, R2022a
  • Anvil: R2020b, R2021b, R2022a
  • Workbench: R2017a, R2018a, R2019a, R2020a, R2022a

Module

You can load the modules by:

module load matlab

MATLAB, Simulink, Compiler, and several of the optional toolboxes are available to faculty, staff, and students. To see the kind and quantity of all MATLAB licenses plus the number that you are currently using you can use the matlab_licenses command:

$ module load matlab
$ matlab_licenses

The MATLAB client can be run in the front-end for application development, however, computationally intensive jobs must be run on compute nodes.

The following sections provide several examples illustrating how to submit MATLAB jobs to a Linux compute cluster.

Matlab Script (.m File)

This section illustrates how to submit a small, serial, MATLAB program as a job to a batch queue. This MATLAB program prints the name of the run host and gets three random numbers.

Prepare a MATLAB script myscript.m, and a MATLAB function file myfunction.m:

% FILENAME:  myscript.m

% Display name of compute node which ran this job.
[c name] = system('hostname');
fprintf('\n\nhostname:%s\n', name);

% Display three random numbers.
A = rand(1,3);
fprintf('%f %f %f\n', A);

quit;

% FILENAME:  myfunction.m

function result = myfunction ()

    % Return name of compute node which ran this job.
    [c name] = system('hostname');
    result = sprintf('hostname:%s', name);

    % Return three random numbers.
    A = rand(1,3);
    r = sprintf('%f %f %f', A);
    result=strvcat(result,r);

end

Also, prepare a job submission file, here named myjob.sub. Run with the name of the script:

#!/bin/bash
# FILENAME:  myjob.sub

echo "myjob.sub"

# Load module, and set up environment for Matlab to run
module load matlab

unset DISPLAY

# -nodisplay:        run MATLAB in text mode; X11 server not needed
# -singleCompThread: turn off implicit parallelism
# -r:                read MATLAB program; use MATLAB JIT Accelerator
# Run Matlab, with the above options and specifying our .m file
matlab -nodisplay -singleCompThread -r myscript

myjob.sub

                            < M A T L A B (R) >
                  Copyright 1984-2011 The MathWorks, Inc.
                    R2011b (7.13.0.564) 64-bit (glnxa64)
                              August 13, 2011

To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

hostname:resource-a001.rcac.purdue.edu
0.814724 0.905792 0.126987

Output shows that a processor core on one compute node (resource-a001) processed the job. Output also displays the three random numbers.

For more information about MATLAB:

Implicit Parallelism

MATLAB implements implicit parallelism which is automatic multithreading of many computations, such as matrix multiplication, linear algebra, and performing the same operation on a set of numbers. This is different from the explicit parallelism of the Parallel Computing Toolbox.

MATLAB offers implicit parallelism in the form of thread-parallel enabled functions. Since these processor cores, or threads, share a common memory, many MATLAB functions contain multithreading potential. Vector operations, the particular application or algorithm, and the amount of computation (array size) contribute to the determination of whether a function runs serially or with multithreading.

When your job triggers implicit parallelism, it attempts to allocate its threads on all processor cores of the compute node on which the MATLAB client is running, including processor cores running other jobs. This competition can degrade the performance of all jobs running on the node.

When you know that you are coding a serial job but are unsure whether you are using thread-parallel enabled operations, run MATLAB with implicit parallelism turned off. Beginning with the R2009b, you can turn multithreading off by starting MATLAB with -singleCompThread:

$ matlab -nodisplay -singleCompThread -r mymatlabprogram

When you are using implicit parallelism, make sure you request exclusive access to a compute node, as MATLAB has no facility for sharing nodes.

For more information about MATLAB's implicit parallelism:

Profile Manager

MATLAB offers two kinds of profiles for parallel execution: the 'local' profile and user-defined cluster profiles. The 'local' profile runs a MATLAB job on the processor core(s) of the same compute node, or front-end, that is running the client. To run a MATLAB job on compute node(s) different from the node running the client, you must define a Cluster Profile using the Cluster Profile Manager.

To prepare a user-defined cluster profile, use the Cluster Profile Manager in the Parallel menu. This profile contains the scheduler details (queue, nodes, processors, walltime, etc.) of your job submission. Ultimately, your cluster profile will be an argument to MATLAB functions like batch().

For your convenience, a generic cluster profile is provided that can be downloaded: myslurmprofile.settings

Please note that modifications are very likely to be required to make myslurmprofile.settings work. You may need to change values for number of nodes, number of workers, walltime, and submission queue specified in the file. As well, the generic profile itself depends on the particular job scheduler on the cluster, so you may need to download or create two or more generic profiles under different names. Each time you run a job using a Cluster Profile, make sure the specific profile you are using is appropriate for the job and the cluster.

To import the profile, start a MATLAB session and select Manage Cluster Profiles... from the Parallel menu. In the Cluster Profile Manager, select Import, navigate to the folder containing the profile, select myslurmprofile.settings and click OK. Remember that the profile will need to be customized for your specific needs. If you have any questions, please contact us.

For detailed information about MATLAB's Parallel Computing Toolbox, examples, demos, and tutorials:

Parallel Computing Toolbox (parfor)

The MATLAB Parallel Computing Toolbox (PCT) extends the MATLAB language with high-level, parallel-processing features such as parallel for loops, parallel regions, message passing, distributed arrays, and parallel numerical methods. It offers a shared-memory computing environment running on the local cluster profile in addition to your MATLAB client. Moreover, the MATLAB Distributed Computing Server (DCS) scales PCT applications up to the limit of your DCS licenses.

This section illustrates the fine-grained parallelism of a parallel for loop (parfor) in a pool job.

The following examples illustrate a method for submitting a small, parallel, MATLAB program with a parallel loop (parfor statement) as a job to a queue. This MATLAB program prints the name of the run host and shows the values of variables numlabs and labindex for each iteration of the parfor loop.

This method uses the job submission command to submit a MATLAB client which calls the MATLAB batch() function with a user-defined cluster profile.

Prepare a MATLAB pool program in a MATLAB script with an appropriate filename, here named myscript.m:

% FILENAME:  myscript.m

% SERIAL REGION
[c name] = system('hostname');
fprintf('SERIAL REGION:  hostname:%s\n', name)
numlabs = parpool('poolsize');
fprintf('        hostname                         numlabs  labindex  iteration\n')
fprintf('        -------------------------------  -------  --------  ---------\n')
tic;

% PARALLEL LOOP
parfor i = 1:8
    [c name] = system('hostname');
    name = name(1:length(name)-1);
    fprintf('PARALLEL LOOP:  %-31s  %7d  %8d  %9d\n', name,numlabs,labindex,i)
    pause(2);
end

% SERIAL REGION
elapsed_time = toc;        % get elapsed time in parallel loop
fprintf('\n')
[c name] = system('hostname');
name = name(1:length(name)-1);
fprintf('SERIAL REGION:  hostname:%s\n', name)
fprintf('Elapsed time in parallel loop:   %f\n', elapsed_time)

The execution of a pool job starts with a worker executing the statements of the first serial region up to the parfor block, when it pauses. A set of workers (the pool) executes the parfor block. When they finish, the first worker resumes by executing the second serial region. The code displays the names of the compute nodes running the batch session and the worker pool.

Prepare a MATLAB script that calls MATLAB function batch() which makes a four-lab pool on which to run the MATLAB code in the file myscript.m. Use an appropriate filename, here named mylclbatch.m:

% FILENAME:  mylclbatch.m

!echo "mylclbatch.m"
!hostname

pjob=batch('myscript','Profile','myslurmprofile','Pool',4,'CaptureDiary',true);
wait(pjob);
diary(pjob);
quit;

Prepare a job submission file with an appropriate filename, here named myjob.sub:

#!/bin/bash
# FILENAME:  myjob.sub

echo "myjob.sub"
hostname

module load matlab

unset DISPLAY

matlab -nodisplay -r mylclbatch

One processor core runs myjob.sub and mylclbatch.m.

Once this job starts, a second job submission is made.

myjob.sub

                            < M A T L A B (R) >
                  Copyright 1984-2013 The MathWorks, Inc.
                    R2013a (8.1.0.604) 64-bit (glnxa64)
                             February 15, 2013

To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

mylclbatch.mresource-a000.rcac.purdue.edu
SERIAL REGION:  hostname:resource-a000.rcac.purdue.edu

                hostname                         numlabs  labindex  iteration
                -------------------------------  -------  --------  ---------
PARALLEL LOOP:  resource-a001.rcac.purdue.edu           4         1          2
PARALLEL LOOP:  resource-a002.rcac.purdue.edu           4         1          4
PARALLEL LOOP:  resource-a001.rcac.purdue.edu           4         1          5
PARALLEL LOOP:  resource-a002.rcac.purdue.edu           4         1          6
PARALLEL LOOP:  resource-a003.rcac.purdue.edu           4         1          1
PARALLEL LOOP:  resource-a003.rcac.purdue.edu           4         1          3
PARALLEL LOOP:  resource-a004.rcac.purdue.edu           4         1          7
PARALLEL LOOP:  resource-a004.rcac.purdue.edu           4         1          8

SERIAL REGION:  hostname:resource-a001.rcac.purdue.edu

Elapsed time in parallel loop:   5.411486

To scale up this method to handle a real application, increase the wall time in the submission command to accommodate a longer running job. Secondly, increase the wall time of myslurmprofile by using the Cluster Profile Manager in the Parallel menu to enter a new wall time in the property SubmitArguments.

For more information about MATLAB Parallel Computing Toolbox:

Parallel Toolbox (spmd)

The MATLAB Parallel Computing Toolbox (PCT) extends the MATLAB language with high-level, parallel-processing features such as parallel for loops, parallel regions, message passing, distributed arrays, and parallel numerical methods. It offers a shared-memory computing environment with a maximum of eight MATLAB workers (labs, threads; versions R2009a) and 12 workers (labs, threads; version R2011a) running on the local configuration in addition to your MATLAB client. Moreover, the MATLAB Distributed Computing Server (DCS) scales PCT applications up to the limit of your DCS licenses.

This section illustrates how to submit a small, parallel, MATLAB program with a parallel region (spmd statement) as a MATLAB pool job to a batch queue.

This example uses the submission command to submit to compute nodes a MATLAB client which interprets a Matlab .m with a user-defined cluster profile which scatters the MATLAB workers onto different compute nodes. This method uses the MATLAB interpreter, the Parallel Computing Toolbox, and the Distributed Computing Server; so, it requires and checks out six licenses: one MATLAB license for the client running on the compute node, one PCT license, and four DCS licenses. Four DCS licenses run the four copies of the spmd statement. This job is completely off the front end.

Prepare a MATLAB script called myscript.m:

% FILENAME:  myscript.m

% SERIAL REGION
[c name] = system('hostname');
fprintf('SERIAL REGION:  hostname:%s\n', name)
p = parpool('4');
fprintf('                    hostname                         numlabs  labindex\n')
fprintf('                    -------------------------------  -------  --------\n')
tic;

% PARALLEL REGION
spmd
    [c name] = system('hostname');
    name = name(1:length(name)-1);
    fprintf('PARALLEL REGION:  %-31s  %7d  %8d\n', name,numlabs,labindex)
    pause(2);
end

% SERIAL REGION
elapsed_time = toc;          % get elapsed time in parallel region
delete(p);
fprintf('\n')
[c name] = system('hostname');
name = name(1:length(name)-1);
fprintf('SERIAL REGION:  hostname:%s\n', name)
fprintf('Elapsed time in parallel region:   %f\n', elapsed_time)
quit;

Prepare a job submission file with an appropriate filename, here named myjob.sub. Run with the name of the script:

#!/bin/bash 
# FILENAME:  myjob.sub

echo "myjob.sub"

module load matlab

unset DISPLAY

matlab -nodisplay -r myscript

Run MATLAB to set the default parallel configuration to your job configuration:

$ matlab -nodisplay
>> parallel.defaultClusterProfile('myslurmprofile');
>> quit;
$

Once this job starts, a second job submission is made.

myjob.sub

                            < M A T L A B (R) >
                  Copyright 1984-2011 The MathWorks, Inc.
                    R2011b (7.13.0.564) 64-bit (glnxa64)
                              August 13, 2011

To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

SERIAL REGION:  hostname:resource-a001.rcac.purdue.edu

Starting matlabpool using the 'myslurmprofile' profile ... connected to 4 labs.
                    hostname                         numlabs  labindex
                    -------------------------------  -------  --------
Lab 2:
  PARALLEL REGION:  resource-a002.rcac.purdue.edu           4         2
Lab 1:
  PARALLEL REGION:  resource-a001.rcac.purdue.edu           4         1
Lab 3:
  PARALLEL REGION:  resource-a003.rcac.purdue.edu           4         3
Lab 4:
  PARALLEL REGION:  resource-a004.rcac.purdue.edu           4         4

Sending a stop signal to all the labs ... stopped.

SERIAL REGION:  hostname:resource-a001.rcac.purdue.edu
Elapsed time in parallel region:   3.382151

Output shows the name of one compute node (a001) that processed the job submission file myjob.sub and the two serial regions. The job submission scattered four processor cores (four MATLAB labs) among four different compute nodes (a001,a002,a003,a004) that processed the four parallel regions. The total elapsed time demonstrates that the jobs ran in parallel.

For more information about MATLAB Parallel Computing Toolbox:

Distributed Computing Server (parallel job)

The MATLAB Parallel Computing Toolbox (PCT) enables a parallel job via the MATLAB Distributed Computing Server (DCS). The tasks of a parallel job are identical, run simultaneously on several MATLAB workers (labs), and communicate with each other. This section illustrates an MPI-like program.

This section illustrates how to submit a small, MATLAB parallel job with four workers running one MPI-like task to a batch queue. The MATLAB program broadcasts an integer to four workers and gathers the names of the compute nodes running the workers and the lab IDs of the workers.

This example uses the job submission command to submit a Matlab script with a user-defined cluster profile which scatters the MATLAB workers onto different compute nodes. This method uses the MATLAB interpreter, the Parallel Computing Toolbox, and the Distributed Computing Server; so, it requires and checks out six licenses: one MATLAB license for the client running on the compute node, one PCT license, and four DCS licenses. Four DCS licenses run the four copies of the parallel job. This job is completely off the front end.

Prepare a MATLAB script named myscript.m :

% FILENAME:  myscript.m

% Specify pool size.
% Convert the parallel job to a pool job.
parpool('4');
spmd

if labindex == 1
    % Lab (rank) #1 broadcasts an integer value to other labs (ranks).
    N = labBroadcast(1,int64(1000));
else
    % Each lab (rank) receives the broadcast value from lab (rank) #1.
    N = labBroadcast(1);
end

% Form a string with host name, total number of labs, lab ID, and broadcast value.
[c name] =system('hostname');
name = name(1:length(name)-1);
fmt = num2str(floor(log10(numlabs))+1);
str = sprintf(['%s:%d:%' fmt 'd:%d   '], name,numlabs,labindex,N);

% Apply global concatenate to all str's.
% Store the concatenation of str's in the first dimension (row) and on lab #1.
result = gcat(str,1,1);
if labindex == 1
    disp(result)
end

end   % spmd
matlabpool close force;
quit;

Also, prepare a job submission, here named myjob.sub. Run with the name of the script:

# FILENAME:  myjob.sub

echo "myjob.sub"

module load matlab

unset DISPLAY

# -nodisplay: run MATLAB in text mode; X11 server not needed
# -r:         read MATLAB program; use MATLAB JIT Accelerator
matlab -nodisplay -r myscript

Run MATLAB to set the default parallel configuration to your appropriate Profile:

$ matlab -nodisplay
>> defaultParallelConfig('myslurmprofile');
>> quit;
$

Once this job starts, a second job submission is made.

myjob.sub

                            < M A T L A B (R) >
                  Copyright 1984-2011 The MathWorks, Inc.
                    R2011b (7.13.0.564) 64-bit (glnxa64)
                              August 13, 2011

To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

>Starting matlabpool using the 'myslurmprofile' configuration ... connected to 4 labs.
Lab 1:
  resource-a006.rcac.purdue.edu:4:1:1000
  resource-a007.rcac.purdue.edu:4:2:1000
  resource-a008.rcac.purdue.edu:4:3:1000
  resource-a009.rcac.purdue.edu:4:4:1000
Sending a stop signal to all the labs ... stopped.
Did not find any pre-existing parallel jobs created by matlabpool.

Output shows the name of one compute node (a006) that processed the job submission file myjob.sub. The job submission scattered four processor cores (four MATLAB labs) among four different compute nodes (a006,a007,a008,a009) that processed the four parallel regions.

To scale up this method to handle a real application, increase the wall time in the submission command to accommodate a longer running job. Secondly, increase the wall time of myslurmprofile by using the Cluster Profile Manager in the Parallel menu to enter a new wall time in the property SubmitArguments.

For more information about parallel jobs:

meep

Description

Meep or MEEP is a free finite-difference time-domain FDTD simulation software package developed at MIT to model electromagnetic systems.

Versions

  • Brown: 1.20.0
  • Negishi: 1.20.0
  • Anvil: 1.20.0

Module

You can load the modules by:

module load meep

modtree

Description

ModuleTree or modtree helps users naviagate between different application stacks and sets up a default compiler and mpi environment.

Versions

  • Bell: deprecated, new
  • Brown: deprecated, new
  • Scholar: deprecated, recent
  • Gilbreth: deprecated, new
  • Negishi: cpu
  • Anvil: cpu, gpu
  • Workbench: deprecated, new

Module

You can load the modules by:

module load modtree

monitor

Description

System resource monitoring tool.

Versions

  • Anvil: 2.3.1
  • Negishi: 2.3.1

Module

You can load the modules by:

module load monitor

mpc

Description

Gnu Mpc is a C library for the arithmetic of complex numbers with arbitrarily high precision and correct rounding of the result.

Versions

  • Bell: 1.1.0
  • Brown: 1.1.0
  • Scholar: 1.1.0
  • Gilbreth: 1.1.0
  • Negishi: 1.1.0
  • Anvil: 1.1.0
  • Workbench: 1.1.0

Module

You can load the modules by:

module load mpc

mpfr

Description

The MPFR library is a C library for multiple-precision floating-point computations with correct rounding.

Versions

  • Bell: 3.1.6
  • Brown: 3.1.6
  • Scholar: 3.1.6
  • Gilbreth: 3.1.6
  • Negishi: 4.0.2
  • Anvil: 4.0.2
  • Workbench: 3.1.6

Module

You can load the modules by:

module load mpfr

mrbayes

Description

MrBayes is a program for Bayesian inference and model choice across a wide range of phylogenetic and evolutionary models. MrBayes uses Markov chain Monte Carlo MCMC methods to estimate the posterior distribution of model parameters.

Versions

  • Anvil: 3.2.7a

Module

You can load the modules by:

module load mrbayes

mxnet

Description

NVIDIA Optimized Deep Learning Framework, powered by Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to mix the flavors of symbolic programming and imperative programming to maximize efficiency and productivity.

Versions

  • Gilbreth: 1.7.0

Module

You can load the modules by:

module load mxnet

namd

Description

NAMDis a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems.

Versions

  • Gilbreth: 2.13
  • Negishi: 2.14
  • Anvil: 2.14

Module

You can load the modules by:

module load namd

nccl

Description

Optimized primitives for collective multi-GPU communication.

Versions

  • Anvil: cuda-11.0_2.11.4, cuda-11.2_2.8.4, cuda-11.4_2.11.4

Module

You can load the modules by:

module load modtree/gpu
module load nccl

ncl

Description

NCL is an interpreted language designed specifically for scientific data analysis and visualization. Supports NetCDF 3/4, GRIB 1/2, HDF 4/5, HDF-EOD 2/5, shapefile, ASCII, binary. Numerous analysis functions are built-in.

Versions

  • Bell: 6.4.0
  • Brown: 6.4.0
  • Scholar: 6.4.0
  • Gilbreth: 6.4.0
  • Anvil: 6.4.0
  • Workbench: 6.4.0

Module

You can load the modules by:

module load ncl

nco

Description

The NCO toolkit manipulates and analyzes data stored in netCDF-accessible formats

Versions

  • Bell: 4.6.7
  • Brown: 4.6.7
  • Scholar: 4.6.7
  • Gilbreth: 4.6.7
  • Negishi: 4.9.3
  • Anvil: 4.9.3
  • Workbench: 4.6.7

Module

You can load the modules by:

module load nco

ncview

Description

Simple viewer for NetCDF files.

Versions

  • Bell: 2.1.7
  • Brown: 2.1.7
  • Scholar: 2.1.7
  • Gilbreth: 2.1.7
  • Anvil: 2.1.8
  • Workbench: 2.1.7

Module

You can load the modules by:

module load ncview

netcdf-c

Description

NetCDF network Common Data Form is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. This is the C distribution.

Versions

  • Negishi: 4.9.0
  • Anvil: 4.7.4

Module

You can load the modules by:

module load netcdf-c

netcdf-cxx4

Description

NetCDF network Common Data Form is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. This is the C++ distribution.

Versions

  • Bell: 4.3.0, 4.3.1
  • Brown: 4.3.0, 4.3.1
  • Scholar: 4.3.0, 4.3.1
  • Gilbreth: 4.3.0, 4.3.1
  • Negishi: 4.3.1
  • Anvil: 4.3.1
  • Workbench: 4.3.0, 4.3.1

Module

You can load the modules by:

module load netcdf-cxx4

netcdf-fortran

Description

NetCDF network Common Data Form is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. This is the Fortran distribution.

Versions

  • Bell: 4.4.4, 4.5.3
  • Brown: 4.4.4, 4.5.2
  • Scholar: 4.4.4, 4.5.2
  • Gilbreth: 4.4.4, 4.5.2
  • Negishi: 4.6.0
  • Anvil: 4.5.3
  • Workbench: 4.4.4, 4.5.2

Module

You can load the modules by:

module load netcdf-fortran

netcdf

Description

NetCDF network Common Data Form is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. This is the C distribution.

Versions

  • Bell: 4.5.0, 4.7.4
  • Brown: 4.5.0, 4.7.0
  • Scholar: 4.5.0, 4.7.0
  • Gilbreth: 4.5.0, 4.7.0
  • Workbench: 4.5.0, 4.7.0

Module

You can load the modules by:

module load netcdf

netlib-lapack

Description

LAPACK version 3.X is a comprehensive FORTRAN library that does linear algebra operations including matrix inversions, least squared solutions to linear sets of equations, eigenvector analysis, singular value decomposition, etc. It is a very comprehensive and reputable package that has found extensive use in the scientific community.

Versions

  • Bell: 3.8.0
  • Brown: 3.6.0
  • Scholar: 3.6.0
  • Gilbreth: 3.6.0
  • Negishi: 3.8.0
  • Anvil: 3.8.0
  • Workbench: 3.6.0

Module

You can load the modules by:

module load netlib-lapack

numactl

Description

Simple NUMA policy support. It consists of a numactl program to run other programs with a specific NUMA policy and a libnuma shared library ("NUMA API") to set NUMA policy in applications.

Versions

  • Negishi: 2.0.14
  • Anvil: 2.0.14

Module

You can load the modules by:

module load numactl

Usage

numactl [ options ] command {arguments ...} command to run the program

Options:

-H, --hardware,
-m nodes, --membind=nodes,     Use the memory on these NUMA nodes
-N nodes, --cpunodebind=nodes,     Use the CPUs on these nodes
-C cpus, --physcpubind=cpus,    Use these CPUs

nwchem

Description

High-performance computational chemistry software

Versions

  • Negishi: 7.0.2
  • Anvil: 7.0.2

Module

You can load the modules by:

module load nwchem

octave

Description

GNU Octave is a high-level language, primarily intended for numerical computations. It provides a convenient command line interface for solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab. It may also be used as a batch-oriented language.

Versions

  • Bell: 4.4.1
  • Brown: 4.4.0
  • Scholar: 4.4.0
  • Negishi: 7.3.0
  • Anvil: 6.3.0
  • Workbench: 4.4.0

Module

You can load the modules by:

module load octave

openblas

Description

OpenBLAS is an open source implementation of the BLAS API with many hand-crafted optimizations for specific processor types

Versions

  • Bell: 0.3.8, 0.3.21
  • Brown: 0.2.20, 0.3.7
  • Scholar: 0.2.20, 0.3.7, 0.3.21
  • Gilbreth: 0.2.20, 0.3.7, 0.3.21
  • Negishi: 0.3.17, 0.3.21
  • Anvil: 0.3.17
  • Workbench: 0.2.20, 0.3.7, 0.3.21

Module

You can load the modules by:

module load openblas

opencv

Description

OpenCV, Open source Computer Vision, is an open-source BSD-licensed library that includes several hundreds of computer vision algorithms. Written in optimized C/C++, the library can take advantage of multi-core processing.

Versions

  • Bell: 4.4.0
  • Gilbreth: 4.5.1

Module

You can load the modules by:

module load learning 
module load opencv

openfoam

Description

OpenFOAM is leading software for computational fluid dynamics (CFD).

Versions

  • Bell: 5.x
  • Brown: 9-20211122
  • Anvil: 8-20210316

Module

You can load the modules by:

module load openfoam

openjdk

Description

The free and open-source java implementation

Versions

  • Negishi: 1.8.0_265-b01, 11.0.17_8
  • Anvil: 11.0.8_10

Module

You can load the modules by:

module load openjdk

panoply

Description

Panoply is a Java-based cross-platform NetCDF, HDF and GRIB Data Viewer.

Versions

  • Bell: 4.11.6
  • Brown: 4.11.0
  • Scholar: 4.11.0
  • Gilbreth: 4.11.0
  • Workbench: 4.11.0

Module

You can load the modules by:

module load panoply

papi

Description

PAPI provides the tool designer and application engineer with a consistent interface and methodology for use of the performance counter hardware found in most major microprocessors. PAPI enables software engineers to see, in near real time, the relation between software performance and processor events. In addition Component PAPI provides access to a collection of components that expose performance measurement opportunities across the hardware and software stack.

Versions

  • Negishi: 6.0.0.1
  • Anvil: 6.0.0.1

Module

You can load the modules by:

module load papi

parafly

Description

Run UNIX commands in parallel

Versions

  • Negishi: r2013

Module

You can load the modules by:

module load parafly

parallel-netcdf

Description

PnetCDF Parallel netCDF is a high-performance parallel I/O library for accessing files in format compatibility with Unidatas NetCDF, specifically the formats of CDF-1, 2, and 5.

Versions

  • Bell: 1.11.2
  • Brown: 1.10.0
  • Scholar: 1.10.0
  • Negishi: 1.11.2
  • Anvil: 1.11.2
  • Workbench: 1.10.0

Module

You can load the modules by:

module load parallel-netcdf

parallel

Description

GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input.

Versions

  • Bell: 20220522
  • Negishi: 20220522
  • Anvil: 20200822

Module

You can load the modules by:

module load parallel

Syntax

# Read commands to be run in parallel from an input file
parallel [OPTIONS] < CMDFILE

# Read command arguments on the command line
parallel [OPTIONS] COMMAND [ARGUMENTS] ::: ARGLIST

# Read command arguments from an input file
parallel [OPTIONS] COMMAND [ARGUMENTS] :::: ARGFILE

paraview

Description

ParaView is an open-source, multi-platform data analysis and visualization application.

Versions

  • Bell: 5.6.2
  • Brown: 5.9.1
  • Anvil: 5.9.1

Module

You can load the modules by:

module load paraview

perl-bioperl

Description

BioPerl is the product of a community effort to produce Perl code which is useful in biology. Examples include Sequence objects, Alignment objects and database searching objects. These objects not only do what they are advertised to do in the documentation, but they also interact - Alignment objects are made from the Sequence objects, Sequence objects have access to Annotation and SeqFeature objects and databases, Blast objects can be converted to Alignment objects, and so on. This means that the objects provide a coordinated and extensible framework to do computational biology.

Versions

  • Anvil: 1.7.6

Commands

  • bp_aacomp
  • bp_bioflat_index
  • bp_biogetseq
  • bp_chaos_plot
  • bp_dbsplit
  • bp_extract_feature_seq
  • bp_fastam9_to_table
  • bp_fetch
  • bp_filter_search
  • bp_find-blast-matches
  • bp_gccalc
  • bp_genbank2gff3
  • bp_index
  • bp_local_taxonomydb_query
  • bp_make_mrna_protein
  • bp_mask_by_search
  • bp_mrtrans
  • bp_mutate
  • bp_nexus2nh
  • bp_nrdb
  • bp_oligo_count
  • bp_process_gadfly
  • bp_process_sgd
  • bp_revtrans-motif
  • bp_search2alnblocks
  • bp_search2gff
  • bp_search2table
  • bp_search2tribe
  • bp_seqconvert
  • bp_seqcut
  • bp_seq_length
  • bp_seqpart
  • bp_seqret
  • bp_seqretsplit
  • bp_split_seq
  • bp_sreformat
  • bp_taxid4species
  • bp_taxonomy2tree
  • bp_translate_seq
  • bp_tree2pag
  • bp_unflatten_seq

Module

You can load the modules by:

module load perl-bioperl

petsc

Description

PETSc is a suite of data structures and routines for the scalable parallel solution of scientific applications modeled by partial differential equations.

Versions

  • Negishi: 3.17.5, 3.18.3
  • Anvil: 3.15.3

Module

You can load the modules by:

module load petsc

picard

Description

Picard is a set of command line tools for manipulating high-throughput sequencing HTS data and formats such as SAM/BAM/CRAM and VCF.

Versions

  • Anvil: 2.25.7

Commands

  • picard

Module

You can load the modules by:

module load picard

Example job

To run picard our our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 20:00:00
#SBATCH -N 1
#SBATCH -n 24
#SBATCH --job-name=picard
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load picard

picard BuildBamIndex -Xmx64g I=19P0126636WES_sorted_md.bam
picard CreateSequenceDictionary -R hg38.fa -O hg38.dict

proj

Description

PROJ is a generic coordinate transformation software, that transforms geospatial coordinates from one coordinate reference system CRS to another. This includes cartographic projections as well as geodetic transformations.

Versions

  • Bell: 5.2.0, 8.1.0, 8.2.1
  • Brown: 5.2.0, 8.1.0, 8.2.1
  • Scholar: 5.2.0, 8.1.0, 8.2.1
  • Gilbreth: 5.2.0, 8.2.1
  • Negishi: 5.2.0, 6.2.0
  • Anvil: 5.2.0, 6.2.0
  • Workbench: 5.2.0, 8.1.0, 8.2.1

Module

You can load the modules by:

module load proj

protobuf

Description

Protocol Buffers (a.k.a., protobuf) are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data.

Versions

  • Bell: 3.11.4
  • Brown: 3.0.2
  • Scholar: 3.0.2
  • Gilbreth: 3.0.2
  • Negishi: 3.11.4, 3.18.0
  • Anvil: 3.11.4
  • Workbench: 3.0.2

Module

You can load the modules by:

module load protobuf

py-mpi4py

Description

mpi4py provides a Python interface to MPI or the Message-Passing Interface. It is useful for parallelizing Python scripts

Versions

  • Anvil: 3.0.3

Module

You can load the modules by:

module load py-mpi4py

python

Description

Native Python 3.9.5 including optimized libraries.

Versions

  • Anvil: 3.9.5

Module

You can load the modules by:

module load python

pytorch

Description

PyTorch is a machine learning library with strong support for neural networks and deep learning. PyTorch also has a large user base and software ecosystem.

Versions

  • Bell: 1.6.0
  • Gilbreth: 1.7.1

Module

You can load the modules by:

module load learning
module load pytorch

qemu

Description

QEMU is a generic and open source machine emulator and virtualizer.

Versions

  • Bell: 2.10.1, 4.1.0
  • Brown: 2.10.1
  • Scholar: 2.10.1
  • Gilbreth: 2.10.1
  • Anvil: 4.1.1
  • Workbench: 2.10.1

Module

You can load the modules by:

module load qemu

qt

Description

Qt is a comprehensive cross-platform C++ application framework.

Versions

  • Bell: 5.12.5
  • Brown: 5.12.5
  • Scholar: 5.12.5
  • Gilbreth: 5.12.5
  • Anvil: 5.15.2
  • Workbench: 5.12.5

Module

You can load the modules by:

module load qt

quantum-espresso

Description

Quantum ESPRESSO is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials.

Versions

  • Bell: 6.6
  • Brown: 6.2.1, 6.3
  • Scholar: 6.2.1, 6.3
  • Negishi: 7.1
  • Anvil: 6.7

Module

You can load the modules by:

module load quantum-espresso

quantumatk

Versions

  • Bell: 2020.09
  • Brown: 2020.09

Module

You can load the modules by:

module load quantumatk

r

Description

Linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc. Please consult the R project homepage for further information.

Versions

  • Bell: 3.6.3, 4.0.0, 4.1.2, 4.2.2
  • Brown: 3.6.1, 3.6.3, 4.0.0, 4.1.2, 4.2.2
  • Scholar: 3.6.1, 3.6.3, 4.0.0, 4.0.5, 4.1.2, 4.2.2
  • Gilbreth: 3.6.1, 3.6.3, 4.0.0, 4.1.2, 4.2.2
  • Negishi: 4.2.2
  • Anvil: 4.0.5, 4.1.0
  • Workbench: 3.6.1, 3.6.3, 4.0.0, 4.1.2, 4.2.2

Module

You can load the modules by:

module load r

Setting Up R Preferences with .Rprofile

Different clusters have different hardware and softwares. So, if you have access to multiple clusters, you must install your R packages separately for each cluster. Each cluster has multiple versions of R and packages installed with one version of R may not work with another version of R. So, libraries for each R version must be installed in a separate directory. You can define the directory where your R packages will be installed using the environment variable R_LIBS_USER.

For your convenience, a sample .Rprofile example file is provided that can be downloaded to your cluster account and renamed into /.Rprofile (or appended to one) to customize your installation preferences. Detailed instructions:

curl -#LO https://www.rcac.purdue.edu/files/knowledge/run/examples/apps/r/Rprofile_example
mv -ib Rprofile_example ~/.Rprofile

The above installation step needs to be done only once on each of the clusters you have access to. Now load the R module and run R to confirm the unique libPaths:

module load r/4.2.2
R
R> .libPaths()                  
[1] "/home/zhan4429/R/bell/4.2.2-gcc-9.3.0-xxbnk6s"                 
[2] "/apps/spack/bell/apps/r/4.2.2-gcc-9.3.0-xxbnk6s/rlib/R/library"

Challenging packages

Below are packages users may have difficulty in installation.

nloptr

In Bell, the installation may fail due to the default `cmake` version is too old. The solution is easy, users just need to load the newer versions of cmake:

module load cmake/3.20.6
module load r
Rscript -e 'install.packages("nloptr")'

In Brown or other older clusters, because our system's cmake and gcc compilers are old, we may not be able to install the latest version of nloptr. The walkaround is that users can install the older versions of nloptr:

module load r
R
 > myrepos = c("https://cran.case.edu")
 > install.packages("devtools", repos = myrepos)
 > library(devtools)
 > install_version("nloptr", version = "> 1.2.2, < 2.0.0", repos = myrepos)

Error: C++17 standard requested but CXX17 is not defined

When users want to install some packages, such as colourvalues, the installation may fail due to Error: C++17 standard requested but CXX17 is not defined. Please follow the below command to fix it:

module load r
module spider gcc
module load gcc/xxx  ## the lateste gcc is recommended
mkdir -p ~/.R
echo 'CXX17 = g++ -std=gnu++17 -fPIC' > ~/.R/Makevars
R
> install.packages("xxxx")

RCurl

Some R packages rely on curl. When you install these packages such as RCurl, you may see such error: checking for curl-config... no Cannot find curl-config To install such packages, you need to load the curl module: 
module load curl
module load r
R
> install.packages("RCurl")

raster, stars and sf

These R packages have some dependencies. To install them, users will need to load several modules. Note that these modules have multiple versions, and the latest version is recommended. However, the default version may not be the latest version. To check the latest version, please run module spider XX. 
module spider gdal
module spider geos
module spider proj
module spider sqlite

module load gdal/XXX geos/XXX proj/XXX sqlite/XXX  ## XXX is the version to use. The latest version is recommended.  
module load r/XXX
R
> install.packages("raster")
     install.packages("stars")
     install.packages("sf")

Running R jobs

This section illustrates how to submit a small R job to a SLURM queue. The example job computes a Pythagorean triple.

Prepare an R input file with an appropriate filename, here named myjob.R:

# FILENAME:  myjob.R

# Compute a Pythagorean triple.
a = 3
b = 4
c = sqrt(a*a + b*b)
c     # display result

Prepare a job submission file with an appropriate filename, here named myjob.sub:

#!/bin/bash
# FILENAME:  myjob.sub

module load r

# --vanilla:
# --no-save: do not save datasets at the end of an R session
R --vanilla --no-save < myjob.R

For other examples or R jobs:

Installing R packages

Challenges of Managing R Packages in the Cluster Environment

  • Different clusters have different hardware and softwares. So, if you have access to multiple clusters, you must install your R packages separately for each cluster.
  • Each cluster has multiple versions of R and packages installed with one version of R may not work with another version of R. So, libraries for each R version must be installed in a separate directory.
  • You can define the directory where your R packages will be installed using the environment variable R_LIBS_USER.
  • For your convenience, a sample ~/.Rprofile example file is provided that can be downloaded to your cluster account and renamed into ~/.Rprofile (or appended to one) to customize your installation preferences. Detailed instructions.

Installing Packages

  • Step 0: Set up installation preferences.
    Follow the steps for setting up your ~/.Rprofile preferences. This step needs to be done only once. If you have created a ~/.Rprofile file previously on a resource, ignore this step.

  • Step 1: Check if the package is already installed.
    As part of the R installations on community clusters, a lot of R libraries are pre-installed. You can check if your package is already installed by opening an R terminal and entering the command installed.packages(). For example,

    module load r/4.1.2
R
    installed.packages()["units",c("Package","Version")]
Package Version 
"units" "0.6-3"
quit()

    If the package you are trying to use is already installed, simply load the library, e.g., library('units'). Otherwise, move to the next step to install the package.

  • Step 2: Load required dependencies. (if needed)
    For simple packages you may not need this step. However, some R packages depend on other libraries. For example, the sf package depends on gdal and geos libraries. So, you will need to load the corresponding modules before installing sf. Read the documentation for the package to identify which modules should be loaded.

    module load gdal
module load geos

  • Step 3: Install the package.
    Now install the desired package using the command install.packages('package_name'). R will automatically download the package and all its dependencies from CRAN and install each one. Your terminal will show the build progress and eventually show whether the package was installed successfully or not.

    R
    install.packages('sf', repos="https://cran.case.edu/")
Installing package into ‘/home/myusername/R/the-resource/4.0.0’
(as ‘lib’ is unspecified)
trying URL 'https://cran.case.edu/src/contrib/sf_0.9-7.tar.gz'
Content type 'application/x-gzip' length 4203095 bytes (4.0 MB)
==================================================
downloaded 4.0 MB
...
...
more progress messages
...
...
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (sf)

The downloaded source packages are in
    ‘/tmp/RtmpSVAGio/downloaded_packages’
  • Step 4: Troubleshooting. (if needed)
    If Step 3 ended with an error, you need to investigate why the build failed. Most common reason for build failure is not loading the necessary modules.

Loading Libraries

Once you have packages installed you can load them with the library() function as shown below:

library('packagename')

The package is now installed and loaded and ready to be used in R.

Example: Installing dplyr

The following demonstrates installing the dplyr package assuming the above-mentioned custom ~/.Rprofile is in place (note its effect in the "Installing package into" information message):

module load r
R
install.packages('dplyr', repos="http://ftp.ussg.iu.edu/CRAN/")
Installing package into ‘/home/myusername/R/the-resource/4.0.0’
(as ‘lib’ is unspecified)
 ...
also installing the dependencies 'crayon', 'utf8', 'bindr', 'cli', 'pillar', 'assertthat', 'bindrcpp', 'glue', 'pkgconfig', 'rlang', 'Rcpp', 'tibble', 'BH', 'plogr'
 ...
 ...
 ...
The downloaded source packages are in 
    '/tmp/RtmpHMzm9z/downloaded_packages'

library(dplyr)

Attaching package: 'dplyr'

For more information about installing R packages:

Loading Data into R

R is an environment for manipulating data. In order to manipulate data, it must be brought into the R environment. R has a function to read any file that data is stored in. Some of the most common file types like comma-separated variable(CSV) files have functions that come in the basic R packages. Other less common file types require additional packages to be installed. To read data from a CSV file into the R environment, enter the following command in the R prompt:

> read.csv(file = "path/to/data.csv", header = TRUE)

When R reads the file it creates an object that can then become the target of other functions. By default the read.csv() function will give the object the name of the .csv file. To assign a different name to the object created by read.csv enter the following in the R prompt:

> my_variable <- read.csv(file = "path/to/data.csv", header = FALSE)

To display the properties (structure) of loaded data, enter the following:

> str(my_variable)

For more functions and tutorials:

Setting Up R Preferences with .Rprofile

For your convenience, a sample ~/.Rprofile example file is provided that can be downloaded to your cluster account and renamed into ~/.Rprofile (or appended to one). Follow these steps to download our recommended ~/.Rprofile example and copy it into place:

curl -#LO https://www.rcac.purdue.edu/files/knowledge/run/examples/apps/r/Rprofile_example
mv -ib Rprofile_example ~/.Rprofile

The above installation step needs to be done only once on ${resource.name}. Now load the R module and run R:

module load r/4.1.2
R
.libPaths()
[1] "/home/myusername/R/the-resource/4.1.2-gcc-6.3.0-ymdumss"
[2] "/apps/spack/the-resource/apps/r/4.1.2-gcc-6.3.0-ymdumss/rlib/R/library"

.libPaths() should output something similar to above if it is set up correctly.

You are now ready to install R packages into the dedicated directory /home/myusername/R/the-resource/4.1.2-gcc-6.3.0-ymdumss.

rocm

Description

ROCm Application for Reporting System Info

Versions

  • Bell: 5.2.0
  • Negishi: 5.2.0

Module

You can load the modules by:

module load rocm

rstudio

Description

This package installs Rstudio desktop from pre-compiled binaries available in the Rstudio website. The installer assumes that you are running on CentOS7/Redhat7/Fedora19. Please fix the download URL for other systems.

Versions

  • Bell: 1.3.959, 1.3.1073, 2021.09, 2022.07
  • Brown: 1.2.1335, 1.3.959, 2021.09, 2022.07
  • Scholar: 1.2.1335, 1.3.959, 2021.09, 2022.07
  • Gilbreth: 1.2.1335, 1.3.959, 2021.09, 2022.07
  • Negishi: 2022.07.2
  • Anvil: 2021.09.0
  • Workbench: 1.2.1335, 1.3.959, 2021.09, 2022.07

Module

You can load the modules by:

module load rstudio

samtools

Description

SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format

Versions

  • Anvil: 1.12

Commands

  • ace2sam
  • blast2sam.pl
  • bowtie2sam.pl
  • export2sam.pl
  • fasta-sanitize.pl
  • interpolate_sam.pl
  • maq2sam-long
  • maq2sam-short
  • md5fa
  • md5sum-lite
  • novo2sam.pl
  • plot-ampliconstats
  • plot-bamstats
  • psl2sam.pl
  • sam2vcf.pl
  • samtools
  • samtools.pl
  • seq_cache_populate.pl
  • soap2sam.pl
  • wgsim
  • wgsim_eval.pl
  • zoom2sam.pl

Module

You can load the modules by:

module load samtools

Example job

To run Samtools on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=samtools
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load samtools

samtools sort my.sam > my_sorted.bam
samtools index my_sorted.bam

sas

Description

SAS is a commercial integrated system for statistical analysis, data mining, and graphics as well as many enterprise oriented additional features.

Versions

  • Bell: 9.4
  • Brown: 9.4
  • Scholar: 9.4
  • Gilbreth: 9.4
  • Workbench: 9.4

Module

You can load the modules by:

module load sas

sentaurus

Description

Sentaurus is a suite of TCAD tools which simulates the fabrication, operation and reliability of semiconductor devices. The Sentaurus simulators use physical models to represent the wafer fabrication steps and device operation, thereby allowing the exploration and optimization of new semiconductor devices.

Versions

  • Bell: 2017.09, 2019.03
  • Brown: 2017.09, 2019.03
  • Workbench: 2017.09, 2019.03

Module

You can load the modules by:

module load sentaurus

spark

Description

Apache Spark is a fast and general engine for large-scale data processing.

Versions

  • Bell: 2.4.4
  • Brown: 2.4.4
  • Scholar: 2.4.4
  • Gilbreth: 2.4.4
  • Negishi: 3.1.1
  • Anvil: 3.1.1
  • Workbench: 2.4.4

Module

You can load the modules by:

module load spark

spss

Description

IBM SPSS Statistics is a powerful statistical software platform. It offers a user-friendly interface and a robust set of features that lets your organization quickly extract actionable insights from your data. Advanced statistical procedures help ensure high accuracy and quality decision making. All facets of the analytics lifecycle are included, from data preparation and management to analysis and reporting.

Versions

  • Workbench: 24

Module

You can load the modules by:

module load spss

sqlite

Description

SQLite3 is an SQL database engine in a C library. Programs that link the SQLite3 library can have SQL database access without running a separate RDBMS process.

Versions

  • Bell: 3.30.1
  • Brown: 3.30.1
  • Scholar: 3.30.1
  • Gilbreth: 3.30.1
  • Workbench: 3.30.1

Module

You can load the modules by:

module load sqlite

sratoolkit

Description

The NCBI SRA Toolkit enables reading dumping of sequencing files from the SRA database and writing loading files into the .sra format.

Versions

  • Anvil: 2.10.9

Module

You can load the modules by:

module load sratoolkit

stata-mp

Description

Stata/MP is the fastest and largest edition of Stata. Stata is a complete, integrated software package that provides all your data science needs—data manipulation, visualization, statistics, and automated reporting.

Versions

  • Bell: 17
  • Scholar: 17

Module

You can load the modules by:

module load stata-mp

stata

Description

Stata is a complete, integrated software package that provides all your data science needs—data manipulation, visualization, statistics, and automated reporting.

Versions

  • Bell: 17
  • Brown: 17
  • Scholar: 17
  • Gilbreth: 17
  • Workbench: 16, 17

Module

You can load the modules by:

module load stata

subversion

Description

Apache Subversion - an open source version control system.

Versions

  • Bell: 1.12.2

Module

You can load the modules by:

module load subversion

swig

Description

SWIG is an interface compiler that connects programs written in C and C++ with scripting languages such as Perl, Python, Ruby, and Tcl. It works by taking the declarations found in C/C++ header files and using them to generate the wrapper code that scripting languages need to access the underlying C/C++ code. In addition, SWIG provides a variety of customization features that let you tailor the wrapping process to suit your application.

Versions

  • Negishi: 4.0.2
  • Anvil: 4.0.2

Module

You can load the modules by:

module load swig

tcl

Description

Tcl Tool Command Language is a very powerful but easy to learn dynamic programming language, suitable for a very wide range of uses, including web and desktop applications, networking, administration, testing and many more. Open source and business-friendly, Tcl is a mature yet evolving language that is truly cross platform, easily deployed and highly extensible.

Versions

  • Bell: 8.6.8
  • Brown: 8.6.8
  • Scholar: 8.6.8
  • Gilbreth: 8.6.8
  • Negishi: 8.6.11, 8.6.12
  • Anvil: 8.6.11
  • Workbench: 8.6.8

Module

You can load the modules by:

module load tcl

tecplot

Description

Tecplot 360 is a Computational Fluid Dynamics (CFD) and numerical simulation software package used in post-processing simulation results. It is also used in chemistry applications to visualize molecule structure by post-processing charge density data.

Versions

  • Bell: 360-2017-R3, 360-2021-R1
  • Brown: 360-2017-R3
  • Scholar: 360-2017-R3
  • Gilbreth: 360-2017-R3
  • Workbench: 360-2017-R3

Module

You can load the modules by:

module load tecplot

tensorflow

Description

TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.

Versions

  • Bell: 2.3.0
  • Gilbreth: 2.4.0

Module

You can load the modules by:

module load learning
module load tensorflow

texinfo

Description

Texinfo is the official documentation format of the GNU project. It was invented by Richard Stallman and Bob Chassell many years ago, loosely based on Brian Reids Scribe and other formatting languages of the time. It is used by many non-GNU projects as well.

Versions

  • Bell: 6.7

Module

You can load the modules by:

module load texinfo

texlive

Description

TeX Live is a free software distribution for the TeX typesetting system. Heads up, its is not a reproducible installation. At any point only the most recent version can be installed. Older versions are included for backward compatibility, i.e., if you have that version already installed.

Versions

  • Bell: 20200406
  • Brown: 20200406
  • Scholar: 20200406
  • Gilbreth: 20200406
  • Negishi: 20220321
  • Anvil: 20200406
  • Workbench: 20200406

Module

You can load the modules by:

module load texlive

tflearn

Description

TFlearn is a modular and transparent deep learning library built on top of Tensorflow. It was designed to provide a higher-level API to TensorFlow in order to facilitate and speed-up experimentations, while remaining fully transparent and compatible with it.

Versions

  • Gilbreth: 0.3.2

Module

You can load the modules by:

module load learning
module load tflearn

theano

Description

Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Theano is most commonly used to perform Deep Learning and has excellent GPU support and integration through PyCUDA.

Versions

  • Bell: 1.0.5
  • Gilbreth: 1.0.5

Module

You can load the modules by:

module load learning
module load theano

thermocalc

Description

Thermo-Calc allows you to calculate the state for a given thermodynamic system to obtain insight.

Versions

  • Bell: 2019b, 2020a, 2021b
  • Brown: 2019b, 2020a, 2021a, 2021b

Module

You can load the modules by:

module load thermocalc

tk

Description

Tk is a graphical user interface toolkit that takes developing desktop applications to a higher level than conventional approaches. Tk is the standard GUI not only for Tcl, but for many other dynamic languages, and can produce rich, native applications that run unchanged across Windows, Mac OS X, Linux and more.

Versions

  • Bell: 8.6.8
  • Brown: 8.6.8
  • Scholar: 8.6.8
  • Gilbreth: 8.6.8
  • Negishi: 8.6.11
  • Anvil: 8.6.11
  • Workbench: 8.6.8

Module

You can load the modules by:

module load tk

tophat

Description

Spliced read mapper for RNA-Seq.

Versions

  • Anvil: 2.1.2

Commands

  • bam2fastx
  • bam_merge
  • bed_to_juncs
  • contig_to_chr_coords
  • fix_map_ordering
  • gtf_juncs
  • gtf_to_fasta
  • juncs_db
  • long_spanning_reads
  • map2gtf
  • prep_reads
  • sam_juncs
  • samtools_0.1.18
  • segment_juncs
  • sra_to_solid
  • tophat
  • tophat2
  • tophat-fusion-post
  • tophat_reports

Module

You can load the modules by:

module load tophat

Example job

To run TopHat on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=tophat
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load tophat

tophat -r 20 test_ref reads_1.fq reads_2.fq

totalview

Description

TotalView is a GUI-based source code defect analysis tool that gives you unprecedented control over processes and thread execution and visibility into program state and variables.

Versions

  • Bell: 2020.2.6, 2021.4.10
  • Brown: 2017.0.12, 2018.2.6, 2019.1.4, 2021.4.10
  • Scholar: 2017.0.12, 2018.2.6, 2019.1.4, 2021.4.10
  • Gilbreth: 2017.0.12, 2018.2.6, 2019.1.4, 2021.4.10
  • Negishi: 2021.4.10
  • Anvil: 2020.2.6
  • Workbench: 2017.0.12, 2018.2.6, 2019.1.4, 2021.4.10

Module

You can load the modules by:

module load totalview

trimmomatic

Description

A flexible read trimming tool for Illumina NGS data.

Versions

  • Anvil: 0.39

Commands

  • trimmomatic

Module

You can load the modules by:

module load trimmomatic

Example job

To run Trimmomatic on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 8
#SBATCH --job-name=trimmomatic
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load trimmomatic

trimmomatic PE -threads 8 \
    input_forward.fq.gz input_reverse.fq.gz \ 
    output_forward_paired.fq.gz output_forward_unpaired.fq.gz \
    output_reverse_paired.fq.gz output_reverse_unpaired.fq.gz \
    ILLUMINACLIP:TruSeq3-PE.fa:2:30:10:2:True LEADING:3 TRAILING:3 MINLEN:36

ucx

Description

a communication library implementing high-performance messaging for MPI/PGAS frameworks

Versions

  • Anvil: 1.11.2

Module

You can load the modules by:

module load ucx

udunits

Description

Automated units conversion

Versions

  • Negishi: 2.2.28

Module

You can load the modules by:

module load udunits

udunits2

Description

Automated units conversion

Versions

  • Bell: 2.2.24
  • Brown: 2.2.24
  • Scholar: 2.2.24
  • Gilbreth: 2.2.24
  • Workbench: 2.2.24

Module

You can load the modules by:

module load udunits2

valgrind

Description

An instrumentation framework for building dynamic analysis.

Versions

  • Bell: 3.15.0
  • Brown: 3.13.0
  • Scholar: 3.13.0
  • Gilbreth: 3.13.0
  • Negishi: 3.19.0
  • Anvil: 3.15.0
  • Workbench: 3.13.0

Module

You can load the modules by:

module load valgrind

vasp

Description

The Vienna Ab initio Simulation Package VASP is a computer program for atomic scale materials modelling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.

Versions

  • Anvil: 5.4.4.pl2, 6.3.0

Module

You can load the modules by:

module load vasp

vcftools

Description

VCFtools is a program package designed for working with VCF files, such as those generated by the 1000 Genomes Project. The aim of VCFtools is to provide easily accessible methods for working with complex genetic variation data in the form of VCF files.

Versions

  • Anvil: 0.1.14

Commands

  • fill-aa
  • fill-an-ac
  • fill-fs
  • fill-ref-md5
  • vcf-annotate
  • vcf-compare
  • vcf-concat
  • vcf-consensus
  • vcf-contrast
  • vcf-convert
  • vcf-fix-newlines
  • vcf-fix-ploidy
  • vcf-indel-stats
  • vcf-isec
  • vcf-merge
  • vcf-phased-join
  • vcf-query
  • vcf-shuffle-cols
  • vcf-sort
  • vcf-stats
  • vcf-subset
  • vcftools
  • vcf-to-tab
  • vcf-tstv
  • vcf-validator

Module

You can load the modules by:

module load vcftools

Example job

To run VCFtools on our clusters:

#!/bin/bash
#SBATCH -A myallocation     # Allocation name 
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --job-name=vcftools
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load vcftools

vcftools --vcf input_data.vcf --chr 1 \
    --from-bp 1000000 --to-bp 2000000

vim

Description

Vim is a highly configurable text editor built to enable efficient text editing. It is an improved version of the vi editor distributed with most UNIX systems. Vim is often called a programmers editor, and so useful for programming that many consider it an entire IDE. Its not just for programmers, though. Vim is perfect for all kinds of text editing, from composing email to editing configuration files.

Versions

  • Bell: 8.1.2141
  • Brown: 7.4.2367
  • Scholar: 7.4.2367
  • Gilbreth: 7.4.2367
  • Workbench: 7.4.2367

Module

You can load the modules by:

module load vim

visit

Description

VisIt is an Open Source, interactive, scalable, visualization, animation and analysis tool. Description

Versions

  • Anvil: 3.1.4

Module

You can load the modules by:

module load visit

vlc

Description

VLC is a free and open source multimedia player for most multimedia formats.

Versions

  • Bell: 3.0.9.2
  • Brown: 3.0.9.2
  • Scholar: 3.0.9.2
  • Gilbreth: 3.0.9.2
  • Anvil: 3.0.9.2
  • Workbench: 3.0.9.2

Module

You can load the modules by:

module load vlc

vmd

Description

VMD is a molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3-D graphics and built-in scripting.

Versions

  • Bell: 1.9.3
  • Brown: 1.9.3
  • Scholar: 1.9.3
  • Gilbreth: 1.9.3
  • Anvil: 1.9.3
  • Workbench: 1.9.3

Module

You can load the modules by:

module load vmd

vscode

Description

Visual Studio Code

Versions

  • Bell: 1.56, 1.59
  • Brown: 1.56, 1.59
  • Scholar: 1.56, 1.59
  • Gilbreth: 1.56, 1.59
  • Anvil: 1.61.2
  • Workbench: 1.56, 1.59

Module

You can load the modules by:

module load vscode

vtk

Description

The Visualization Toolkit VTK is an open-source, freely available software system for 3D computer graphics, image processing and visualization.

Versions

  • Negishi: 9.0.0
  • Anvil: 9.0.0

Module

You can load the modules by:

module load vtk

wannier90

Description

Wannier90 is an open-source code released under GPLv2 for generating maximally-localized Wannier functions and using them to compute advanced electronic properties of materials with high efficiency and accuracy.

Versions

  • Anvil: 3.1.0

Module

You can load the modules by:

module load wannier90

xalt

Versions

  • Bell: 1.1.2
  • Brown: 1.1.2
  • Scholar: 1.1.2, 2.7.1
  • Gilbreth: 1.1.2

Module

You can load the modules by:

module load xalt

zlib

Description

A free, general-purpose, legally unencumbered lossless data-compression library.

Versions

  • Bell: 1.2.11
  • Brown: 1.2.11, 1.2.11-generic
  • Scholar: 1.2.11, 1.2.11-generic
  • Gilbreth: 1.2.11, 1.2.11-generic
  • Negishi: 1.2.13
  • Anvil: 1.2.11
  • Workbench: 1.2.11-generic

Module

You can load the modules by:

module load zlib

zstd

Description

Zstandard, or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library.

Versions

  • Brown: 1.4.3

Module

You can load the modules by:

module load zstd

nextflow

Description

Nextflow is a bioinformatics workflow manager that enables the development of portable and reproducible workflows. It supports deploying workflows on a variety of execution platforms including local, HPC schedulers, AWS Batch, Google Cloud Life Sciences, and Kubernetes. Additionally, it provides support for manage your workflow dependencies through built-in support for Conda, Spack, Docker, Podman, Singularity, Modules, and more.

Versions

  • Negishi: 22.10.1

Module

You can load the modules by:

module load nextflow

Note: Docker is not available on Purdue clusters, so use "-profile singularity", environment modules, or conda for running NextFlow pipelines.

Running Nextflow can be computing or memory intensive. Please do not run it on login nodes, as this might affect other users sharing the same login node with you.

Wrap nextflow into slurm jobscript

The easiest method to use nextflow on clusters is to place the nextflow run command into a batch script and submitting it to Slurm with sbatch. The manager process will run on the allocated compute node, and all tasks are configured to use the local executor.

#!/bin/bash
#SBATCH -A myQueue
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 12
#SBATCH --job-name=nextflow
#SBATCH --mail-type=FAIL,BEGIN,END
#SBATCH --error=%x-%J-%u.err
#SBATCH --output=%x-%J-%u.out

module load nextflow

nextflow run main.nf -profile singularity

Nextflow submits tasks as slurm jobs

Nextflow can also submit its tasks to Slurm instead of running them on the local host. Place the following file named nextflow.config in your Nextflow working directory: 
process {
        executor = 'slurm'
        queueSize = 50
        pollInterval = '1 min'
        queueStatInterval = '5 min'
        submitRateLimit = '10 sec'
}

Please do not change the above default configuration. Nextflow workflow manager process can generate a disruptive amount of communication requests to Slurm and the configuration file is used to reduce the frequency of those requests.

clusterOptions

Inside the individual process definitions in your scripts, you will need to specify the clusterOptions variable to provide your queue and computing resources appropriate for that task. This can be done by adding something in the pattern of clusterOptions='-A standby -N1 -n1 -c12 -t 1:00:00' to the top of your task process blocks.

 

Below is a simple example to run Fastqc: 
nextflow.enable.dsl=2
  
process FASTQC {
   clusterOptions='-A standby -N1 -n1 -c4 -t 00:30:00'
   input:
   path reads
   script:
   """
   mkdir -p fastqc_out
   module load biocontainers fastqc
   fastqc -o fastqc_out ${reads}
   """
}
reads_ch = Channel.fromPath( 'reads/fastq/*.fastq.gz' )

workflow {
  FASTQC(reads_ch)
}
Using clusterOptions='-A standby -N1 -n1 -c4 -t 00:30:00' , each nextflow task will be submitted to standby queue requesting 4 cores and 30 mins walltime.

nf-core

Description

A community effort to collect a curated set of analysis pipelines built using Nextflow and tools to run the pipelines.

Home page: https://nf-co.re

Versions

  • Anvil: 2.7.2, 2.8
  • Negishi: 2.7.2, 2.8

Commands

          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.8 - https://nf-co.re


                                                                                                    
 Usage: nf-core [OPTIONS] COMMAND [ARGS]...                                                         
                                                                                                    
 nf-core/tools provides a set of helper tools for use with nf-core Nextflow pipelines.              
 It is designed for both end-users running pipelines and also developers creating new pipelines.    
                                                                                                    
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
│ --version                        Show the version and exit.                                      │
│ --verbose        -v              Print verbose output to the console.                            │
│ --hide-progress                  Don't show progress bars.                                       │
│ --log-file       -l    Save a verbose log to a file.                                   │
│ --help           -h              Show this message and exit.                                     │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands for users ─────────────────────────────────────────────────────────────────────────────╮
│ list        List available nf-core pipelines with local info.                                    │
│ launch      Launch a pipeline using a web GUI or command line prompts.                           │
│ download    Download a pipeline, nf-core/configs and pipeline singularity images.                │
│ licences    List software licences for a given workflow (DSL1 only).                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands for developers ────────────────────────────────────────────────────────────────────────╮
│ create            Create a new pipeline using the nf-core template.                              │
│ lint              Check pipeline code against nf-core guidelines.                                │
│ modules           Commands to manage Nextflow DSL2 modules (tool wrappers).                      │
│ subworkflows      Commands to manage Nextflow DSL2 subworkflows (tool wrappers).                 │
│ schema            Suite of tools for developers to manage pipeline schema.                       │
│ bump-version      Update nf-core pipeline version number.                                        │
│ sync              Sync a pipeline TEMPLATE branch with the nf-core template.                     │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

Module

You can load the modules by:

module load nf-core

List available pipelines

To check all available pipelines:

$ nf-core list
                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.8 - https://nf-co.re

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Pipeline Name        ┃ Stars ┃ Latest Release ┃      Released ┃ Last Pulled ┃ Have latest release?  ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━┩
│ funcscan             │    29 │          1.1.0 │    2 days ago │           - │ -                     │
│ smrnaseq             │    49 │          2.2.0 │    4 days ago │           - │ -                     │
│ rnafusion            │    95 │          2.3.4 │    4 days ago │           - │ -                     │
│ rnaseq               │   604 │         3.11.2 │    5 days ago │  5 days ago │ No (dev - 4b7695a)    │
│ demultiplex          │    26 │          1.2.0 │    5 days ago │           - │ -                     │
│ differentialabundan… │    19 │          1.2.0 │   2 weeks ago │  2 days ago │ Yes (v1.2.0)          │
│ mhcquant             │    21 │          2.4.1 │   3 weeks ago │           - │ -                     │
│ viralintegration     │     8 │          0.1.0 │  1 months ago │           - │ -                     │
│ quantms              │     8 │          1.1.1 │  1 months ago │           - │ -                     │
│ viralrecon           │    93 │          2.6.0 │  1 months ago │           - │ -                     │
│ airrflow             │    24 │            3.0 │  1 months ago │           - │ -                     │
│ scrnaseq             │    81 │          2.2.0 │  1 months ago │           - │ -                     │
│ epitopeprediction    │    25 │          2.2.1 │  1 months ago │           - │ -                     │
│ isoseq               │    12 │          1.1.4 │  2 months ago │           - │ -                     │
│ taxprofiler          │    49 │          1.0.0 │  2 months ago │ 2 weeks ago │ No (master - c3f1adf) │
│ nanoseq              │   109 │          3.1.0 │  2 months ago │           - │ -                     │
│ cutandrun            │    41 │            3.1 │  2 months ago │           - │ -                     │
│ circdna              │    12 │          1.0.2 │  2 months ago │           - │ -                     │
│ ampliseq             │   111 │          2.5.0 │  2 months ago │           - │ -                     │
│ mag                  │   126 │          2.3.0 │  2 months ago │           - │ -                     │
│ nascent              │     8 │          2.1.1 │  2 months ago │           - │ -                     │
│ phyloplace           │     3 │          1.0.0 │  2 months ago │           - │ -                     │
│ proteinfold          │    21 │          1.0.0 │  3 months ago │           - │ -                     │
│ crisprseq            │     8 │            1.0 │  3 months ago │           - │ -                     │
│ hic                  │    48 │          2.0.0 │  3 months ago │ 2 weeks ago │ Yes (v2.0.0)          │
│ sarek                │   235 │          3.1.2 │  4 months ago │           - │ -                     │
│ fetchngs             │    78 │            1.9 │  4 months ago │           - │ -                     │
│ methylseq            │   104 │          2.3.0 │  4 months ago │           - │ -                     │
│ atacseq              │   134 │            2.0 │  5 months ago │           - │ -                     │
│ eager                │    91 │          2.4.6 │  5 months ago │  2 days ago │ Yes (v2.4.6)          │
│ coproid              │     7 │          1.1.1 │  6 months ago │           - │ -                     │
│ hgtseq               │    16 │          1.0.0 │  6 months ago │           - │ -                     │
│ hlatyping            │    41 │          2.0.0 │  6 months ago │           - │ -                     │
│ chipseq              │   144 │          2.0.0 │  7 months ago │           - │ -                     │
│ rnavar               │    16 │          1.0.0 │ 10 months ago │           - │ -                     │
│ mnaseseq             │     9 │          1.0.0 │ 11 months ago │           - │ -                     │
│ hicar                │     3 │          1.0.0 │ 12 months ago │           - │ -                     │
│ bamtofastq           │     8 │          1.2.0 │   1 years ago │           - │ -                     │
│ bacass               │    42 │          2.0.0 │   2 years ago │  5 days ago │ Yes (v2.0.0)          │
│ bactmap              │    41 │          1.0.0 │   2 years ago │           - │ -                     │
│ metaboigniter        │    10 │          1.0.1 │   2 years ago │           - │ -                     │
│ diaproteomics        │    10 │          1.2.4 │   2 years ago │           - │ -                     │
│ clipseq              │    13 │          1.0.0 │   2 years ago │           - │ -                     │
│ pgdb                 │     3 │          1.0.0 │   2 years ago │           - │ -                     │
│ dualrnaseq           │    12 │          1.0.0 │   2 years ago │           - │ -                     │
│ cageseq              │     9 │          1.0.2 │   2 years ago │           - │ -                     │
│ proteomicslfq        │    29 │          1.0.0 │   3 years ago │           - │ -                     │
│ imcyto               │    20 │          1.0.0 │   3 years ago │           - │ -                     │
│ slamseq              │     4 │          1.0.0 │   3 years ago │           - │ -                     │
│ callingcards         │     1 │            dev │             - │           - │ -                     │
│ circrna              │    27 │            dev │             - │           - │ -                     │
│ fastquorum           │     8 │            dev │             - │           - │ -                     │
│ genomeannotator      │     9 │            dev │             - │           - │ -                     │
│ genomeassembler      │    12 │            dev │             - │           - │ -                     │
│ gwas                 │    12 │            dev │             - │           - │ -                     │
│ lncpipe              │    25 │            dev │             - │           - │ -                     │
│ metapep              │     3 │            dev │             - │           - │ -                     │
│ metatdenovo          │     2 │            dev │             - │           - │ -                     │
│ nanostring           │     2 │            dev │             - │           - │ -                     │
│ pangenome            │    23 │            dev │             - │ 2 weeks ago │ No (a_brave_new_world │
│                      │       │                │               │             │ - 6aa9b39)            │
│ radseq               │     0 │            dev │             - │           - │ -                     │
│ raredisease          │    37 │            dev │             - │           - │ -                     │
│ rnadnavar            │     0 │            dev │             - │           - │ -                     │
│ rnasplice            │     3 │            dev │             - │           - │ -                     │
│ scflow               │    19 │            dev │             - │           - │ -                     │
│ spatialtranscriptom… │    19 │            dev │             - │           - │ -                     │
│ spinningjenny        │     0 │            dev │             - │           - │ -                     │
│ variantcatalogue     │     3 │            dev │             - │           - │ -                     │
└──────────────────────┴───────┴────────────────┴───────────────┴─────────────┴───────────────────────┘

Download pipelines

It is highly recommended to download pipelines to clusters before running them. Using singularity containers to run these pipelines are also recommended. Befor downloading, please set up environment variables NXF_SINGULARITY_CACHEDIR for singularity cache or NXF_CONDA_CACHEDIR for conda cache. Below is an example you can add to .bashrc for bash users.

export NXF_SINGULARITY_CACHEDIR="$SCRATCH/singularity/cache"
export NXF_CONDA_CACHEDIR="$SCRATCH/conda/cache"

Below is the example to download the rnaseq pipeline:

$ nf-core download rnaseq


                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.8 - https://nf-co.re


? Select release / branch: (Use arrow keys)
 » 3.11.2  [release]
   3.11.1  [release]
   3.11.0  [release]
   3.10.1  [release]
   3.10  [release]
   3.9  [release]
   3.8.1  [release]
   3.8  [release]
   3.7  [release]
   3.6  [release]
   3.5  [release]
   3.4  [release]
   3.3  [release]
   3.2  [release]
   3.1  [release]
   3.0  [release]
   2.0  [release]
   1.4.2  [release]
   1.4.1  [release]
   1.4  [release]
   1.3  [release]
   1.2  [release]
   1.1  [release]
   1.0  [release]

Run the pipeline

If users have downloaded the rnaseq pipeline to a folder called nf-core under $HOME, users can use it to run RNAseq analysis. Here is an example for running RNAseq analysis with human samples. 
#!/bin/bash
#SBATCH -A XXXX
#SBATCH --job-name=rnaseq    
#SBATCH --output=slurm-%A.%a.out 
#SBATCH --error=slurm-%A.%a.err  
#SBATCH --nodes=1                
#SBATCH --ntasks=1               
#SBATCH --cpus-per-task=64       
#SBATCH --time=24:00:00          
#SBATCH --mail-type=all  

module load nextflow

nextflow run $HOME/nf-core/nf-core-rnaseq-3.11.2/workflow/ \
              --input samplesheet.csv --outdir results \ 
              --genome GRCh37 -profile singularity

grace

Description

Grace is a WYSIWYG 2D plotting tool for the X Window System and M*tif.

Versions

  • Negishi: 5.1.25

Module

You can load the modules by:

module load grace

imagemagick

Description

ImageMagick is a software suite to create, edit, compose, or convert bitmap images.

Versions

  • Negishi: 7.0.8-7

Module

You can load the modules by:

module load imagemagick

xcb-util-image

Description

The XCB util modules provides a number of libraries which sit on top of libxcb, the core X protocol library, and some of the extension libraries. These experimental libraries provide convenience functions and interfaces which make the raw X protocol more usable. Some of the libraries also provide client-side code which is not strictly part of the X protocol but which have traditionally been provided by Xlib.

Versions

  • Negishi: 0.3.9

Module

You can load the modules by:

module load xcb-util-image

xcb-util-keysyms

Description

The XCB util modules provides a number of libraries which sit on top of libxcb, the core X protocol library, and some of the extension libraries. These experimental libraries provide convenience functions and interfaces which make the raw X protocol more usable. Some of the libraries also provide client-side code which is not strictly part of the X protocol but which have traditionally been provided by Xlib.

Versions

  • Negishi: 0.4.0

Module

You can load the modules by:

module load xcb-util-keysyms

xcb-util-renderutil

Description

The XCB util modules provides a number of libraries which sit on top of libxcb, the core X protocol library, and some of the extension libraries. These experimental libraries provide convenience functions and interfaces which make the raw X protocol more usable. Some of the libraries also provide client-side code which is not strictly part of the X protocol but which have traditionally been provided by Xlib.

Versions

  • Negishi: 0.4.0

Module

You can load the modules by:

module load xcb-util-renderutil

xcb-util-wm

Description

The XCB util modules provides a number of libraries which sit on top of libxcb, the core X protocol library, and some of the extension libraries. These experimental libraries provide convenience functions and interfaces which make the raw X protocol more usable. Some of the libraries also provide client-side code which is not strictly part of the X protocol but which have traditionally been provided by Xlib.

Versions

  • Negishi: 0.4.1

Module

You can load the modules by:

module load xcb-util-wm

libxp

Description

libXp - X Print Client Library.

Versions

  • Negishi: 1.0.3

Module

You can load the modules by:

module load libxp

libxscrnsaver

Description

XScreenSaver - X11 Screen Saver extension client library

Versions

  • Negishi: 1.2.2

Module

You can load the modules by:

module load libxscrnsaver

libxslt

Description

Libxslt is the XSLT C library developed for the GNOME project. XSLT itself is a an XML language to define transformation for XML. Libxslt is based on libxml2 the XML C library developed for the GNOME project. It also implements most of the EXSLT set of processor-portable extensions functions and some of Saxons evaluate and expressions extensions.

Versions

  • Negishi: 1.1.33

Module

You can load the modules by:

module load libxslt

mesa-glu

Description

This package provides the Mesa OpenGL Utility library.

Versions

  • Negishi: 9.0.2

Module

You can load the modules by:

module load mesa-glu

motif

Description

Motif - Graphical user interface GUI specification and the widget toolkit

Versions

  • Negishi: 2.3.8

Module

You can load the modules by:

module load motif

Singularity

Note: Singularity was originally a project out of Lawrence Berkeley National Laboratory. It has now been spun off into a distinct offering under a new corporate entity under the name Sylabs Inc. This guide pertains to the open source community edition, SingularityCE.

What is Singularity?

Singularity is a new feature of the Community Clusters allowing the portability and reproducibility of operating system and application environments through the use of Linux containers. It gives users complete control over their environment.

Singularity is like Docker but tuned explicitly for HPC clusters. More information is available from the project’s website.

Features

  • Run the latest applications on an Ubuntu or Centos userland
  • Gain access to the latest developer tools
  • Launch MPI programs easily
  • Much more

Singularity’s user guide is available at: sylabs.io/guides/3.8/user-guide

Example

Here is an example using an Ubuntu 16.04 image on ${resource.name}:

singularity exec /depot/itap/singularity/ubuntu1604.img cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"

Here is another example using a Centos 7 image:

singularity exec /depot/itap/singularity/centos7.img cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

Purdue Cluster Specific Notes

All service providers will integrate Singularity slightly differently depending on site. The largest customization will be which default files are inserted into your images so that routine services will work.

Services we configure for your images include DNS settings and account information. File systems we overlay into your images are your home directory, scratch, Data Depot, and application file systems.

Here is a list of paths:

  • /etc/resolv.conf
  • /etc/hosts
  • /home/$USER
  • /apps
  • /scratch
  • /depot

This means that within the container environment these paths will be present and the same as outside the container. The /apps, /scratch, and /depot directories will need to exist inside your container to work properly.

Creating Singularity Images

Due to how singularity containers work, you must have root privileges to build an image. Once you have a singularity container image built on your own system, you can copy the image file up to the cluster (you do not need root privileges to run the container).

You can find information and documentation for how to install and use singularity on your system:

We have version 3.8.0-1.el7 on the cluster. You will most likely not be able to run any container built with any singularity past that version. So be sure to follow the installation guide for version 3.8 on your system.

singularity --version
singularity version 3.8.0-1.el7

Everything you need on how to build a container is available from their user-guide. Below are merely some quick tips for getting your own containers built for ${resource.name}.

You can use a Definition File to both build your container and share its specification with collaborators (for the sake of reproducibility). Here is a simplistic example of such a file:

# FILENAME: Buildfile

Bootstrap: docker
From: ubuntu:18.04

%post
    apt-get update && apt-get upgrade -y
    mkdir /apps /depot /scratch

To build the image itself:

sudo singularity build ubuntu-18.04.sif Buildfile

The challenge with this approach however is that it must start from scratch if you decide to change something. In order to create a container image iteratively and interactively, you can use the --sandbox option.

sudo singularity build --sandbox ubuntu-18.04 docker://ubuntu:18.04

This will not create a flat image file but a directory tree (i.e., a folder), the contents of which are the container's filesystem. In order to get a shell inside the container that allows you to modify it, user the --writable option.

sudo singularity shell --writable ubuntu-18.04
Singularity: Invoking an interactive shell within container...

Singularity ubuntu-18.04.sandbox:~>

You can then proceed to install any libraries, software, etc. within the container. Then to create the final image file, exit the shell and call the build command once more on the sandbox.

sudo singularity build ubuntu-18.04.sif ubuntu-18.04

Finally, copy the new image to ${resource.name} and run it.

Apptainer

Note: Apptainer was formerly known as Singularity and is now a part of the Linux Foundation. When migrating from Singularity see the user compatibility documentation.

What is Apptainer?

Apptainer is an open-source container platform designed to be simple, fast, and secure. It allows the portability and reproducibility of operating systems and application environments through the use of Linux containers. It gives users complete control over their environment.

Apptainer is like Docker but tuned explicitly for HPC clusters. More information is available on the project’s website.

Features

  • Run the latest applications on an Ubuntu or Centos userland
  • Gain access to the latest developer tools
  • Launch MPI programs easily
  • Much more

Apptainer’s user guide is available at: apptainer.org/docs/user/main/introduction.html

Example

Here is an example using an Ubuntu 16.04 image on ${resource.name}:

apptainer exec /depot/itap/singularity/ubuntu1604.img cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"

Here is another example using a Centos 7 image:

apptainer exec /depot/itap/singularity/centos7.img cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)

Purdue Cluster Specific Notes

All service providers will integrate Apptainer slightly differently depending on site. The largest customization will be which default files are inserted into your images so that routine services will work.

Services we configure for your images include DNS settings and account information. File systems we overlay into your images are your home directory, scratch, Data Depot, and application file systems.

Here is a list of paths:

  • /etc/resolv.conf
  • /etc/hosts
  • /home/$USER
  • /apps
  • /scratch
  • /depot

This means that within the container environment these paths will be present and the same as outside the container. The /apps, /scratch, and /depot directories will need to exist inside your container to work properly.

Creating Apptainer Images

You can build on your system or straight on the cluster (you do not need root privileges to build or run the container).

You can find information and documentation for how to install and use Apptainer on your system:

We have version 1.1.6 (or newer) on the cluster. Please note that installed versions may change throughout cluster life time, so when in doubt, please check exact version with a --version command line flag:

apptainer --version
apptainer version 1.1.6-1

Everything you need on how to build a container is available from their user guide. Below are merely some quick tips for getting your own containers built for ${resource.name}.

You can use a Definition File to both build your container and share its specification with collaborators (for the sake of reproducibility). Here is a simplistic example of such a file:

# FILENAME: Buildfile

Bootstrap: docker
From: ubuntu:18.04

%post
    apt-get update && apt-get upgrade -y
    mkdir /apps /depot /scratch

To build the image itself:

apptainer build ubuntu-18.04.sif Buildfile

The challenge with this approach however is that it must start from scratch if you decide to change something. In order to create a container image iteratively and interactively, you can use the --sandbox option.

apptainer build --sandbox ubuntu-18.04 docker://ubuntu:18.04

This will not create a flat image file but a directory tree (i.e., a folder), the contents of which are the container's filesystem. In order to get a shell inside the container that allows you to modify it, user the --writable option.

apptainer shell --writable ubuntu-18.04
Apptainer>

You can then proceed to install any libraries, software, etc. within the container. Then to create the final image file, exit the shell and call the build command once more on the sandbox.

apptainer build ubuntu-18.04.sif ubuntu-18.04

Finally, copy the new image to ${resource.name} and run it.

Helpful?

Thanks for letting us know.

Please don't include any personal information in your comment. Maximum character limit is 250.
250
Thanks for your feedback.