Purdue University

Link to update at July 20, 2022 5:57pm EDT UPDATE: July 20, 2022 5:57pm EDT

As of 5:54PM, Gilbreth maintenance has concluded and scheduling of jobs has been resumed. Please be advised that if you had not yet migrated your home directory prior to the start of maintenance, your new home directory will appear empty and the contents of your old home directory will be located at /home-old/$USER. You can copy your old home directory to your new home directory in its entirety using the copy-rcac-home command, or you can manually copy only the files you want using the file paths above. For more information, see our user guide on transferring homes.

Link to update at July 8, 2022 3:09pm EDT UPDATE: July 8, 2022 3:09pm EDT

The scope of this maintenance has changed from what was communicated earlier. Due to delays with hardware delivery caused by global supply chain shortages, we will not be performing new GPUs addition and queues re-configuration during July 20 maintenance.

GPU addition and queues conversion will be performed at a later date (TBD pending hardware delivery). Please continue using your current queues after the maintenance on July 20th.

Other announced changes will be carried out as planned. Please see updated details below.

Link to original posting ORIGINAL: July 20, 2022 8:00am - 5:00pm EDT

Gilbreth will be unavailable due to maintenance on July 20th 8:00am-5:00pm to allow for an expansion to Gilbreth’s resources. In response to the growing demand for hardware which can facilitate GPU-accelerated workloads, Gilbreth is being expanded to include many more modern A100, A30, and A10 GPUs.

At the same time, there will be significant other changes made to extend all the latest features of other community clusters to Gilbreth. Here are all of the major changes along with links to additional resources:

1. New pricing structure: Researchers may now purchase dedicated access to individual GPUs on Gilbreth at a reduced rate.
2. New home directories: Home directories on Gilbreth will be separated from the legacy home filesystem shared amongst several other clusters. A separate detailed announcement to this regard will be sent very shortly.
3. Replacement of notebook.gilbreth: Researchers will be able to use the Open OnDemand service to launch Jupyter notebooks and gain exclusive access to GPUs. The older Python notebook service at notebook.gilbreth.rcac.purdue.edu will be retired.
4. CUDA upgrade: CUDA drivers on Gilbreth will be upgraded to version 11.7.
5. Toolchain upgrades: Compilers and MPI libraries will be upgraded, and older versions of these libraries will be removed. Any existing Anaconda environments or user-installed applications which used deprecated libraries may need to be rebuilt.

These originally scheduled items are postponed to a later date due to vendor delivery delay:

• New A10, A30, and A100 GPUs: A total of 48 new Nvidia A10 GPUs, 72 new Nvidia A30 GPUs, and 28 new Nvidia A100-80GB GPUs are being incorporated into Gilbreth. This will increase Gilbreth GPU count by 60%.
• New queue names: All users will be able to submit jobs to new lab/PI-specific queues as well as a new “standby” queue. The “partner”, “highmem” and “long” queues will no longer be available. In order to move to these new queues, all jobs not finished prior to the July 20th maintenance will be deleted.

Any Slurm jobs which request a walltime which would take them past Wednesday, July 20th, 2022 at 8:00am EDT will not start before this maintenance work.