Outages
-
The Bell cluster continues to experience issues with Hardware. Engineers are currently diagnosing the issues and are working with vendors to schedule and perform repairs as quickly as possible. Job scheduling continues, but you may experience longer...
-
A section of the Bell cluster compute nodes began experiencing issues with power feed and cooling around 2:30pm EDT. Engineers have powered down affected nodes and are working to identify a fix. Some jobs may have ended up terminated or requeued. Jo...
-
The Brown cluster began experiencing issues with its scratch filesystem around 11:20am EDT. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed. We will pr...
-
MATH data center cooling outage
The Math building data center began experience issues with its cooling system around 1:40pm EDT. To minimize thermal load on the cooling infrastructure, job scheduling has been paused and all idle compute nodes on Anvil, Bell, Geddes, Gilbreth, and...
-
Bell Scratch Degraded Storage (Returned to Service)
Bell Scratch is near capacity and performance is degraded. As of this morning, Bell Scratch was 94% full. This afternoon we paused scheduling as scratch was not responding consistently. We have more drives on order, but with global supply chain issue...
-
The Halstead cluster began experiencing issues with its scratch file system around 8:00am EDT. The problem manifests as various I/O errors or hangs when reading, writing or listing scratch directories. Engineers are currently diagnosing the issue and...
-
Scheduling Paused on Brown and Hammer
Beginning around 2:00pm EDT, the ailing cooling systems for Brown and Hammer began experiencing issues. To reduce the thermal load on the systems, scheduling of new jobs has been paused on Brown and Hammer and will not be resumed until after tomorrow...
-
Unscheduled campus power outage
Several Research Computing resources became affected by a campus power outage around 7:00pm EDT. Multiple login and compute nodes may have powered down, leading to jobs fail and/or requeue with a NODE_FAIL or similar status. Engineers are currently d...
-
The Bell cluster began experiencing issues with its scratch filesystem around 9:00pm EDT on Saturday, April 9th, 2022. Access to files in scratch may appear severely delayed or frozen. Engineers are currently diagnosing the issue and are working to...
-
Unscheduled Weber ingress/egress outage
The Weber cluster's data transfer server (weber-sftp.rcac.purdue.edu) suffered a cooling fan failure around 8:30pm EDT on Saturday, April 9th, 2022. The cluster remains operation with the exception of ingress/egress of files via the affected server....
-
Gilbreth scratch degraded performance
Following last night's scratch outage, the Gilbreth scratch filesystem is currently functional but operates with partially degraded performance. Engineers have opened a support ticket with the vendor and monitor the state of the filesystem continuou...
-
Unscheduled Gilbreth cluster outage
The Gilbreth cluster began experiencing issues with its scratch filesystem around 7:00pm EDT. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed. We will...
-
Unscheduled Data Depot Slowdown on Community Clusters
As of 9:00am EDT, users of community clusters may experience slowness while trying to access Data Depot (including loading modules, starting applications or reading data) . The symptoms appear on both login and compute nodes. System engineers are act...
-
Unscheduled Math Data Center Cooling Outage
The Math building data center began experience issues with its cooling system around 11:40am EDT. As one of manifestations, users may experience issues while logging in to the Anvil, Bell, Gilbreth, and Halstead clusters. To minimize thermal load on...
-
Unscheduled Math data center cooling outage
The Math building data center began experience issues with its cooling system around 11:40am EST. As one of manifestations, users may experience issues while logging in to the Anvil, Bell, Gilbreth, Halstead, Workbench, and Data Depot clusters. To m...
-
As of 8:00pm EST on Friday, February 11th, 2022 the Data Depot filesystem outage has been resolved and scheduling has been resumed on all clusters. The Bell, Brown, Gilbreth, Halstead, Scholar, Workbench, and Data Depot cluster began experiencing i...
-
The Gilbreth cluster began experiencing issues with its Data Depot mounts around 9:00am EST. The /depot filesystem is not visible on some of the login and compute nodes. Engineers are currently diagnosing the issue and are working to identify a fix....
-
The Bell cluster began experiencing issues with scheduler database around 11:35am EST. The problem manifests as freezing and/or "socket timed out" and "Unable to contact slurm controller" error messages upon the usual Slurm comman...
-
The Weber cluster began experiencing issues with weber-sftp subsystem around 2:00pm EST. The problem affects ingress/egress path to the cluster. Engineers are currently diagnosing the issue and are working to identify a fix. We will provide an upda...
-
The Bell cluster began experiencing issues with its scratch filesystem around 6:30pm EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed. We will prov...