Outages and Maintenance
-
Scratch system failure on Rice, Snyder, Hammer
*** Update *** As of 7:00 pm, the problem on the scratch system has been corrected, and scheduling has resumed on all three affected clusters - Rice, Snyder, and Hammer. Update Storage engineers are working with the system vendor to evaluate a propos...
-
As of 2:35 pm, Conte cluster is returned to service. Scheduling is resumed in all queues. Update The source of the problem has been identified and the fix is underway. We anticipate returning Conte to service by 3pm today. Original message The Conte...
-
Emergency Maintenance on Rice, Snyder, Hammer
As of 7:15pm, all queues on these clusters have resumed scheduling. Nodes will continue to be upgraded as they finish current jobs and become available. In the interim, the clusters will run in a degraded state, but will continue to start new jobs...
-
The Data Depot file system was sporadically available for 2 hours today. Some jobs running on the Community Clusters paused during the instability but have resumed. We expect no job loss to have occurred. This issue is now resolved.
-
Update: April 13, 2017 5:02pm The EXRC cluster has been returned to service. Original Message: The EXRC cluster will be unavailable beginning at Thursday, April 13th, 2017 at 8:00am EDT, for scheduled maintenance. The cluster will return to full prod...
-
Rice, Snyder, Hammer, Scholar Maintenance
The Hammer, Rice, Scholar, and Snyder clusters have been returned to service. Please note that Thinlinc clients and web browser access can be found at: Rice: desktop.rice.rcac.purdue.edu Hammer: desktop.hammer.rcac.purdue.edu Snyder: desktop.snyder.r...
-
The Halstead cluster will be unavailable beginning at Thursday, April 6th, 2017 at 8:00am EDT, for scheduled maintenance. The cluster will return to full production by Thursday, April 6th, 2017 at 11:59pm EDT. During this time, Halstead will have the...
-
Halstead nodes continue to come back online. While the cluster is operating normally, the total amount of available nodes is not yet at full capacity. We will update on the situation by 6:00pm. Update: Scheduling has been restarted and jobs are cur...
-
Emergency Carter Cluster Maintenance
Update: Owner queues on Carter have been restarted. While Carter is currently deemed stable, performance is still impacted. Engineers are closely monitoring the situation and will take corrective action if necessary. Update: At this time, only Carter...
-
UPDATE: As of 11:45a, the Scholar cluster maintenance was completed. Cluster is back in service. The Scholar cluster will be unavailable beginning on Wednesday, March 15th, 2017 at 8:00am EDT, for scheduled maintenance. The cluster will return to f...
-
UPDATE: At this time, the maintenance has been completed and is back in service. The Thinlinc cluster will be unavailable starting at 5pm on March 14th until midnight for necessary maintenance and upgrades. During this time, the remote desktop servic...
-
The Fortress archival storage system is currently experiencing intermittent connectivity. We expect the situation to be resolved by approximately 1pm. UPDATE: Storage engineers have resolved the connectivity problems and Fortress is back in full prod...
-
Partial scratch outages on Rice, Snyder, Carter, Scholar and Hammer
The scratch filesystems serving Carter, Hammer, Rice, Scholar, and Snyder started behaving abnormally this morning. This may have affected some jobs, and anyone using one of the login nodes for these clusters may have had sessions freeze or seen dela...
-
The Research Data Depot has been restored to service. A portion of the systems serving the Research Data Depot have suffered a failure. Some systems using Depot have been affected, particularly research clusters and users accessing the Depot over NFS...
-
Conte and Hathi Cluster Maintenance
The Conte and Hathi clusters have been updated and returned to full production. This is a gentle reminder that the Conte and Hathi clusters will be undergoing a scheduled maintenance beginning at Tuesday, February 21st, 2017 at 8:00am EST. Please sa...
-
Unscheduled scratch outage on Rice, Snyder, and Hammer
The scratch filesystem serving Hammer, Rice, and Snyder is currently unavailable. Both currently running jobs and attempts to access files in scratch will block until the filesystem is back online. Job scheduling on Hammer, Rice, and Snyder has been...
-
Halstead MPI problem, scheduling paused
Following the security updates on Halstead, an issue was discovered that prevented multi-node MPI jobs from running properly. Scheduling on Halstead has been stopped, and systems engineers are working on fixing the issue. We will provide further stat...
-
Emergency Security Patching of RCAC Clusters
Due to a recent security vulnerability, the Carter, Halstead, Hammer, Radon, Rice, Scholar, and Snyder clusters will have their operating system upgraded to a newer version during February 2, 2017 5:00pm - March 2, 2017 5:00pm EST. Unlike other cl...
-
Unscheduled scratch outage on Conte
The scratch filesystem serving Conte is currently unavailable. Both currently running jobs and attempts to access files in scratch will block until the filesystem is back online. Job scheduling on Conte has been paused while storage engineers addres...
-
Connectivity issues to Research Data Depot
System monitoring has revealed intermittent issues connecting to the Research Data Depot on Thursday January 19. When this issue occurs, users will experience pauses when working in a UNIX shell on community cluster systems, or as interrupted or drop...