Article #848: Math Cluster Maintenance
An issue was discovered shortly after Conte, Hansen, Hathi, and Radon were brought back online with the /group path on several front-ends and nodes. A...
Outages and Maintenance
-
-
Article #853: Campus network outage
Networking to and from campus, and around large parts of campus are down. Many services are unreachable at the moment. We will provide updates as they...
-
Article #854: Campus networking outage
Networking to and from campus, and around large parts of campus are down. Many services are unreachable at the moment. We will provide updates as they...
-
Article #852: Network issues affecting Snyder cluster
The problem is now RESOLVED after the reboot of a router. ======= The network serving Snyder is currently experiencing issues. Attempts to log in to t...
-
Article #851: Unscheduled Storage Outage
The Isilon filesystem was restored to normal service and all affected clusters had it remounted as quickly as was sustainable by the filesystem. This...
-
Article #845: Conte Cluster Maintenance
Conte has been returned to normal operations as of Wednesday, May 18th, 2016 at 5:55am EDT. All upgrades were completed, though a small number of nod...
-
Article #844: Unscheduled Scratch Outage on Carter
UPDATE: The issue with Carter's scratch filesystem has been resolved. The filesystem is now available. Job scheduling on the cluster has been resum...
-
Article #837: Unscheduled Scheduling Outage on Hansen
Job scheduling on Hansen has returned to normal. This concludes the outage. Original Message: Hansen is not currently scheduling any new jobs. A file...
-
Article #836: Unscheduled Scratch Outage on Carter
The scratch storage on Carter and Scholar has been returned to normal operations. The rebuild process will be continuing in the background, so we wil...
-
Article #835: Standby queues paused on Hansen Cluster
Update As of 5:20pm the standby and standby-c queues have been started and their jobs are being scheduled for execution. The standby queues on Hansen...
-
Article #834: Unscheduled outage on Peregrine-1
Outage RESOLVED A misconfiguration that caused an unneeded IB driver to be loaded was fixed. Peregrine-1 is back online. Job scheduling is on. Origi...
-
Article #831: Unscheduled outage for Peregrine1
As of Monday, March 7th, 2016 at 12:30pm EST, the Peregrine1 cluster is unavailable due to a failed network switch in its datacenter. This switch is...
-
Article #828: ECN services outage - ITaP Research Computing systems impacted
Engineering Computing Network (ECN) will be performing staged patching and reboots of all of ECN's RedHat Linux workstations and servers to protect ag...
-
Article #825: Unscheduled Outage on Data Depot
The Depot filesystem checks have all completed cleanly and the Depot has been fully returned to normal operations. All queues on all clusters are sch...
-
Article #822: Unscheduled outage on Rice and Snyder
As of 9:15 PM, the Snyder and Rice clusters have been brought back into service after cooling was brought back online. Front-ends are operational and...
-
Article #821: Unscheduled scratch outage on Carter
There was an issue with the cluster's gateway switches, causing infiniband traffic to be incapable of IP over infiniband. This also caused an instabil...
-
Article #804: Cluster Maintenance - Conte
The scheduler issue has been resolved, and Conte has been returned to normal operations as of Wednesday, February 10th, 2016 at 9:30pm EST. Update: Fe...
-
Article #819: Unscheduled outage on Carter
The cause of this turned out to be a power loss to Carter's scratch filesystem and portions of the Data Depot, which has been restored now. Carter no...
-
Article #820: Unscheduled Outage in Math Data Center
Most of the impact of this turned out to be to the Depot storage system, which has now been restored to normal operations. All the other affected sys...
-
Article #809: Hathi & WinHPC Power Maintenance
The Hathi and WinHPC clusters will be unavailable beginning at Thursday, February 4th, 2016 at 6:00am EST, for scheduled maintenance to the power feed...