Outages and Maintenance
-
Partial outage affecting some Coates queues
Update - 6:45 pm Tuesday, 10 April 2012 ITaP engineers have found and repaired the network issue that was affecting Coates nodes type B, C and E. Job scheduling has been resumed for all queues. If you encounter any problems, please report them to rc...
-
Unscheduled outage to MATH datacenter
Update - 9:30pm, 4/1/2012: As of about 9:30pm, Sunday, 1 April, ITaP systems staff have returned Hansen to production status, and job scheduling is re-enabled. The scratch filesystem on Hansen has been restored with no apparent loss of files; if you...
-
PBS unavailable on Rossmann cluster
Due to a network issue, the server running the PBS software for Rossmann is unavailable. While the server is unavailable, attempts to use PBS commands ("qsub", "qstat", "pbsnodes") will fail with error messages like: qst...
-
Unscheduled outage to Rossmann cluster
At approximately 10:50pm, Thursday, March 15, the power distribution to large portions of the Rossmann cluster failed. These feeds also power the login nodes for the cluster, which, while unavailable, renders Rossmann unavailable for use. Power was r...
-
System Maintenance - Spring Break 2012
During the week of spring break, 2012, the Steele, Coates, and Rossmann clusters will each be down for maintenance for one day to install OS patches and update the PBS batch software to version 11.1. Additionally, the Radon cluster will be unavailabl...
-
Lustre unavailable on Hansen cluster
Update: As of 9:45pm, Lustre is back in production and scheduling has resumed on Hansen. Original Notice: As of approximately 8:00pm February 7, an issue was found the Lustre filesystem on Hansen making the filesystem unavailable for use. ITaP engine...
-
This morning, the PBS system on Coates developed an issue with the storage holding its internal state.While systems engineers are working on recovering it from backup, any new job submissions will not be possible, nor will you be able to query job st...
-
Fortress: ADIC Scalar 10k tape robot unavailable (1/4/2012)
Update - 1/9/2012 The repairs to the ADIC tape library have been completed and Fortress' tape functionality is back in operation. Update - 1/6/2012 Following work today by vendor engineers, the latest estimate for the ADIC tape robot's return to serv...
-
Hansen: unscheduled outage to Lustre scratch
Update The error condition on the Lustre filesystem has been cleared, and Hansen is back in production and accepting new jobs. Jobs already running should have resumed at the point where they were blocked waiting when the Lustre error occurred. This...
-
Fortress: ADIC Scalar 10k tape robot unavailable
Update 12/2/11 (4:15pm) The tape robot has been returned to service and Fortress is back in production. Please contact us at rcac-help@purdue.edu if you encounter further issues. Update 12/2/11: The ADIC Scalar 10K robot is temporarily down again wit...
-
Updated 11/30/11: Network engineers have identified the cause of the network issue in question, and have applied a workaround, which has restored the Hansen network to full functionality. The next maintenance window to address to address the root cau...
-
The LustreA scratch filesystem, used by Rossmann and Coates, suffered an unknown failure sometime in the early morning of November 15, 2011. LustreA was returned to normal operation at about 10:30am. Any jobs on those systems run overnight before t...
-
Fortress archive upgrade to HPSS
UPDATE: Archive Conversion moved to Oct. 7-14 Information Technology at Purdue (ITaP) is upgrading the research computing archival storage system Fortress. Currently based upon EMC's DiskXtender (DXUL), Fortress is being upgraded to new, more powerfu...
-
This week, ITaP engineers have been troubleshooting issues with the Coates cluster, with the most common symptom being PBS jobs that abort or restart after some period of run time. Late yesterday afternoon, a change was made to the cluster's networki...
-
ITaP research computers to be down two weekends in August for building upgrade
ITaP’s research computing systems will be shut down at least part of August 6-7 and August 13-14 because of an ongoing power upgrade project at the Mathematical Sciences Building. Some of the research computing systems also might be down or have to r...
-
Major research computing systems down during Aug. 5-17 for building upgrades
Major ITaP research computing systems will be shut down beginning at 5 p.m. Friday, Aug 5, including the Rossmann, Coates, Moffett and Radon clusters. The supercomputers are scheduled to be off until Wednesday, Aug. 17. An outage to complete power an...
-
MATH Datacenter upgrades, starting Friday, August 5
Beginning at 5:00 pm, Friday, August 5th, the Coates and Rossmann supercomputer clusters will be unavailable due to work to complete a power and cooling upgrade to the Math Sciences building datacenter. We estimate that these clusters will be unavail...
-
Aug. 5-17 research computing system outage FAQ
What’s happening? ITaP’s research computing systems will be shut down beginning at 5 p.m. Friday, Aug 5, including the Rossmann, Coates, Moffett and Radon clusters. The supercomputers are scheduled to be off until Wednesday, Aug. 17. Why? An outage r...
-
All RCAC systems unavailable some portion of Tue-Fri, 3/29-4/1
All RCAC systems will be unavailable on Tuesday, March 29th from 3:00am – 6:00pm. The Rossmann, Coates, Radon, and Moffett clusters will remain down through 6:00pm Thursday, March 31st. Update, 9:00am, March 29: Power has been restored to the Math b...
-
ITaP research computers to be down during building upgrades
What’s happening? ITaP’s research computing systems will be shut down beginning at 3 a.m. Tuesday, March 29. The Coates and Rossmann cluster supercomputers could be off through 6 p.m. Thursday, March 31. Why? An outage related to an ongoing power and...