RCAC - Outages https://www.rcac.purdue.edu/index.php/news/rss/Outages en Sat, 07 Mar 2026 22:38:57 EST <![CDATA[MATH Datacenter Cooling issue - Job scheduling paused on Anvil/Gautschi]]> https://www.rcac.purdue.edu/index.php/news/7625 https://www.rcac.purdue.edu/index.php/news/7625 The MATH datacenter started experiencing issues with cooling systems around 12pm. Job scheduling on the Anvil and Gautschi clusters was paused shortly after and scheduling resumed at 1:30pm.

]]> Wed, 04 Mar 2026 12:00:00 -0500 Outages <![CDATA[Data Depot Filesystem issue: Scheduling Resumed]]> https://www.rcac.purdue.edu/index.php/news/7611 https://www.rcac.purdue.edu/index.php/news/7611 An internal portion of the Data Depot filesystem is currently offline, as a result, all scheduling has been paused until this issue is resolved.

Impact to you Attempts to read files that are on the affected storage may result in error messages

Our IT team is actively working with the vendor to restore service as quickly as possible. We will send an update as soon as more information is available.

]]> Wed, 11 Feb 2026 14:30:00 -0500 Outages <![CDATA[Gilbreth Remains Unavailable – Update Expected by 10:00 AM]]> https://www.rcac.purdue.edu/index.php/news/7608 https://www.rcac.purdue.edu/index.php/news/7608 The scheduled Math Data Center maintenance is complete. The Gilbreth cluster remains unavailable as we continue post‑maintenance work. The next update will be provided by 10:00 AM tomorrow (Friday), or sooner if available.

We appreciate your patience and understanding as we work to restore Gilbreth service.

]]> Fri, 06 Feb 2026 00:15:00 -0500 Outages <![CDATA[Unscheduled Data Depot Service Outage]]> https://www.rcac.purdue.edu/index.php/news/7607 https://www.rcac.purdue.edu/index.php/news/7607 RCAC’s Data Depot storage system is currently unavailable. Our team is actively investigating and working to restore service as quickly as possible.

We will provide updates as the situation develops and once service is fully restored.

We appreciate your patience and understanding.

]]> Thu, 05 Feb 2026 08:00:00 -0500 Outages <![CDATA[Globus access to Depot degraded; slow Depot logins and Depot access on clusters]]> https://www.rcac.purdue.edu/index.php/news/7605 https://www.rcac.purdue.edu/index.php/news/7605 Users of Data Depot on RCAC clusters are currently experiencing degraded performance, and some Globus transfers to and from Depot are failing or running slowly. In addition, some users may see slow Globus logins or be temporarily unable to log in to Globus when accessing Depot collections.

System monitoring has identified an issue where heavy job activity was overloading the Data Depot filesystem used by the clusters and Globus.

You may see the following impacts:

  • Globus transfers to and from Depot collections may fail, stall, or run much more slowly than usual.
  • Globus logins may be slow or occasionally fail when accessing Depot endpoints.
  • Jobs on RCAC clusters that read from or write to Depot may experience slow file access, delayed directory listings, or timeouts.

Our engineers are investigating the high load from a large number of concurrent jobs and are working to reduce the impact on Depot, Globus, and cluster workloads. Existing jobs will continue to run, but any that are heavily Depot‑I/O‑bound may run more slowly or see I/O errors until performance improves. We will provide another update by 5:00PM EST or sooner if the issue is resolved.

]]> Fri, 30 Jan 2026 15:00:00 -0500 Outages <![CDATA[Geddes Storage Outage]]> https://www.rcac.purdue.edu/index.php/news/7603 https://www.rcac.purdue.edu/index.php/news/7603 The storage system for Geddes started experiencing issues overnight. Any deployments using the geddes-standard-multinode storage class will experience issues accessing data. Engineers are currently investigating.

]]> Thu, 29 Jan 2026 08:00:00 -0500 Outages <![CDATA[Gilbreth Scheduling paused due to Scratch filesystem issue]]> https://www.rcac.purdue.edu/index.php/news/7597 https://www.rcac.purdue.edu/index.php/news/7597 We’re currently addressing an issue with the Scratch filesystem on Gilbreth, and job scheduling has been temporarily paused while we work on a fix.

Our team is actively investigating and will post updates as soon as more information is available or 4pm EST. Thank you for your patience and understanding!

]]> Thu, 22 Jan 2026 14:00:00 -0500 Outages <![CDATA[Unscheduled Cluster & Data Depot Outage]]> https://www.rcac.purdue.edu/index.php/news/7596 https://www.rcac.purdue.edu/index.php/news/7596 Data Depot and clusters began experiencing issues around 8:00AM EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.

We will provide an update by 2:00PM EST today.

]]> Tue, 20 Jan 2026 08:00:00 -0500 Outages <![CDATA[Unscheduled Weber outage]]> https://www.rcac.purdue.edu/index.php/news/7562 https://www.rcac.purdue.edu/index.php/news/7562 The Weber cluster began experiencing issues around 8:00 AM EST. Engineers are currently diagnosing the issue and are working to identify a fix. The cluster is unreachable at this time. No ETA is currently available for return to service

We will provide an update by 2:00 PM EST or sooner.

]]> Wed, 03 Dec 2025 08:00:00 -0500 Outages <![CDATA[Unscheduled gilbreth outage]]> https://www.rcac.purdue.edu/index.php/news/7560 https://www.rcac.purdue.edu/index.php/news/7560 The gilbreth cluster began experiencing issues around 9:00 AM EST. Engineers are currently working on a solution and expect for return to service by 11:30 AM EST.

]]> Tue, 02 Dec 2025 09:30:00 -0500 Outages <![CDATA[Unscheduled Data Depot Outage]]> https://www.rcac.purdue.edu/index.php/news/7443 https://www.rcac.purdue.edu/index.php/news/7443 The Data Depot storage system began experiencing issues starting around 4:30pm EDT today. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.

We will provide an update by 9pm.

]]> Sat, 01 Nov 2025 16:30:00 -0400 Outages <![CDATA[Geddes Storage Issues]]> https://www.rcac.purdue.edu/index.php/news/7439 https://www.rcac.purdue.edu/index.php/news/7439 The Geddes cluster began experiencing issues with its storage system around 4 AM this morning. Some users may be experiencing no space left on device errors or other storage related errors on their deployments. Engineers are currently diagnosing the issue and are working to identify a fix.

We will provide an update by 11 AM.

]]> Thu, 30 Oct 2025 08:30:00 -0400 Outages <![CDATA[Unscheduled Data Depot outage]]> https://www.rcac.purdue.edu/index.php/news/7430 https://www.rcac.purdue.edu/index.php/news/7430 Edit:

The Data Depot file system has returned to full service and scheduling has resumed on all clusters.

The Data Depot storage system began experience issues starting around 9am EDT this morning. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.

We will provide an update by 12pm (noon).

]]> Fri, 17 Oct 2025 09:00:00 -0400 Outages <![CDATA[Unscheduled Data Depot outage]]> https://www.rcac.purdue.edu/index.php/news/7421 https://www.rcac.purdue.edu/index.php/news/7421 Edit:

Data Depot functionality has been restored.

The Data Depot file system began experiencing issues with writes around 2:30pm EDT. The data migration process currently ongoing from Data Depot 2 to Data Depot 3 ran into an unexpected problem. Engineers have identified the problem and are correcting it. Users may have seen "no space left on device" for approximately 30 minutes. Job scheduling has been paused while this issue is being addressed.

We will provide an update by 5 PM.

]]> Wed, 15 Oct 2025 14:30:00 -0400 Outages <![CDATA[Problem For New User/Projects On Anvil]]> https://www.rcac.purdue.edu/index.php/news/7302 https://www.rcac.purdue.edu/index.php/news/7302 Anvil is experiencing a problem with new user and allocation propagation.

Our engineers are working on the fix, and will keep this updated.

The problem has been fixed at 5 pm.

]]> Mon, 18 Aug 2025 11:00:00 -0400 Outages <![CDATA[Unscheduled Gautschi outage]]> https://www.rcac.purdue.edu/index.php/news/7271 https://www.rcac.purdue.edu/index.php/news/7271 The Gautschi cluster began experiencing issues with cooling around 2:00pm EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.

No ETA at this time. We will update as soon as possible or by 5:00pm EST.

]]> Thu, 31 Jul 2025 14:00:00 -0400 Outages <![CDATA[Incorrect Account Email]]> https://www.rcac.purdue.edu/index.php/news/7255 https://www.rcac.purdue.edu/index.php/news/7255 Today, RCAC user management systems sent incorrect email messages to many faculty partners and their resource managers. Please ignore any recent email about expirations or removals. You may verify who has access to your resources through our site www.rcac.purdue.edu, at any time, or email rcac-help@purdue.edu if you have concerns..

Thank you!

]]> Tue, 15 Jul 2025 14:30:00 -0400 Outages <![CDATA[Power outage after thunderstorm]]> https://www.rcac.purdue.edu/index.php/news/7227 https://www.rcac.purdue.edu/index.php/news/7227 A power outage affected our datacenters at approx 3:15pm. It appears cluster services are restored, however access and authentication may be slowed while we work to get other core infrastructure back online.

OoD and desktop of Negishi and gilbreth may affect due to node unexpectedly rebooted.

Our engineer is working on the back up right now.

Update on 3:46 pm: Negishi and Gilbreth Nodes are back to normal.

]]> Wed, 18 Jun 2025 15:15:00 -0400 Outages <![CDATA[Unscheduled Weber Login Interruption]]> https://www.rcac.purdue.edu/index.php/news/7224 https://www.rcac.purdue.edu/index.php/news/7224 The Weber cluster began experiencing issues with logins around 10:00am EDT. Engineers are currently diagnosing the issue and are working to identify a fix.

We will provide an update by 2:00pm EDT.

]]> Tue, 10 Jun 2025 10:00:00 -0400 Outages <![CDATA[Anvil Network Interruption]]> https://www.rcac.purdue.edu/index.php/news/7223 https://www.rcac.purdue.edu/index.php/news/7223 Beginning at 10:00 AM EST on June 10th, the Anvil cluster experience a brief network interruption to fix an issue related to network connectivity. We are continuing to work on it and expected return to service is 11:00 AM EST the end of the day.

Update: At 19:00 tonight, engineer team will be swapping out Anvil's skyway. The interruption is expected to last until 19:30 tonight.

Update: At 07:30 AM on June 11th, engineer completed the repair & test. Anvil return to service.

]]> Tue, 10 Jun 2025 10:00:00 -0400 Outages