Outages
-
Anvil is experiencing more issues related to the power outage yesterday in the Purdue Data Center. Users are currently unable to login via any method, SSH, Open On Demand, etc. Engineers have been dispatched to resolve the issue. This post will be up...
-
Update: As of 3:45pm, the Bell cluster has returned to production status. Scheduling is still paused on the Negishi cluster, and we will have an update by 5:00pm EDT The Bell and Negishi clusters began experiencing issues with power around 1:00pm EDT...
-
Data Depot degraded performance on RCAC clusters
Users of Data Depot on RCAC clusters are currently experiencing significant performance degradation. The symptoms manifest as delays in listing or accessing files in /depot, significant lags in terminal sessions (especially if you have Data Depot in...
-
Unscheduled Hammer OnDemand Outage
Open OnDemand services for the Hammer cluster are currently offline. Engineers are investigating a boot disk failure on the server that hosts the gateway.hammer.rcac.purdue.edu virtual machine.
-
Unscheduled Hammer Slurm outage
The Hammer cluster began experiencing issues with the Slurm scheduler around 5:00am, Thursday, July 6th. The Slurm scheduler is non-responsive, as a result, jobs will fail to schedule. Desktop and SSH access to Hammer login nodes is still available,...
-
The Geddes cluster began experiencing issues overnight. Engineers are currently diagnosing the issue and are working to identify a fix. Workloads will be unavailable while this issue is being addressed. We will provide an update by 12 PM.
-
Data Depot partial outage (network drives)
The Data Depot began experiencing issues with its network drive mapping capability around 1:30pm EDT. The symptoms manifest as users being unable to map their Data Depot spaces as network drives on their Windows, Mac or Linux laptops and workstation...
-
Around 2:10p EST, the Brown cluster began experiencing issues with home directory mounts. Job scheduling on the Brown cluster has been paused while engineers investigate the issue. We will provide an update by 5pm.
-
The Anvil cluster began experiencing issues with its scratch filesystem around 6:45pm EDT. Access to scratch directories may be slow or hang. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paus...
-
Unscheduled Scholar partial outage
The Scholar cluster began experiencing issues with its Thinlinc remote desktop (desktop.scholar.rcac.purdue.edu) and its RStudio Server (rstudio.scholar.rcac.purdue.edu) around 8:30am EDT. Engineers are currently diagnosing the issue and are working...
-
The Gilbreth cluster began experiencing issues with its scheduler spool filesystem around 10:30pm EDT on Saturday, March 18th, 2023. The problem manifests as an I/O error during new batch job submissions and in Open OnDemand gateway applications. Int...
-
The Bell cluster began experiencing issues with its scratch filesystem around 7:50pm EDT. File access operations (e.g. ls) may appear hanging. Logins to the Open OnDemand gateway (gateway.bell.rcac.purdue.edu) may appear sluggish or hanging as well....
-
The Bell cluster began experiencing issues with its scratch filesystem around 12:55pm EST. File access operations (e.g. ls) may appear hanging. Logins to the Open OnDemand gateway ( gateway.bell.rcac.purdue.edu) may appear sluggish or hanging as wel...
-
The Anvil cluster began experiencing issues with its scratch and project file system around 10:00am EST. Access to scratch and project directories may be slow or hang. Engineers are currently diagnosing the issue and are working with the vendor to id...
-
The Bell cluster began experiencing issues with its Lustre scratch filesystem around 12:30pm EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed. We w...
-
Data Depot Degraded Performance
The data depot began experiencing issues around 9:50am EST. While engineers work to diagnose and fix this issue, users may notice degraded performance in the form of sluggish I/O operations performed on Data Depot. This may also cause slow logins for...
-
Scheduling paused on multiple clusters
The Bell, Brown, Gilbreth, Halstead, and Scholar clusters began experiencing issues with their Data Depot mounts around 9:50am EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while...
-
Scheduling Paused on Brown, Gilbreth, Halstead, and Hammer
As of 11:30am EDT, the Brown, Gilbreth, Halstead, and Hammer clusters began experiencing issues with their filesystems which may cause login failures. Engineers are currently investigating the root cause, and in the interim, job scheduling has been p...
-
Unscheduled Scholar Gateway outage
The Scholar cluster began experiencing issues with its OnDemand Gateway around Sunday, October 16th, 2022 at 9:00pm EDT. The issue manifests as connection to gateway.scholar.purdue.edu timing out. Engineers are currently diagnosing the issue with the...
-
The Bell Gateway began experiencing issues following the Oct 4-6th Bell Maintenance. In particular, gateway applications have been observed to fail upon attempting to connect to the application after launching the job. Our engineers are investigating...