Bell
-
The Bell cluster began experiencing issues with its home and scratch directories filesystem around 12:40pm EDT. Problems manifest as hanging new logins and unresponsive established sessions. Engineers are currently diagnosing the issue and are workin...
-
Scheduling Paused on Multiple Clusters
At about 4:00 pm today (Wednesday, 21 July, 2021) System Engineers found an issue with the schedulers on the Bell, Brown, Gilbreth, Halstead, and Scholar clusters. Job scheduling has been paused while this is being investigated. Symptoms of this pro...
-
RCAC Whole-Floor Downtime and Power Work
The majority of the Research Computing computational resources will be unavailable July 30, 2021 7:00am - August 1, 2021 12:00pm EDT for a whole-floor downtime due to electrical power work in MATH and POD data centers. Along with a required preven...
-
Unscheduled Data Depot outage on multiple clusters
The Bell, Brown, Gilbreth, Halstead, Scholar, and Workbench clusters began experiencing issues with mounting old Data Depot filesystem around 12:30am EDT. Multiple nodes are flagged offline by an automatic check, and bioinformatics application suite...
-
Unscheduled Data Depot and community clusters outage
At about 9:30am EDT, Data Depot servers started experiencing a ramping high load. Coupled with an ongoing scaling issues with the metadata subsystem, this caused Data Depot to become increasingly unresponsive for both community clusters and network d...
-
Unscheduled Bell, Brown, Gilbreth, Halstead, Hammer, Scholar, and Data Depot outage
The Bell, Brown, Gilbreth, Halstead, Hammer, Scholar, and Data Depot cluster began experiencing issues with Data Depot mounting around 7:00am EDT. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been...
-
Unscheduled multiple clusters and Data Depot outage
The Bell, Brown, Gilbreth, Halstead, Hammer, Scholar, Workbench clusters and Data Depot servers began experiencing issues with Data Depot mounting on Wednesday, September 29th, 2021 around 4:40pm EDT. Engineers are currently diagnosing the issue and...
-
Unscheduled multiple clusters and Data Depot outage
The Bell, Brown, Gilbreth, Halstead, Hammer, Scholar, Workbench clusters and Data Depot began experiencing issues with intermittent high load on the Data Depot servers around 4:30pm EDT. Engineers are currently diagnosing the issue and are working to...
-
The Bell cluster began experiencing issues with high load and sluggish performance on the scratch filesystem around 1:20pm EDT. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this...
-
Research Computing Holiday Break
Research Computing personnel will observe the university winter break from 5:00pm EST EST on Wednesday, December 22nd, 2021, and will resume normal business hours on Monday, January 3rd, 2022. During this time, Research Computing services will conti...
-
The Bell cluster began experiencing issues with its scratch filesystem around 6:30pm EST. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed. We will prov...
-
The Bell cluster began experiencing issues with scheduler database around 11:35am EST. The problem manifests as freezing and/or "socket timed out" and "Unable to contact slurm controller" error messages upon the usual Slurm comman...
-
As of 8:00pm EST on Friday, February 11th, 2022 the Data Depot filesystem outage has been resolved and scheduling has been resumed on all clusters. The Bell, Brown, Gilbreth, Halstead, Scholar, Workbench, and Data Depot cluster began experiencing i...
-
Unscheduled Math data center cooling outage
The Math building data center began experience issues with its cooling system around 11:40am EST. As one of manifestations, users may experience issues while logging in to the Anvil, Bell, Gilbreth, Halstead, Workbench, and Data Depot clusters. To m...
-
Whole-Floor Cluster Maintenance
The majority of Research Computing computational resources (Bell, Brown, Geddes, Gilbreth, Halstead, Hammer, Scholar, Weber, and Workbench clusters) will be unavailable March 15, 2022 4:00pm - March 16, 2022 12:00pm EDT during Whole-Floor Data Depo...
-
Unscheduled Math Data Center Cooling Outage
The Math building data center began experience issues with its cooling system around 11:40am EDT. As one of manifestations, users may experience issues while logging in to the Anvil, Bell, Gilbreth, and Halstead clusters. To minimize thermal load on...
-
Unscheduled Data Depot Slowdown on Community Clusters
As of 9:00am EDT, users of community clusters may experience slowness while trying to access Data Depot (including loading modules, starting applications or reading data) . The symptoms appear on both login and compute nodes. System engineers are act...
-
The Bell cluster began experiencing issues with its scratch filesystem around 9:00pm EDT on Saturday, April 9th, 2022. Access to files in scratch may appear severely delayed or frozen. Engineers are currently diagnosing the issue and are working to...
-
Unscheduled campus power outage
Several Research Computing resources became affected by a campus power outage around 7:00pm EDT. Multiple login and compute nodes may have powered down, leading to jobs fail and/or requeue with a NODE_FAIL or similar status. Engineers are currently d...
-
Bell Scratch Degraded Storage (Returned to Service)
Bell Scratch is near capacity and performance is degraded. As of this morning, Bell Scratch was 94% full. This afternoon we paused scheduling as scratch was not responding consistently. We have more drives on order, but with global supply chain issue...