Unscheduled Brown outage
Replacement hardware for Brown scratch has arrived and has been deployed. Engineers are performing validation and configuration to ensure the replacement and the filesystem are functioning as expected.
We do not have an ETA at the moment, and will provide next update by 6pm tomorrow (Wednesday, August 10th) or as soon as we have any new information. We greatly appreciate your patience.
Replacement hardware for the Brown scratch is expected to arrive on Tuesday, August 9.
We will provide an update by 6pm on Tuesday, August 9 or sooner as we have more information.
We currently expect vendor-provided replacement hardware for the Brown scratch to be delivered on Tuesday. The scratch filesystem remains down, and scheduling remains paused while engineers evaluate available options.
We appreciate your patience. We will provide an update by 6pm on Monday, August 8 or sooner as we have more information.
The vendor diagnosed defects in filesystem controllers and is shipping replacement hardware.
Brown scratch filesystem remains unavailable. We will provide next update by 9pm.
Work continues on troubleshooting the issue and bringing Brown back to normal operation. Engineers are working with the vendor to identify the source of the problem with the scratch file system.
Brown scratch remains unavailable, and any job that relies on scratch will not be able to perform I/O operations there.
We will provide another update by noon tomorrow, August 7.
The Brown cluster began experiencing issues with its scratch filesystem around 11:20am EDT. Engineers are currently diagnosing the issue and are working to identify a fix. Job scheduling has been paused while this issue is being addressed.
We will provide an update by 6pm.