As of about 11:00 am, the problem with the chilled water has been corrected, and scheduling has resumed on all RCAC clusters. Thank you for your patience. If you encounter any issues or have questions, please contact us at firstname.lastname@example.org
The temperature of the chilled water supply in the data center has exceeded its optimum, which affects the cooling for most of our research clusters. Physical plant and RCAC personnel are evaluating the problem and we expect more information by 11:00am.
In the meantime, we have temporarily stopped scheduling on the following clusters: Coates, Hansen, Rossmann, Carter, and Radon. Currently running jobs will continue to run, but newly submitted jobs will wait in the queue until we are sure the hardware temperatures will not exceed workable limits.