Unscheduled scratch outage on Rice, Scholar and Snyder

October 20, 2017  8:20am – 5:50pm
Rice, Scholar, Snyder

UPDATE: October 20, 2017  5:52pm

Scratch storage has been fully restored on all the affected clusters, and queues restarted. This concludes the outage.

UPDATE: October 20, 2017  2:11pm

Storage engineers continue working with the vendor on bringing Rice, Scholar, and Snyder scratch back to normal operation. We will provide further update no later than 6pm.

UPDATE: October 20, 2017  10:49am

Work continues on bringing Rice, Scholar, and Snyder scratch filesystem back to normal operation. Storage engineers are working with the vendor to identify the source of the problem. We will provide another update by 2pm.

ORIGINAL: October 20, 2017  9:24am

The scratch filesystem serving Rice, Scholar, and Snyder is currently unavailable.

Both currently running jobs and attempts to access files in scratch will block until the filesystem is back online.

Job scheduling on Rice, Scholar, and Snyder has been paused while storage engineers address the issue.

Originally posted: October 20, 2017  9:24am