Overview of Fortress

The Fortress system is a large, long-term, multi-tiered file caching and storage system utilizing both online disk and robotic tape drives. ITaP upgraded Fortress from DXUL to HPSS in October of 2011.

Detailed Hardware Specification

Fortress uses an ADIC Scalar 10K robotic tape library from Quantum with a maximum capacity of 6.6 PB. Currently, Fortress has a usable capacity of 1.3 PB.

Storage Subsystem Current Capacity Hardware
Disk Cache 25 TB 2 IBM DS3512 Storage arrays
File Metadata Storage 7 TB IBM DS3524 Storage
with EXP3524 expansion tray
Long-Term Storage 1.3 PB LTO-IV Robotic Tape Library

All files stored on Fortress appear in at least two separate storage devices:

  • One copy is permanently on tape.
  • Recently used or files smaller than 100MB have their primary copy stored on a conventional spinning disk storage array (disk cache). Disk cache provides a rapid restore time.
Both primary and secondary copies of larger files reside on separate tape cartridges in the robotic tape library. After a period of inactivity, HPSS will migrate files from disk cache to tape.

To ensure optimal performance for all users, and to keep the Fortress system healthy, please remember the following tips:

  • Fortress operates most effectively with large files - 1GB or larger. If your data is comprised of smaller files, use HTAR to directly create archives in Fortress.
  • When working with files on cluster head nodes, use your home directory or a scratch file system, rather than editing or computing on files directly in Fortress. Copy any data you wish to archive to Fortress after computation is complete.
  • The HPSS software does not handle sparse files (files with empty space) in an optimal manner. Therefore, if you must copy a sparse file into HPSS, use HSI rather than the cp or mv commands.
  • Due to the sparse files issue, the rsync command should not be used to copy data into Fortress through NFS, as this may cause problems with the system.

Fortress runs Red Hat Enterprise Linux, version 5.6 and uses HPSS 7.3.3p1 from IBM and the HPSS Collaboration.