Storage upgrade will mean faster, more reliable data access for Purdue researchers

September 27, 2011

An upgrade of the software underlying the data storage system used by many Purdue researchers will provide faculty and their students quicker, more robust and more secure access to their data.

ITaP and its Rosen Center for Advanced Computing will be upgrading the research computing archival storage system Fortress to the new, more powerful High Performance Storage System (HPSS) the week of Oct. 3-10. ITaP staff will migrate the data currently in Purdue's archive, a move that should be transparent to users once the migration is complete. The conversion will begin on Monday, Oct. 3, with the existing system placed in a read-only mode to allow for the migration. Fortress will be unavailable for the weekend starting on Friday, Oct. 7, with full service expected to return by 5 p.m. on Monday, Oct. 10.

The Rosen Center will hold a training session for those who would like to know more about the new system at 2:30 p.m. Tuesday, Oct. 4, in Stewart Center, Room 214A. If you plan to attend, please email bmadsen@purdue.edu.

“I believe that users will find their archive experience much improved with more robust and secure interfaces, faster access to data and the ability to better manage their data sets,” says Ramon Williamson, senior storage engineer for the Rosen Center.

The new software provides flexible and scalable hierarchical storage management that keeps recently used data on disk, for faster access, and less recently used data on tape. Benefits from the switch to HPSS will include increased transfer speed when using parallel FTP (PFTP) and the ability to provide scalable capacity and performance. HPSS is able to bring the power of multiple servers to bear on data flow bottlenecks.

HPSS also can aggregate the capacity and performance of many computers, disks and tape drives into a single virtual file system of exceptional size and versatility. This approach is able to meet otherwise unachievable demands of total storage capacity, file sizes, data rates and number of objects stored.

In addition, Williamson says, two add-on utilities implemented in the conversion project, HTAR and HSI, should help users access their data more easily and manage it efficiently.

Purdue’s research data archive use has increased five-fold in the past five years, making the old DiskXtender software less stable and straining its ability to access data at speeds now possible with modern networking and hardware. Moreover, DiskXtender vendor EMC stopped supporting the software as of August 2011.

HPSS is the result of a decade of collaboration among U.S. Department of Energy laboratories and IBM, with significant contributions by universities and other labs worldwide.

“HPSS is a proven standard in archival software, and is in use in many of the major research facilities around the country and the world.” Williamson says.

To find out more, see the rcac-help@purdue.edu.

Originally posted: September 27, 2011