Fortress User Guide

Collapse Topics

    Overview of Fortress
        Fortress Overview

    Accounts
        Accounts on Fortress

    File Storage and Transfer
        File Transfer
            Globus
            Windows Network Drive / SMB
            HSI
            HTAR


    Common error messages
        HSI/HTAR: put: Error -5 on transfer

    Common Questions
        How can my collaborators outside Purdue get access to Fortress?
        What is the best way to access my data?
        Do I need to do anything to my firewall to access Fortress?
        Can I set up a shared space for my research group to share data?
        Can I download HSI or HTAR binaries for my OS platform?
        How can I verify the contents of my archives while using HTAR?
        What limitiations does Fortress have?


Overview of Fortress Fortress Overview

Fortress Overview

The Fortress system is a large, long-term, multi-tiered file caching and storage system utilizing both online disk and robotic tape drives.

Fortress uses an IBM T3584 robotic tape library with a capacity of over 10 PB.

Storage Subsystem Current Capacity Hardware
Disk Cache Over 200 TB 3 IBM DS3512 Storage arrays
Long-Term Storage Over 10 PB LTO-6 Robotic Tape Library

All files stored on Fortress appear in at least two separate storage devices:

  • One copy is permanently on tape.
  • Recently used or files smaller than 100MB have their primary copy stored on a conventional spinning disk storage array (disk cache). Disk cache provides a rapid restore time.

Both primary and secondary copies of larger files reside on separate tape cartridges in the robotic tape library. After a period of inactivity, HPSS will migrate files from disk cache to tape.

Fortress writes two copies of every file either to two tapes, or to disk and a tape, to protect against medium errors. Unfortunately, Fortress does not automatically switch to the alternate copy when it has trouble accessing the primary. If it seems to be taking an extraordinary amount of time to retrieve a file (hours), please either email rcac-help@purdue.edu. We can then investigate why it is taking so long. If it is an error on the primary copy, we will instruct Fortress to switch to the alternate copy as the primary and recreate a new alternate copy.

Fortress Storage Quota

There is currently no quota on Fortress disk use. Fortress userswill receive a monthly email report showing your current Fortress usage.

Files belonging to deleted accounts will also be retained, but inaccessible except by special request after the accounts have been terminated. The files will be kept for no more than ten years or the usability of the media on which they are stored, whichever comes first.

Fortress File Recovery

Data on Fortress is not backed up elsewhere in a traditional sense. New and modified files in the disk cache are migrated to tape within 30 minutes, and Fortress maintains two copies of every file on different media to protect against media failures, but there is no backup protecting against accidental deletions.

If you remove or overwrite a file on Fortress, it is gone. You cannot request to have it retrieved.

Accounts

Accounts on Fortress

Obtaining an Account

All Purdue faculty, staff, and students with the approval of their advisor may request access to Fortress. Refer to the Accounts / Access page for more details on how to request access.

Outside Collaborators

Your Departmental Business Office can submit a Request for Privileges (R4P) to provide access to collaborators outside Purdue, including recent graduates. Instructions are at http://www.purdue.edu/hr/pdf/r4pRequestorInstructions.pdf and the Request form is at https://www.purdue.edu/apps/account/r4p

More Accounts Information

    Accounts on Fortress

Accounts Accounts on Fortress

Accounts on Fortress

Obtaining an Account

All Purdue faculty, staff, and students participating in the Community Cluster program have access to Fortress along with their cluster nodes and scratch space.

Additionally, all Purdue faculty, staff, and students with the approval of their advisor may also request access to Fortress. Refer to the Accounts / Access page for more details on how to request access.

Research groups may, upon request, be assigned a group data storage space within Fortress to facilitate sharing of research data. Access to this space is controlled a UNIX group assigned to the research group. Faculty should contact ITaP at rcac-help@purdue.edu to create a shared space for their research group.

ITaP research computing resources are not intended to store data protected by Federal privacy and security laws (e.g., HIPAA, ITAR, classified, etc.). It is the responsibility of the faculty partner to ensure that no protected data is stored on the systems.

  • Particularly in the case of group storage, please keep in mind that such spaces are, by design, accessible by others and should not be used to store private information such as grades, login credentials, or personal data.

Outside Collaborators

Your Departmental Business Office can submit a Request for Privileges (R4P) to provide access to collaborators outside Purdue, including recent graduates. Instructions are at http://www.purdue.edu/hr/pdf/r4pRequestorInstructions.pdf and the Request form is at https://www.purdue.edu/apps/account/r4p

Login & Keytabs

It is not possible to login directly to Fortress via SSH, SCP, or SFTP. You may access your files there efficiently using HSI or HTAR. Windows Network Drive/SMB access is possible, though with significant performance loss.

A Kerberos keytab file is required to log into Fortress via HSI or HTAR. However, all ITaP research systems may access Fortress without any Kerberos keytab preparation. If for some reason you lose your keytab, you may easily regenerate one on any ITaP research system by running the command fortresskey.

However, to access Fortress from a personal or departmental computer, you will need to first copy your Kerberos keytab file to the computer you wish to use. This keytab can be found in your research home directory, within the hidden subdirectory named ".private" as the file "hpss.keytab" (.private/hpss.keytab). This keytab will allow you to access HPSS services without needing to type a password and will remain valid for 90 days. Your keytab on ITaP research systems will automatically be regenerated after this time, and you will need to re-copy the new keytab file to any other computers you use to directly access Fortress then.

If you do not have an account on any ITaP research systems other than Fortress, you will need to generate a keytab file using the web interface:

File Storage and Transfer

File Storage and Transfer for Fortress

Your home directory on Fortress is the default directory that in which your archive files are stored.

On Fortress, your home directory will appear as /home/myusername, but this is not the same directory as your home directory on any other ITaP system. Your home directory on Fortress is your long-term storage directory for all ITaP systems.

The following link will take you to more information about transferring files in and out of Fortress.

    File Transfer
        Globus
        Windows Network Drive / SMB
        HSI
        HTAR


File Storage and Transfer File Transfer

File Transfer

Fortress supports several methods for file transfer. Use the links below to learn more about these methods.

Fortress does not support SCP, FTP, or SFTP.

    Globus
    Windows Network Drive / SMB
    HSI
    HTAR

File Storage and Transfer File Transfer Globus

Globus

Globus, previously known as Globus Online, is a powerful and easy to use file transfer service that is useful for transferring files virtually anywhere. It works within ITaP's various research storage systems; it connects between ITaP and remote research sites running Globus; and it connects research systems to personal systems. You may use Globus to connect to your home, scratch, and Fortress storage directories. Since Globus is web-based, it works on any operating system that is connected to the internet. The Globus Personal client is available on Windows, Linux, and Mac OS X. It is primarily used as a graphical means of transfer but it can also be used over the command line.

Globus Web:

  • Navigate to http://transfer.rcac.purdue.edu
  • Click "Proceed" to log in with your Purdue Career Account.
  • On your first login it will ask to make a connection to a Globus account. If you already have one - sign in to associate with your Career Account. Otherwise, click the link to create a new account.
  • Now you're at the main screen. Click "File Transfer" which will bring you to a two-endpoint interface.
  • The endpoint for disk-based storage is named "purdue#rcac", however, you can start typing "purdue" and it will autocomplete.
  • The paths to research storage are the same as they are when you're logged into the clusters, but are provided below for reference.
    • Research Data Depot directory: /depot/mygroupname where mygroupname is the name of your group.
    • Fortress can be accessed at the "purdue#fortress" endpoint.
  • For the second endpoint, you can choose any other Globus endpoint, such as another research site, or a Globus Personal endpoint, which will allow you to transfer to a personal workstation or laptop.

Globus Personal Client setup:

  • On the endpoint page from earlier, click "Get Globus Connect Personal" or download it from here: Globus Connect Personal
  • Name this particular personal system and click "Generate Setup Key" on this page: Create Gloubs Personal endpoint
  • Copy the key and paste it into the setup box when installing the client for your system.
  • Your personal system is now available as an endpoint within the Globus transfer interface.

Globus Command Line:

For more information, please see Globus Support.

File Storage and Transfer File Transfer Windows Network Drive / SMB

Windows Network Drive / SMB

SMB (Server Message Block), also known as CIFS, is an easy to use file transfer protocol that is useful for transferring files between ITaP research systems and a desktop or laptop. You may use SMB to connect to your home, scratch, and Fortress storage directories. The SMB protocol is available on Windows, Linux, and Mac OS X. It is primarily used as a graphical means of transfer but it can also be used over the command line.

Note: to access Fortress through SMB file sharing, you must be on a Purdue campus network or connected through VPN.

Windows:

  • Windows 7: Click Windows menu > Computer, then click Map Network Drive in the top bar
  • Windows 8.1: Tap the Windows key, type computer, select This PC, click Computer > Map Network Drive in the top bar
  • In the folder location enter the following information and click Finish:

    • To access your home directory, enter \\samba.rcac.purdue.edu\myusername where myusername is your career account name.
    • To access your scratch space on Fortress, enter \\samba.rcac.purdue.edu\scratch. Once mapped, you will be able to navigate to data\m\myusername where m is the first letter of your username and myusername is your career account name. You may also navigate to any of the other cluster scratch directories from this drive mapping.
    • To access your Fortress long-term storage home directory, enter \\fortress-smb.rcac.purdue.edu\myusername where myusername is your career account name.
    • To access a shared Fortress group storage directory, enter \\fortress-smb.rcac.purdue.edu\group\mygroupname where mygroupname is the name of the shared group space.

  • You may be prompted for login information. Enter your username as onepurdue\myusername and your account password. If you forget the onepurdue prefix it will prevent you from logging in.
  • Your home, scratch, or Fortress directory should now be mounted as a drive in the Computer window.

Mac OS X:

  • In the Finder, click Go > Connect to Server
  • In the Server Address enter the following information and click Connect:

    • To access your home directory, enter smb://samba.rcac.purdue.edu/myusername where myusername is your career account name.
    • To access your scratch space on Fortress, enter smb://samba.rcac.purdue.edu\scratch. Once connected, you will be able to navigate to data\m\myusername where m is the first letter of your username and myusername is your career account name. You may also navigate to any of the other cluster scratch directories from this mount.
    • To access your Fortress long-term storage home directory, enter smb://fortress-smb.rcac.purdue.edu/myusername where myusername is your career account name.
    • To access a shared Fortress group storage directory, enter smb://fortress-smb.rcac.purdue.edu/group/mygroupname where mygroupname is the name of the shared group space.

  • You may be prompted for login information. Enter your username, password and for the domain enter onepurdue or it will prevent you from logging in.

Linux:

  • There are several graphical methods to connect in Linux depending on your desktop environment. Once you find out how to connect to a network server on your desktop environment, choose the Samba/SMB protocol and adapt the information from the Mac OS X section to connect.
  • If you would like access via samba on the command line you may install smbclient which will give you ftp-like access and can be used as shown below. SCP or SFTP is recommended over this use case. For all the possible ways to connect look at the Mac OS X instructions.
    smbclient //fortress-smb.rcac.purdue.edu/myusername -U myusername

File Storage and Transfer File Transfer HSI

HSI

HSI, the Hierarchical Storage Interface, is the preferred method of transferring files to and from Fortress. HSI is designed to be a friendly interface for users of the High Performance Storage System (HPSS). It provides a familiar Unix-style environment for working within HPSS while automatically taking advantage of high-speed, parallel file transfers without requiring any special user knowledge.

HSI is provided on all ITaP research systems as the command hsi. HSI is also available for download for many operating systems.

Interactive usage:

$ hsi

*************************************************************************
*                    Purdue University
*                  High Performance Storage System (HPSS)
*************************************************************************
* This is the Purdue Data Archive, Fortress.  For further information
* see http://www.rcac.purdue.edu/storage/fortress/
*
*   If you are having problems with HPSS, please call IT/Operational
*   Services at 49-44000 or send E-mail to rcac-help@purdue.edu.
*
*************************************************************************
Username: myusername  UID: 12345  Acct: 12345(12345) Copies: 1 Firewall: off [hsi.3.5.8 Wed Sep 21 17:31:14 EDT 2011]

[Fortress HSI]/home/myusername->put data1.fits
put  'test' : '/home/myusername/test' ( 1024000000 bytes, 250138.1 KBS (cos=11))

[Fortress HSI]/home/myusername->lcd /tmp

[Fortress HSI]/home/myusername->get data1.fits
get  '/tmp/data1.fits' : '/home/myusername/data1.fits' (2011/10/04 16:28:50 1024000000 bytes, 325844.9 KBS )

[Fortress HSI]/home/myusername->quit

Batch transfer file:

put data1.fits
put data2.fits
put data3.fits
put data4.fits
put data5.fits
put data6.fits
put data7.fits
put data8.fits
put data9.fits

Batch usage:

$ hsi < my_batch_transfer_file
*************************************************************************
*                    Purdue University
*                  High Performance Storage System (HPSS)
*************************************************************************
* This is the Purdue Data Archive, Fortress.  For further information
* see http://www.rcac.purdue.edu/storage/fortress/
*
*   If you are having problems with HPSS, please call IT/Operational
*   Services at 49-44000 or send E-mail to rcac-help@purdue.edu.
*
*************************************************************************
Username: myusername  UID: 12345  Acct: 12345(12345) Copies: 1 Firewall: off [hsi.3.5.8 Wed Sep 21 17:31:14 EDT 2011]
put  'data1.fits' : '/home/myusername/data1.fits' ( 1024000000 bytes, 250200.7 KBS (cos=11))
put  'data2.fits' : '/home/myusername/data2.fits' ( 1024000000 bytes, 258893.4 KBS (cos=11))
put  'data3.fits' : '/home/myusername/data3.fits' ( 1024000000 bytes, 222819.7 KBS (cos=11))
put  'data4.fits' : '/home/myusername/data4.fits' ( 1024000000 bytes, 224311.9 KBS (cos=11))
put  'data5.fits' : '/home/myusername/data5.fits' ( 1024000000 bytes, 323707.3 KBS (cos=11))
put  'data6.fits' : '/home/myusername/data6.fits' ( 1024000000 bytes, 320322.9 KBS (cos=11))
put  'data7.fits' : '/home/myusername/data7.fits' ( 1024000000 bytes, 253192.6 KBS (cos=11))
put  'data8.fits' : '/home/myusername/data8.fits' ( 1024000000 bytes, 253056.2 KBS (cos=11))
put  'data9.fits' : '/home/myusername/data9.fits' ( 1024000000 bytes, 323218.9 KBS (cos=11))
EOF detected on TTY - ending HSI session

For more information about HSI:

File Storage and Transfer File Transfer HTAR

HTAR

HTAR (short for "HPSS TAR") is a utility program that writes TAR-compatible archive files directly onto Fortress, without having to first create a local file. Its command line was originally based on tar, with a number of extensions added to provide extra features.

HTAR is provided on all ITaP Research Computing systems as the command htar. HTAR is also available for download for many operating systems.

Usage:

Create a tar archive on Fortress named data.tar including all files with the extension ".fits":

  
$ htar -cvf data.tar *.fits
HTAR: a   data1.fits
HTAR: a   data2.fits
HTAR: a   data3.fits
HTAR: a   data4.fits
HTAR: a   data5.fits
HTAR: a   /tmp/HTAR_CF_CHK_17953_1317760775
HTAR Create complete for data.tar. 5,120,006,144 bytes written for 5 member files, max threads: 3 Transfer time: 16.457 seconds (311.121 MB/s)
HTAR: HTAR SUCCESSFUL

Unpack a tar archive on Fortress named data.tar into a scratch directory for use in a batch job:

$ cd $RCAC_SCRATCH/job_dir
$ htar -xvf data.tar
HTAR: x data1.fits, 1024000000 bytes, 2000001 media blocks
HTAR: x data2.fits, 1024000000 bytes, 2000001 media blocks
HTAR: x data3.fits, 1024000000 bytes, 2000001 media blocks
HTAR: x data4.fits, 1024000000 bytes, 2000001 media blocks
HTAR: x data5.fits, 1024000000 bytes, 2000001 media blocks
HTAR: Extract complete for data.tar, 5 files. total bytes read: 5,120,004,608 in 18.841 seconds (271.749 MB/s )
HTAR: HTAR SUCCESSFUL

Look at the contents of the data.tar HTAR archive on Fortress:

$ htar -tvf data.tar
HTAR: -rw-r--r--  myusername/pucc 1024000000 2011-10-04 16:30  data1.fits
HTAR: -rw-r--r--  myusername/pucc 1024000000 2011-10-04 16:35  data2.fits
HTAR: -rw-r--r--  myusername/pucc 1024000000 2011-10-04 16:35  data3.fits
HTAR: -rw-r--r--  myusername/pucc 1024000000 2011-10-04 16:35  data4.fits
HTAR: -rw-r--r--  myusername/pucc 1024000000 2011-10-04 16:35  data5.fits
HTAR: -rw-------  myusername/pucc        256 2011-10-04 16:39  /tmp/HTAR_CF_CHK_17953_1317760775
HTAR: Listing complete for data.tar, 6 files 6 total objects
HTAR: HTAR SUCCESSFUL

Unpack a single file, "data5.fits", from the tar archive on Fortress named data.tar into a scratch directory.:

$ htar -xvf data.tar data5.fits
HTAR: x data5.fits, 1024000000 bytes, 2000001 media blocks
HTAR: Extract complete for data.tar, 1 files. total bytes read: 1,024,000,512 in 3.642 seconds (281.166 MB/s )
HTAR: HTAR SUCCESSFUL

HTAR Archive Verification

HTAR allows different types of content verification while creating archives. Users can ask HTAR to verify the contents of an archive during (or after) creation using the '-Hverify' switch. The syntax of this option is:

htar -Hverify=option[,option...] ... other arguments ... 

where option can be any of the following:

Option Explanation
info Compares tar header info with the corresponding values in the index.
crc Enables CRC checking of archive files for which a CRC was generated when the file is added to the archive.
compare Enables a byte-by-byte comparison of archive member files and their local file counterparts.
nocrc Disables CRC checking of archive files.
nocompare Disables a byte-by-byte comparison of archive member files and their local file counterparts.

Users can use a comma-separated list of options shown above, or a numeric value, or the wildcard all to specify the degree of verification. The numeric values for Hverify can be interpreted as follows:

0: Enables "info" verification.
1: Enables level 0 + "crc" verification.
2: Enables level 1 + "compare" verification.
all: Enables all comparison options.

An example to verify an archive during creation using checksums (crc):

htar -Hverify=1 -cvf abc.tar ./abc

An example to verify a previously created archive using checksums (crc):

htar -Hverify=1 -Kvf abc.tar

Please note that the time for verifying an archive increases as you increase the verification level. Carefully choose the option that suits your dataset best.

For details please see the HTAR Man Page.

For more information about HTAR:

Common error messages

Common Problems

Some common errors are listed below with their solutions.

    HSI/HTAR: put: Error -5 on transfer

Common error messages HSI/HTAR: put: Error -5 on transfer

HSI/HTAR: put: Error -5 on transfer

First, check your firewall settings, and ensure that there are no firewall rules interfering with connecting to Fortress. If firewalls are not responsible:

Open the file named /etc/hosts on your workstation, especially if you run a Debian or Ubuntu Linux distribution. Look for a line like:

127.0.1.1  hostname.dept.purdue.edu hostname

Replace the IP address 127.0.1.1 with the real IP address for your system. If you don't know your IP address, you can find it with the command:

host `hostname --fqdn`

Common Questions How can my collaborators outside Purdue get access to Fortress?

How can my collaborators outside Purdue get access to Fortress?

Your Departmental Business Office can submit a Request for Privileges (R4P) to provide access to collaborators outside Purdue, including recent graduates. Instructions are available and the request can be made on the request form. Once the R4P process is complete, you will need to add your outside collaborators to Fortress as you would any for any Purdue collaborator.

Common Questions What is the best way to access my data?

What is the best way to access my data?

HSI and HTAR: HSI provides a FTP-style interface taking advantage of the power of HPSS without requiring any special user knowledge. HTAR is a utility to aggregate a set of files into a single tar archive directly into Fortress, without requiring space to first create an archive.

Common Questions Do I need to do anything to my firewall to access Fortress?

Do I need to do anything to my firewall to access Fortress?

Yes, any machines using HSI or HTAR must have all firewalls (local and departmental) configured to allow open access from the following IP addresses:

128.210.251.141
128.210.251.142
128.210.251.143
128.210.251.144
128.210.251.145
128.210.251.146
128.210.251.148

Firewall issues may manifest with error messages like "put: Error -50 on transfer." If you are unsure of how to modify your firewall settings, please consult with your department's IT support or the documentation for your operating system. Access to Fortress is restricted to on-campus networks. If you need to directly access Fortress from off-campus, please use the Purdue VPN service before connecting.

Common Questions Can I set up a shared space for my research group to share data?

Can I set up a shared space for my research group to share data?

Yes, research groups may, upon request, be assigned a group data storage space within Fortress to facilitate sharing of research data. Access to this space is controlled through a UNIX group assigned to the research group.

ITaP research resources are not intended to store data protected by Federal privacy and security laws (e.g., HIPAA, ITAR, classified, etc.). It is the responsibility of the faculty partner to ensure that no protected data is stored on the systems.

Please keep in mind that such spaces are, by design, accessible by others and should not be used to store private information such as grades, login credentials, or personal data. Contact us at rcac-help@purdue.edu to create a group space for your group.

Common Questions Can I download HSI or HTAR binaries for my OS platform?

Can I download HSI or HTAR binaries for my OS platform?

Yes, visit the Downloads page to download HSI or HTAR packages for your operating system.

Note: If your username on your desktop does not match your career account username, HSI and HTAR require configuration to connect using your career account username:

  • For HSI, use the -l careeraccount option on the hsi command line.
  • For HTAR, set the HPSS_PRINCIPAL environment variable to your career account username:
    bash: export HPSS_PRINCIPAL=careeraccount
    csh/tcsh: setenv HPSS_PRINCIPAL=careeraccount

Common Questions How can I verify the contents of my archives while using HTAR?

How can I verify the contents of my archives while using HTAR?

You can ask HTAR to verify the contents of an archive during/after creation using the '-Hverify' switch. Please see the Fortress User Guide for details.

Common Questions What limitiations does Fortress have?

What limitiations does Fortress have?

Fortress has a few limitations that you should keep in mind:

  • Fortress does not support direct FTP, SFTP or SCP transfers.
  • Fortress does not support Unicode filenames. All filenames must contain only ASCII characters.
  • Fortressdoes not support sparse files.
  • HTAR has an individual file size limit of 64GB. If any files you are trying to archive with HTAR are greater than 64GB, then HTAR will immediately fail. This does not limit the number of files in the archive or the total overall size of the archive. To get around this limitation, try using the htar_large command. It is slower than using HTAR but it will work around the 64GB file size limit.