Software: HTAR
Description
The HTAR program manipulates HPSS-resident archives, or archives that reside on a remote system (subject to the restrictions noted below), by writing files to, or retrieving files from, either HPSS or a remote FTP server.
Available Versions
| Unspecified: | groff version 1.19.2 |
|---|
# htar
HTAR — HPSS TAR Utility
## NAME
HTAR − HPSS TAR Utility
## PURPOSE
Manipulates HPSS-resident tar-format archives.
## SYNOPSIS
```bash
htar -{c|t|x|K|X} [-?] -f <Archive> [-B] [-d <debuglevel>] [-E]
[-F [user@]FTP_server[#port]] [-H <opt[:opt...]>] [-h]
[-I {IndexFile | .suffix}] [-L <inputlist>] [-M <maxfiles>] [-m]
[-n <days>] [-O] [-o] [-P] [-p] [-q] [-S <Bufsize>] [-T <Max Threads>]
[-V] [-v] [-w] [-Y [<Archive COS ID>][:<Index File COS ID>]] [Filespec | Directory ...]
```
## DESCRIPTION
The HTAR program manipulates HPSS-resident archives, or archives that reside on a remote system, by writing files to, or retrieving files from, either HPSS or a remote FTP server. Files are in POSIX tar format and may be read by standard tar programs.
HTAR can operate on remote FTP-resident archives if the executable was built with that capability and the remote FTP server supports the HPSS parallel transfer protocol.
Files specified by Filespec may be directories; directories are processed recursively. The -f Archive flag is required.
## WHY USE HTAR?
HTAR is optimized to create archive files directly in HPSS using multiple threads and buffering; it is often faster than creating local archives and copying them to HPSS. HTAR also creates a separate index file for random access into archives.
## "Archive" and "Member" files
"Archive file" refers to the tar-format file named by -f. "Member file" refers to individual files contained within the archive.
## Member File Size Constraints
Individual member files are limited to 68719476735 bytes due to POSIX tar size field limits.
## HTAR Index File
HTAR creates an index file (default suffix ".idx") pointing to member file positions within the archive. Use -X to build an index for non-HTAR tar files. The -I option can specify a different index filename or suffix.
## Use of Absolute Pathnames
HTAR strips leading / when extracting so extracted files are relative to the current working directory. The -K verify action uses absolute paths unless -Hrelpaths is specified.
## HTAR Consistency File
HTAR writes a consistency-check file as the last member; it is used to verify the archive and index. The -Hnocfchk option disables checks that use the consistency file for certain actions.
## Tar File Restrictions
POSIX USTAR limits path component lengths (prefix 154 chars, name 99 chars effective). If a path cannot be split, it cannot be archived.
## HPSS Default Directories
By default, the archive and index reside in the HPSS home directory. Use -P when creating archive paths containing non-existent subdirectories.
## Local Temporary Directory
HTAR uses TMPDIR (or /tmp) for temporary files.
## HTAR Command Options
Action flags: `-c`, `-t`, `-x`, `-X`, `-K`, `-D`, `-U` (one action required).
Filespec may be WildcardPath, Pathname, or Filename. Shell expands wildcards before passing to HTAR.
### Action Flags
- `-c` — Create a new HPSS-resident archive (overwrites existing archive without prompting).
- `-t` — List member files.
- `-x` — Extract member files (recurses directories). If no File parameter is given, extracts all files.
- `-K` — Verify archive contents according to verification options.
- `-X` — Build a new index by reading the entire tar file.
- `-D` — Soft-delete files by marking them deleted in the index.
- `-U` — Undelete files previously marked deleted in the index.
### Options (selected)
- `-?` — Displays verbose help.
- `-B` — Display block numbers in listings (debugging).
- `-d <debuglevel>` — Set debug level (0–5).
- `-E` — Use a local file for the Archive instead of HPSS.
- `-F [user@]FTP_server[#port]` — Archive resides on a remote FTP server (requires build support).
- `-f [Archive]` — Archive name (required). Use `-` for stdin/stdout (special handling; -I required when writing to stdout).
- `-G hsm|backup` — GPFS/HPSS interface operation (admin use).
- `-H opt[:opt...]` — HTAR-specific options; multiple colon-separated options allowed (acct, cif, copies, exfile, family, ghi_fsid, okfile, nocfchk, nostage, crc, nocrc, port, rmlocal, server, tss, umask, verify, relpaths, etc.).
- `-I index_name` — Specify index filename or suffix.
- `-L InputList` — Read file list from InputList.
- `-M maxfiles` — Set maximum number of member files when creating an archive.
- `-m` — Use extraction time as modification time.
- `-n time` — Include only files newer than specified time for create action.
- `-O` — Write extracted files to stdout.
- `-o` — Compatibility with older tar: extracted files owned by invoking user.
- `-p` — Preserve file modes (ignore umask).
- `-P` — Create intermediate subdirectories when creating archive.
- `-q` — Quiet mode.
- `-S bufsize` — Buffer size for transfers (supports k/K, m/M suffixes).
- `-T MaxThreads` — Max threads for local file workers (default build-time value, e.g., 15).
- `-V` — Slightly verbose progress in interactive mode.
- `-v` — Verbose mode (prints operation flags per file: `a` add, `x` extract, `i` index entry).
- `-w` — Confirm each action interactively.
- `-Y auto|<ArchiveCosID>[:<IndexCosID>]` — Select HPSS Class of Service ID; `auto` uses size hints. Can be set via `HTAR_COS` env var.
## HTAR Memory Restrictions
HTAR uses temporary files and in-memory tables; errors occur if temp files cannot be created or memory is insufficient.
## Using HTAR with Firewalls
Set one of these environment variables to specify HPSS port ranges used by movers:
```text
HPSS_PFTPC_PORT_RANGE
RPC_RESTRICTED_PORTS
```
Format: `ncacn_ip_tcp[start_port-end_port]`.
Example (Korn shell):
```bash
export HPSS_PFTPC_PORT_RANGE=ncacn_ip_tcp[20300-20500]
```
If ports are not set correctly, transfers will fail with -50 (ECONN).
## Authentication
HTAR supports several authentication mechanisms (site-dependent). On Kerberos-enabled sites, HTAR may obtain credentials via `kinit` if needed.
## Use of a NETRC File
HTAR supports FTP-style NETRC files for username/password authentication. See the HSI web site for [details:](http://www.mgleicher.us)
## HTAR Installation Path
Typical install locations:
```bash
/usr/local/bin/htar
/usr/local/bin/htar.exe
/tmp/tar*
$HPSS_CFG_FILE_PATH/HPSS.conf
```
## HTAR Execution Environment
HTAR is often a wrapper script that sets environment variables and execs the binary. Relevant environment variables include `HTAR_COS`, `HPSS_SERVER_HOST`, `HPSS_HOSTNAME`, `HPSS_CFG_FILE_PATH`, `TMPDIR`, and `HOME`.
## Notes
1. Maximum single member size ~64 GB due to tar header limits.
2. HTAR writes only regular files; not suitable for magnetic tape devices.
## Exit Status
```bash
0 Successful completion.
>0 An error occurred.
```
## Examples
Write files to a new archive in HPSS:
```bash
htar -cf files.tar file1 file2
```
Write files to an archive on a remote FTP server:
```bash
htar -cf files.tar -F blue.pacific.llnl.gov file1 file2
```
Extract files from a specific directory in an archive:
```bash
htar -xm -f proj1.tar project1/src
```
List files in an archive:
```bash
htar -vtf out.tar
```
## Files
```bash
/usr/local/bin/htar # wrapper script
/usr/local/bin/htar.exe # executable
/tmp/tar* # temporary files
$HPSS_CFG_FILE_PATH/HPSS.conf # network options
```
## Related Information
File archivers: `cat`, `dd`, `tar`, `pax`. HPSS transfer programs: `hsi`, `pftp`, `nft`.
## Bugs and Limitations
- Cannot specify relative index file paths not rooted in the Archive directory without using absolute paths.
- HTAR does not support append, update, or removal of files within an archive.
## Web site
[Reach Us](http://www.mgleicher.us)