Software Stack Changes in May

May 19, 2014
Carter, Conte, Hansen, Peregrine1, Radon, Rossmann

On May 19th, there will be changes made to the software stack on ITaP's community clusters. The software stack will be updated to include the latest Intel compiler, version 14.0.2.144. Support for Intel versions 12.0.084 and 13.0.1.117 compilations will be ended along with support for PGI compilations of software on Intel clusters. Some miscellaneous software will also be updated to the latest versions available on the clusters.

WHAT SOFTWARE WILL BE IMPACTED?

The largest of these changes will be to the software stack built with Intel compilers (FFTW, NetCDF, MPI libraries, etc). The stack will be expanded to include support for the latest Intel compiler, version 14.0.2.144. In addition to adding new versions, the older software stack built with version 12.0.084 of the Intel compiler will be removed. Support for Intel version 13.0.1.117 will also be ended. At this time, no Intel 13.0.1.117 built software will be removed, however, we will no longer build new versions of software to use this compiler. Software built with 13.1.1.163 will remain the default Intel software choice.

By May 20th the following software will be available, built with Intel 14.0.2.144.

  • fftw/3.3.3_intel-14.0.2.144
  • hdf5/1.8.7_intel-14.0.2.144
  • mpich2/1.4.1p1_intel-14.0.2.144 (Coates, Rossmann, Hansen, Radon)
  • mvapich2/1.9_intel-14.0.2.144
  • netcdf/3.6.3_intel-14.0.2.144
  • netcdf/4.1.1_intel-14.0.2.144
  • openmpi/1.6.3_intel-14.0.2.144

Please try out the new Intel compilations to see how they work with your software and code. If you encounter any issues or have any other feedback, please let us know. If Intel 14 works well for researchers, the default software stack will be upgraded to Intel 14 in a future software stack update. Your feedback will help guide the direction of software stack changes.

In addition to the compiler changes, we will also be upgrading the oldest version of FFTW, HDF5, MVAPICH2, and OpenMPI. This is being done to prepare for even newer versions of these software in future software stack updates. The following software will be upgraded and will no longer be available for use.

  • FFTW 3.3.1 upgraded to 3.3.3
  • HDF5 1.8.5 upgraded to 1.8.7
  • OpenMPI 1.4.4 upgraded to 1.6.3
  • MVAPICH2 1.8 upgraded to 1.9

We will also be ending support for PGI built software on our Intel clusters (Carter, Conte, Peregrine1, Radon). The compilers will remain available for software that requires the PGI compilers, however, the PGI built versions of FFTW, HDF5, NetCDF, and MPI libraries will be removed and no new ones will be built.

Several older, infrequently used, versions of miscellaneous software will also be upgraded to their latest, currently available versions. A couple other miscellaneous software will have their default versions upgraded to the latest. These particular changes should go unnoticed by most researchers.

If any software you are using will be impacted by these changes you will see a notice message being printed to your session or in your job output files when loading an affected module. This notice will provide recommendations on the latest version. For software versions being removed it is recommended that you switch to the latest version, however, these older versions can be made available for use via the "unsupported" module archive upon request. For software versions undergoing a default upgrade you can continue using the old default by switching your "module load" to explicitly load the version.

HOW DO I KNOW IF MY WORKFLOW WILL BE IMPACTED?

Whenever a module that will be impacted is loaded a notice is printed to your screen or job output log. Please take a look at your job output over the next couple of weeks and make note of any changes being advertised. You may continue using these modules as-is until May 20th to allow time to make any changes necessary. Users are encouraged to make any changes necessary before then to avoid disruption when changes are made.

WHAT IF AN IMPACTED MODULE IS REQUIRED BY MY RESEARCH?

We understand some users may not be able to change compilers or MPI libraries in the middle of a research project. Modules involved in a default version update will continue to be available, however, you will need to update your job scripts to request the specific version of the module. If you are already loading specific versions no changes are necessary.

Older versions of software being removed can be made available in the “unsupported” module archive upon request. Several modules that are most likely to be requested are already populated in the archive. You may use the archive by adding "module load unsupported" to your job scripts (prior to loading the affected modules).

WHY ARE YOU CHANGING THE SOFTWARE STACK?

ITaP aims to provide a software stack that allows for optimal use (performance and stability) of the clusters. This necessitates periodic updates to the stack as compilers, libraries, and software are improved over time. By removing older modules from the main stack we help ensure the selection is simple and easy for users to find the best compilers and libraries to use.

If no modules were removed the selection would become difficult to navigate as well become difficult for ITaP staff to manage. Any major changes will be coordinated with scheduled maintenance periods to minimize impact.

If you have any questions or concerns with the upcoming changes please contact rcac-help@purdue.edu

Originally posted: April 22, 2014