Software

Environment Modules, or short Modules are the means by which most of the installed scientific software is provided on the NHR@KIT systems.

The use of different compilers, libraries and software packages requires users to set up a specific session environment suited for the program they want to run. HoreKa cluster provides users with the possibility to load and unload complete environments for compilers, libraries and software packages by a single command.

Description¶

The Environment ''Modules'' package enables dynamic modification of your environment by the use of so-called ''modulefiles''. A ''modulefile'' contains information to configure the shell for a program/software . Typically, a modulefile contains instructions that alter or set shell environment variables, such as PATH and MANPATH, to enable access to various installed software.

One of the key features of using the Environment ''Modules'' software is to allow multiple versions of the same software to be used in your environment in a controlled manner. For example, two different versions of the Intel C compiler can be installed on the system at the same time - the version used is based upon which Intel C compiler modulefile is loaded.

The software stack of HoreKa cluster provides a number of modulefiles. You can also create your own modulefiles. ''Modulefiles'' may be shared by many users on a system, and users may have their own collection of modulefiles to supplement or replace the shared modulefiles.

A modulefile does not provide configuration of your environment until it is explicitly loaded, i.e., the specific modulefile for a software product or application must be loaded in your environment before the configuration information in the modulefile is effective.

If you want to see which modules are loaded you must execute module list.

$ module list
Currently Loaded Modules:
  1) dot   2) compiler/intel/19.1   3) numlib/mkl/2020   4) mpi/openmpi/4.0

Usage¶

On HoreKa Lmod: A New Environment Module System from http://lmod.readthedocs.org/en/latest/ is installed.

Documentation¶

Execute module help or man module for help on how to use ''Modules'' software.

For help on particular version of ''Module'', e.g. Intel default compiler version, execute module help compiler/intel.

$ module help compiler/intel
---------------- Module Specific Help for "compiler/intel/19.1" ----------------
Intel(R) Compilers 19.1 for Linux*
For details see: https://software.intel.com/en-us/intel-compilers
In case of problems, please contact: Hartmut Häfner <hartmut.haefner@kit.edu>
SCC support end: 2022-12-31

Online Documentation¶

Lmod: A New Environment Module System

Display all available Modules¶

Available ''Module'' are modulefiles that can be loaded by the user. A ''Module'' must be loaded before it provides changes to your environment, as described in the introduction to this section. You can display all available ''Modules'' on the system by executing:

$ module avail

The short form the command is:

$ module av

Available ''Modules'' can be also displayed in different modes, such as

each ''Module'' per one line
```
$ module -t avail
```
Some modules may not be available right now, because their requirements are not met. To get a complete list of all possible modules use the [[#Display all possible Modules|module spider command]].

Module categories, versions and defaults¶

The NHR@KIT cluster systems provides a large variety of software and software versions. Therefore ''Modules'' are divided in category folders containing subfolders of modulefiles again containing modulefile versions, and must be addressed as follows: category/softwarename/version For instance all versions of the Intel compiler belong to the category of compilers, thus the corresponding modulefiles are placed under the category ''compiler'' and ''intel''.

In case of multiple software versions, one version will be always defined as the default version. The ''Module'' of the default can be addressed by simply omitting the version number: category/softwarename

Finding software Modules¶

Currently all software packages are assigned to the following ''Module'' categories:

bio
cae
compiler
devel
jupyter
lib
math
mpi
numlib
phys
system
toolkit
vis

You can selectively list software in one of those categories using, e.g. for the category "compiler"

$ module avail compiler/

Searches are looking for a substring starting at the begin of the name, so this would list all software in categories starting with a "c"

$ module avail c

while this would find nothing

$ module avail hem

Loading Modules¶

You can load a ''Module'' software in to your environment to enable easier access to software that you want to use by executing:

$ module load category/softwarename/version

or

$ module add category/softwarename/version

Loading a ''Module'' in this manner affects ONLY your environment for the current session.

Loading conflicts¶

You can not load different versions of the same software at the same time! Loading the Intel compiler in version X while Intel compiler in version Y is loaded leads to an automatic unloading of Intel compiler in version Y.

Showing the changes introduced by a Module¶

Loading a ''Module'' will change the environment of the current shell session. For instance the $PATH variable will be expanded by the software's binary directory. Other ''Module'' variables may even change the behavior of the current shell session or the software program(s) in a more drastic way.

All the changes to the current shell session to be invoked by loading the ''Module'' can be reviewed using

module show category/softwarename/version.

Example (Intel compiler)

$ module show compiler/intel
------------------------------------------------------------------------------------------------------------------------------------
   /software/all/lmod/modulefiles/Core/compiler/intel/19.1.lua:
------------------------------------------------------------------------------------------------------------------------------------
setenv("INTEL_LICENSE_FILE","28518@scclic1.scc.kit.edu")
setenv("AR","/opt/intel/compilers_and_libraries_2020/linux/bin/intel64/xiar")
setenv("CC","/opt/intel/compilers_and_libraries_2020/linux/bin/intel64/icc")
setenv("CXX","/opt/intel/compilers_and_libraries_2020/linux/bin/intel64/icpc")
setenv("F77","/opt/intel/compilers_and_libraries_2020/linux/bin/intel64/ifort")
setenv("FC","/opt/intel/compilers_and_libraries_2020/linux/bin/intel64/ifort")
setenv("CFLAGS","-O2 -xCORE-AVX2")
setenv("CXXFLAGS","-O2 -xCORE-AVX2")
setenv("FFLAGS","-O2 -xCORE-AVX2")
setenv("FCFLAGS","-O2 -xCORE-AVX2")
setenv("INTEL_VERSION","19.1.0.166")
setenv("INTEL_HOME","/opt/intel/compilers_and_libraries_2020/linux")
setenv("INTEL_BIN_DIR","/opt/intel/compilers_and_libraries_2020/linux/bin/intel64")
setenv("INTEL_LIB_DIR","/opt/intel/compilers_and_libraries_2020/linux/lib/intel64")
setenv("INTEL_INC_DIR","/opt/intel/compilers_and_libraries_2020/linux/include")
setenv("INTEL_MAN_DIR","/opt/intel/compilers_and_libraries_2020/linux/man/common")
setenv("INTEL_DOC_DIR","/opt/intel/compilers_and_libraries_2020/linux/documentation/en")
setenv("GDB_VERSION","19.1.0.166")
setenv("GDB_HOME","/opt/intel/debugger_2020/gdb/intel64")
setenv("GDB_BIN_DIR","/opt/intel/debugger_2020/gdb/intel64/bin")
setenv("GDB_LIB_DIR","/opt/intel/debugger_2020/libipt/intel64/lib")
setenv("GDB_INC_DIR","/opt/intel/debugger_2020/gdb/intel64/include")
setenv("GDB_INF_DIR","/opt/intel/documentation_2020/en/debugger/gdb-ia/info")
setenv("GDB_MAN_DIR","/opt/intel/documentation_2020/en/debugger/gdb-ia/man")
setenv("KMP_AFFINITY","noverbose,granularity=core,respect,warnings,compact,1")
prepend_path("PATH","/opt/intel/compilers_and_libraries_2020/linux/bin/intel64")
prepend_path("MANPATH","/opt/intel/compilers_and_libraries_2020/linux/man/common")
prepend_path("LD_LIBRARY_PATH","/opt/intel/compilers_and_libraries_2020/linux/lib/intel64")
whatis("Sets up Intel C/C++ and Fortran compiler version 19.1 (Intel(R) Compilers 19.1 for Linux*) - supported by SCC till 2022-12-31!")
help([[Intel(R) Compilers 19.1 for Linux*
For details see: https://software.intel.com/en-us/intel-compilers
In case of problems, please contact: Hartmut Häfner <hartmut.haefner@kit.edu>
SCC support end: 2022-12-31]])
prepend_path("MODULEPATH","/software/all/lmod/modulefiles/Compiler/intel/19.1")
family("compiler")

'module show' does not load the ''Module''!

Modules depending on Modules¶

Some program ''Modules'' depend on libraries to be loaded to the user environment. Therefore the corresponding ''Modules'' of the software must be loaded together with the ''Modules'' of the libraries.

By default such software ''Modules'' try to load required ''Modules'' and corresponding versions automatically.

Unloading Modules¶

To unload or remove a software ''Module'' execute:

$ module unload category/softwarename/version

or

$ module remove category/softwarename/version

Unloading all loaded modules¶

Purge¶

Unloading a ''Module'' that has been loaded by default makes it inactive for the current session only - it will be reloaded the next time you log in.

In order to remove all previously loaded software modules from your environment issue the command 'module purge'.

Example

$ module list
Currently Loaded Modules:
  1) dot   2) compiler/intel/19.1   3) numlib/mkl/2020   4) mpi/openmpi/4.0
$ module purge
$ module list
No modules loaded

Beware!

'module purge' is working without any further inquiry.

Display your loaded Modules¶

All ''Modules'' that are currently loaded for you can be displayed by the command module list. [[#Purge|See example above]].

Note: You only have to load further ''Modules'', if you want to use additional software packages or to change the version of an already loaded software.

Display all possible Modules¶

Modulefiles can be searched by the user. You can dipslay all possible modules by executing:

$ module spider

module spider modulename : You can search using a modulename. The search lists all possible versions of that module.

$ module spider gnu

------------------------------------------------------------------------------------------------------------------------------
  compiler/gnu:
------------------------------------------------------------------------------------------------------------------------------
     Versions:
        compiler/gnu/8
        compiler/gnu/8.3.1
        compiler/gnu/9
        compiler/gnu/10

------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "compiler/gnu" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider compiler/gnu/10
------------------------------------------------------------------------------------------------------------------------------

module spider string : You can search using a string. The search lists all modules which contain "string":

$ module spider net

Moreover, you can see the dependencies of the module with using the same command. For example, if the following is executed, you can see which modules need to be loaded before loading the module mpi/openmpi/3.1

$ module spider mpi/openmpi/4.0
------------------------------------------------------------------------------------------------------------------------------
  mpi/openmpi: mpi/openmpi/4.0
------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "mpi/openmpi/4.0" module is available to load.

      compiler/gnu/10
      compiler/gnu/8
      compiler/gnu/9
      compiler/intel/18.0
      compiler/intel/19.0
      compiler/intel/19.1
      compiler/llvm/10.0

    Help:
      Open MPI (http://www.openmpi.org) is an open source implementation of the
      Message Passing Interface specification (http://www.mpi-forum.org/) developed
      and maintained by a consortium of research, academic, and industry partners.

      Open MPI also includes an implementation of the OpenSHMEM parallel
      programming API (http://www.openshmem.org/).  OpenSHMEM is a
      Partitioned Global Address Space (PGAS) abstraction layer, which
      provides fast inter-process communication using one-sided
      communication techniques.

      This RPM contains all the tools necessary to compile, link, and run
      Open MPI and OpenSHMEM jobs.

      To select an OpenMPI version with Valgrind support, the environment variable
      VALGRIND must be set to true before loading the module:
      export VALGRIND=true

      In case of problems, please contact: Hartmut Häfner <hartmut.haefner@kit.edu>
      SCC support end: As soon as this version is declared as "Older version" on
                       the website http://www.openmpi.org/software/ompi

How do Modules work?¶

The default shell on the NHR@KIT systems is bash, so explanations and examples will be shown for bash. In general, programs cannot modify the environment of the shell they are being run from, so how can the module command do exactly that?

The module command is not a program, but a bash-function. You can view its content using:

$ type module

and you will get the following result:

$ type module
module is a function
module ()
{
    eval $(/software/all/admin/modules/module-wrapper/modulecmd bash $*)
}

In this function, lmod is called. Its output to stdout is then executed inside your current shell using the bash-internal ''eval'' command. As a consequence, all output that you see from the module is transmitted via stderr (output handle 2) or in some cases even stdin (output handle 0).

Last update: November 29, 2022