User Tools

Site Tools


setting_up_environment_modules

Setting up environment (Modules)

To run jobs on Mogon, you need to set up your environment correctly. This wiki page will help you to do so.

What are modulefiles or environment modules?

Environment modules, a.k.a modulefiles, are a way to set up a working environment (under Linux/Unix) dynamically. HPC system administration has to be rather conservative: In order to provide a consistent environment it is not impossible to install the newest (untested) piece of software (or kernel stuff) system-wide. On the other hand the requirement to run state of the art scientific software has to be fulfilled. This is where environment modules step in rather handy:

So, loading a particular module will set up an environment? This is mainly done by exporting environment variables, particularly PATH and (LD)_LIBRARY_PATH and others. Usually modulefiles contain code to sanitize the setup, e.g. reporting conflicts with other loaded modules or deprecation warnings.

Working with modulefiles

List of useful module commands

Before we really start, the most frequently used commands at a glance:

Command Description
module -h|help shows an overview of all available commands
module avail shows all available modules
module load|add [module] loads the module [module]
module list shows all loaded modules
module rm|unload [module] unloads the module [module]
module purge unloads all loaded modules

Show modules

In many cases there are several versions of the same compiler/program installed. You have to check that your desired version is available. Yo can get an overview of all available modules using the command module avail 1).

As module reports on standard error, a grep for a particular module can be done like:

$ module avail |& grep <your_module_you_are_looking_for>

Area Specific Modules

EasyBuild, our software installation framework provides prefixes – at least the way we install software – for modulefiles: bio, chem and phys. This classification is a bit arbitrary at times, but you can search for the respective strings and look whether your desired application is already available.

Other classes include lib and compiler. Of particular relevance is sometimes the lang category, e.g. for providing scripting languages with their correct version (see for example the entries on Python, R and BioConductor and Perl).

Loading modules

Now, knowing the modules that are available you can go on by loading the module(s) you need. This will happen using the command module load [module] or module add [module] 2).

Example: Maybe you want to load gcc (version 6.3.0) and OpenMPI, so you would do:

$ module load mpi/OpenMPI/2.0.2-GCC-6.3.0

That's all - OpenMPI (version 2.0.2) was compiled with gcc 6.3.0 as the version string indicates, but mpicc depends on the specific compiler (and libraries), hence gcc is also provided in the correct version.

We strongly suggest to avoid loading modules in your .bashrc-file, because we frequently observed that this leads to conflicting environments in the long run. Alternatives are:

  • Loading modules in submission scripts and
  • sourcing environment setup scripts interactively, depending on a particular workflow.

Which module is the right?

  • You sometimes see module load <category>/<entry> – not followed by a version string. This will take the most recent version (alphabetically).
  • It is sufficient to load one module per application/library/scripting language.
  • If in doubt contact the HPC-team.

Listing active modules

If you want to check your loaded modules, simply use module list.

From the example above:

$ module list
Currently Loaded Modulefiles:
  1) compiler/GCCcore/6.3.0        3) mpi/OpenMPI/2.0.2-GCC-6.3.0
  2) compiler/GCC/6.3.0

Unload modules

To expel a specific module you can use the command module rm [module] or module unload [module]. If you want to unload all your loaded modules, use module purge.

Example:

$ module unload compiler/GCCcore/6.3.0
$ module purge
1)
In the past we had a dynamic list of available modulefiles linked, here. This list has been removed, because it was impossible to keep track of the changed settings on two clusters.
2)
They are equal. It's yours to chose which you like more
setting_up_environment_modules.txt · Last modified: 2017/11/28 05:38 by meesters