Setting Up Environment (Modules)
To run jobs on MOGON, you need to set up your environment correctly. This wiki page will help you to do so.
What are modulefiles or environment modules?
Environment modules, a.k.a modulefiles, are a way to set up a working environment (under Linux/Unix) dynamically. HPC system administration has to be rather conservative: In order to provide a consistent environment it is not impossible to install the newest (untested) piece of software (or kernel stuff) system-wide. On the other hand the requirement to run state of the art scientific software has to be fulfilled. This is where environment modules step in rather handy:
So, loading a particular module will set up an environment? This is mainly done by exporting environment variables, particularly PATH and (LD)_LIBRARY_PATH and others. Usually modulefiles contain code to sanitize the setup, e.g. reporting conflicts with other loaded modules or deprecation warnings.
Working with modulefiles
List of useful module commands
Before we really start, the most frequently used commands at a glance:
| Command | Description |
|---|---|
module -h|help | shows an overview of all available commands |
module avail | shows all available modules |
module load|add [module] | loads the module [module] |
module list | shows all loaded modules |
module rm|unload [module] | unloads the module [module] |
module purge | unloads all loaded modules |
Show modules
In many cases there are several versions of the same compiler/program installed. You have to check that your desired version is available.
Yo can get an overview of all available modules using the command module avail 1).
As module reports on standard error, a grep for a particular module can be done like:
$ module avail |& grep <your_module_you_are_looking_for>
Area Specific Modules
EasyBuild, our software installation framework provides prefixes – at least the way we install software – for modulefiles: bio, chem and phys. This classification is a bit arbitrary at times, but you can search for the respective strings and look whether your desired application is already available.
Other classes include lib and compiler. Of particular relevance is sometimes the lang category, e.g. for providing scripting languages with their correct version (see for example the entries on Python, R and BioConductor and Perl).
Loading modules
Now, knowing the modules that are available you can go on by loading the module(s) you need. This will happen using the command module load [module] or module add [module] 2).
Example: Maybe you want to load gcc (version 6.3.0) and OpenMPI, so you would do:
$ module load mpi/OpenMPI/2.0.2-GCC-6.3.0
That's all - OpenMPI (version 2.0.2) was compiled with gcc 6.3.0 as the version string indicates, but mpicc depends on the specific compiler (and libraries), hence gcc is also provided in the correct version.
.bashrc-file, because we frequently observed that this leads to conflicting environments in the long run. Alternatives are:- Loading modules in submission scripts and
- sourcing environment setup scripts interactively, depending on a particular workflow.
- You sometimes see
module load <category>/<entry>– not followed by a version string. This will take the most recent version (alphabetically). - It is sufficient to load one module per application/library/scripting language.
- If in doubt contact the HPC-team.
Listing active modules
If you want to check your loaded modules, simply use module list.
From the example above:
$ module list Currently Loaded Modulefiles: 1) compiler/GCCcore/6.3.0 3) mpi/OpenMPI/2.0.2-GCC-6.3.0 2) compiler/GCC/6.3.0
Unload modules
To expel a specific module you can use the command module rm [module] or module unload [module].
If you want to unload all your loaded modules, use module purge.
Example:
$ module unload compiler/GCCcore/6.3.0 $ module purge