Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | |||
start:working_on_mogon:workflow_organization:setting_up_environment_modules [2021/11/24 11:22] jrutte02 removed |
— (current) | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Setting Up Environment (Modules) ====== | ||
- | |||
- | To run jobs on MOGON, you need to set up your environment correctly. This wiki page will help you to do so. | ||
- | |||
- | ===== What are modulefiles or environment modules? ===== | ||
- | |||
- | [[https:// | ||
- | |||
- | So, loading a particular module will set up an environment? | ||
- | |||
- | ======= Lmod: A New Environment Module System ======= | ||
- | |||
- | < | ||
- | Lmod is a Lua based module system that easily handles the MODULEPATH Hierarchical problem. Environment Modules provide a convenient way to dynamically change the users’ environment through modulefiles. This includes easily adding or removing directories to the PATH environment variable. Modulefiles for Library packages provide environment variables that specify where the library and header files can be found. | ||
- | |||
- | < | ||
- | </ | ||
- | |||
- | |||
- | |||
- | ==== Lmod Opt-Out ==== | ||
- | |||
- | Create the file '' | ||
- | |||
- | <code bash> | ||
- | touch ~/.nolmod | ||
- | </ | ||
- | |||
- | ====== Working with Lmod ====== | ||
- | |||
- | ==== Frequently Used Commands ==== | ||
- | |||
- | ^ Command | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | |||
- | |||
- | ===== Finding Modules ===== | ||
- | |||
- | In many cases there are several versions of the same compiler/ | ||
- | |||
- | ==== module avail ==== | ||
- | |||
- | You can get an overview of **all available** modules using the command | ||
- | |||
- | <code shell> | ||
- | module avail | ||
- | </ | ||
- | or | ||
- | <code shell> | ||
- | module av | ||
- | </ | ||
- | |||
- | You can use the search capabilities to refine your search, e.g.: | ||
- | |||
- | <code bash> | ||
- | module av mathematica | ||
- | </ | ||
- | |||
- | gives | ||
- | |||
- | <code shell> | ||
- | JGU HPC Modules | ||
- | |||
- | ------------------- / | ||
- | | ||
- | |||
- | Where: | ||
- | | ||
- | Questions? contact us at hpc@uni-mainz.de | ||
- | yet more ... | ||
- | </ | ||
- | |||
- | |||
- | ==== Area Specific Modules ==== | ||
- | |||
- | EasyBuild, our software installation framework provides prefixes – at least the way we install software – for modulefiles: | ||
- | |||
- | Other classes include lib and compiler. Of particular relevance is sometimes the lang category, e.g. for providing scripting languages with their correct version (see for example the entries on Python, R and BioConductor and Perl). | ||
- | |||
- | For example with: | ||
- | <code bash> | ||
- | module avail/av phys | ||
- | </ | ||
- | |||
- | you get all the Modules in '' | ||
- | |||
- | <code shell> | ||
- | [joe_user@login23 ~]$ module avail phys | ||
- | JGU HPC Modules | ||
- | |||
- | ------------------- / | ||
- | | ||
- | | ||
- | | ||
- | | ||
- | | ||
- | | ||
- | | ||
- | | ||
- | | ||
- | |||
- | Where: | ||
- | | ||
- | Questions? contact us at hpc@uni-mainz.de | ||
- | yet more ... | ||
- | </ | ||
- | |||
- | ==== module spider ==== | ||
- | |||
- | The command '' | ||
- | |||
- | <code shell> | ||
- | module spider mathematica | ||
- | </ | ||
- | |||
- | gives | ||
- | |||
- | <code shell> | ||
- | ---------------------------------------------------------------------------- | ||
- | math/ | ||
- | ---------------------------------------------------------------------------- | ||
- | Description: | ||
- | Mathematica is a computational software program used in many | ||
- | scientific, engineering, | ||
- | |||
- | | ||
- | math/ | ||
- | math/ | ||
- | |||
- | ---------------------------------------------------------------------------- | ||
- | For detailed information about a specific " | ||
- | Note that names that have a trailing (E) are extensions provided by other modules. | ||
- | For example: | ||
- | |||
- | $ module spider math/ | ||
- | ---------------------------------------------------------------------------- | ||
- | </ | ||
- | |||
- | ===== Loading Modules ===== | ||
- | |||
- | Now, knowing the modules that are available you can go on by loading the module(s) you need. This will happen using the command '' | ||
- | |||
- | Example: Maybe you want to load gcc (version 10.2.0) and OpenMPI, so you would do | ||
- | |||
- | <code bash> | ||
- | module load mpi/ | ||
- | </ | ||
- | |||
- | That's all - OpenMPI (version 4.0.5) was compiled with gcc 10.2.0 as the version string indicates, but mpicc depends on the specific compiler (and libraries), hence gcc is also provided in the correct version. | ||
- | |||
- | ===== Listing active Modules ===== | ||
- | |||
- | <code shell> | ||
- | module list | ||
- | </ | ||
- | From the example above: | ||
- | |||
- | <code shell> | ||
- | Currently Loaded Modules: | ||
- | 1) compiler/ | ||
- | 2) compiler/ | ||
- | 3) lib/ | ||
- | </ | ||
- | |||
- | ===== Unload Modules ===== | ||
- | |||
- | To expel a specific module you can use the command | ||
- | |||
- | <code shell> | ||
- | module rm [module] | ||
- | </ | ||
- | |||
- | or | ||
- | |||
- | <code shell> | ||
- | module unload [module] | ||
- | </ | ||
- | |||
- | If you want to unload **all** your loaded modules, use | ||
- | |||
- | <code shell> | ||
- | module purge | ||
- | </ | ||
- | |||
- | ===== Displaying contents of a Module ===== | ||
- | |||
- | <code bash> | ||
- | module show | ||
- | </ | ||
- | |||
- | ===== ml - A convenient Tool ===== | ||
- | |||
- | < | ||
- | |||
- | For those of you who can’t type the //mdoule//, moduel, err //module// command correctly, Lmod has a tool for you. With '' | ||
- | < | ||
- | </ | ||
- | |||
- | |||
- | <code bash> | ||
- | ml | ||
- | </ | ||
- | |||
- | means '' | ||
- | |||
- | <code bash> | ||
- | ml foo | ||
- | </ | ||
- | |||
- | means '' | ||
- | |||
- | <code bash> | ||
- | ml -bar | ||
- | </ | ||
- | |||
- | means '' | ||
- | |||
- | <code bash> | ||
- | ml foo -bar | ||
- | </ | ||
- | |||
- | means '' | ||
- | |||
- | You can do **all** the module commands: | ||
- | |||
- | <code bash> | ||
- | ml spider | ||
- | ml avail | ||
- | ml show foo | ||
- | </ | ||
- | |||
- | ===== User Collections ===== | ||
- | |||
- | User defined initial list of login modules | ||
- | |||
- | <code bash> | ||
- | module load module1 module2 ... | ||
- | </ | ||
- | |||
- | On MOGON, users can create as many collections as they like. You can save to a nemed collection with: | ||
- | |||
- | <code bash> | ||
- | module save < | ||
- | </ | ||
- | |||
- | The collection is restored with: | ||
- | |||
- | <code bash> | ||
- | module restore < | ||
- | </ | ||
- | |||
- | You can display the contents of a name collection with | ||
- | |||
- | <code bash> | ||
- | module describe < | ||
- | </ | ||
- | |||
- | Use the following command to list the collections you already have: | ||
- | |||
- | <code bash> | ||
- | module savelist | ||
- | </ | ||
- | |||
- | Finally, you can deactivate a collection with: | ||
- | |||
- | <code bash> | ||
- | module disable < | ||
- | </ | ||
- | |||
- | If no ''< | ||
- | |||
- | <code bash> | ||
- | cd ~/.lmod.d; mv foo~ foo | ||
- | </ | ||
- | |||
- | |||
- | ====== Working with Environment Modules ====== | ||
- | |||
- | In order to use the old Environment Modules system, the file '' | ||
- | |||
- | ===== List of useful module commands ===== | ||
- | |||
- | Before we really start, the most frequently used commands at a glance: | ||
- | |||
- | ^ Command ^ Description ^ | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | | '' | ||
- | |||
- | ===== Show modules ===== | ||
- | |||
- | In many cases there are several versions of the same compiler/ | ||
- | Yo can get an overview of all available modules using the command '' | ||
- | |||
- | As '' | ||
- | <code bash> | ||
- | $ module avail |& grep < | ||
- | </ | ||
- | |||
- | ===== Area Specific Modules ===== | ||
- | |||
- | [[https:// | ||
- | |||
- | Other classes include '' | ||
- | |||
- | ===== Loading modules ===== | ||
- | |||
- | Now, knowing the modules that are available you can go on by loading the module(s) you need. This will happen using the command '' | ||
- | |||
- | Example: | ||
- | Maybe you want to load gcc (version 6.3.0) and OpenMPI, so you would do: | ||
- | <code bash> | ||
- | $ module load mpi/ | ||
- | </ | ||
- | |||
- | That's all - OpenMPI (version 2.0.2) was compiled with gcc 6.3.0 as the version string indicates, but '' | ||
- | |||
- | <callout type=" | ||
- | We strongly suggest to avoid loading modules in your '' | ||
- | * Loading modules in submission scripts and | ||
- | * sourcing environment setup scripts interactively, | ||
- | </ | ||
- | <callout type=" | ||
- | Which module is the right? | ||
- | * You sometimes see '' | ||
- | * It is sufficient to load one module per application/ | ||
- | * If in doubt contact the HPC-team. | ||
- | </ | ||
- | |||
- | |||
- | ===== Listing active modules ===== | ||
- | |||
- | If you want to check your loaded modules, simply use '' | ||
- | |||
- | From the example above: | ||
- | <code bash> | ||
- | $ module list | ||
- | Currently Loaded Modulefiles: | ||
- | 1) compiler/ | ||
- | 2) compiler/ | ||
- | </ | ||
- | |||
- | ===== Unload modules ===== | ||
- | |||
- | To expel a specific module you can use the command '' | ||
- | If you want to unload all your loaded modules, use '' | ||
- | |||
- | Example: | ||
- | <code bash> | ||
- | $ module unload compiler/ | ||
- | $ module purge | ||
- | </ | ||
- | |||