Detection of loaded modules¶
Since EasyBuild v3.3.0, the eb
command performs a check to see if any (EasyBuild-generated)
modules have been loaded in the current environment.
This check can be controlled via the --detect-loaded-modules
and --allow-loaded-modules
configuration options.
In addition any unknown $EBROOT*
environment variables are detected and acted upon, according to how
the --check-ebroot-env-vars
configuration option was set.
Contents
Motivation¶
Running EasyBuild in an environment where one or more (EasyBuild-generated) modules are loaded may interfere with the software installations performed by EasyBuild, i.e.:
- they may cause installations failures, for example due to incompatibilities with the modules being loaded during the installation procedure being performed;
- they may cause installations to work in that particular environment, for example by providing a necessary dependency
Since manually loading modules may affect the reproducibility of software installations, it should be discouraged.
In EasyBuild versions before v3.3.0, having a loaded module for the same software packages as the one being installed resulted in an EasyBuild error message.
Since EasyBuild v3.3.0 a more extensive detection mechanism is available
and the action that should be taken for loaded modules can be controlled via --detect-loaded-modules
.
Having a module loaded for any software package that is being installed still results in a hard error.
Detection mechanism¶
Detecting loaded EasyBuild-generated modules is done by checking the environment for defined $EBROOT*
environment variables. If any are found, the corresponding loaded module is determined for better reporting.
In case defined $EBROOT*
environment variables are found that do not match a loaded modules,
action is taken as well; see Checking of $EBROOT* environment variables.
Action to take if loaded modules are detected¶
The action that should be taken when one or more loaded (EasyBuild-generated) modules are detected
can be specified via the --detect-loaded-modules
configuration option.
Supported values include:
- error: report error and stop EasyBuild upon detection of loaded modules
- ignore: ignore any loaded modules
- purge: run ‘module purge’ to clean environment of loaded modules
- unload: unload loaded modules
- warn: print warning and continue upon detection of loaded modules (current default)
The specified action is only taken for non-allowed loaded modules, see Allowing particular loaded modules.
error
: report error and stop EasyBuild upon detection of loaded modules¶
When EasyBuild is configured with --detect-loaded-modules=error
, it will print a clear error and stop when
any (non-allowed) loaded modules are detected.
For example:
$ eb example.eb --detect-loaded-modules=error
== temporary log file in case of crash /tmp/eb-UlKMRN/easybuild-MgfEHi.log
ERROR: Found one or more non-allowed loaded (EasyBuild-generated) modules in current environment:
* Spack/0.10.0
To make EasyBuild allow particular loaded modules, use the --allow-loaded-modules configuration option.
Use --detect-loaded-modules={error,ignore,purge,unload,warn} to specify action to take when loaded modules are detected.
See http://easybuild.readthedocs.io/en/latest/Detecting_loaded_modules.html for more information.
ignore
: ignore any loaded modules¶
With --detect-loaded-modules=ignore
in place, any loaded modules are simply ignored and EasyBuild continues silently.
This is equivalent to the behaviour of EasyBuild versions prior to version 3.3.0.
Note
This is not recommended!
purge
: run ‘module purge
’ to clean environment of loaded modules¶
Using --detect-loaded-modules=purge
, EasyBuild will run module purge
if any (non-allowed) loaded modules
are detected, in an attempt to restore the environment to a clean state before starting software installations.
A short warning message is printed in case module purge
was used to clean up the environment:
$ export EASYBUILD_DETECT_LOADED_MODULES=purge
$ eb example.eb
== temporary log file in case of crash /tmp/eb-QLTV9v/easybuild-6mOmIN.log
WARNING: Found non-allowed loaded (EasyBuild-generated) modules (Spack/0.10.0), running 'module purge'
...
Note
Whether or not module purge
is a suitable action is site-specific, since this will unload all loaded modules
(except for ‘sticky’ modules when Lmod is used),
including modules that were not installed with EasyBuild and which may be always required.
Configuring EasyBuild to use module purge
when (non-allowed) loaded modules are found should not be
done on Cray systems, since it will result in a broken environment.
unload
: unload loaded modules¶
When --detect-loaded-modules=unload
is used, EasyBuild will only unload the (non-allowed & EasyBuild-generated) modules.
The modules are unloaded in reverse order, i.e. the last loaded module is unloaded first.
This is an alternative to using module purge
, in case some other (allowed) modules are loaded and should remain loaded.
A short warning message is printed when loaded modules are unloaded:
$ eb example.eb --detect-loaded-modules=unload
== temporary log file in case of crash /tmp/eb-JyyaEF/easybuild-WyGqZs.log
WARNING: Unloading non-allowed loaded (EasyBuild-generated) modules: Spack/0.10.0
...
warn
: print warning and continue upon detection of loaded modules¶
When EasyBuild is configured with --detect-loaded-modules=warn
, EasyBuild will print a warning
mentioning that one or more loaded (EasyBuild-generated) were detected, before continuing as normal.
The warning is intended to make the user aware that the environment in which EasyBuild is being run is not clean.
For example:
$ export EASYBUILD_DETECT_LOADED_MODULES=warn
$ eb example.eb
== temporary log file in case of crash /tmp/eb-9HD20m/easybuild-WAYzK2.log
WARNING: Found one or more non-allowed loaded (EasyBuild-generated) modules in current environment:
* Spack/0.10.0
To make EasyBuild allow particular loaded modules, use the --allow-loaded-modules configuration option.
Use --detect-loaded-modules={error,ignore,purge,unload,warn} to specify action to take when loaded modules are detected.
See http://easybuild.readthedocs.io/en/latest/Detecting_loaded_modules.html for more information.
...
Note
This is the default behaviour in EasyBuild v3.3.0 and newer.
Allowing particular loaded modules¶
By default, only EasyBuild itself is considered as an allowed module.
EasyBuild can be configured to allow particular modules to be loaded via --allow-loaded-modules
,
by specifying a comma-seperated list of software names.
For example:
$ export EASYBUILD_DETECT_LOADED_MODULES=error
$ export EASYBUILD_ALLOW_LOADED_MODULES=EasyBuild,GC3Pie
$ module load EasyBuild
$ module load GC3Pie
$ eb example.eb
...
When --allow-loaded-modules
is used, the EasyBuild module is no more allowed by default and must
be explicitly listed if you want to allow an EasyBuild installation provided through a module.
Checking of $EBROOT*
environment variables¶
The detection mechanism for loaded modules relies on defined $EBROOT*
environment variables, and
determining by which loaded module they were set.
When one or more $EBROOT*
environment variables are found for which the corresponding loaded modules can not
be found, this can lead to problems.
Hence, EasyBuild will detect this and act on it, according to the value specified to --check-ebroot-env-vars
:
- error: report error and stop EasyBuild on unknown $EBROOT* environment variables
- ignore: ignore unknown $EBROOT* environment variables
- unset: unset unknown $EBROOT* environment variables and print warning
- warn: print warning for unknown $EBROOT* environment variables and continue (current default)
error
: report error and stop EasyBuild on unknown $EBROOT*
environment variables¶
When configured with --check-ebroot-env-vars=error
, EasyBuild will stop with a clear error message
when it discovers one or more $EBROOT*
environment variables that do not correspond to a loaded module:
$ export EBROOTUNKNOWN=just_an_example
$ eb example.eb --check-ebroot-env-vars=error
== temporary log file in case of crash /tmp/eb-cNqOzG/easybuild-xmV8vc.log
ERROR: Found defined $EBROOT* environment variables without matching loaded module: $EBROOTUNKNOWN
(control action via --check-ebroot-env-vars={error,ignore,unset,warn})
ignore
: ignore unknown $EBROOT*
environment variables¶
To simply ignore any defined $EBROOT*
environment variables that do not correspond with a loaded module,
configure EasyBuild with --check-ebroot-env-vars=ignore
.
unset
: unset unknown $EBROOT*
environment variables and print warning¶
If you are confident that the defined $EBROOT*
environment variables for which no matching loaded module was found
are harmless, you can specify that EasyBuild should clean up the environment by unsetting them, before continuing.
A clear warning message will be printed whenever this occurs:
$ export EBROOTUNKNOWN=just_an_example
$ eb bzip2-1.0.6.eb --check-ebroot-env-vars=unset
== temporary log file in case of crash /tmp/eb-IvXik8/easybuild-cjHjhs.log
WARNING: Found defined $EBROOT* environment variables without matching loaded module: $EBROOTUNKNOWN; unsetting them
...
Note that these unknown $EBROOT*
will only be unset in the environment of the running EasyBuild session,
not in the current session in which eb
is being run.
warn
: print warning for unknown $EBROOT*
environment variables and continue¶
If EasyBuild was configured with --check-ebroot-env-vars=warn
, a warning will be printed
when one or more defined $EBROOT*
environment variables are encountered for which no corresponding loaded module
was found:
$ export EBROOTUNKNOWN=just_an_example
$ export EASYBUILD_CHECK_EBROOT_ENV_VARS=warn
$ eb example.eb
== temporary log file in case of crash /tmp/eb-1h_LQ9/easybuild-BHrPk4.log
WARNING: Found defined $EBROOT* environment variables without matching loaded module: $EBROOTUNKNOWN
(control action via --check-ebroot-env-vars={error,ignore,unset,warn})
...
Note
This is the default behaviour in EasyBuild v3.3.0 and newer.