easybuild.tools.module_generator module

Generating module files.

Authors:

• Stijn De Weirdt (Ghent University)
• Dries Verdegem (Ghent University)
• Kenneth Hoste (Ghent University)
• Pieter De Baets (Ghent University)
• Jens Timmerman (Ghent University)
• Fotis Georgatos (Uni.Lu, NTUA)
• Damian Alvarez (Forschungszentrum Juelich GmbH)

class easybuild.tools.module_generator.ModuleGenerator(application, fake=False)

Bases: object

Class for generating module files.

CHARS_TO_ESCAPE = None

DOT_MODULERC = '.modulerc'

INDENTATION = ' '

MODULE_FILE_EXTENSION = None

MODULE_SHEBANG = None

SYNTAX = None

append_paths(key, paths, allow_abs=False, expand_relpaths=True)

Generate append-path statements for the given list of paths.

Parameters:

• key – environment variable to append paths to
• paths – list of paths to append
• allow_abs – allow providing of absolute paths
• expand_relpaths – expand relative paths into absolute paths (by prefixing install dir)

check_group(group, error_msg=None)

Generate a check of the software group and the current user, and refuse to load the module if the user don’t belong to the group

Parameters:

• group – string with the group name
• error_msg – error message to print for users outside that group

check_version(minimal_version_maj, minimal_version_min, minimal_version_patch='0')

Check the minimal version of the modules tool in the module file :param minimal_version_maj: the major version to check :param minimal_version_min: the minor version to check :param minimal_version_patch: the patch version to check

comment(msg)

Return given string formatted as a comment.

conditional_statement(conditions, body, negative=False, else_body=None, indent=True, cond_or=False, cond_tmpl=None)

Return formatted conditional statement, with given condition and body.

Parameters:

• conditions – (list of) string(s) containing the statement(s) for the if condition (in correct syntax)
• body – (multiline) string with if body (in correct syntax, without indentation)
• negative – boolean indicating whether the (individual) condition(s) should be negated
• else_body – optional body for ‘else’ part
• indent – indent if/else body
• cond_or – combine multiple conditions using ‘or’ (default is to combine with ‘and’)
• cond_tmpl – template for condition expression (default: ‘%s’)

create_symlinks(mod_symlink_paths, fake=False)

Create moduleclass symlink(s) to actual module file.

define_env_var(env_var)

Determine whether environment variable with specified name should be defined or not.

Parameters:env_var – name of environment variable to check

det_installdir(modfile)

Determine installation directory used by given module file

det_user_modpath(user_modpath)

Determine user-specific modules subdirectory, to be used in ‘use’ statements (cfr. implementation of use() method).

get_description(conflict=True)

Generate a description.

get_module_filepath(fake=False, mod_path_suffix=None)

Return path to module file.

get_modules_path(fake=False, mod_path_suffix=None)

Return path to directory where module files should be generated in.

getenv_cmd(envvar, default=None)

Return module-syntax specific code to get value of specific environment variable.

is_loaded(mod_names)

Generate (list of) expression(s) to check whether specified module(s) is (are) loaded.

Parameters:mod_names – (list of) module name(s) to check load status for

load_module(mod_name, recursive_unload=False, depends_on=False, unload_modules=None, multi_dep_mods=None)

Generate load statement for specified module.

Parameters:

• mod_name – name of module to generate load statement for
• recursive_unload – boolean indicating whether the ‘load’ statement should be reverted on unload
• depends_on – use depends_on statements rather than (guarded) load statements
• unload_modules – name(s) of module to unload first
• multi_dep_mods – list of module names in multi_deps context, to use for guarding load statement

modulerc(module_version=None, filepath=None, modulerc_txt=None)

Generate contents of .modulerc file, in Tcl syntax (compatible with all module tools, incl. Lmod). If ‘filepath’ is specified, the .modulerc file will be written as well.

Parameters:

• module_version – specs for module-version statement (dict with ‘modname’, ‘sym_version’ & ‘version’ keys)
• filepath – location where .modulerc file should be written to
• modulerc_txt – contents of .modulerc to use

Returns:

contents of .modulerc file

msg_on_load(msg)

Add a message that should be printed when loading the module.

prepend_paths(key, paths, allow_abs=False, expand_relpaths=True)

Generate prepend-path statements for the given list of paths.

Parameters:

• key – environment variable to append paths to
• paths – list of paths to append
• allow_abs – allow providing of absolute paths
• expand_relpaths – expand relative paths into absolute paths (by prefixing install dir)

set_alias(key, value)

Generate set-alias statement in modulefile for the given key/value pair.

set_as_default(module_dir_path, module_version, mod_symlink_paths=None)

Set generated module as default module

Parameters:

• module_dir_path – module directory path, e.g. $HOME/easybuild/modules/all/Bison
• module_version – module version, e.g. 3.0.4
• mod_symlink_paths – list of paths in which symlinks to module files must be created

set_environment(key, value, relpath=False)

Generate a quoted setenv statement for the given key/value pair.

Parameters:

• key – name of environment variable to define
• value – value to define environment variable with
• relpath – value is path relative to installation prefix

start_module_creation()

Prepares creating a module and returns the file header (shebang) if any including the newline

Meant to be used in a with statement:

with generator.start_module_creation() as txt:

# Write txt

swap_module(mod_name_out, mod_name_in, guarded=True)

Generate swap statement for specified module names.

Parameters:

• mod_name_out – name of module to unload (swap out)
• mod_name_in – name of module to load (swap in)
• guarded – guard ‘swap’ statement, fall back to ‘load’ if module being swapped out is not loaded

unload_module(mod_name)

Generate unload statement for specified module.

Parameters:mod_name – name of module to generate unload statement for

unpack_setenv_value(env_var_name, env_var_val)

Unpack value that specifies how to define an environment variable with specified name.

update_paths(key, paths, prepend=True, allow_abs=False, expand_relpaths=True)

Generate prepend-path or append-path statements for the given list of paths.

Parameters:

• key – environment variable to prepend/append paths to
• paths – list of paths to prepend
• prepend – whether to prepend (True) or append (False) paths
• allow_abs – allow providing of absolute paths
• expand_relpaths – expand relative paths into absolute paths (by prefixing install dir)

use(paths, prefix=None, guarded=False, user_modpath=None)

Generate module use statements for given list of module paths. :param paths: list of module path extensions to generate use statements for; paths will be quoted :param prefix: optional path prefix; not quoted, i.e., can be a statement :param guarded: use statements will be guarded to only apply if path exists :param user_modpath: user-specific modules subdirectory to include in use statements

class easybuild.tools.module_generator.ModuleGeneratorLua(*args, **kwargs)

Bases: easybuild.tools.module_generator.ModuleGenerator

Class for generating Lua module files.

CHARS_TO_ESCAPE = []

END_STR = ']==]'

INSTALLDIR_REGEX = '^local root\\s+=\\s+"(?P<installdir>.*)"'

IS_LOADED_TEMPLATE = 'isloaded("%s")'

LOAD_REGEX = '^\\s*(?:load|depends_on)\\("(\\S+)"'

LOAD_TEMPLATE = 'load("%(mod_name)s")'

LOAD_TEMPLATE_DEPENDS_ON = 'depends_on("%(mod_name)s")'

MODULE_FILE_EXTENSION = '.lua'

MODULE_SHEBANG = ''

PATH_JOIN_TEMPLATE = 'pathJoin(root, "%s")'

START_STR = '[==['

SYNTAX = 'Lua'

UPDATE_PATH_TEMPLATE = '%s_path("%s", %s)'

check_group(group, error_msg=None)

Generate a check of the software group and the current user, and refuse to load the module if the user don’t belong to the group

Parameters:

• group – string with the group name
• error_msg – error message to print for users outside that group

check_str(txt)

Check whether provided string has any unwanted substrings in it.

check_version(minimal_version_maj, minimal_version_min, minimal_version_patch='0')

Check the minimal version of the moduletool in the module file :param minimal_version_maj: the major version to check :param minimal_version_min: the minor version to check :param minimal_version_patch: the patch version to check

comment(msg)

Return string containing given message as a comment.

conditional_statement(conditions, body, negative=False, else_body=None, indent=True, cond_or=False, cond_tmpl=None)

Return formatted conditional statement, with given condition and body.

Parameters:

• conditions – (list of) string(s) containing the statement(s) for the if condition (in correct syntax)
• body – (multiline) string with if body (in correct syntax, without indentation)
• negative – boolean indicating whether the (individual) condition(s) should be negated
• else_body – optional body for ‘else’ part
• indent – indent if/else body
• cond_or – combine multiple conditions using ‘or’ (default is to combine with ‘and’)
• cond_tmpl – template for condition expression (default: ‘%s’)

det_user_modpath(user_modpath)

Determine user-specific modules subdirectory, to be used in ‘use’ statements (cfr. implementations of use() method).

get_description(conflict=True)

Generate a description.

getenv_cmd(envvar, default=None)

Return module-syntax specific code to get value of specific environment variable.

load_module(mod_name, recursive_unload=None, depends_on=False, unload_modules=None, multi_dep_mods=None)

Generate load statement for specified module.

Parameters:

• mod_name – name of module to generate load statement for
• recursive_unload – boolean indicating whether the ‘load’ statement should be reverted on unload (if None: enable if recursive_mod_unload build option or depends_on is True)
• depends_on – use depends_on statements rather than (guarded) load statements
• unload_modules – name(s) of module to unload first
• multi_dep_mods – list of module names in multi_deps context, to use for guarding load statement

modulerc(module_version=None, filepath=None, modulerc_txt=None)

Generate contents of .modulerc(.lua) file, in Lua syntax (but only if Lmod is recent enough, i.e. >= 7.7.38)

Parameters:

• module_version – specs for module-version statement (dict with ‘modname’, ‘sym_version’ & ‘version’ keys)
• filepath – location where .modulerc file should be written to
• modulerc_txt – contents of .modulerc to use

Returns:

contents of .modulerc file

msg_on_load(msg)

Add a message that should be printed when loading the module.

set_alias(key, value)

Generate set-alias statement in modulefile for the given key/value pair.

set_as_default(module_dir_path, module_version, mod_symlink_paths=None)

Create a symlink named ‘default’ inside the package’s module folder in order to set the default module version

Parameters:

• module_dir_path – module directory path, e.g. $HOME/easybuild/modules/all/Bison
• module_version – module version, e.g. 3.0.4
• mod_symlink_paths – list of paths in which symlinks to module files must be created

set_environment(key, value, relpath=False)

Generate a quoted setenv statement for the given key/value pair.

Parameters:

• key – name of environment variable to define
• value – value to define environment variable with
• relpath – value is path relative to installation prefix

swap_module(mod_name_out, mod_name_in, guarded=True)

Generate swap statement for specified module names.

Parameters:

• mod_name_out – name of module to unload (swap out)
• mod_name_in – name of module to load (swap in)
• guarded – guard ‘swap’ statement, fall back to ‘load’ if module being swapped out is not loaded

unload_module(mod_name)

Generate unload statement for specified module.

Parameters:mod_name – name of module to generate unload statement for

update_paths(key, paths, prepend=True, allow_abs=False, expand_relpaths=True)

Generate prepend_path or append_path statements for the given list of paths

Parameters:

• key – environment variable to prepend/append paths to
• paths – list of paths to prepend/append
• prepend – whether to prepend (True) or append (False) paths
• allow_abs – allow providing of absolute paths
• expand_relpaths – expand relative paths into absolute paths (by prefixing install dir)

use(paths, prefix=None, guarded=False, user_modpath=None)

Generate module use statements for given list of module paths. :param paths: list of module path extensions to generate use statements for; paths will be quoted :param prefix: optional path prefix; not quoted, i.e., can be a statement :param guarded: use statements will be guarded to only apply if path exists :param user_modpath: user-specific modules subdirectory to include in use statements

class easybuild.tools.module_generator.ModuleGeneratorTcl(application, fake=False)

Bases: easybuild.tools.module_generator.ModuleGenerator

Class for generating Tcl module files.

CHARS_TO_ESCAPE = ['$']

INSTALLDIR_REGEX = '^set root\\s+(?P<installdir>.*)'

IS_LOADED_TEMPLATE = 'is-loaded %s'

LOAD_REGEX = '^\\s*(?:module\\s+load|depends-on)\\s+(\\S+)'

LOAD_TEMPLATE = 'module load %(mod_name)s'

LOAD_TEMPLATE_DEPENDS_ON = 'depends-on %(mod_name)s'

MODULE_FILE_EXTENSION = ''

MODULE_SHEBANG = '#%Module'

SYNTAX = 'Tcl'

check_group(group, error_msg=None)

Generate a check of the software group and the current user, and refuse to load the module if the user don’t belong to the group

Parameters:

• group – string with the group name
• error_msg – error message to print for users outside that group

comment(msg)

Return string containing given message as a comment.

conditional_statement(conditions, body, negative=False, else_body=None, indent=True, cond_or=False, cond_tmpl=None)

Return formatted conditional statement, with given condition and body.

Parameters:

• conditions – (list of) string(s) containing the statement(s) for the if condition (in correct syntax)
• body – (multiline) string with if body (in correct syntax, without indentation)
• negative – boolean indicating whether the (individual) condition(s) should be negated
• else_body – optional body for ‘else’ part
• indent – indent if/else body
• cond_or – combine multiple conditions using ‘or’ (default is to combine with ‘and’)
• cond_tmpl – template for condition expression (default: ‘%s’)

det_user_modpath(user_modpath)

Determine user-specific modules subdirectory, to be used in ‘use’ statements (cfr. implementation of use() method).

get_description(conflict=True)

Generate a description.

getenv_cmd(envvar, default=None)

Return module-syntax specific code to get value of specific environment variable.

load_module(mod_name, recursive_unload=None, depends_on=False, unload_modules=None, multi_dep_mods=None)

Generate load statement for specified module.

Parameters:

• mod_name – name of module to generate load statement for
• recursive_unload – boolean indicating whether the ‘load’ statement should be reverted on unload (if None: enable if recursive_mod_unload build option or depends_on is True)
• depends_on – use depends_on statements rather than (guarded) load statements
• unload_modules – name(s) of module to unload first
• multi_dep_mods – list of module names in multi_deps context, to use for guarding load statement

msg_on_load(msg)

Add a message that should be printed when loading the module.

set_alias(key, value)

Generate set-alias statement in modulefile for the given key/value pair.

set_as_default(module_dir_path, module_version, mod_symlink_paths=None)

Create a .version file inside the package module folder in order to set the default version for TMod

Parameters:

• module_dir_path – module directory path, e.g. $HOME/easybuild/modules/all/Bison
• module_version – module version, e.g. 3.0.4
• mod_symlink_paths – list of paths in which symlinks to module files must be created

set_environment(key, value, relpath=False)

Generate a quoted setenv statement for the given key/value pair.

Parameters:

• key – name of environment variable to define
• value – value to define environment variable with
• relpath – value is path relative to installation prefix

swap_module(mod_name_out, mod_name_in, guarded=True)

Generate swap statement for specified module names.

Parameters:

• mod_name_out – name of module to unload (swap out)
• mod_name_in – name of module to load (swap in)
• guarded – guard ‘swap’ statement, fall back to ‘load’ if module being swapped out is not loaded

unload_module(mod_name)

Generate unload statement for specified module.

Parameters:mod_name – name of module to generate unload statement for

update_paths(key, paths, prepend=True, allow_abs=False, expand_relpaths=True)

Generate prepend-path or append-path statements for the given list of paths.

Parameters:

• key – environment variable to prepend/append paths to
• paths – list of paths to prepend
• prepend – whether to prepend (True) or append (False) paths
• allow_abs – allow providing of absolute paths
• expand_relpaths – expand relative paths into absolute paths (by prefixing install dir)

use(paths, prefix=None, guarded=False, user_modpath=None)

Generate module use statements for given list of module paths. :param paths: list of module path extensions to generate use statements for; paths will be quoted :param prefix: optional path prefix; not quoted, i.e., can be a statement :param guarded: use statements will be guarded to only apply if path exists :param user_modpath: user-specific modules subdirectory to include in use statements

easybuild.tools.module_generator.avail_module_generators()

Return all known module syntaxes.

easybuild.tools.module_generator.dependencies_for(mod_name, modtool, depth=None)

Obtain a list of dependencies for the given module, determined recursively, up to a specified depth (optionally) :param depth: recursion depth (default is None, which corresponds to infinite recursion depth)

easybuild.tools.module_generator.module_generator(app, fake=False)

Return ModuleGenerator instance that matches the selected module file syntax to be used

easybuild.tools.module_generator.module_load_regex(modfilepath)

Return the correct (compiled) regex to extract dependencies, depending on the module file type (Lua vs Tcl)