easybuild.tools.config module

EasyBuild configuration (paths, preferences, etc.)

Authors:

  • Stijn De Weirdt (Ghent University)
  • Dries Verdegem (Ghent University)
  • Kenneth Hoste (Ghent University)
  • Pieter De Baets (Ghent University)
  • Jens Timmerman (Ghent University)
  • Toon Willems (Ghent University)
  • Ward Poelmans (Ghent University)
  • Damian Alvarez (Forschungszentrum Juelich GmbH)
  • Andy Georges (Ghent University)
  • Maxime Boissonneault (Compute Canada)
class easybuild.tools.config.BuildOptions(*args, **kwargs)

Bases: abc.BaseBuildOptions

Representation of a set of build options, acts like a dictionary.

KNOWN_KEYS = ['aggregate_regtest', 'backup_modules', 'banned_linked_shared_libs', 'checksum_priority', 'container_config', 'container_image_format', 'container_image_name', 'container_template_recipe', 'container_tmpdir', 'cuda_cache_dir', 'cuda_cache_maxsize', 'cuda_compute_capabilities', 'download_timeout', 'dump_test_report', 'easyblock', 'envvars_user_modules', 'extra_modules', 'filter_deps', 'filter_ecs', 'filter_env_vars', 'filter_rpath_sanity_libs', 'force_download', 'git_working_dirs_path', 'github_user', 'github_org', 'group', 'hide_deps', 'hide_toolchains', 'http_header_fields_urlpat', 'hooks', 'ignore_dirs', 'insecure_download', 'job_backend_config', 'job_cores', 'job_deps_type', 'job_max_jobs', 'job_max_walltime', 'job_output_dir', 'job_polling_interval', 'job_target_resource', 'locks_dir', 'modules_footer', 'modules_header', 'mpi_cmd_template', 'only_blocks', 'optarch', 'package_tool_options', 'parallel', 'pr_branch_name', 'pr_commit_msg', 'pr_descr', 'pr_target_repo', 'pr_title', 'regtest_output_dir', 'rpath_filter', 'rpath_override_dirs', 'required_linked_shared_libs', 'silence_deprecation_warnings', 'skip', 'stop', 'subdir_user_modules', 'sysroot', 'test_report_env_filter', 'testoutput', 'umask', 'wait_on_lock', 'zip_logs', 'add_dummy_to_minimal_toolchains', 'add_system_to_minimal_toolchains', 'allow_modules_tool_mismatch', 'backup_patched_files', 'consider_archived_easyconfigs', 'container_build_image', 'debug', 'debug_lmod', 'dump_autopep8', 'enforce_checksums', 'experimental', 'extended_dry_run', 'force', 'generate_devel_module', 'group_writable_installdir', 'hidden', 'ignore_checksums', 'ignore_index', 'ignore_locks', 'ignore_test_failure', 'install_latest_eb_release', 'logtostdout', 'minimal_toolchains', 'module_extensions', 'module_only', 'package', 'parallel_extensions_install', 'read_only_installdir', 'rebuild', 'remove_ghost_install_dirs', 'rpath', 'sanity_check_only', 'sequential', 'set_default_module', 'set_gid_bit', 'skip_extensions', 'skip_test_cases', 'skip_test_step', 'sticky_bit', 'trace', 'unit_testing_mode', 'upload_test_report', 'update_modules_tool_cache', 'use_ccache', 'use_existing_modules', 'use_f90cache', 'wait_on_lock_limit', 'cleanup_builddir', 'cleanup_easyconfigs', 'cleanup_tmpdir', 'extended_dry_run_ignore_errors', 'fixed_installdir_naming_scheme', 'lib_lib64_symlink', 'lib64_fallback_sanity_check', 'lib64_lib_symlink', 'map_toolchains', 'modules_tool_version_check', 'mpi_tests', 'pre_create_installdir', 'show_progress_bar', 'accept_eula_for', 'from_pr', 'include_easyblocks_from_pr', 'robot', 'search_paths', 'check_ebroot_env_vars', 'detect_loaded_modules', 'local_var_naming_check', 'strict', 'container_type', 'pr_target_branch', 'env_for_shebang', 'index_max_age', 'job_eb_cmd', 'max_fail_ratio_adjust_permissions', 'minimal_build_env', 'package_release', 'package_tool', 'package_type', 'pr_target_account', 'suffix_modules_path', 'default_opt_level', 'allow_loaded_modules', 'wait_on_lock_interval', 'output_style', 'build_specs', 'command_line', 'external_modules_metadata', 'pr_paths', 'robot_path', 'valid_module_classes', 'valid_stops', 'dry_run', 'mod_depends_on', 'recursive_mod_unload', 'retain_all_deps', 'silent', 'try_to_generate', 'check_osdeps', 'validate']
class easybuild.tools.config.ConfigurationVariables(*args, **kwargs)

Bases: abc.BaseConfigurationVariables

This is a dict that supports legacy config names transparently.

KNOWN_KEYS = ['buildpath', 'config', 'containerpath', 'installpath', 'installpath_modules', 'installpath_software', 'job_backend', 'logfile_format', 'moduleclasses', 'module_naming_scheme', 'module_syntax', 'modules_tool', 'packagepath', 'package_naming_scheme', 'prefix', 'repository', 'repositorypath', 'sourcepath', 'subdir_modules', 'subdir_software', 'tmp_logdir']
REQUIRED = ['buildpath', 'config', 'containerpath', 'installpath', 'installpath_modules', 'installpath_software', 'job_backend', 'logfile_format', 'moduleclasses', 'module_naming_scheme', 'module_syntax', 'modules_tool', 'packagepath', 'package_naming_scheme', 'prefix', 'repository', 'repositorypath', 'sourcepath', 'subdir_modules', 'subdir_software', 'tmp_logdir']
get_items_check_required()
For all known/required keys, check if exists and return all key/value pairs.
no_missing: boolean, when True, will throw error message for missing values
class easybuild.tools.config.Singleton

Bases: abc.ABCMeta

Serves as metaclass for classes that should implement the Singleton pattern.

See http://stackoverflow.com/questions/6760685/creating-a-singleton-in-python

easybuild.tools.config.build_option(key, **kwargs)

Obtain value specified build option.

easybuild.tools.config.build_path()

Return the build path

easybuild.tools.config.container_path()

Return the path for container recipes & images

easybuild.tools.config.find_last_log(curlog)

Find location to last log file that is still available.

Parameters:curlog – location to log file of current session
Returns:path to last log file (or None if no log files were found)
easybuild.tools.config.get_build_log_path()

Return (temporary) directory for build log

easybuild.tools.config.get_job_backend()

Return job execution backend (PBS, GC3Pie, …)

easybuild.tools.config.get_log_filename(name, version, add_salt=False, date=None, timestamp=None)

Generate a filename to be used for logging

Parameters:
  • name – software name (‘%(name)s’)
  • version – software version (‘%(version)s’)
  • add_salt – add salt (5 random characters)
  • date – string representation of date to use (‘%(date)s’)
  • timestamp – timestamp to use (‘%(time)s’)
easybuild.tools.config.get_module_naming_scheme()

Return module naming scheme (EasyBuildMNS, HierarchicalMNS, …)

easybuild.tools.config.get_module_syntax()

Return module syntax (Lua, Tcl)

easybuild.tools.config.get_modules_tool()

Return modules tool (EnvironmentModulesC, Lmod, …)

easybuild.tools.config.get_output_style()

Return output style to use.

easybuild.tools.config.get_package_naming_scheme()

Return the package naming scheme

easybuild.tools.config.get_pretend_installpath()

Get the installpath when –pretend option is used

easybuild.tools.config.get_repository()

Return the repository (git, svn or file)

easybuild.tools.config.get_repositorypath()

Return the repository path

easybuild.tools.config.init(options, config_options_dict)

Gather all variables and check if they’re valid Variables are read in this order of preference: generaloption > legacy environment > legacy config file

easybuild.tools.config.init_build_options(build_options=None, cmdline_options=None)

Initialize build options.

easybuild.tools.config.install_path(typ=None)

Returns the install path - subdir ‘software’ for actual installation (default) - subdir ‘modules’ for environment modules (typ=’mod’)

easybuild.tools.config.log_file_format(return_directory=False, ec=None, date=None, timestamp=None)

Return the format for the logfile or the directory

Parameters:
  • ec – dict-like value that provides values for %(name)s and %(version)s template values
  • date – string representation of date to use (‘%(date)s’)
  • timestamp – timestamp to use (‘%(time)s’)
easybuild.tools.config.log_format(ec=None)

Return the logfilename format

easybuild.tools.config.log_path(ec=None)

Return the log path

easybuild.tools.config.mk_full_default_path(name, prefix='/home/docs/.local/easybuild')

Create full path, avoid ‘/’ at the end.

easybuild.tools.config.module_classes()

Return list of module classes specified in config file.

easybuild.tools.config.package_path()

Return the path where built packages are copied to

easybuild.tools.config.read_environment(env_vars, strict=False)

NO LONGER SUPPORTED: use read_environment from easybuild.tools.environment instead

easybuild.tools.config.source_path()

NO LONGER SUPPORTED: use source_paths instead

easybuild.tools.config.source_paths()

Return the list of source paths

easybuild.tools.config.update_build_option(key, value)

Update build option with specified name to given value.

WARNING: Use this with care, the build options are not expected to be changed during an EasyBuild session!

easybuild.tools.config.update_build_options(key_value_dict)

Update build options as specified by the given dictionary (where keys are assumed to be build option names). Returns dictionary with original values for the updated build options.