EasyBuild documentation

Welcome to the documentation of EasyBuild, a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

This documentation is intended for EasyBuild version 4.7.0, and was last rebuilt on Tue, 28 Feb 2023 13:34:28.

Note

Please consult the overview of major changes in EasyBuild v4.0 before upgrading!

Introductory topics

• What is EasyBuild?
• Concepts and terminology
  • EasyBuild framework
  • Easyblocks
  • Toolchains
  • Easyconfig files
  • Extensions
• Typical workflow example: building and installing WRF
  • Searching for available easyconfigs files
  • Getting an overview of planned installations
  • Installing a software stack

Getting started

• Installing EasyBuild
  • Requirements
  • Using pip to Install EasyBuild
  • Installing EasyBuild with EasyBuild
  • Dependencies
  • Sources
  • In case of installation issues…
• Configuring EasyBuild
  • Supported configuration types
  • Overview of current configuration (--show-config, --show-full-config)
  • Available configuration settings
• Common toolchains
  • Definition and motivation
  • Versioning scheme for common toolchains
  • Update cycle for common toolchains
  • Overview of common toolchains
  • Overview of common toolchains (deprecated versions)
  • Customizing common toolchains
  • More information about toolchains

Basic usage topics

• Using the EasyBuild command line
  • Specifying builds
  • Commonly used command line options
• Writing easyconfig files: the basics
  • What is an easyconfig (file)?
  • Available easyconfig parameters
  • Mandatory easyconfig parameters
  • Common easyconfig parameters
  • Tweaking existing easyconfig files
  • Dynamic values for easyconfig parameters
  • Version-specific documentation relevant to easyconfigs
  • Contributing easyconfigs
• Understanding EasyBuild logs
  • Basic information
  • Navigating log files

Advanced usage topics

• Archived easyconfigs
  • Toolchain deprecation
  • Using --consider-archived-easyconfigs
• Backing up of existing modules (--backup-modules)
  • Disabling automatic backup of modules
  • Example
• Generating container recipes & images
  • Requirements
  • Usage
  • Configuration
  • ‘Stacking’ container images
  • Seeding in source files for container build process
• Contributing
  • How to contribute
  • Pull requests
  • Review process for contributions
• Controlling compiler optimization flags
  • Controlling target architecture specific optimizations via --optarch
• EasyBuild on Cray
  • Test systems
  • EasyBuild toolchains
  • What works already?
  • Required EasyBuild configuration
  • Major supported/tested applications
• Detection of loaded modules
  • Motivation
  • Detection mechanism
  • Action to take if loaded modules are detected
  • Allowing particular loaded modules
  • Checking of $EBROOT* environment variables
• Local variables in easyconfig files
  • Motivation & context
  • Changes in EasyBuild v4.0 w.r.t. local variables in easyconfig files
  • Recommended naming scheme for local variables in easyconfig files
  • Warning for local variables that do not follow the recommended naming scheme
  • Specifying what should be done when non-confirming local variables are found via --local-var-naming-check
  • Renaming local variables to match the recommended naming scheme using eb --fix-deprecated-easyconfigs
• Using an index to speed up searching for easyconfigs
  • Creating an index (--create-index)
  • Updating an existing index (--create-index --force)
  • Using index files
  • Ignoring indices (--ignore-index)
  • Controlling how long the index is valid (--index-max-age)
  • Index included with EasyBuild releases
  • Should I create an index?
• Easystack files
  • The basics
  • Usage
  • Structure of an easystack file
  • Combining command line options with options in an easystack file
  • To be developed
• Experimental features
• Extended dry run
  • Important notes
  • Overview of dry run mechanism
  • Guidelines for easyblocks
  • Example output
• Hooks
  • What are hooks?
  • Configuring EasyBuild to use your hook implementations
  • Available hooks
  • Implementing hooks
  • Caveats
  • Examples of hook implementations
• Implementing easyblocks
  • The basics
  • Easyblocks vs easyconfigs
  • Naming scheme for easyblocks
  • Structure of an easyblock
  • Deriving from existing (generic) easyblocks
  • Specific aspects of easyblocks
  • Using new/custom easyblocks
  • Testing easyblocks
  • Use case: an easyblock for Tensorflow
• Including additional Python modules (--include-*)
  • General aspects of --include-* options
  • Including additional easyblocks (--include-easyblocks)
  • Including additional module naming schemes (--include-module-naming-schemes)
  • Including additional toolchains (--include-toolchains)
• Installing extensions in parallel (experimental!)
  • Configuration
  • Requirements
  • Caveats and Known issues
• Integration with GitHub
  • Requirements
  • Configuration
  • Checking status of GitHub integration (--check-github)
  • Using easyconfigs from pull requests (--from-pr)
  • Using easyblocks from pull requests (--include-easyblocks-from-pr)
  • Uploading test reports (--upload-test-report)
  • Reviewing easyconfig pull requests (--review-pr)
  • Merging easyconfig pull requests (--merge-pr)
  • Submitting new and updating pull requests (--new-pr, --update-pr)
• Locks to prevent duplicate installations running at the same time
  • Locking implementation details
  • Removing locks
  • Configuration options related to installation locks
  • Locks directory
• Manipulating dependencies
  • Filtering out dependencies using --filter-deps
  • Installing dependencies as hidden modules using --hide-deps
  • Using minimal toolchains for dependencies
• Packaging support
  • Prerequisites
  • Configuration options
  • Usage
  • Packaging existing installations
• Partial installations
  • Stopping the installation procedure after a step using -s/--stop
  • Fetching sources with --fetch
  • Installing additional extensions using -k/-skip
  • Only (re)generating (additional) module files using --module-only
• Progress bars
  • General information on progress bars
  • Types of progress bars
  • Relevant configuration options
• Support for RPATH
  • What is RPATH?
  • Why RPATH?
  • Enabling RPATH linking
  • Implementation
  • Filtering RPATH entries via --rpath-filter
  • Relation to $LD_LIBRARY_PATH
• Submitting jobs using --job
  • Quick introduction to --job
  • Configuring --job
  • Usage of --job
  • Examples
• Tracing progress
  • Trace output
  • Example
• Using external modules
  • Using external modules as dependencies
  • Metadata for external modules
• Wrapping dependencies

Other topics

• Code style
  • Notes
• Unit tests
  • What the unit tests are not
  • Available unit test suites
  • Applications
  • Usage
• Useful scripts
  • fix_broken_easyconfigs.py
  • install-EasyBuild-develop.sh
  • clean_gists.py
• Deprecated easyconfigs
  • Symptoms
  • Reasons for deprecation
  • Implications
  • Deprecated toolchains
• Deprecated functionality
  • Overview of deprecated functionality in EasyBuild version 4.7.0
  • Deprecation policy
  • How to check for use of deprecated functionality
• Removed functionality
  • Overview of removed functionality since EasyBuild v4.0
  • Overview of removed functionality since EasyBuild v3.0
  • Overview of removed functionality since EasyBuild v2.0
• EasyBuild maintainers
  • Criteria
  • Roles

Overview of version specific (auto-generated) documentation pages

• Available config file constants
• Available easyconfig parameters
• Constants available for easyconfig files
• Demos
• EasyBuild framework API
• License constants available for easyconfig files
• List of available easyblocks
• List of available toolchain options
• List of known toolchains
• List of supported software
• Overview of configuration options (eb –help)
• Overview of generic easyblocks
• Templates available for easyconfig files

Getting help

Having trouble? We’d like to help!

• Search this documentation collection
• Search for information in the archives of the easybuild@lists.ugent.be mailing list or subscribe to post a question.
• Did you try eb --help?
• Ask a question in the #easybuild IRC channel on the Freenode network, or in the EasyBuild Slack channel https://easybuild.slack.com/ (subscribe via https://easybuild-slack.herokuapp.com)
• Consider participating to an EasyBuild conference call
• Report issues with EasyBuild framework in our framework ticket tracker.
• Report issues with EasyBuild easyblocks in our easyblocks ticket tracker.
• Report issues with EasyBuild easyconfigs in our easyconfigs ticket tracker.
• Report issues with EasyBuild documentation or other aspects in our general ticket tracker.

---

Lists and tables

As of version EasyBuild version 4.7.0:

• The complete table of available toolchains is visible at List of known toolchains
• The list of available easyblocks is visible at List of easyblocks
• The list of available (generic) easyconfig parameters is visible at easyconfigs_parameters

Appendices

• Documentation changelog
• EasyBuild release notes
• Search
• Useful Links