Environment Modules (software)

Last updated
Environment Modules
Stable release
5.4.0 / February 20, 2024;0 days ago (2024-02-20)
Repository
Operating system Unix-like
License GNU General Public License#Version 2
Website modules.sourceforge.net

The Environment Modules system is a tool to help users manage their Unix or Linux shell environment, by allowing groups of related environment-variable settings to be made or removed dynamically.

Contents

Modules has been around since the early 1990s [1] and is used at some of the largest computer centers [2] to deploy multiple versions of different software tools to users. The National Energy Research Scientific Computing Center (NERSC) reports that they use Environment Modules to manage nearly all software. [3] Environment Modules is specified as a Baseline Configuration requirement of the DoD High Performance Computing Modernization Program (HPCMP) [4] Project Baseline Configuration [5] team for participating DoD Supercomputing Resource Centers (DSRCs).

modulefiles

The modules system is based on modulefiles, [6] which specify groups of environment settings that need to be made together. Modulefiles can be installed in a central location for general use, or in a user directory for personal use. Environment Modules modulefiles are written in the Tcl (Tool Command Language) and are interpreted by the modulecmd program via the module [7] user interface.

The key advantage of Environment Modules is that it is shell independent and supports all major shells such as bash, ksh, zsh, sh, tcsh, and csh. The second key advantage is that it allows to use multiple versions of the program or package from the same account by just loading proper module. Those two advantages were instrumental in making Environment Modules a part of most HPC cluster setups. It also inspired several alternative implementation such as lmod from University of Texas, which is written in Lua instead of TCL.

modulefiles are created on per application per version basis. They can be dynamically loaded, unloaded, or switched. Along with the capability of using multiple versions of the same software it also can be used to implement site policies regarding the access and use of applications.

Default modulefiles directory

The default modules search path is in a hidden configuration file you can display with:

$ cat${MODULESHOME}/init/.modulespath /etc/modulefiles # CentOS, Scientific Linux, RHEL/usr/local/Modules/modulefiles # Build from source default

The /etc/modulefiles directory used by some distributions (or any other directory) can be used after a build from source by modifying the ${MODULESHOME}/init/.modulespath file.

Add your own modules to the path

The ${MODULESHOME}/modulefiles/use.own module essentially performs these steps:

mkdir$HOME/privatemodules cp${MODULESHOME}/modulefiles/null$HOME/privatemodules moduleuse--append$HOME/privatemodules

Use a version file within a hierarchical organization

The commands in this section require read/write/execute access to the /etc/modulefiles directory. The $HOME/privatemodules or another directory can be used instead along with "module use --append" or modification of the configuration file specifying the default modules search path.

The default modulefiles directory is empty initially. Copy the null module to the default modulefiles directory to have it shown by "module avail". The following uses the null and module-info modules to show use of a version file within a hierarchical organization and their effect on module avail and module show:

mkdir/etc/modulefiles/test cp${MODULESHOME}/modulefiles/null/etc/modulefiles/test/2.0 cp${MODULESHOME}/modulefiles/module-info/etc/modulefiles/test/1.0 moduleavail moduleshowtest

Set the first version as the default:

echo'#%Module'>/etc/modulefiles/test/.version echo'set ModulesVersion "1.0"'>>/etc/modulefiles/test/.version moduleavail moduleshowtestmoduleshowtest/2.0

Switch to the newer version as the default with:

rm/etc/modulefiles/test/.version echo'#%Module'>/etc/modulefiles/test/.version echo'set ModulesVersion "2.0"'>>/etc/modulefiles/test/.version moduleavail moduleshowtest

After the above following a fresh install from source you would see:

$ moduleavail  -------------------- /usr/local/Modules/3.2.10/modulefiles ---------------------dot         module-git  module-info modules     null        use.own------------------------------- /etc/modulefiles -------------------------------null              test/1.0          test/2.0(default)--------------------------- /home/user/privatemodules --------------------------null$ moduleshowtest-------------------------------------------------------------------/etc/modulefiles/test/2.0:module-whatis    does absolutely nothing -------------------------------------------------------------------$ moduleshowtest/1.0 -------------------------------------------------------------------/etc/modulefiles/test/1.0:module-whatis    returns all various module-info values +++ module-info +++++++++++++++++++++++++++++++flags                   = 2mode                    = displayname                    = test/1.0specified               = test/1.0shell                   = bashshelltype               = shversion                 = test/1.0user                    = advancedtrace                   = 0tracepat                = -.*symbols                 = *undef*+++ info ++++++++++++++++++++++++++++++++++++++hostname                = localhostlevel                   = 1loaded null             = 0library                 = /usr/local/lib/tcl8.6nameofexecutable        =sharedlibextension      = .sotclversion              = 8.6patchlevel              = 8.6.1+++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------------------

Automatic modules initialization

Environment Modules on Scientific Linux, CentOS, and RHEL distributions in the environment-modules package include modules.csh and modules.sh scripts for the /etc/profile.d directory that make modules initialization part of the default shell initialization. One of the advantages of Environment Modules is a single modulefile that supports bash, ksh, zsh, sh as well as tcsh and csh shell users for environment setup and initialization. This makes managing complex environments a bit less complicated.

For a source build the automation for all users can be manually configured.

bash, ksh, zsh, sh automatic modules initialization

/etc/profile.d/modules.sh from the etc/global/profile.modules file in the 3.2.10 modules build directory.

trap""123case"$0"in-bash|bash|*/bash)./usr/local/Modules/default/init/bash;;-ksh|ksh|*/ksh)./usr/local/Modules/default/init/ksh;;-zsh|zsh|*/zsh)./usr/local/Modules/default/init/zsh;;*)./usr/local/Modules/default/init/sh;;# defaultesactrap123

Copy the etc/global/profile.modules file from the 3.2.10 modules build directory to the system initialization directory:

$ sudocpetc/global/profile.modules/etc/profile.d/modules.sh

Add a version 3.2.10 symbolic link for the above generic addresses:

$ cd/usr/local/Modules $ sudoln-sT3.2.10default

tcsh, csh automatic modules initialization

A /etc/profile.d/modules.csh symbolic link to the etc/global/profile.modules file in the 3.2.10 modules build directory can enable automatic modules initialization for these users.

Installing on Linux

Installing Environment Modules on Linux using yum and rpm

On Scientific Linux, CentOS, and RHEL distributions Environment Modules is in the environment-modules package which can be installed with:

sudo yum install environment-modules

Once installed the package information can be viewed with:

rpm -qi environment-modules rpm -ql environment-modules

Installing Environment Modules on Linux using apt and dpkg

On Ubuntu or systems using apt-get, Environment Modules can be installed with:

sudo apt-get install environment-modules

Once installed the package information can be viewed with:

dpkg -L environment-modules

Installing Environment Modules on Linux from source

Although installing from a Linux distributions repository using that distributions update manager is the easiest the software can be installed from source. Resolve dependencies is the most difficult task for an installation from source. The typical configure, make, install cycle can become painfully slow as each configure improvement reveals another dependency not available in your default environment. This section includes the steps to install the Environment Modules package on source including compiling the Tcl (Tool Command Language) from source as a dependency.

8.6.1 version of Tcl (Tool Command Language) built from /usr/local/src

Although the PCLinuxOS 64-bit repositories include Tcl (Tool Command Language) they do not include a development package with the configuration files required to build Environment Modules on Linux from source. Building Tcl from source will make the required files available.

Tcl Source: http://sourceforge.net/projects/tcl/files/Tcl/

Extract source after downloading

cd/usr/local/src tar-xvzf~/Downloads/tcl8.6.1-src.tar.gz

Configure, make, install

cd/usr/local/src/tcl8.6.1 unix/configure make sudomakeinstall

3.2.10 version of modules built from /usr/local/src

Extract source after downloading

cd/usr/local/src tar-xvzf~/Downloads/modules-3.2.10.tar.gz

Configure, make, install

cd/usr/local/src/modules-3.2.10 CPPFLAGS="-DUSE_INTERP_ERRORLINE"./configure make sudomakeinstall

See also

Related Research Articles

<span class="mw-page-title-main">Bash (Unix shell)</span> GNU replacement for the Bourne shell

Bash is a Unix shell and command language written by Brian Fox for the GNU Project as a free software replacement for the Bourne shell. The shell's name is an acronym for Bourne-Again SHell, a pun on the name of the Bourne shell that it replaces and the notion of being "born again". First released in 1989, it has been used as the default login shell for most Linux distributions and it was one of the first programs Linus Torvalds ported to Linux, alongside GCC. It is available on nearly all modern operating systems.

<span class="mw-page-title-main">Cygwin</span> Unix subsystem for Windows machines

Cygwin is a Unix-like environment and command-line interface for Microsoft Windows.

<span class="mw-page-title-main">Shell script</span> Script written for the shell, or command line interpreter, of an operating system

A shell script is a computer program designed to be run by a Unix shell, a command-line interpreter. The various dialects of shell scripts are considered to be scripting languages. Typical operations performed by shell scripts include file manipulation, program execution, and printing text. A script which sets up the environment, runs the program, and does any necessary cleanup or logging, is called a wrapper.

<span class="mw-page-title-main">Unix shell</span> Command-line interpreter for Unix operating system

A Unix shell is a command-line interpreter or shell that provides a command line user interface for Unix-like operating systems. The shell is both an interactive command language and a scripting language, and is used by the operating system to control the execution of the system using shell scripts.

<span class="mw-page-title-main">Z shell</span> Unix shell

The Z shell (Zsh) is a Unix shell that can be used as an interactive login shell and as a command interpreter for shell scripting. Zsh is an extended Bourne shell with many improvements, including some features of Bash, ksh, and tcsh.

<span class="mw-page-title-main">GoboLinux</span> Linux distribution

GoboLinux is a Linux distribution whose most prominent feature is a reorganization of the traditional Linux file system. Rather than following the Filesystem Hierarchy Standard like most Unix-like systems, each program in a GoboLinux system has its own subdirectory tree, where all of its files may be found. Thus, a program "Foo" has all of its specific files and libraries in /Programs/Foo, under the corresponding version of this program at hand. For example, the commonly known GCC compiler suite version 8.1.0, would reside under the directory /Programs/GCC/8.1.0.

Unix security refers to the means of securing a Unix or Unix-like operating system. A secure environment is achieved not only by the design concepts of these operating systems, but also through vigilant user and administrative practices.

Maven is a build automation tool used primarily for Java projects. Maven can also be used to build and manage projects written in C#, Ruby, Scala, and other languages. The Maven project is hosted by The Apache Software Foundation, where it was formerly part of the Jakarta Project.

<span class="mw-page-title-main">Comparison of command shells</span>

A command shell is a command-line interface to interact with and manipulate a computer's operating system.

modprobe is a Linux program originally written by Rusty Russell and used to add a loadable kernel module to the Linux kernel or to remove a loadable kernel module from the kernel. It is commonly used indirectly: udev relies upon modprobe to load drivers for automatically detected hardware.

<span class="mw-page-title-main">MacPorts</span> Package Manager for macOS

MacPorts, formerly DarwinPorts, is a package manager for macOS and Darwin. It is an open-source software project that aims to simplify the installation of other open source software. It is similar in function to Fink and the BSD ports collections.

Tkinter is a Python binding to the Tk GUI toolkit. It is the standard Python interface to the Tk GUI toolkit, and is Python's de facto standard GUI. Tkinter is included with standard Linux, Microsoft Windows and macOS installs of Python.

<span class="mw-page-title-main">CMake</span> Cross-platform, compiler-independent build system generator

In software development, CMake is cross-platform free and open-source software for build automation, testing, packaging and installation of software by using a compiler-independent method. CMake is not a build system itself; it generates another system's build files. It supports directory hierarchies and applications that depend on multiple libraries. It can invoke native build environments such as Make, Qt Creator, Ninja, Android Studio, Apple's Xcode, and Microsoft Visual Studio. It has minimal dependencies, requiring only a C++ compiler on its own build system.

In a Unix shell, the full stop called the dot command (.) is a command that evaluates commands in a computer file in the current execution context. In the C shell, a similar functionality is provided as the source command, and this name is seen in "extended" POSIX shells as well.

NixOS is a free and open-source Linux distribution based on the purely functional Nix package manager. NixOS is composed using modules and packages defined in the nixpkgs project.

pip (package manager) Package management system for Python

pip is a package-management system written in Python and is used to install and manage software packages. The Python Software Foundation recommends using pip for installing Python applications and its dependencies during deployment. Pip connects to an online repository of public packages, called the Python Package Index. Pip can be configured to connect to other package repositories, provided that they comply to Python Enhancement Proposal 503.

cdist Software configuration management tool

cdist is a free software configuration management tool for Unix-like systems. It manages nodes over SSH using the Bourne Shell, and does not require any additional software to be installed on target nodes.

Grunt is a JavaScript task runner, a tool used to automatically perform frequent tasks such as minification, compilation, unit testing, and linting. It uses a command-line interface to run custom tasks defined in a file. Grunt was created by Ben Alman and is written in Node.js. It is distributed via npm. As of October 2022, there were more than 6,000 plugins available in the Grunt ecosystem.

doas is a program to execute commands as another user. The system administrator can configure it to give specified users privileges to execute specified commands. It is free and open-source under the ISC license and available in Unix and Unix-like operating systems.

<span class="mw-page-title-main">Termux</span> Terminal emulator for Android

Termux is a free and open-source terminal emulator for Android which allows for running a Linux environment on an Android device. Termux installs a minimal base system automatically; additional packages are available using its package manager, based on Debian's.

References

  1. ↑ Furlani, John L. (September 30 – October 3, 1991). "Modules: Providing a Flexible User Environment" (PDF). Proceedings of the Fifth Large Installation Systems Administration Conference (LISA V). San Diego, CA: 141–152. Retrieved 9 February 2014.
  2. ↑ "NICS User Support Kraken Modules Description". The National Institute for Computational Sciences (NICS). Oak Ridge National Laboratory - Department of Energy. 22 May 2011. Archived from the original on 22 February 2014. Retrieved 9 February 2014.
  3. ↑ "NERSC Modules Software Environment". The National Energy Research Scientific Computing Center (NERSC). Office of Science - Department of Energy. 2014-02-07. Retrieved 19 February 2014.
  4. ↑ "Use of Modules for Accessing Multiple Versions of Software". Baseline Configuration. DoD High Performance Computing Modernization Program (HPCMP). 20 Mar 2008. Archived from the original on 22 February 2014. Retrieved 9 February 2014.
  5. ↑ "DoD High Performance Computing Modernization Program Baseline Configuration". Baseline Configuration. DoD High Performance Computing Modernization Program (HPCMP). Archived from the original on 22 February 2014. Retrieved 9 February 2014.
  6. ↑ John L. Furlani & Peter W. Osel (July 2009). "modulefile - files containing Tcl code for the Modules package". SourceForge. man page. Retrieved 9 February 2014.
  7. ↑ John L. Furlani & Peter W. Osel (July 2009). "module - command interface to the Modules package". SourceForge. man page. Retrieved 9 February 2014.
  1. http://modules.sourceforge.net/ - home page for the Environment Modules project
  2. John L. Furlani, Modules: Providing a Flexible User Environment Proceedings of the Fifth Large Installation Systems Administration Conference (LISA V), pp. 141–152, San Diego, CA, September 30 - October 3, 1991.
  3. http://lmod.sourceforge.net - alternative implementation using LUA instead of Tcl
  4. http://www.lysator.liu.se/cmod/ - alternative implementation using C only
  5. PennState Environment Modules User Guide
  6. Drag your design environment kicking and screaming into the '90s with Modules! - SNUB Boston 2001 - Erich Whitney, Axiowave Networks, Mark Sprague, ATI Research
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.