Deployed 83a33cb with MkDocs version: 1.6.1

Author: klust

2025-LUST/2_Using/2_02_creating_easyconfig_files/index.html

@@ -4559,10 +4559,15 @@ <h2 id="what-are-external-modules">What are external modules?<a class="headerlin
 <p>For such dependencies, EasyBuild will:</p>
 <ul>
 <li>
-<p>load the module before initiating the software build and install procedure</p>
+<p>load the module before initiating the software build and install procedure,</p>
 </li>
 <li>
-<p>include a <code>module load</code> statement in the generated module file (for runtime dependencies)</p>
+<p>include a <code>module load</code> statement in the generated module file (for runtime dependencies),</p>
+</li>
+<li>
+<p>create <code>EBROOT*</code> and <code>EBVERSION*</code> environment variables if proper metadata is defined 
+  for the external module (see later in this section). The latter can be picked up by
+  easyblocks just as the corresponding variables set by regular EasyBuild-installed modules.</p>
 </li>
 </ul>
 <div class="admonition note">
@@ -4600,8 +4605,12 @@ <h2 id="easybuild-metadata-for-external-modules">EasyBuild Metadata for external
 <span class="na">prefix</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">FFTW_DIR/..</span>
 <span class="na">version</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">3.3.8.10</span>
 </code></pre></div></p>
-<p>The environment variable <code>$EBROOTFFTW</code> will also be defined according to the <code>prefix</code> specified in the metadata file.</p>
-<p>On LUMI, users in generally don't need to be too concerned about the metadata file as the EasyBuild-user (and other hidden
+<p>The environment variable <code>$EBROOTFFTW</code> will also be defined according to the <code>prefix</code> specified in the metadata file,
+which is always pointing to an environment variable and starting a path relative from there.</p>
+<p>As a result of this definition, to a correctly written easyblock, the <code>cray-fftw/3.3.8.10</code> module will appear as
+if it is the EasyBuild-installed <code>FFTW/3.3.8.10</code> module and during the installation process, the environment 
+variables <code>EBROOTFFTW</code> and <code>EBVERSIONFFTW</code> will be defined as expected.</p>
+<p>On LUMI, users don't need to be too concerned about the metadata file as the EasyBuild-user (and other hidden
 EasyBuild configuration modules) take care of pointing to the right metadata file, which is specific for each version of the
 Cray PE and hence each version of the LUMI software stack.</p>
 <hr />
@@ -4626,7 +4635,7 @@ <h2 id="easybuild-metadata-for-external-modules">EasyBuild Metadata for external
     <span class="md-icon" title="Last update">
       <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg>
     </span>
-    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="April 30, 2025 09:07:39">April 30, 2025</span>
+    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="May 6, 2025 15:20:25">May 6, 2025</span>
   </span>
 
 

2025-LUST/2_Using/2_03_external_modules/index.html

@@ -800,6 +800,15 @@
     <nav class="md-nav" aria-label="Easyconfig parameters">
       <ul class="md-nav__list">
 
+          <li class="md-nav__item">
+  <a href="#reading-parameters" class="md-nav__link">
+    <span class="md-ellipsis">
+      Reading parameters
+    </span>
+  </a>
+  
+</li>
+        
           <li class="md-nav__item">
   <a href="#updating-parameters" class="md-nav__link">
     <span class="md-ellipsis">
@@ -4783,6 +4792,15 @@
     <nav class="md-nav" aria-label="Easyconfig parameters">
       <ul class="md-nav__list">
 
+          <li class="md-nav__item">
+  <a href="#reading-parameters" class="md-nav__link">
+    <span class="md-ellipsis">
+      Reading parameters
+    </span>
+  </a>
+  
+</li>
+        
           <li class="md-nav__item">
   <a href="#updating-parameters" class="md-nav__link">
     <span class="md-ellipsis">
@@ -4955,7 +4973,7 @@ <h2 id="generic-vs-software-specific-easyblocks">Generic vs software-specific ea
 Typically this involves highly customised steps, for example specifying dedicated configuration options, creating
 or adjusting specific files, executing non-standard shell commands, etc. Usually a custom implementation of the
 sanity check is also included. Much of the work done in software-specific easyblocks can often also be done 
-in generic easyblocks using parameters such as <code>confdigopts</code> etc., but a software-specific easyblock can
+in generic easyblocks using parameters such as <code>configopts</code> etc., but a software-specific easyblock can
 hide some of that complexity from the user. Other software-specific easyblocks implement very specific
 installation procedures that do not fit in one of the generic ones.</p>
 <p>Using a generic easyblock requires specifying the <code>easyblock</code> parameter in the easyconfig file.
@@ -5005,7 +5023,7 @@ <h3 id="python-module-name-and-location">Python module name and location<a class
 <ul>
 <li><code>gcc.py</code> (for <em>GCC</em>)</li>
 <li><code>netcdf_fortran.py</code> (for <em>netCDF-Fortran</em>)</li>
-<li><code>gamess_us.py</code> (for <em>GAMESS (US)</em>)</li>
+<li><code>gamess_us.py</code> (for <em>GAMESS-US</em>)</li>
 </ul>
 <p>The <code>get_module_path</code> function provided by the EasyBuild framework in the
 <code>easybuild.framework.easyconfig.easyconfig</code> module returns the (full)
@@ -5029,8 +5047,8 @@ <h3 id="python-module-name-and-location">Python module name and location<a class
 configuration option. As long as both the filename of the Python module and the name of the Python class
 are correct, EasyBuild will use these easyblocks when needed.</p>
 <p>On LUMI, the EasyBuild configuration modules take care of setting this parameter (using the corresponding environment
-variable), pointing to custom easyblocks in the LUMI software stack itself and a repo (with a fixed name) that users
-can create themselves. At this moment it does not yet include possible other easyblock repositories in other repositories.</p>
+variable), pointing to custom easyblocks in the LUMI software stack itself, the repo for contributed easyconfigs,
+and a repo (with a fixed name) that users can create themselves. </p>
 <h2 id="structure-of-an-easyblock">Structure of an easyblock<a class="headerlink" href="#structure-of-an-easyblock" title="Permanent link">&para;</a></h2>
 <p>The example below shows the overall structure of an easyblock:</p>
 <div class="highlight"><pre><span></span><code><span class="kn">from</span><span class="w"> </span><span class="nn">easybuild.framework.easyblock</span><span class="w"> </span><span class="kn">import</span> <span class="n">EasyBlock</span>
@@ -5080,6 +5098,7 @@ <h2 id="deriving-from-existing-easyblocks">Deriving from existing easyblocks<a c
         <span class="nb">super</span><span class="p">(</span><span class="n">EB_Example</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">configure_step</span><span class="p">()</span>
 </code></pre></div>
 <h2 id="easyconfig-parameters">Easyconfig parameters<a class="headerlink" href="#easyconfig-parameters" title="Permanent link">&para;</a></h2>
+<h3 id="reading-parameters">Reading parameters<a class="headerlink" href="#reading-parameters" title="Permanent link">&para;</a></h3>
 <p>All of the easyconfig parameters that are defined in an easyconfig file
 are available via the <code>EasyConfig</code> instance that can be accessed through <code>self.cfg</code> in an easyblock.
 For instance, if the easyconfig file specifies</p>
@@ -5207,6 +5226,8 @@ <h2 id="executing-shell-commands">Executing shell commands<a class="headerlink"
 <p><code>std_qa=&lt;dict&gt;</code> to specify regular expression patterns for common questions, and the matching answer;</p>
 </li>
 </ul>
+<p>In EasyBuild 5, the new function <code>run_shell_cmd</code> is introduced to replace these functions,
+and both functions are deprecated and scheduled to be removed in EasyBuild 6.</p>
 <h2 id="manipulating-environment-variables">Manipulating environment variables<a class="headerlink" href="#manipulating-environment-variables" title="Permanent link">&para;</a></h2>
 <p>To (re)define environment variables, the <code>setvar</code> function provided by the
 <a href="https://docs.easybuild.io/en/latest/api/easybuild.tools.environment.html"><code>easybuild.tools.environment</code></a>
@@ -5585,7 +5606,7 @@ <h3 id="exercise-i2">Exercise I.2<a class="headerlink" href="#exercise-i2" title
     <span class="md-icon" title="Last update">
       <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg>
     </span>
-    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="April 30, 2025 09:07:39">April 30, 2025</span>
+    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="May 6, 2025 15:20:25">May 6, 2025</span>
   </span>
 
 

2025-LUST/2_Using/2_04_implementing_easyblocks/index.html

@@ -4743,12 +4743,16 @@ <h2 id="why-hooks">Why hooks?<a class="headerlink" href="#why-hooks" title="Perm
 <p>Some site have really large hook files to implement policies and modify standard EasyBuild build recipes, e.g.,</p>
 <ul>
 <li><a href="https://github.com/easybuilders/JSC/blob/2022/Custom_Hooks/eb_hooks.py">The hooks file from Jülich Supercomputing Centre</a></li>
-<li>ComputeCanada <a href="https://github.com/ComputeCanada/easybuild-computecanada-config/blob/main/cc_hooks_common.py">cc_hooks_common.py</a>
-    and <a href="https://github.com/ComputeCanada/easybuild-computecanada-config/blob/main/cc_hooks_gentoo.py">cc_hooks_gento.py</a></li>
+<li>Digital Research Alliance of Canada (formerly ComputeCanada) 
+    <a href="https://github.com/ComputeCanada/easybuild-computecanada-config/blob/main/cc_hooks_common.py">cc_hooks_common.py</a>
+    and <a href="https://github.com/ComputeCanada/easybuild-computecanada-config/blob/main/2023/cc_hooks.py">2023/cc_hooks.py</a>
+    files</li>
 <li><a href="https://github.com/EESSI/software-layer/blob/main/eb_hooks.py">The hooks file from the EESSI software stack</a>. 
-   <a href="https://www.eessi-hpc.org/">EESSI</a> is an effort to build a software stack distributed via 
-   <a href="https://cernvm.cern.ch/fs/">CernVM-FS</a> using EasyBuild to build all software.</li>
+    <a href="https://www.eessi.io/">EESSI</a> is an effort to build a software stack distributed via 
+    <a href="https://cernvm.cern.ch/fs/">CernVM-FS</a> using EasyBuild to build all software.</li>
 </ul>
+<p>On LUMI we also use hooks, but the definition file is rather small. e/g/.
+<a href="https://github.com/Lumi-supercomputer/LUMI-SoftwareStack/blob/main/easybuild/hooks/LUMI_site_hooks-22.08.py">the LUMI hooks file in use since LUMI/22.08</a>.</p>
 <h2 id="implementing-and-using-hooks">Implementing and using hooks<a class="headerlink" href="#implementing-and-using-hooks" title="Permanent link">&para;</a></h2>
 <p>Using hooks is done by implementing functions with specific names in a Python module,
 and configuring EasyBuild to use them by specifying the path to the file that contains them
@@ -4778,6 +4782,8 @@ <h3 id="startend-hooks">Start/end hooks<a class="headerlink" href="#startend-hoo
 <p>These hooks can be implementing by defining a function named <code>start_hook</code> and <code>end_hook</code>
 in the Python module file that is provided via the <code>hooks</code> configuration option.
 No arguments are provided when calling these hooks.</p>
+<p><em>On LUMI, an end hook is used to clear the Lmod cache after installing software as otherwise
+a newly installed module may not be found immediately.</em></p>
 <h3 id="parse-hook">Parse hook<a class="headerlink" href="#parse-hook" title="Permanent link">&para;</a></h3>
 <p>The <code>parse</code> hook is triggered right after an easyconfig file is being parsed,
 <em>before</em> EasyBuild sets up its internal data structures based on the parsed easyconfig.</p>
@@ -4883,7 +4889,7 @@ <h3 id="manipulating-easyconfig-parameters">Manipulating easyconfig parameters<a
     <span class="md-icon" title="Last update">
       <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg>
     </span>
-    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="April 30, 2025 09:07:39">April 30, 2025</span>
+    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="May 6, 2025 16:08:56">May 6, 2025</span>
   </span>
 
 

2025-LUST/3_Advanced/3_02_hooks/index.html

@@ -4854,6 +4854,7 @@ <h3 id="job-backend">Job backend<a class="headerlink" href="#job-backend" title=
 <div class="highlight"><pre><span></span><code><span class="nb">export</span><span class="w"> </span><span class="nv">EASYBUILD_JOB_BACKEND</span><span class="o">=</span><span class="s1">&#39;Slurm&#39;</span>
 </code></pre></div>
 <p>On LUMI this is taken care of in the EasyBuild configuration modules such as <code>EasyBuild-user</code>.</p>
+<p>In EasyBuild v5.x the default is already Slurm, and GC3Pie is deprecated.</p>
 <h3 id="job-resources">Job resources<a class="headerlink" href="#job-resources" title="Permanent link">&para;</a></h3>
 <p>To submit an installation as a job, simply use <code>eb --job</code>:</p>
 <div class="highlight"><pre><span></span><code>eb<span class="w"> </span>example.eb<span class="w"> </span>--job
@@ -4915,8 +4916,9 @@ <h3 id="temporary-log-files-and-build-directories">Temporary log files and build
 <p>The problems for the temporary log files are twofold. First, they may end up in a place
 that is not available on the compute nodes. E.g., for the same reasons as for the build
 path, the LUMI EasyBuild configuration will place the temporary files in a subdirectory of
-<code>$XDG_RUNTIME_DIR</code> on the loginnodes but a subdirectory of <code>/dev/shm/$USER</code> on the
-compute nodes. The second problem however is that if an installation fails, those log files are
+<code>$XDG_RUNTIME_DIR</code> on the loginnodes but a subdirectory of <code>/tmp/$sLURM_JOBID</code> on the
+compute nodes (as that one should be cleaned up at the end of the job). 
+The second problem however is that if an installation fails, those log files are
 not even accessible anymore which may leave you wondering about the actual cause of the failing 
 installation...</p>
 <p>To remedy this, there are a couple of EasyBuild configuration options you can use:</p>
@@ -5075,7 +5077,7 @@ <h3 id="final-result">Final result<a class="headerlink" href="#final-result" tit
     <span class="md-icon" title="Last update">
       <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg>
     </span>
-    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="April 30, 2025 09:07:39">April 30, 2025</span>
+    <span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="May 6, 2025 16:08:56">May 6, 2025</span>
   </span>