Skip to content

Commit fb136dc

Browse files
klustklust
committed
Deployed 55d11c1 with MkDocs version: 1.6.1
1 parent 1752dc7 commit fb136dc

4 files changed

Lines changed: 317 additions & 140 deletions

File tree

‎2025-LUST/4_EasyBuild_on_LUMI/4_02_tips_and_tricks/index.html‎

Lines changed: 178 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1020,6 +1020,15 @@
10201020
</label>
10211021
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
10221022

1023+
<li class="md-nav__item">
1024+
<a href="#creating-a-full-working-copy-of-the-software-stack-to-develop-in-the-core-stack" class="md-nav__link">
1025+
<span class="md-ellipsis">
1026+
Creating a full working copy of the software stack to develop in the core stack
1027+
</span>
1028+
</a>
1029+
1030+
</li>
1031+
10231032
<li class="md-nav__item">
10241033
<a href="#long-lists-of-elements-for-preconfigopts-or-options-for-configopts" class="md-nav__link">
10251034
<span class="md-ellipsis">
@@ -4628,6 +4637,15 @@
46284637
</label>
46294638
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
46304639

4640+
<li class="md-nav__item">
4641+
<a href="#creating-a-full-working-copy-of-the-software-stack-to-develop-in-the-core-stack" class="md-nav__link">
4642+
<span class="md-ellipsis">
4643+
Creating a full working copy of the software stack to develop in the core stack
4644+
</span>
4645+
</a>
4646+
4647+
</li>
4648+
46314649
<li class="md-nav__item">
46324650
<a href="#long-lists-of-elements-for-preconfigopts-or-options-for-configopts" class="md-nav__link">
46334651
<span class="md-ellipsis">
@@ -4672,6 +4690,165 @@
46724690

46734691

46744692
<h1 id="tips-tricks">Tips &amp; tricks<a class="headerlink" href="#tips-tricks" title="Permanent link">&para;</a></h1>
4693+
<h2 id="creating-a-full-working-copy-of-the-software-stack-to-develop-in-the-core-stack">Creating a full working copy of the software stack to develop in the core stack<a class="headerlink" href="#creating-a-full-working-copy-of-the-software-stack-to-develop-in-the-core-stack" title="Permanent link">&para;</a></h2>
4694+
<p>This can be a lot of work, as it implies you'd also have to compile the toolchains you
4695+
want to use. For rare development, it may be better to use a copy in the LUST project (which
4696+
existed but was not maintained as noone used it).</p>
4697+
<p>Basic process:</p>
4698+
<ul>
4699+
<li>
4700+
<p>Create a directory where the software stack will be installed. Call this <code>&lt;DIR1&gt;</code></p>
4701+
</li>
4702+
<li>
4703+
<p>In that directory:</p>
4704+
<div class="highlight"><pre><span></span><code><span class="nb">cd</span><span class="w"> </span>&lt;DIR1&gt;
4705+
git<span class="w"> </span>clone<span class="w"> </span>git@github.com:Lumi-supercomputer/LUMI-SoftwareStack.git
4706+
git<span class="w"> </span>clone<span class="w"> </span>git@github.com:Lumi-supercomputer/LUMI-EasyBuild-contrib.git
4707+
<span class="c1"># Next one not strictly needed unless you want to process documentation</span>
4708+
git<span class="w"> </span>clone<span class="w"> </span>git@github.com:Lumi-supercomputer/LUMI-EasyBuild-docs.git
4709+
git<span class="w"> </span>clone<span class="w"> </span>git@github.com:Lumi-supercomputer/LUMI-EasyBuild-containers.git
4710+
mkdir<span class="w"> </span>appl-local-containers
4711+
<span class="nb">cd</span><span class="w"> </span>appl-local-containers
4712+
ln<span class="w"> </span>-s<span class="w"> </span>../LUMI-EasyBuild-containers
4713+
ln<span class="w"> </span>-s<span class="w"> </span>/appl/local/containers/sif-images
4714+
ln<span class="w"> </span>-s<span class="w"> </span>/appl/local/containers/easybuild-sif-images
4715+
<span class="nb">cd</span><span class="w"> </span>-
4716+
</code></pre></div>
4717+
<p>(And in fact, you may want to make a directory <code>easybuild-sif-images</code> instead and link to the individual
4718+
files in the system as then you can add additional containers for testing without putting them already in
4719+
<code>/appl/local/containers/easybuild-sif-images</code>.)</p>
4720+
</li>
4721+
<li>
4722+
<p>Also create a separate directory for your local user stack in which you want to experiment with
4723+
building on top of the "central" stack in <code>&lt;DIR1&gt;</code>. Call this directory <code>&lt;DIR2&gt;</code>.</p>
4724+
</li>
4725+
<li>
4726+
<p>In that directory:</p>
4727+
<div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>git@github.com:Lumi-supercomputer/LUMI-EasyBuild-contrib.git
4728+
</code></pre></div>
4729+
<p>and you could clone a repository with your personal easyconfigs also and call this <code>UserRepo</code>.</p>
4730+
</li>
4731+
<li>
4732+
<p>You need a script to set up the environment to use that copy of the software stack instead.
4733+
E.g., with a bash function like</p>
4734+
<div class="highlight"><pre><span></span><code><span class="k">function</span><span class="w"> </span>init-lumi-h<span class="w"> </span><span class="o">{</span><span class="w"> </span><span class="c1"># (Re-)initialize my personal LUMI test environment</span>
4735+
4736+
<span class="w"> </span><span class="c1">###################################################################</span>
4737+
<span class="w"> </span><span class="c1">#</span>
4738+
<span class="w"> </span><span class="c1"># First part: Ensure our initialisation variables are set in case</span>
4739+
<span class="w"> </span><span class="c1"># init-lumi would be configured to preload an environment,</span>
4740+
<span class="w"> </span><span class="c1">#</span>
4741+
4742+
<span class="w"> </span><span class="c1"># Place of the installation</span>
4743+
<span class="w"> </span><span class="c1"># installroot=&quot;$HOME/LUMI&quot;</span>
4744+
<span class="w"> </span><span class="nv">installroot</span><span class="o">=</span><span class="s2">&quot;&lt;DIR1&gt;&quot;</span>
4745+
4746+
<span class="w"> </span><span class="c1"># Our matching LUMI-user</span>
4747+
<span class="w"> </span><span class="nb">export</span><span class="w"> </span><span class="nv">EBU_USER_PREFIX</span><span class="o">=</span><span class="s2">&quot;&lt;DIR2&gt;&quot;</span>&gt;<span class="s2">&quot;</span>
4748+
4749+
<span class="s2"> # Use the test container repository</span>
4750+
<span class="s2"> export LUMI_CONTAINER_REPOSITORY_ROOT=&#39;&lt;DIR1&gt;/appl-local-containers&#39;</span>
4751+
4752+
<span class="s2"> # Force partition L as the others aren&#39;t ready yet</span>
4753+
<span class="s2"> # export LUMI_OVERWRITE_PARTITION=&#39;L&#39;</span>
4754+
4755+
<span class="s2"> # LMOD to use</span>
4756+
<span class="s2"> installroot_lmod=&quot;</span>/opt/cray/pe/lmod/lmod<span class="s2">&quot;</span>
4757+
4758+
4759+
<span class="s2"> ###################################################################</span>
4760+
<span class="s2"> #</span>
4761+
<span class="s2"> # Second part: LMOD initialisation</span>
4762+
<span class="s2"> #</span>
4763+
4764+
<span class="s2"> # Clear LMOD. We will restart it.</span>
4765+
<span class="s2"> # As it is a function we use eval.</span>
4766+
<span class="s2"> #eval &#39;clearLmod&#39;</span>
4767+
<span class="s2"> eval &#39;module --force purge&#39;</span>
4768+
<span class="s2"> eval </span><span class="k">$(</span><span class="nv">$LMOD_DIR</span>/clearLMOD_cmd<span class="w"> </span>--shell<span class="w"> </span>bash<span class="w"> </span>--full<span class="k">)</span>
4769+
<span class="s2"> unset LUMI_INIT_FIRST_LOAD</span>
4770+
4771+
<span class="s2"> # Clear the lmod cache as we may be switching between versions of Lmod.</span>
4772+
<span class="s2"> [ -d </span><span class="nv">$HOME</span><span class="s2">/.lmod.d/.cache ] &amp;&amp; /bin/rm -rf </span><span class="nv">$HOME</span><span class="s2">/.lmod.d/.cache # System Lmod 8.3.1</span>
4773+
<span class="s2"> [ -d </span><span class="nv">$HOME</span><span class="s2">/.cache/lmod ] &amp;&amp; /bin/rm -rf </span><span class="nv">$HOME</span><span class="s2">/.cache/lmod # Lmod 8.7.x</span>
4774+
4775+
<span class="s2"> # Resource the program environment initialisation</span>
4776+
<span class="s2"> # source /appl/lumi/LUMI-SoftwareStack/Setup/cray-pe-configuration.sh</span>
4777+
<span class="s2"> source /etc/cray-pe.d/cray-pe-configuration.sh</span>
4778+
4779+
<span class="s2"> # Correct the path in some variables read from the system file.</span>
4780+
<span class="s2"> sysroot=&#39;/appl/lumi&#39;</span>
4781+
<span class="s2"> mpaths=&quot;</span><span class="si">${</span><span class="nv">mpaths</span><span class="p">//</span><span class="nv">$sysroot</span><span class="p">/</span><span class="nv">$installroot</span><span class="si">}</span><span class="s2">&quot;</span>
4782+
<span class="s2"> LMOD_PACKAGE_PATH=&quot;</span><span class="si">${</span><span class="nv">LMOD_PACKAGE_PATH</span><span class="p">/</span><span class="nv">$sysroot</span><span class="p">/</span><span class="nv">$installroot</span><span class="si">}</span><span class="s2">&quot;</span>
4783+
<span class="s2"> LMOD_RC=&quot;</span><span class="si">${</span><span class="nv">LMOD_RC</span><span class="p">/</span><span class="nv">$sysroot</span><span class="p">/</span><span class="nv">$installroot</span><span class="si">}</span><span class="s2">&quot;</span>
4784+
<span class="s2"> LMOD_ADMIN_FILE=&quot;</span><span class="si">${</span><span class="nv">LMOD_ADMIN_FILE</span><span class="p">/</span><span class="nv">$sysroot</span><span class="p">/</span><span class="nv">$installroot</span><span class="si">}</span><span class="s2">&quot;</span>
4785+
4786+
<span class="s2"> # Initialise LMOD</span>
4787+
<span class="s2"> source </span><span class="si">${</span><span class="nv">installroot_lmod</span><span class="si">}</span><span class="s2">/init/profile</span>
4788+
4789+
<span class="s2"> # Build MODULEPATH</span>
4790+
<span class="s2"> mod_paths=&quot;</span>/opt/cray/pe/lmod/modulefiles/core
4791+
<span class="w"> </span>/opt/cray/pe/lmod/modulefiles/craype-targets/default
4792+
<span class="w"> </span><span class="nv">$mpaths</span>
4793+
<span class="w"> </span>/opt/cray/modulefiles
4794+
<span class="w"> </span>/opt/modulefiles<span class="s2">&quot;</span>
4795+
<span class="s2"> MODULEPATH=&#39;&#39;</span>
4796+
<span class="s2"> for p in </span><span class="k">$(</span><span class="nb">echo</span><span class="w"> </span><span class="nv">$mod_paths</span><span class="k">)</span><span class="s2"> ; do</span>
4797+
<span class="s2"> if [ -d </span><span class="nv">$p</span><span class="s2"> ] ; then</span>
4798+
<span class="s2"> MODULEPATH=</span><span class="nv">$MODULEPATH</span><span class="s2">:</span><span class="nv">$p</span>
4799+
<span class="s2"> fi</span>
4800+
<span class="s2"> done</span>
4801+
<span class="s2"> export MODULEPATH=</span><span class="si">${</span><span class="nv">MODULEPATH</span><span class="p">/:/</span><span class="si">}</span><span class="s2"> # Export and remove the leading :.</span>
4802+
4803+
<span class="s2"> # Build LMOD_SYSTEM_DEFAULT_MODULES</span>
4804+
<span class="s2"> LMOD_SYSTEM_DEFAULT_MODULES=</span><span class="k">$(</span><span class="nb">echo</span><span class="w"> </span><span class="si">${</span><span class="nv">init_module_list</span><span class="k">:-</span><span class="nv">PrgEnv</span><span class="p">-</span><span class="nv">$default_prgenv</span><span class="si">}</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>sed<span class="w"> </span><span class="s2">&quot;s_ *_:_g&quot;</span><span class="k">)</span>
4805+
<span class="s2"> export LMOD_SYSTEM_DEFAULT_MODULES</span>
4806+
<span class="s2"> # Need eval on the next line as it is a shell function.</span>
4807+
<span class="s2"> eval &quot;</span>module<span class="w"> </span>--initial_load<span class="w"> </span>--no_redirect<span class="w"> </span>restore<span class="s2">&quot;</span>
4808+
4809+
4810+
<span class="s2"> ###################################################################</span>
4811+
<span class="s2"> #</span>
4812+
<span class="s2"> # Third part: Personal finishing touches</span>
4813+
<span class="s2"> #</span>
4814+
4815+
<span class="s2"> # Set some aliases</span>
4816+
<span class="s2"> alias cdesr=&quot;</span><span class="nb">cd</span><span class="w"> </span><span class="nv">$installroot</span><span class="s2">&quot;</span>
4817+
<span class="s2"> alias pdesr=&quot;</span><span class="nb">pushd</span><span class="w"> </span><span class="nv">$installroot</span><span class="s2">&quot;</span>
4818+
<span class="s2"> alias cdeur=&#39;cd </span><span class="nv">$EBU_USER_PREFIX</span><span class="s2">&#39;</span>
4819+
<span class="s2"> alias pdeur=&#39;pushd </span><span class="nv">$EBU_USER_PREFIX</span><span class="s2">&#39;</span>
4820+
<span class="s2"> alias cdes=&quot;</span><span class="nb">cd</span><span class="w"> </span><span class="nv">$installroot</span>/LUMI-SoftwareStack/easybuild/easyconfigs<span class="s2">&quot;</span>
4821+
<span class="s2"> alias pdes=&quot;</span><span class="nb">pushd</span><span class="w"> </span><span class="nv">$installroot</span>/LUMI-SoftwareStack/easybuild/easyconfigs<span class="s2">&quot;</span>
4822+
<span class="s2"> alias cdec=&quot;</span><span class="nb">cd</span><span class="w"> </span><span class="nv">$installroot</span>/LUMI-EasyBuild-contrib/easybuild/easyconfigs<span class="s2">&quot;</span>
4823+
<span class="s2"> alias pdec=&quot;</span><span class="nb">pushd</span><span class="w"> </span><span class="nv">$installroot</span>/LUMI-EasyBuild-contrib/easybuild/easyconfigs<span class="s2">&quot;</span>
4824+
<span class="s2"> alias cdecc=&quot;</span><span class="nb">cd</span><span class="w"> </span><span class="nv">$LUMI_CONTAINER_REPOSITORY_ROOT</span>/LUMI-EasyBuild-containers/easybuild/easyconfigs<span class="s2">&quot;</span>
4825+
<span class="s2"> alias pdecc=&quot;</span><span class="nb">pushd</span><span class="w"> </span><span class="nv">$LUMI_CONTAINER_REPOSITORY_ROOT</span>/LUMI-EasyBuild-containers/easybuild/easyconfigs<span class="s2">&quot;</span>
4826+
<span class="s2"> alias cdeu=&#39;cd </span><span class="nv">$EBU_USER_PREFIX</span><span class="s2">/UserRepo/easybuild/easyconfigs&#39;</span>
4827+
<span class="s2"> alias pdeu=&#39;pushd </span><span class="nv">$EBU_USER_PREFIX</span><span class="s2">/UserRepo/easybuild/easyconfigs&#39;</span>
4828+
<span class="s2"> alias upgrade-tc=&quot;</span><span class="nv">$installroot</span>/LUMI-SoftwareStack/tools/upgrade-tc.py<span class="s2">&quot;</span>
4829+
<span class="s2"> alias upgrade-locals=&quot;</span><span class="nv">$installroot</span>/LUMI-SoftwareStack/tools/upgrade-locals.lua<span class="s2">&quot;</span>
4830+
<span class="s2"> echo -e &quot;</span><span class="se">\n</span>Aliases<span class="w"> </span>introduced<span class="w"> </span>by<span class="w"> </span>this<span class="w"> </span>command:<span class="se">\n</span><span class="s2">&quot; \</span>
4831+
<span class="s2"> &quot;</span>cdesr/pdesr<span class="w"> </span>:<span class="w"> </span>system<span class="w"> </span>install<span class="w"> </span>root<span class="w"> </span>:<span class="w"> </span><span class="nv">$installroot</span><span class="se">\n</span><span class="s2">&quot; \</span>
4832+
<span class="s2"> &quot;</span>cdeur/pdeur<span class="w"> </span>:<span class="w"> </span>user<span class="w"> </span>install<span class="w"> </span>root<span class="w"> </span>:<span class="w"> </span><span class="nv">$EBU_USER_PREFIX</span><span class="se">\n</span><span class="s2">&quot; \</span>
4833+
<span class="s2"> &quot;</span>cdes/pdes<span class="w"> </span>:<span class="w"> </span>system<span class="w"> </span>EasyConfigs<span class="w"> </span>:<span class="w"> </span><span class="nv">$installroot</span>/LUMI-SoftwareStack/easybuild/easyconfigs<span class="se">\n</span><span class="s2">&quot; \</span>
4834+
<span class="s2"> &quot;</span>cdec/pdec<span class="w"> </span>:<span class="w"> </span>contributed<span class="w"> </span>EasyConfigs<span class="w"> </span>:<span class="w"> </span><span class="nv">$installroot</span>/LUMI-EasyBuild-contrib/easybuild/easyconfigs<span class="se">\n</span><span class="s2">&quot; \</span>
4835+
<span class="s2"> &quot;</span>cdecc/pdecc<span class="w"> </span>:<span class="w"> </span>container<span class="w"> </span>EasyConfigs<span class="w"> </span>:<span class="w"> </span><span class="nv">$LUMI_CONTAINER_REPOSITORY_ROOT</span>/LUMI-EasyBuild-containers/easybuild/easyconfigs<span class="se">\n</span><span class="s2">&quot; \</span>
4836+
<span class="s2"> &quot;</span>cdeu/pdeu<span class="w"> </span>:<span class="w"> </span>user<span class="w"> </span>EasyConfigs<span class="w"> </span>:<span class="w"> </span><span class="nv">$EBU_USER_PREFIX</span>/UserRepo/easybuild/easyconfig<span class="se">\n</span><span class="s2">&quot; \</span>
4837+
<span class="s2"> &quot;</span>upgrade-tc<span class="w"> </span>:<span class="w"> </span>CSCS<span class="w"> </span>script<span class="w"> </span>to<span class="w"> </span>upgrade<span class="w"> </span>a<span class="w"> </span>toolchain<span class="w"> </span><span class="k">in</span><span class="w"> </span>an<span class="w"> </span>EasyConfig<span class="se">\n</span><span class="s2">&quot; \</span>
4838+
<span class="s2"> &quot;</span>upgrade-locals<span class="w"> </span>:<span class="w"> </span>script<span class="w"> </span>to<span class="w"> </span>upgrade<span class="w"> </span>the<span class="w"> </span>local_*_version<span class="w"> </span>lines<span class="w"> </span><span class="k">in</span><span class="w"> </span>an<span class="w"> </span>EasyConfig<span class="se">\n\n</span><span class="s2">&quot; \</span>
4839+
<span class="s2"> &quot;</span>Lmod<span class="w"> </span>version:<span class="w"> </span><span class="k">$(</span><span class="nb">eval</span><span class="w"> </span><span class="s1">&#39;module --version&#39;</span><span class="w"> </span><span class="p">|&amp;</span><span class="w"> </span>grep<span class="w"> </span>Version<span class="w"> </span><span class="p">|</span><span class="w"> </span>sed<span class="w"> </span>-e<span class="w"> </span><span class="s1">&#39;s|.*Version *\(8.[[:digit:]]*.[[:digit:]]*\).*|\1|&#39;</span><span class="k">)</span><span class="se">\n</span><span class="s2">&quot;</span>
4840+
4841+
<span class="s2">}</span>
4842+
</code></pre></div>
4843+
<p>but of course adapt <code>&lt;DIR1&gt;</code> and <code>&lt;DIR2&gt;</code>.</p>
4844+
</li>
4845+
<li>
4846+
<p>Now you can follow the steps in the <a href="https://lumi-supercomputer.github.io/LUMI-SoftwareStack/procedures/">"Some procedures" page</a>
4847+
of the <a href="https://lumi-supercomputer.github.io/LUMI-SoftwareStack/">LUMI Software Stack technical documentation</a>.</p>
4848+
<p>This documentation is in fact generated from files in the <code>docs</code> subdirectory in the <code>LUMI-SoftwareStack</code> repository
4849+
with a similar mkdocs environment as the LUMI Software Library.</p>
4850+
</li>
4851+
</ul>
46754852
<h2 id="long-lists-of-elements-for-preconfigopts-or-options-for-configopts">Long lists of elements for <code>preconfigopts</code> or options for <code>configopts</code>.<a class="headerlink" href="#long-lists-of-elements-for-preconfigopts-or-options-for-configopts" title="Permanent link">&para;</a></h2>
46764853
<p>An example showing both is in recent <a href="https://github.com/Lumi-supercomputer/LUMI-EasyBuild-contrib/tree/main/easybuild/easyconfigs/n/NAMD">LUMI NAMD EasyConfigs</a>, e.g.,
46774854
<a href="https://github.com/Lumi-supercomputer/LUMI-EasyBuild-contrib/blob/main/easybuild/easyconfigs/n/NAMD/NAMD-3.0-cpeGNU-24.03-rocm-gpu-resident.eb">"NAMD-3.0-cpeGNU-24.03-rocm-gpu-resident.eb"</a>.</p>
@@ -4772,7 +4949,7 @@ <h2 id="more-to-follow">More to follow....<a class="headerlink" href="#more-to-f
47724949
<span class="md-icon" title="Last update">
47734950
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg>
47744951
</span>
4775-
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="May 26, 2025 12:10:59">May 26, 2025</span>
4952+
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="May 28, 2025 16:02:54">May 28, 2025</span>
47764953
</span>
47774954

47784955

‎search/search_index.json‎

Lines changed: 1 addition & 1 deletion
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)