docker: tutorial cluster interactions

Author: falkmielke

content/tutorials/development_containers/index.md

@@ -29,9 +29,13 @@ output:
 You might have heard about "containerization" with [**Docker**](https://docs.docker.com).
 Docker has been labeled "the *Holy Grail* of reproducibility" in [The Open Science Manual by Claudio Zandonella Callegher and Davide Massidda (2023)](https://arca-dpss.github.io/manual-open-science/docker-chapter.html).
 Although containerization is an immensely useful Open Science tool worth striving for, the *Holy Grail* is an inaccurate metaphor, because
-(i) Unlike The Grail, Docker is easy to find and accessible.
-(ii) Docker alone does not make a reproducible workflow; some of its capability is occasionally confused with package version management.
-(iii) Docker has issues, some of them mitigated by configuration adjustment or switching to "Podman".
+
+-  (i) Unlike The Grail, Docker is easy to find and accessible.
+-  (ii) Docker alone does not make a reproducible workflow; some of its capability is occasionally confused with package version management.
+-  (iii) Docker has issues, some of them mitigated by configuration adjustment or switching to "Podman".
+
+Time to explore what containers really are, and what they are not.
+
 
 # Overview
 
@@ -47,15 +51,15 @@ or store and publish a whole image right away.
 In this notebook, you will find **installation instructions**, <a href="#sec-commands" class="quarto-xref"><b>useful commands</b></a>, references, and a loose assembly of general and almost philosophical topics to prime you on the **complications and misconceptions** surrounding containerization.
 
 There are numerous useful build instructions and container images already out there, which you can **simply `pull` and `run`**.
-This is an easy, entry level application of container software like Docker, [covered in an introductory tutorial](../../../tutorials/TODO).
+This is an easy, entry level application of container software like Docker, [covered in an introductory tutorial](../../tutorials/development_containers_run).
 
-A second step is to set up and deploy a **self-`build` custom container** I demonstrate step-by-step [in a slightly more advanced tutorial](../../../tutorials/TODO).
+A second step is to set up and deploy a **self-`build` custom container** I demonstrate step-by-step [in a slightly more advanced tutorial](../../tutorials/development_containers_build).
 This is intended to be a rather general test case, enabling you to later configure more specific container solutions for your own purpose.
 For example, you will learn how to spin up an existing `rocker/rstudio` container, and even modify it with additional system components and libraries.
 
 For relevant INBO-specific use cases, make sure to [check out the `containbo` repository](https://github.com/inbo/containbo) which documents **even more tipps and tricks** assembled during my humble (but mostly succesful) attempts to get INBO R packages to run in a container environment.
 
-I also present **Podman** as a [full replacement for Docker](../../../tutorials/TODO), and recommend to give it a try.
+I also present **Podman** as a [full replacement for Docker](../../tutorials/development_containers_podman), and recommend to give it a try.
 
 On Windows, installation, configuration, and management of containers runs via the `docker desktop` app.
 However, this series of tutorials also covers (and in fact focuses on) the terminal-centered steps to be executed on a Linux computer or within a WSL.
@@ -173,7 +177,7 @@ docker --version
 
 Congratulations: now the fun starts!
 
-With docker installed, the next step is to run a container image which someone else has prepared and hosted online, [which you can read about in the next tutorial](../../../tutorials/TODO).
+With docker installed, the next step is to run a container image which someone else has prepared and hosted online, [which you can read about in the next tutorial](../../tutorials/development_containers_run).
 
 # The Holy Grail?
 
@@ -253,7 +257,7 @@ Historically, Docker could not run "rootless", i.e. without elevated privileges
 [This seems to have changed](https://docs.docker.com/engine/security/rootless), according to Docker.
 Some caution is still warranted: the setup procedure requires downloading and running shell scripts (which must be checked); the deamon still builds on `systemd` (*usually* root level); some functionality is limited.
 
-On the other hand, there is Podman (cf. the [Podman tutorial](../../../tutorials/TODO)).
+On the other hand, there is Podman (cf. the [Podman tutorial](../../tutorials/development_containers_podman)).
 It *used to* require almost the same extra steps as the `docker-rootless` to work rootless, but we found that these requirements are now met per default.
 It seems that, at the time of writing, Docker and Podman have identical capabilities in terms of rootless containerization.
 The remaining difference is that Podman seems to have more sensible default settings.

content/tutorials/development_containers/index.qmd

@@ -1,5 +1,5 @@
 ---
-title: "Containers with Docker and Podman"
+title: "Containers: An Overview"
 description: "Introduction to containerization and the practical use of Docker-like tools."
 date: "2025-02-21"
 authors: [falkmielke]
@@ -28,10 +28,12 @@ output:
 You might have heard about "containerization" with [**Docker**](https://docs.docker.com).
 Docker has been labeled "the *Holy Grail* of reproducibility" in [The Open Science Manual by Claudio Zandonella Callegher and Davide Massidda (2023)](https://arca-dpss.github.io/manual-open-science/docker-chapter.html).
 Although containerization is an immensely useful Open Science tool worth striving for, the *Holy Grail* is an inaccurate metaphor, because 
-(i) Unlike The Grail, Docker is easy to find and accessible.
-(ii) Docker alone does not make a reproducible workflow; some of its capability is occasionally confused with package version management.
-(iii) Docker has issues, some of them mitigated by configuration adjustment or switching to "Podman".
 
+- (i) Unlike The Grail, Docker is easy to find and accessible.
+- (ii) Docker alone does not make a reproducible workflow; some of its capability is occasionally confused with package version management.
+- (iii) Docker has issues, some of them mitigated by configuration adjustment or switching to "Podman".
+
+Time to explore what containers really are, and what they are not.
 
 # Overview
 
@@ -49,18 +51,18 @@ In this notebook, you will find **installation instructions**, [**useful command
 
 
 There are numerous useful build instructions and container images already out there, which you can **simply `pull` and `run`**.
-This is an easy, entry level application of container software like Docker, [covered in an introductory tutorial](/tutorials/TODO).
+This is an easy, entry level application of container software like Docker, [covered in an introductory tutorial](../../tutorials/development_containers_run).
 
 
-A second step is to set up and deploy a **self-`build` custom container** I demonstrate step-by-step [in a slightly more advanced tutorial](/tutorials/TODO).
+A second step is to set up and deploy a **self-`build` custom container** I demonstrate step-by-step [in a slightly more advanced tutorial](../../tutorials/development_containers_build).
 This is intended to be a rather general test case, enabling you to later configure more specific container solutions for your own purpose.
 For example, you will learn how to spin up an existing `rocker/rstudio` container, and even modify it with additional system components and libraries.
 
 
 For relevant INBO-specific use cases, make sure to [check out the `containbo` repository](https://github.com/inbo/containbo) which documents **even more tipps and tricks** assembled during my humble (but mostly succesful) attempts to get INBO R packages to run in a container environment.
 
 
-I also present **Podman** as a [full replacement for Docker](/tutorials/TODO), and recommend to give it a try.
+I also present **Podman** as a [full replacement for Docker](../../tutorials/development_containers_podman), and recommend to give it a try.
 
 
 On Windows, installation, configuration, and management of containers runs via the `docker desktop` app. 
@@ -192,7 +194,7 @@ docker --version
 Congratulations: now the fun starts!
 
 
-With docker installed, the next step is to run a container image which someone else has prepared and hosted online, [which you can read about in the next tutorial](/tutorials/TODO).
+With docker installed, the next step is to run a container image which someone else has prepared and hosted online, [which you can read about in the next tutorial](../../tutorials/development_containers_run).
 
 
 # The Holy Grail? 
@@ -282,7 +284,7 @@ Historically, Docker could not run "rootless", i.e. without elevated privileges.
 Some caution is still warranted: the setup procedure requires downloading and running shell scripts (which must be checked); the deamon still builds on `systemd` (*usually* root level); some functionality is limited.
 
 
-On the other hand, there is Podman (cf. the [Podman tutorial](/tutorials/TODO)).
+On the other hand, there is Podman (cf. the [Podman tutorial](../../tutorials/development_containers_podman)).
 It *used to* require almost the same extra steps as the `docker-rootless` to work rootless, but we found that these requirements are now met per default.
 It seems that, at the time of writing, Docker and Podman have identical capabilities in terms of rootless containerization. 
 The remaining difference is that Podman seems to have more sensible default settings.

content/tutorials/development_containers_build/index.md

@@ -25,16 +25,20 @@ output:
 ---
 
 
-By now, you [will have successfully installed](../tutorials/development_containers) Docker or [Podman](../tutorials/development_containers_podman).
-You hopefully succeeded in [running others' containers](../tutorials/development_containers_run), e.g. from a container repository.
+By now, you [will have successfully installed](../../tutorials/development_containers) Docker or [Podman](../../tutorials/development_containers_podman).
+You hopefully succeeded in [running others' containers](../../tutorials/development_containers_run), e.g. from a container repository.
 
 Next, it is time to customize your container.
 
 To give you a metaphor to work on: imagine you have a nice little DIY project for your garage workshop.
 This time, you would like to build your own [Matryoshka dolls](https://en.wikipedia.org/wiki/Matryoshka_doll) (матрёшка, stacking dolls, a great allegory for recursion).
 
-https://images.unsplash.com/photo-1586010135736-c16373adf060?q=80
-(Photo by <a href="https://unsplash.com/@ilmatar?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Iza Gawrych</a> on <a href="https://unsplash.com/photos/a-group-of-blue-and-gold-vases-sitting-on-top-of-a-table-oL3O2PybLoo?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>)
+<figure>
+<img src="https://images.unsplash.com/photo-1586010135736-c16373adf060?q=80" alt="build" />
+<figcaption aria-hidden="true">(Photo by <a href="https://unsplash.com/@ilmatar?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Iza Gawrych</a> on <a href="https://unsplash.com/photos/a-group-of-blue-and-gold-vases-sitting-on-top-of-a-table-oL3O2PybLoo?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>)</figcaption>
+</figure>
+
+
 
 Like all good DIY, you do not fully start from scratch: you start with a blueprint which someone else has created, or general building instructions.
 You usually do not grow your own trees to get the wood, you buy wooden blocks of approximately the right size; neither do you mix you own paint from elemental ingredients, you assemble what others have to offer.
@@ -274,10 +278,10 @@ When designing and building Dockerfiles, you effectively craft your own DIY Matr
 This may involve tinkering, some sawdust will fall off on the sides, but often the end product is quite presentable.
 
 And that is one of the main purposes of a custom docker image: you can store a given set of interrelated software building blocks for later use (reproducibility).
-Some of these sets are rather rough, abstract, or general (like the images you get on image repositories, which you can [simply pull and run](../tutorials/development_containers_run)).
+Some of these sets are rather rough, abstract, or general (like the images you get on image repositories, which you can [simply pull and run](../../tutorials/development_containers_run)).
 Others are bespoke, containing exact requirements for a given task.
-Both functions are important building blocks of open science, and I elaborate more about this framework [in the main article on containerization](../tutorials/development_containers).
-Docker is a specific implementation of the container concept, and you might also want to [try out Podman](../tutorials/development_containers_podman) as an alternative.
+Both functions are important building blocks of open science, and I elaborate more about this framework [in the main article on containerization](../../tutorials/development_containers).
+Docker is a specific implementation of the container concept, and you might also want to [try out Podman](../../tutorials/development_containers_podman) as an alternative.
 
 Good luck with all your DIY projects, and thank you for reading!
 

content/tutorials/development_containers_build/index.qmd

@@ -34,8 +34,11 @@ Next, it is time to customize your container.
 To give you a metaphor to work on: imagine you have a nice little DIY project for your garage workshop. 
 This time, you would like to build your own [Matryoshka dolls](https://en.wikipedia.org/wiki/Matryoshka_doll) (матрёшка, stacking dolls, a great allegory for recursion).
 
-https://images.unsplash.com/photo-1586010135736-c16373adf060?q=80
-(Photo by <a href="https://unsplash.com/@ilmatar?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Iza Gawrych</a> on <a href="https://unsplash.com/photos/a-group-of-blue-and-gold-vases-sitting-on-top-of-a-table-oL3O2PybLoo?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>)
+<figure>
+<img src="https://images.unsplash.com/photo-1586010135736-c16373adf060?q=80" alt="build" />
+<figcaption aria-hidden="true">(Photo by <a href="https://unsplash.com/@ilmatar?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Iza Gawrych</a> on <a href="https://unsplash.com/photos/a-group-of-blue-and-gold-vases-sitting-on-top-of-a-table-oL3O2PybLoo?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>)</figcaption>
+</figure>
+
 
 
 Like all good DIY, you do not fully start from scratch: you start with a blueprint which someone else has created, or general building instructions.