Skip to content

Commit 5c7a044

Browse files
LisaBockLisaBock
committed
update episode 4
1 parent a7a80c8 commit 5c7a044

1 file changed

Lines changed: 154 additions & 12 deletions

File tree

‎_episodes/04-recipe.md‎

Lines changed: 154 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
title: "Running your first recipe"
33
teaching: 15
44
exercises: 15
5-
compatibility: ESMValTool v2.13.0
5+
compatibility: ESMValTool v2.14.0
66

77
questions:
88
- "How to run a recipe?"
@@ -50,12 +50,153 @@ output to the command line. The final message should be "Run was successful".
5050
The exact output varies depending on your machine, but it should look something
5151
like the example log output below.
5252

53-
```
54-
55-
{% include example_output.txt %}
53+
> ## Output
54+
>
55+
> ```yaml
56+
>
57+
> 2026-04-02 12:02:54,972 UTC [3235444] INFO
58+
> ______________________________________________________________________
59+
> _____ ____ __ ____ __ _ _____ _
60+
> | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | |
61+
> | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| |
62+
> | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | |
63+
> |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_|
64+
> ______________________________________________________________________
65+
>
66+
> Earth System Model Evaluation Tool
67+
>
68+
> A community tool for the evaluation of Earth system models.
69+
>
70+
> https://esmvaltool.org
71+
>
72+
> The Earth System Model Evaluation Tool (ESMValTool) is a community
73+
> diagnostics and performance metrics tool for the evaluation of Earth
74+
> System Models (ESMs) that allows for routine comparison of single or
75+
> multiple models, either against predecessor versions or against
76+
> observations.
77+
>
78+
> Tutorial: https://tutorial.esmvaltool.org
79+
> Documentation: https://docs.esmvaltool.org
80+
> Contact: esmvaltool-dev@listserv.dfn.de
81+
>
82+
> If you find this software useful for your research, please cite it using
83+
> https://doi.org/10.5281/zenodo.3387139 for ESMValCore or
84+
> https://doi.org/10.5281/zenodo.3401363 for ESMValTool or
85+
> any of the reference papers listed at https://esmvaltool.org/references/.
86+
>
87+
> Have fun!
88+
>
89+
> 2026-04-02 12:02:54,973 UTC [3235444] INFO Package versions
90+
> 2026-04-02 12:02:54,973 UTC [3235444] INFO ----------------
91+
> 2026-04-02 12:02:54,973 UTC [3235444] INFO ESMValCore: 2.15.0.dev9+g412767341
92+
> 2026-04-02 12:02:54,973 UTC [3235444] INFO ESMValTool: 2.15.0.dev4+g74d7837f0
93+
> 2026-04-02 12:02:54,973 UTC [3235444] INFO ----------------
94+
> 2026-04-02 12:02:54,974 UTC [3235444] INFO Reading configuration files from:
95+
> /home/b/username/ESMValTool/ESMValCore/esmvalcore/config/configurations/defaults (defaults)
96+
> /home/b/username/.config/esmvaltool (default user configuration directory)
97+
> 2026-04-02 12:02:54,974 UTC [3235444] INFO Writing program log files to:
98+
> /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/main_log.txt
99+
> /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/main_log_debug.txt
100+
> /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/cmor_log.txt
101+
> 2026-04-02 12:02:56,104 UTC [3235444] INFO Starting the Earth System Model Evaluation Tool at time: 2026-04-02 12:02:56 UTC
102+
> 2026-04-02 12:02:56,104 UTC [3235444] INFO ----------------------------------------------------------------------
103+
> 2026-04-02 12:02:56,105 UTC [3235444] INFO RECIPE = /home/b/username/ESMValTool/ESMValTool/esmvaltool/recipes/examples/recipe_python.yml
104+
> 2026-04-02 12:02:56,105 UTC [3235444] INFO RUNDIR = /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run
105+
> 2026-04-02 12:02:56,105 UTC [3235444] INFO WORKDIR = /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/work
106+
> 2026-04-02 12:02:56,105 UTC [3235444] INFO PREPROCDIR = /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/preproc
107+
> 2026-04-02 12:02:56,105 UTC [3235444] INFO PLOTDIR = /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/plots
108+
> 2026-04-02 12:02:56,105 UTC [3235444] INFO ----------------------------------------------------------------------
109+
> 2026-04-02 12:02:56,163 UTC [3235444] INFO Running tasks using at most 256 processes
110+
> 2026-04-02 12:02:56,163 UTC [3235444] INFO If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory.
111+
> 2026-04-02 12:02:56,163 UTC [3235444] INFO If you experience memory problems, try reducing 'max_parallel_tasks' in your configuration.
112+
> 2026-04-02 12:02:56,808 UTC [3235444] INFO Creating tasks from recipe
113+
> 2026-04-02 12:02:56,808 UTC [3235444] INFO Creating tasks for diagnostic map
114+
> 2026-04-02 12:02:56,808 UTC [3235444] INFO Creating diagnostic task map/script1
115+
> 2026-04-02 12:02:56,810 UTC [3235444] INFO Creating preprocessor task map/tas
116+
> 2026-04-02 12:02:56,810 UTC [3235444] INFO Creating preprocessor 'to_degrees_c' task for variable 'tas'
117+
> 2026-04-02 12:02:56,821 UTC [3235444] INFO Found input files for Dataset: tas, Amon, CMIP6, BCC-ESM1, historical, r1i1p1f1, mon, gn, v20181214
118+
> 2026-04-02 12:02:56,826 UTC [3235444] INFO Found input files for Dataset: tas, Amon, CMIP5, bcc-csm1-1, historical, r1i1p1, mon, v1
119+
> 2026-04-02 12:02:56,827 UTC [3235444] INFO PreprocessingTask map/tas created.
120+
> 2026-04-02 12:02:56,827 UTC [3235444] INFO Creating tasks for diagnostic timeseries
121+
> 2026-04-02 12:02:56,827 UTC [3235444] INFO Creating diagnostic task timeseries/script1
122+
> 2026-04-02 12:02:56,828 UTC [3235444] INFO Creating preprocessor task timeseries/tas_amsterdam
123+
> 2026-04-02 12:02:56,828 UTC [3235444] INFO Creating preprocessor 'annual_mean_amsterdam' task for variable 'tas_amsterdam'
124+
> 2026-04-02 12:02:56,833 UTC [3235444] INFO Found input files for Dataset: tas, Amon, CMIP6, BCC-ESM1, historical, r1i1p1f1, mon, gn, v20181214
125+
> 2026-04-02 12:02:56,837 UTC [3235444] INFO Found input files for Dataset: tas, Amon, CMIP5, bcc-csm1-1, historical, r1i1p1, mon, v1
126+
> 2026-04-02 12:02:56,838 UTC [3235444] INFO PreprocessingTask timeseries/tas_amsterdam created.
127+
> 2026-04-02 12:02:56,838 UTC [3235444] INFO Creating preprocessor task timeseries/tas_global
128+
> 2026-04-02 12:02:56,838 UTC [3235444] INFO Creating preprocessor 'annual_mean_global' task for variable 'tas_global'
129+
> 2026-04-02 12:02:56,845 UTC [3235444] INFO Found input files for Dataset: tas, Amon, CMIP6, BCC-ESM1, historical, r1i1p1f1, mon, gn, v20181214, supplementaries: areacella, fx, 1pctCO2, fx, v20190613
130+
> 2026-04-02 12:02:56,849 UTC [3235444] INFO Found input files for Dataset: tas, Amon, CMIP5, bcc-csm1-1, historical, r1i1p1, mon, v1, supplementaries: areacella, fx, r0i0p0, fx
131+
> 2026-04-02 12:02:56,850 UTC [3235444] INFO PreprocessingTask timeseries/tas_global created.
132+
> 2026-04-02 12:02:56,850 UTC [3235444] INFO These tasks will be executed: timeseries/tas_amsterdam, map/tas, map/script1, timeseries/script1, timeseries/tas_global
133+
> 2026-04-02 12:02:56,880 UTC [3235444] INFO Wrote recipe with version numbers and wildcards to:
134+
> file:///scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/recipe_python_filled.yml
135+
> 2026-04-02 12:03:15,435 UTC [3235444] INFO Using Dask distributed scheduler (address: tcp://127.0.0.1:46079, dashboard link: http://127.0.0.1:8787/status)
136+
> 2026-04-02 12:03:15,436 UTC [3235444] INFO Running 5 tasks using 5 processes
137+
> 2026-04-02 12:03:15,519 UTC [3236556] INFO Starting task map/tas in process [3236556]
138+
> 2026-04-02 12:03:15,519 UTC [3236558] INFO Starting task timeseries/tas_amsterdam in process [3236558]
139+
> 2026-04-02 12:03:15,520 UTC [3236560] INFO Starting task timeseries/tas_global in process [3236560]
140+
> 2026-04-02 12:03:15,606 UTC [3235444] INFO Progress: 3 tasks running, 2 tasks waiting for ancestors, 0/5 done
141+
> 2026-04-02 12:03:15,805 UTC [3236558] INFO Extracting data for Amsterdam, Noord-Holland, Nederland (52.3730796 °N, 4.8924534 °E)
142+
> 2026-04-02 12:03:16,177 UTC [3236558] INFO Extracting data for Amsterdam, Noord-Holland, Nederland (52.3730796 °N, 4.8924534 °E)
143+
> 2026-04-02 12:03:16,239 UTC [3236556] INFO Computing and saving data for preprocessing task map/tas
144+
> 2026-04-02 12:03:16,384 UTC [3236558] INFO Generated PreprocessorFile: /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/preproc/timeseries/tas_amsterdam/MultiModelMean_historical_Amon_tas_1850-2000.nc
145+
> 2026-04-02 12:03:17,250 UTC [3236556] INFO map/tas [########################################] | 100% Completed | 1.01 s
146+
> 2026-04-02 12:03:17,252 UTC [3236560] INFO Computing and saving data for preprocessing task timeseries/tas_global
147+
> 2026-04-02 12:03:17,256 UTC [3236556] INFO Successfully completed task map/tas (priority 1) in 0:00:01.736625
148+
> 2026-04-02 12:03:17,414 UTC [3235444] INFO Progress: 2 tasks running, 2 tasks waiting for ancestors, 1/5 done
149+
> 2026-04-02 12:03:17,442 UTC [3236562] INFO Starting task map/script1 in process [3236562]
150+
> 2026-04-02 12:03:17,447 UTC [3236562] INFO Running command ['/work/bd0854/username/utils/mambaforge/envs/esmvaltool/bin/python3.12', '/home/b/username/ESMValTool/ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.py', '/scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/map/script1/settings.yml']
151+
> 2026-04-02 12:03:17,447 UTC [3236562] INFO Writing output to /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/work/map/script1
152+
> 2026-04-02 12:03:17,447 UTC [3236562] INFO Writing plots to /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/plots/map/script1
153+
> 2026-04-02 12:03:17,447 UTC [3236562] INFO Writing log to /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/map/script1/log.txt
154+
> 2026-04-02 12:03:17,447 UTC [3236562] INFO To re-run this diagnostic script, run:
155+
> cd /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/map/script1; MPLBACKEND="Agg" /work/bd0854/username/utils/mambaforge/envs/esmvaltool/bin/python3.12 /home/b/username/ESMValTool/ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.py /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/map/script1/settings.yml
156+
> 2026-04-02 12:03:17,514 UTC [3235444] INFO Progress: 3 tasks running, 1 tasks waiting for ancestors, 1/5 done
157+
> 2026-04-02 12:03:18,367 UTC [3236560] INFO timeseries/tas_global [########################################] | 100% Completed | 1.10 s
158+
> 2026-04-02 12:03:18,370 UTC [3236558] INFO Computing and saving data for preprocessing task timeseries/tas_amsterdam
159+
> 2026-04-02 12:03:18,394 UTC [3236560] INFO Successfully completed task timeseries/tas_global (priority 4) in 0:00:02.873452
160+
> 2026-04-02 12:03:18,675 UTC [3235444] INFO Progress: 2 tasks running, 1 tasks waiting for ancestors, 2/5 done
161+
> 2026-04-02 12:03:20,566 UTC [3236558] INFO timeseries/tas_amsterdam [########################################] | 100% Completed | 2.18 s
162+
> 2026-04-02 12:03:20,591 UTC [3236558] INFO Successfully completed task timeseries/tas_amsterdam (priority 3) in 0:00:05.071412
163+
> 2026-04-02 12:03:20,712 UTC [3235444] INFO Progress: 1 tasks running, 1 tasks waiting for ancestors, 3/5 done
164+
> 2026-04-02 12:03:20,730 UTC [3236564] INFO Starting task timeseries/script1 in process [3236564]
165+
> 2026-04-02 12:03:20,757 UTC [3236564] INFO Running command ['/work/bd0854/username/utils/mambaforge/envs/esmvaltool/bin/python3.12', '/home/b/username/ESMValTool/ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.py', '/scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/timeseries/script1/settings.yml']
166+
> 2026-04-02 12:03:20,757 UTC [3236564] INFO Writing output to /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/work/timeseries/script1
167+
> 2026-04-02 12:03:20,757 UTC [3236564] INFO Writing plots to /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/plots/timeseries/script1
168+
> 2026-04-02 12:03:20,757 UTC [3236564] INFO Writing log to /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/timeseries/script1/log.txt
169+
> 2026-04-02 12:03:20,758 UTC [3236564] INFO To re-run this diagnostic script, run:
170+
> cd /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/timeseries/script1; MPLBACKEND="Agg" /work/bd0854/username/utils/mambaforge/envs/esmvaltool/bin/python3.12 /home/b/username/ESMValTool/ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.py /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/timeseries/script1/settings.yml
171+
> 2026-04-02 12:03:20,812 UTC [3235444] INFO Progress: 2 tasks running, 0 tasks waiting for ancestors, 3/5 done
172+
> 2026-04-02 12:03:28,198 UTC [3236562] INFO Maximum memory used (estimate): 0.3 GB
173+
> 2026-04-02 12:03:28,199 UTC [3236562] INFO Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
174+
> 2026-04-02 12:03:28,274 UTC [3236564] INFO Maximum memory used (estimate): 0.3 GB
175+
> 2026-04-02 12:03:28,275 UTC [3236564] INFO Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
176+
> 2026-04-02 12:03:28,618 UTC [3236562] INFO Successfully completed task map/script1 (priority 0) in 0:00:11.175930
177+
> 2026-04-02 12:03:28,820 UTC [3235444] INFO Progress: 1 tasks running, 0 tasks waiting for ancestors, 4/5 done
178+
> 2026-04-02 12:03:29,008 UTC [3236564] INFO Successfully completed task timeseries/script1 (priority 2) in 0:00:08.278011
179+
> 2026-04-02 12:03:29,020 UTC [3235444] INFO Progress: 0 tasks running, 0 tasks waiting for ancestors, 5/5 done
180+
> 2026-04-02 12:03:29,020 UTC [3235444] INFO Successfully completed all tasks.
181+
> 2026-04-02 12:03:29,716 UTC [3235444] INFO Wrote recipe with version numbers and wildcards to:
182+
> file:///scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/recipe_python_filled.yml
183+
> 2026-04-02 12:03:30,651 UTC [3235444] INFO Wrote recipe output to:
184+
> file:///scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/index.html
185+
> 2026-04-02 12:03:30,651 UTC [3235444] INFO It looks like you are connected to a remote machine via SSH. To show the output html file, you can try the following command on your local machine:
186+
> server=136.172.124.6 && port=31415 && ssh -t -L ${port}:localhost:${port} username@${server} /work/bd0854/username/utils/mambaforge/envs/esmvaltool/bin/python3.12 -m http.server ${port} -d /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254
187+
> Then visit http://localhost:31415 in your browser
188+
> 2026-04-02 12:03:30,651 UTC [3235444] INFO If the port 31415 is already in use, you can replace it with any other free one (e.g., 12789). If you are connected through a jump host, replace the server IP address 136.172.124.6 with your SSH server name
189+
> 2026-04-02 12:03:30,652 UTC [3235444] INFO Ending the Earth System Model Evaluation Tool at time: 2026-04-02 12:03:30 UTC
190+
> 2026-04-02 12:03:30,652 UTC [3235444] INFO Time for running the recipe was: 0:00:34.547424
191+
> 2026-04-02 12:03:31,701 UTC [3235444] INFO Maximum memory used (estimate): 8.3 GB
192+
> 2026-04-02 12:03:31,708 UTC [3235444] INFO Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
193+
> 2026-04-02 12:03:31,733 UTC [3235444] WARNING Input data is not (fully) CMOR-compliant, see /scratch/b/username/esmvaltool_output/recipe_python_20260402_120254/run/cmor_log.txt for details
194+
> 2026-04-02 12:03:31,733 UTC [3235444] INFO Run was successful
195+
>
196+
> ```
197+
>
198+
{: .solution}
56199
57-
```
58-
{: .output}
59200
60201
> ## Pro tip: ESMValTool search paths
61202
>
@@ -75,17 +216,18 @@ Let's dissect what's happening here.
75216
>
76217
> After the banner and general information, the output starts with some important locations.
77218
>
78-
> 1. Did ESMValTool use the right config file?
219+
> 1. Did ESMValTool use the right config files?
79220
> 1. What is the path to the example recipe?
80221
> 1. What is the main output folder generated by ESMValTool?
81222
> 1. Can you guess what the different output directories are for?
82223
> 1. ESMValTool creates two log files. What is the difference?
83224
>
84225
> > ## Answers
85226
> >
86-
> > 1. The config file should be the one we edited in the previous episode,
87-
> > something like `/home/<username>/.config/esmvaltool/config-user.yml` or
88-
`~/esmvaltool_tutorial/config-user.yml`.
227+
> > 1. The list of directories involves the directory with all default config files
228+
> > (e.g., `../ESMValCore/esmvalcore/config/configurations/defaults`)
229+
> > and should list the (default) user configuration directory with our edited files
230+
> > from the previous episode (default: `~/.config/esmvaltool`).
89231
> > 1. ESMValTool found the recipe in its installation directory,
90232
>> something like
91233
> > `/home/users/username/mambaforge/envs/esmvaltool/bin/esmvaltool/recipes/examples/`
@@ -98,11 +240,11 @@ Let's dissect what's happening here.
98240
> > folder is made inside the output directory specified in the previous
99241
> > episode: `~/esmvaltool_tutorial/esmvaltool_output`.
100242
> > 1. There should be four output folders:
101-
> > - `plots/`: this is where output figures are stored.
102-
> > - `preproc/`: this is where pre-processed data are stored.
103243
> > - `run/`: this is where esmvaltool stores general information about the
104244
> > run, such as log messages and a copy of the recipe file.
105245
> > - `work/`: this is where output files (not figures) are stored.
246+
> > - `preproc/`: this is where pre-processed data are stored.
247+
> > - `plots/`: this is where output figures are stored.
106248
> > 1. The log files are:
107249
> > - `main_log.txt` is a copy of the command-line output
108250
> > - `main_log_debug.txt` contains more detailed information that may be

0 commit comments

Comments
 (0)