Skip to content

Commit f5be0c4

Browse files
committed
Update submission scripts docs add note
1 parent 6402468 commit f5be0c4

2 files changed

Lines changed: 26 additions & 20 deletions

File tree

docs/platforms/example_scripts.rst

Lines changed: 25 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,22 @@ Example Scheduler Submission Scripts
22
====================================
33

44
Below are example submission scripts used to configure and launch libEnsemble
5-
on a variety of high-powered systems. See :ref:`here<platform-index>` for more
6-
information about the respective systems and configuration.
5+
on a variety of high-powered systems. See :ref:`Running on HPC Systems<platform-index>`
6+
for more information about the respective systems and configuration.
7+
8+
.. note::
9+
It is **highly recommended** that the directive lines (e.g., #SBATCH) in batch
10+
submission scripts do **NOT** specify processor, task, or GPU configuration info
11+
--- these lines should only specify the number of nodes required.
12+
13+
For example, do not specify ``#SBATCH --gpus-per-node=4`` in order to use four
14+
GPUs on the node, when each worker may use less than this, as this may assign
15+
all of the GPUs to a single MPI invocation. Instead, the configuration should
16+
be supplied either
17+
:doc:`in the simulation function<../examples/sim_funcs/forces_simf_gpu>`
18+
or, if using dynamic resources,
19+
:doc:`in the generator<../examples/sim_funcs/forces_simf_gpu_vary_resources>`.
20+
721

822
General examples
923
----------------
@@ -55,33 +69,26 @@ System Examples
5569
:caption: /examples/libE_submission_scripts/submit_pbs_polaris.sh
5670
:language: bash
5771

58-
.. dropdown:: Bridges - Central Mode
72+
.. dropdown:: Bebop - Central Mode
5973

60-
.. literalinclude:: ../../examples/libE_submission_scripts/bridges_submit_slurm_central.sh
61-
:caption: /examples/libE_submission_scripts/bridges_submit_slurm_central.sh
74+
.. literalinclude:: ../../examples/libE_submission_scripts/bebop_submit_pbs_central.sh
75+
:caption: /examples/libE_submission_scripts/bebop_submit_pbs_central.sh
6276
:language: bash
6377

64-
.. dropdown:: Bebop - Central Mode
78+
.. dropdown:: Bridges - MPI / Central Mode
6579

66-
.. literalinclude:: ../../examples/libE_submission_scripts/bebop_submit_slurm_central.sh
67-
:caption: /examples/libE_submission_scripts/bebop_submit_slurm_central.sh
80+
.. literalinclude:: ../../examples/libE_submission_scripts/bridges_submit_slurm_central.sh
81+
:caption: /examples/libE_submission_scripts/bridges_submit_slurm_central.sh
6882
:language: bash
6983

70-
.. dropdown:: Bebop - Distributed Mode
84+
.. dropdown:: SLURM - MPI / Distributed Mode (co-locate workers & MPI applications)
7185

72-
.. literalinclude:: ../../examples/libE_submission_scripts/bebop_submit_pbs_distrib.sh
73-
:caption: /examples/libE_submission_scripts/bebop_submit_pbs_distrib.sh
86+
.. literalinclude:: ../../examples/libE_submission_scripts/submit_distrib_mpi4py.sh
87+
:caption: /examples/libE_submission_scripts/submit_distrib_mpi4py.sh
7488
:language: bash
7589

7690
.. dropdown:: Summit (Decommissioned) - On Launch Nodes with Multiprocessing
7791

7892
.. literalinclude:: ../../examples/libE_submission_scripts/summit_submit_mproc.sh
7993
:caption: /examples/libE_submission_scripts/summit_submit_mproc.sh
8094
:language: bash
81-
82-
.. dropdown:: Cobalt - Intermediate node with Multiprocessing
83-
84-
.. literalinclude:: ../../examples/libE_submission_scripts/cobalt_submit_mproc.sh
85-
:caption: /examples/libE_submission_scripts/cobalt_submit_mproc.sh
86-
:language: bash
87-

examples/libE_submission_scripts/submit_perlmutter.sh

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,9 @@
77

88
# This script is using GPU partition
99
export MPICH_GPU_SUPPORT_ENABLED=1
10-
export SLURM_EXACT=1
1110

1211
# One worker for generator and 4 for sims (one GPU each)
1312
python libe_calling_script.py -n 5
1413

1514
# Or if libE_specs option gen_on_manager=True
16-
python libe_calling_script.py -n 4
15+
# python libe_calling_script.py -n 4

0 commit comments

Comments
 (0)