|
1 | | -# Using a Custom Production Image |
| 1 | +# Building a Custom Production Image |
2 | 2 |
|
3 | 3 | Using a container has many advantages and one of them is the ability to develop code on one machine (e.g. your personal laptop), |
4 | 4 | but the deploy the _exact same code_ to run on several other computers (e.g. SLAC batch computing). |
5 | | -This page details a process you can follow to generate your own production image that has your developments of ldmx-sw inside of it and has two big parts. |
6 | | - |
7 | | -1. Building a custom production image with your developments |
8 | | -2. Running any production image (with specific focus on using `singularity`). |
9 | | - |
10 | | -## 1. Building a Custom Production Image |
| 5 | +This page details a process you can follow to generate your own production image that has your developments of ldmx-sw inside of it. |
| 6 | +Refer to the [Batch Computing](../using/batch.md) for how to use these production images. |
11 | 7 |
|
12 | 8 | Building a docker image is complicated, but hopefully you can get one of the two methods listed below to work for your specific case. |
13 | 9 | The common denominator in these methods is that you *need* to have a DockerHub repository that you have administrator access to. |
@@ -61,34 +57,3 @@ _Note: If you haven't yet, you may need to `docker login` on your computer for t |
61 | 57 | ``` |
62 | 58 | docker push docker-user-name/docker-repo-name:some-tag |
63 | 59 | ``` |
64 | | - |
65 | | -## 2. Running the Production Image on the Batch Computer |
66 | | -0. Decide where you want to save the production image: `export LDMX_PRODUCTION_IMG=$(pwd -P)/ldmx_my_production_image_some_tag.sif` |
67 | | -1. Pull the docker container down from Docker Hub and build it into a `.sif` image file. _Note: This step requires your Docker Hub repository to be public._ |
68 | | -``` |
69 | | -singularity build ${LDMX_PRODUCTION_IMG} docker://docker-user-name/docker-repo-name:some-tag |
70 | | -``` |
71 | | -2. Now you can run a configuration script with your developments in the container using |
72 | | -``` |
73 | | -singularity run --no-home ${LDMX_PRODUCTION_IMG} . config.py |
74 | | -``` |
75 | | -This is the command you want to be giving to `bsub` or some other submission program. |
76 | | -The only files it needs access to are the configuration script that you want to run and the `.sif` image file; |
77 | | -both of which are only used at the start-up of the container. |
78 | | - |
79 | | -_Note: On SLAC computers, the default singularity cache directory is $HOME, but SLAC users are not given very much space in $HOME. It may help your singularity build and run commands if you change the cache directory 'SINGULARITY_CACHEDIR' to somewhere with more space._ |
80 | | - |
81 | | -## 3. Submission Script |
82 | | -It is best practice to write a "submission script" that handles the running of this command _and_ any pre- or post- run actions. |
83 | | -A lot of different submission scripts have been written in `bash` and `python`, but they all have a similar structure: |
84 | | -1. Setup the batch environment (e.g. Find singularity image file and turn off email notifications) |
85 | | -2. Configure or write a job script which does all the pre- and post- run actions as well as the `singularity run` command. |
86 | | - - Go to a scratch or temporary directory to work |
87 | | - - Pre-Run Actions: copying over input file, inserting parameters into configuration script, etc... |
88 | | - - Run `singularity run` command |
89 | | - - Post-Run Actions: copying output files to output directory cleaning up scractch directory |
90 | | -3. Submit the job script using the submission program (e.g. `bsub` or `condor`) however many times |
91 | | - |
92 | | -The `batch` directory in the [LDMX-Software/ldmx-sw-scripts](https://github.com/LDMX-Software/ldmx-sw-scripts) |
93 | | -repository offers some examples of these submission scripts, although they tend to be a little old |
94 | | -and will need to be updated. |
0 commit comments