Skip to content

Commit afe4d61

Browse files
committed
Formatting
1 parent b09d3b5 commit afe4d61

1 file changed

Lines changed: 16 additions & 31 deletions

File tree

README.md

Lines changed: 16 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -109,38 +109,28 @@ sequencers:
109109

110110
## How It Works
111111

112-
### Run Processing Pipeline
113-
114112
1. **Discovery**: Scans configured sequencing directories for run folders
115113
2. **Validation**: Confirms run ID matches expected format for the sequencer type
116114
3. **Transfer Phases**:
117-
- **Sequencing Phase**: Starts continuous background rsync transfer while sequencing is ongoing (when the final sequencing file doesn't exist). Uploads status and metadata files (`metadata_for_statusdb`) to database.
115+
- **Sequencing Phase**: Starts continuous background rsync transfer while sequencing is ongoing (when the final sequencing file doesn't exist). Uploads status and metadata files (specified for each sequencer type in the config with `metadata_for_statusdb`) to database.
118116
- **Final Transfer**: After sequencing completes (final sequencing file appears), initiates final rsync transfer and captures exit code.
119-
- **Completion**: Updates database when transfer succeeds.
117+
- **Completion**: Updates database when transfer was successful.
120118

121119
### Status Tracking
122120

123121
Run status is tracked in CouchDB with events including:
124122

125-
**Status:** `sequencing_started`
126-
**Meaning:** Sequencing is ongoing
127-
**Occurs when:** A run folder exists but the final sequencing file has not been created yet
128-
129-
**Status:** `transfer_started`
130-
**Meaning:** Intermediate transfer was initiated
131-
**Occurs when:** A run folder exists but the final sequencing file has not been created yet.
132-
133-
**Status:** `sequencing_finished`
134-
**Meaning:** Sequencing has completed
135-
**Occurs when:** A run folder exists and the final sequencing file has been created
123+
| Status | Meaning | Occurs when |
124+
| ------------------------ | ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
125+
| `sequencing_started` | Sequencing is ongoing | A run folder exists but the final sequencing file has not been created yet |
126+
| `transfer_started` | Intermediate transfer was initiated | Sequencing is ongoing and an rsync has been started |
127+
| `sequencing_finished` | Sequencing has completed | A run folder exists and the final sequencing file has been created |
128+
| `final_transfer_started` | Final sync has started | A run folder exists and the final sequencing file has been created, but the final rsync exit code file has not yet been created or contains a non-zero exit code |
129+
| `transferred_to_hpc` | Transfer completed successfully | A run folder exists, the final sequencing file has been created, and the final rsync exit code file contains a 0 exit code |
136130

137-
**Status:** `final_transfer_started`
138-
**Meaning:** Final sync has started
139-
**Occurs when:** A run folder exists and the final sequencing file has been created, but the final rsync exit code file has not yet been created or contains a non-zero exit code
131+
### Flow chart
140132

141-
**Status:** `transferred_to_hpc`
142-
**Meaning:** Transfer completed successfully
143-
**Occurs when:** A run folder exists, the final sequencing file has been created, and the final rsync exit code file contains a 0 exit code
133+
![Flow chart for Dataflow transfer](/docs/Dataflow_Transfer_flowchart.svg)
144134

145135
### Key Components
146136

@@ -161,13 +151,8 @@ Run status is tracked in CouchDB with events including:
161151

162152
The logic of the script relies on the following status files:
163153

164-
- run.final_file
165-
- The final file written by each sequencing machine. Used to indicate when the sequencing has completed.
166-
- final_rsync_exitcode - Used to indicate when the final rsync is done, so that the final rsync can be run in the background. This is especially useful for restarts after long pauses of the cronjob.
167-
168-
### Flow chart
169-
170-
![Flow chart for Dataflow transfer](/docs/Dataflow_Transfer_flowchart.svg)
154+
- `run.final_file` - The final file written by each sequencing machine. Used to indicate when the sequencing has completed.
155+
- `final_rsync_exitcode` - Used to indicate when the final rsync is done, so that the final rsync can be run in the background. This is especially useful for restarts after long pauses of the cronjob.
171156

172157
## Development
173158

@@ -209,9 +194,9 @@ dataflow_transfer/
209194
To add support for a new sequencer, add the following to dataflow_transfer:
210195

211196
1. Add a new class for the sequencer in one of the run classes files below. Make sure it inherits from the manufacturer class (IlluminaRun, ElementRun, ONTRun)
212-
a. `dataflow_transfer/run_classes/illumina_runs.py`
213-
b. `dataflow_transfer/run_classes/element_runs.py`
214-
c. `dataflow_transfer/run_classes/ont_runs.py`
197+
- `dataflow_transfer/run_classes/illumina_runs.py`
198+
- `dataflow_transfer/run_classes/element_runs.py`
199+
- `dataflow_transfer/run_classes/ont_runs.py`
215200
2. Import the new class in `dataflow_transfer/run_classes/__init__.py`
216201
3. Add a test fixture for the new run in `dataflow_transfer/tests/test_run_classes.py` and include it in the relevant tests
217202
4. Add a section for the sequencer in the config file

0 commit comments

Comments
 (0)