You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/guides/guide-admin/crypt4gh_to_protes.md
+17-21Lines changed: 17 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,26 +1,26 @@
1
-
# Setting up Crypt4GH encryption/decryption in Funnel
1
+
# Crypt4GH Integration with proTES: A Guide to Secure Genomic Analysis
2
2
3
-
This guide explains how to configure and deploy an environment that enables collaborative research on sensitive genomic data. Data holders can securely provide encrypted data for analysis while researchers process it through TES/[Funnel](https://github.com/ohsu-comp-bio/funnel) and [proTES](https://github.com/elixir-cloud-aai/proTES), where automatic decryption occurs within secure containers without granting researchers direct access to the sensitive data. This setup leverages [GA4GH TES](https://github.com/ga4gh/task-execution-schemas)standards for scalable and secure task execution.
3
+
This guide explains how to configure and deploy an environment that enables collaborative research on sensitive genomic data. Data holders can securely provide encrypted data for analysis while researchers process it through [Funnel](https://github.com/ohsu-comp-bio/funnel) and [proTES](https://github.com/elixir-cloud-aai/proTES), where automatic decryption occurs within secure containers without granting researchers direct access to the sensitive data. This setup leverages [GA4GH TES](https://github.com/ga4gh/task-execution-schemas)standard for scalable and secure task execution.
4
4
5
5
## Use Case
6
6
7
-
A data holder needs to provide sensitive genomic data for analysis to researchers in a cloud environment. The data must remain encrypted during storage and transfer, with decryption occurring only within a secure computational environment (container), without granting direct data access to the researcher.
7
+
A data holder needs to provide sensitive genomic data for analysis to a researcher in a cloud environment. The data must remain encrypted during storage and transfer, with decryption occurring only within a secure computational environment (container), without granting direct data access to the researcher.
8
8
9
9
1. The data holder encrypts sensitive data using Crypt4GH and stores them at a secure storage (e.g. S3 buckets).
10
10
2. The researcher submits a GA4GH TES task to `proTES` for analysis of the encrypted data.
11
11
3. The installed `proTES middleware` automatically detects the encrypted data and decrypts them using Crypt4GH keys that are managed by `proTES`.
12
12
4. The researcher's task command is executed on the decrypted data.
13
13
5. The analysis results are stored at a dedicated storage accessible to the researcher
14
14
15
-
`Note` all computational steps are done in a secure containerized environment.
15
+
`Note:` all computational steps are done in a secure containerized environment.
16
16
17
17
This approach allows collaborative research where sensitive data can be processed in cloud environments without provisioning data access to the researcher but instead utilizing a combination of `Crypt4GH` and `proTES` for data encryption, decryption, and analysis.
18
18
Additionally, the researcher can repeat the analysis with adjusted parameters anytime without further action of the data holder.
19
19
20
20
21
21
## Overview
22
22
23
-
[Crypt4GH](https://crypt4gh.readthedocs.io/) is a standard for encrypting sensitive genomic data. This setup demonstrates:
23
+
[Crypt4GH](https://crypt4gh.readthedocs.io/) is a standard for encrypting sensitive data. This setup demonstrates:
24
24
25
25
- Generating cryptographic key pairs for data exchange between parties (data holder and researcher)
26
26
- Encrypting files using the data holder's private key and researcher's public key
@@ -29,7 +29,7 @@ Additionally, the researcher can repeat the analysis with adjusted parameters an
29
29
30
30
**Security Note:** Private keys should be stored in secure locations and used only for encryption/decryption. Consider using signed URLs for transferring private keys to the TES instance.
31
31
32
-
**Goal of this tutorial:** You'll have a setup which encrypts sensitive data, stores them in a secure storage, automatic detection of encrypted data triggers decryption followed by processing, ensuring that sensitive data remains protected.
32
+
**Goal of this tutorial:** You'll have a setup which encrypts sensitive data, stores them in a secure storage, automatic detection of encrypted data, decryption followed by processing, and ensuring that sensitive data remains protected.
33
33
34
34
## Setup
35
35
@@ -70,7 +70,7 @@ Run the following commands on both the Funnel server and worker VMs:
70
70
sudo apt update
71
71
sudo apt install -y make golang-go protobuf-compiler
72
72
73
-
# Install Go protocol buffer plugins
73
+
# Install Go protocol buffer plugins (use pinned versions for reproducibility)
74
74
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
75
75
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
76
76
export PATH=$PATH:$(go env GOPATH)/bin
@@ -176,7 +176,9 @@ Verify that both services are running by checking the logs or accessing the Funn
176
176
177
177
### Step 5: Configure ProTES
178
178
179
-
ProTES acts as a gateway and provides middleware for automatic Crypt4GH decryption. Follow the [proTES](https://github.com/elixir-cloud-aai/proTES) installation guide to deploy proTES on your third VM.
179
+
ProTES acts as a gateway and provides middleware for automatic Crypt4GH decryption. Follow the [proTES](https://github.com/elixir-cloud-aai/proTES) installation guide to deploy proTES on your third VM.
180
+
181
+
For detailed middleware installation, refer to the [protes-middleware-crypt4gh](https://github.com/elixir-cloud-aai/protes-middleware-crypt4gh).
180
182
181
183
Once installed, configure the Crypt4GH middleware by editing the `pro_tes/config.yaml` file:
- `CryptMiddleware`: Automatically detects and decrypts `.c4gh` files during task execution
192
-
193
-
For detailed middleware installation, refer to the [protes-middleware-crypt4gh](https://github.com/elixir-cloud-aai/protes-middleware-crypt4gh).
194
-
195
191
## Usage Examples
196
192
197
193
The following examples demonstrate the complete encryption/decryption workflow using three sequential tasks.
@@ -267,16 +263,16 @@ Create a file named `task1_keygen.json`:
267
263
268
264
- Generates two key pairs: one for the data holder and one for the researcher
269
265
- Keys are generated without encryption (`--nocrypt`) for demonstration purposes
270
-
- The researcher's public key is copied to the data holder's directory for use in encryption
266
+
- The researcher's public key is copied to the data holder's directory for use during encryption
271
267
- All keys are exported to local storage via TES outputs
272
268
273
269
### Task 2: Encrypt a File
274
270
275
-
This task retrieves a file, encrypts it using Crypt4GH, and stores both the encrypted file and metadata. Create a file named `task2_encrypt_file.json`:
271
+
This task retrieves a file, encrypts it using Crypt4GH keya, and stores both the encrypted file in a distinct location. Create a file named `task2_encrypt_file.json`:
276
272
277
273
```json
278
274
{
279
-
"name": "Encrypt file with crypt4gh",
275
+
"name": "Encrypt stat file with crypt4gh",
280
276
"description": "Retrieve a file, record its size, and encrypt it using data holder and researcher keys",
281
277
"inputs": [
282
278
{
@@ -331,11 +327,11 @@ This task retrieves a file, encrypts it using Crypt4GH, and stores both the encr
331
327
332
328
**Key Details:**
333
329
334
-
- Takes the data holder's private key and researcher's public key as inputs
335
330
- Downloads a sample file from a URL
336
-
- Records the original file size for verification
331
+
- Records the original file size
332
+
- Takes the data holder's private key and researcher's public key as inputs
337
333
- Encrypts the file using Crypt4GH, producing a `.c4gh` encrypted file
338
-
- Stores both the encrypted file and size metadata
334
+
- Stores the encrypted file
339
335
340
336
### Task 3: Decrypt and Process File
341
337
@@ -369,7 +365,7 @@ Create a file named `task3_decrypt_and_write_size.json`:
369
365
"outputs": [
370
366
{
371
367
"name": "decrypted_file",
372
-
"description": "Decrypted size text file",
368
+
"description": "MD5 checksum of automatically decrypted file",
Copy file name to clipboardExpand all lines: docs/guides/guide-admin/sensitive_data_analysis.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -107,7 +107,7 @@ In this tutorial, we assume a single LS-Login service for all the deployed tools
107
107
108
108
### LS-Login in MinIO
109
109
110
-
LS-Login can be activated in MinIO either by using the MinIO console using the OIDC configuration or by setting environmental variables, as described in the MinIO [OIDC Documentation](https://min.io/docs/minio/linux/operations/external-iam/configure-openid-external-identity-management.html). There are detailed instruction in the [ELIXIR-Cloud-AAI documentation](https://elixir-cloud-aai.github.io/guides/guide-admin/services_to_ls_aai/) for using MinIO with LS-Login.
110
+
LS-Login can be activated in MinIO either by using the MinIO console using the OIDC configuration or by setting environmental variables, as described in the MinIO [OIDC Documentation](https://min.io/docs/minio/linux/operations/external-iam/configure-openid-external-identity-management.html). There are detailed instructions in the [ELIXIR-Cloud-AAI documentation](https://elixir-cloud-aai.github.io/guides/guide-admin/services_to_ls_aai/) for using MinIO with LS-Login.
0 commit comments