Skip to content

Commit 201efcd

Browse files
JoeBatt1989JoeBatt1989
authored
Merge pull request #83 from Project-MONAI/e2e-test-results
Added further test evidence and data
2 parents 77386a4 + 13ec226 commit 201efcd

4 files changed

Lines changed: 140 additions & 11 deletions

File tree

‎e2e-testing/README.md‎

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@ The below matrix was created to replicate and test real life scenarios.
1919
| MONAI\_E2E\_006 | argo | Execution of a workflow (US) | Workflow is triggered and report is exported to PACS. | Study\_4 | argo\_export\_workflow.json | argo\_workflow\_1 | Passed |
2020
| MONAI\_E2E\_007 | argo | Execution of a workflow (Xray) | Workflow is triggered and report is exported to PACS. | Study\_5 | argo\_export\_workflow.json | argo\_workflow\_1 | Passed |
2121
| MONAI\_E2E\_008 | argo | Execution of a workflow for a multi-series study | Multi-series study is sent through as one association which results in a single report back to PACS. | Study\_6 | argo\_export\_workflow.json | argo\_workflow\_1 | Passed |
22-
| MONAI\_E2E\_009 | argo\_gpu | Execution of a workflow where the Argo task requires GPU | Workflow is triggered with a model which required GPU access. | Study\_1 | liver-seg.json | argo\_workflow\_5 | Not Yet Run |
23-
| MONAI\_E2E\_010 | argo\_map | Execution of a workflow where the Argo task uses a MAP | Workflow is triggered with a model which is created as a MAP. | Study\_1 | argo\_map\_workflow.json | argo\_workflow\_5 | Not Yet Run |
22+
| MONAI\_E2E\_009 | argo\_gpu | Execution of a workflow where the Argo task requires GPU | Workflow is triggered with a model which required GPU access. | Study\_7 | liver-seg.json | argo\_workflow\_5 | Passed |
23+
| MONAI\_E2E\_010 | argo\_map | Execution of a workflow where the Argo task uses a MAP | Workflow is triggered with a model which is created as a MAP. | Study\_7 | liver-seg.json | argo\_workflow\_5 | Passed |
2424
| MONAI\_E2E\_011 | workflow\_execution | No execution when there isnt a match on a workflow AETitle | No workflows are triggered when a workflow request does not match on AETitle. | Study\_3 | alt\_aetitle.json | argo\_workflow\_1 | Passed |
2525
| MONAI\_E2E\_012 | router | Router task filters out execution based on DICOM modality | Router task filters out payloads which should not be executed on. | Study\_1 | router\_workflow\_1.json | argo\_workflow\_1 | Passed |
2626
| MONAI\_E2E\_013 | router | Router task forwards to correct Task Destination | Router task directs study down the MR branch of the workflow | Study\_3 | router\_workflow\_1.json | argo\_workflow\_1 | Passed |
@@ -65,15 +65,15 @@ TBC
6565
The [test-data](/e2e-testing/test-data) folder contains all the tests data to support the execution of the End to End tests. This includes Argo workflows Clinical Workflows and DICOM studies used.
6666

6767
#### Argo Workflows
68-
| Name | Description |
69-
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
70-
| argo\_workflow\_1 | Both operators of the mean pixel calc (test\_operator & test\_reporter) can be executed as a single Argo Task |
71-
| argo\_workflow\_2 | test\_operator of the mean pixel calc can be executed in isolation as a single Argo Task |
72-
| argo\_workflow\_3 | reporter\_operator of the mean pixel calc can be executed in isolation as a single Argo Task |
73-
| argo\_workflow\_exception | Both operators of the mean pixel calc (test\_operator & test\_reporter) can be executed as a single Argo Task. Image used for test\_operator will throw an exception |
74-
| argo\_workflow\_4 | Both operators of the mean pixel calc (test\_operator & test\_reporter) can be executed as a single Argo Task. Hard coded [thread.sl](http://thread.sl/)eep(5 mins) |
75-
| argo\_workflow\_5 | Argo Task with GPU enabled |
76-
| argo\_workflow\_6 | Argo Task with MAP
68+
| Name | Description |
69+
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
70+
| argo\_workflow\_1 | Both operators of the mean pixel calc (test\_operator & test\_reporter) can be executed as a single Argo Task |
71+
| argo\_workflow\_1\_new | Both operators of the mean pixel calc (test\_operator & test\_reporter) can be executed as a single Argo Task |
72+
| argo\_workflow\_2 | test\_operator of the mean pixel calc can be executed in isolation as a single Argo Task |
73+
| argo\_workflow\_3 | reporter\_operator of the mean pixel calc can be executed in isolation as a single Argo Task |
74+
| argo\_workflow\_exception | Image used for test\_operator will throw an exception |
75+
| argo\_workflow\_sleep<br> | Both operators of the mean pixel calc (test\_operator & test\_reporter) can be executed as a single Argo Task. Hard coded [thread.sl](http://thread.sl/)eep(5 mins) |
76+
| argo\_workflow\_5 | Argo Task executed MAP on GPU
7777

7878
#### DICOM Studies
7979
| Name | Description |
@@ -84,6 +84,7 @@ The [test-data](/e2e-testing/test-data) folder contains all the tests data to su
8484
| Study\_4 | US Study |
8585
| Study\_5 | Xray Study |
8686
| Study\_6 | Multi-series |
87+
| Study\_7 | Liver CT |
8788

8889
### Deployment
8990
TBC

‎e2e-testing/test-evidence/MONAI_E2E_009_010_Passed.zip‎

6.26 MB
Binary file not shown.

‎e2e-testing/test-scenarios/Argo_Workflows/argo_workflow_5.yaml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
metadata:
2+
name: liver-seg-argo-template
3+
generateName: liver-seg-argo-template-
4+
namespace: argo
5+
spec:
6+
templates:
7+
- name: demo-pipeline
8+
inputs: {}
9+
outputs: {}
10+
metadata: {}
11+
steps:
12+
- - name: liver-segmentation
13+
template: liver-seg
14+
arguments:
15+
artifacts:
16+
- name: input-dicom
17+
s3:
18+
accessKeySecret:
19+
name: my-minio-cred
20+
key: accesskey
21+
secretKeySecret:
22+
name: my-minio-cred
23+
key: secretkey
24+
key: argo/inputs/artifacts/input-dicom
25+
- name: liver-seg
26+
inputs:
27+
artifacts:
28+
- name: input-dicom
29+
path: /var/monai/input
30+
s3:
31+
accessKeySecret:
32+
name: my-minio-cred
33+
key: accesskey
34+
secretKeySecret:
35+
name: my-minio-cred
36+
key: secretkey
37+
key: argo/inputs/artifacts/input-dicom
38+
outputs:
39+
artifacts:
40+
- name: output
41+
path: /var/monai/output
42+
s3:
43+
bucket: bucket1
44+
accessKeySecret:
45+
name: my-minio-cred
46+
key: accesskey
47+
secretKeySecret:
48+
name: my-minio-cred
49+
key: secretkey
50+
key: >-
51+
00000000-1000-0000-0000-000000000000/report-pdf/liver-seg/report.pdf
52+
metadata: {}
53+
container:
54+
name: ''
55+
image: ghcr.io/mmelqin/monai_ai_livertumor_seg_app:1.0
56+
command:
57+
- /bin/bash
58+
- '-c'
59+
args:
60+
- python3 -u /opt/monai/app/app.py -m /opt/monai/models/model/model.ts
61+
resources:
62+
limits:
63+
nvidia.com/gpu: '1'
64+
entrypoint: demo-pipeline
65+
arguments: {}

‎e2e-testing/test-scenarios/Clinical_Workflows/liver_seg.json‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
{
2+
"name": "liver-seg",
3+
"version": "1.0.0",
4+
"description": "MONAI AI Liver Seg",
5+
"informatics_gateway": {
6+
"ae_title": "MONAI",
7+
"export_destinations": [
8+
"ORTHANC"
9+
]
10+
},
11+
"tasks": [
12+
{
13+
"id": "liver-seg",
14+
"description": "liver segmentation",
15+
"type": "argo",
16+
"args": {
17+
"namespace": "argo",
18+
"workflow_template_name": "liver-seg-argo-template",
19+
"server_url": "https://argo-server.argo:2746",
20+
"allow_insecure": "true"
21+
},
22+
"task_destinations": [
23+
{
24+
"name": "export-report"
25+
}
26+
],
27+
"artifacts": {
28+
"input": [
29+
{
30+
"name": "input-dicom",
31+
"value": "{{ context.input.dicom }}",
32+
"mandatory": true
33+
}
34+
],
35+
"output": [
36+
{
37+
"name": "output"
38+
}
39+
]
40+
},
41+
"input_parameters": null
42+
},
43+
{
44+
"id": "export-report",
45+
"description": "export report",
46+
"type": "export",
47+
"export_destinations": [
48+
{
49+
"name": "ORTHANC"
50+
}
51+
],
52+
"artifacts": {
53+
"input": [
54+
{
55+
"name": "output",
56+
"value": "{{ context.executions.liver-seg.artifacts.output }}",
57+
"mandatory": true
58+
}
59+
]
60+
}
61+
}
62+
]
63+
}

0 commit comments

Comments
 (0)