Skip to content

Commit a770984

Browse files
JoeBatt1989JoeBatt1989
authored
Merge pull request #81 from Project-MONAI/e2e-test-results
Added additonal E2E test data and evidence
2 parents aedeb02 + 84c0073 commit a770984

3 files changed

Lines changed: 43 additions & 3 deletions

File tree

‎e2e-testing/README.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ This document describes the test coverage, test data, status of their execution
88

99
## Test Scenarios
1010
The below matrix was created to replicate and test real life scenarios.
11+
1112
| ID | Category | Test Description | Summary | Dicom Study | Clinical Workflow Ref | Argo Workflow | Status |
1213
| --------------- | ---------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | ----------------------------------------------- | -------------------------------------------- | -------------------------------------- |
1314
| MONAI\_E2E\_001 | workflow\_management | Create and trigger workflow | Create and trigger a workflow by sending DICOM data to MIG. | Study\_1 | argo\_workflow.json | argo\_workflow\_1 | Passed |
@@ -38,7 +39,7 @@ The below matrix was created to replicate and test real life scenarios.
3839
| MONAI\_E2E\_026 | output\_artefacts | Non Mandatory output artefact is missing | Workflow has a non mandatory output artefact specified which is not generated. Workflow completes without issue. | Study\_1 | output\_artefacts\_non\_mandatory\_missing.json | argo\_workflow\_1 | Passed |
3940
| MONAI\_E2E\_027 | export | Files within folder are exported | Export Task will export a folder back to PACS | Study\_3 | export\_folder.json | argo\_workflow\_1 | Passed |
4041
| MONAI\_E2E\_028 | task\_failed | Model execution throws exception | Workfow references argo\_workflow\_exception which will thrown a known exception. This marks the workflow as failed. | Study\_1 | model\_exception\_workflow.json | argo\_workflow\_exception | Passed |
41-
| MONAI\_E2E\_029 | task\_failed | Model exceeds timeout and task is marked as failed | Workfow references an argo task which runs longer than is specified in the clincial workflow timeout. | Study\_1 | timeout\_exception\_workflow.json | argo\_workflow\_sleep | Not Yet Run |
42+
| MONAI\_E2E\_029 | task\_failed | Model exceeds timeout and task is marked as Cancelled | Workfow references an argo task which runs longer than is specified in the clincial workflow timeout. | Study\_1 | timeout\_exception\_workflow.json | argo\_workflow\_sleep | Fail |
4243
| MONAI\_E2E\_030 | parallel\_execution | Parellel execution of seperate Argo tasks | Workflow has a router task that triggers two argo tasks in parallel. This is to replicate 2 models within the same workflow executing on the same input | Study\_1 | router\_workflow\_2.json | argo-workflow-sleep-a | Passed |
4344
| MONAI\_E2E\_031 | parallel\_execution | Parellel execution of a workflow by sending multiple different studies | Trigger workflow multiple times by sending 5 studies in quick succession. Execution should not be linear and containers are created on demand. | Study\_1 | argo\_workflow\_long\_running.json | argo-workflow-sleep-a | Passed |
4445
| MONAI\_E2E\_032 | resiliency | MinIO is not online for initial input via MIG | MinIO is offline/disconnected. argo\_workflow\_1 is executed and will retry a configurable number of times. MinIO is reconnected in this time and the workflow completes. | Study\_1 | argo\_export\_workflow.json | argo\_workflow\_1 | Not Yet Run |
@@ -48,8 +49,8 @@ The below matrix was created to replicate and test real life scenarios.
4849
| MONAI\_E2E\_036 | resiliency | Rabbit is not online. Model execution fails and payload is lost | Rabbit is offline/disconnected. A payload is sent. Payload is not sent after retries fail. | Study\_1 | argo\_export\_workflow.json | argo\_workflow\_1 | Passed |
4950
| MONAI\_E2E\_037 | resiliency | MIG is not online and attempt to send data from PACS | PACS cannot send data to MIG and association fails | Study\_3 | argo\_export\_workflow.json | argo\_workflow\_1 | Not Yet Run |
5051
| MONAI\_E2E\_038 | resiliency | MWM is not online and send data from PACS | WorkflowRequestEvent sits on the queue until the Workflow Manager comes back online. | Study\_1 | argo\_export\_workflow.json | argo\_workflow\_1 | Not Yet Run |
51-
| MONAI\_E2E\_039 | resiliency | PACS is down and try to export results from MIG | | Study\_1 | argo\_export\_workflow.json | argo\_workflow\_2 | Not Yet Run |
52-
| MONAI\_E2E\_040 | task\_failed | Clinical workflow references a Argo Workflow Template that does not exist. | | any | | n/a | Not Yet Run |
52+
| MONAI\_E2E\_039 | resiliency | PACS is down and try to export results from MIG | PACS is down which causes the export request to fail | Study\_1 | argo\_export\_workflow.json | argo\_workflow\_2 | Not Yet Run |
53+
| MONAI\_E2E\_040 | task\_failed | Clinical workflow references a Argo Workflow Template that does not exist. | Argo workflow does not exist which causes the Task and Workflow to fail | any | no\_argo\_exists.json | n/a | Not Yet Run |
5354

5455
### Creating additional Test Coverage
5556
If additional tests are required please ensure:

‎e2e-testing/test-evidence/MONAI_E2E_040_Passed.zip‎

1.92 MB
Binary file not shown.

‎e2e-testing/test-scenarios/Clinical_Workflows/no_argo_exists.json‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
{
2+
"name": "argo_workflow",
3+
"version": "1.0.0",
4+
"description": "Workflow with Argo task",
5+
"informatics_gateway": {
6+
"ae_title": "MONAI",
7+
"export_destinations": [
8+
"ORTHANC"
9+
]
10+
},
11+
"tasks": [
12+
{
13+
"id": "mean-pixel-calc",
14+
"description": "mean-pixel-calc operator and reporter",
15+
"type": "argo",
16+
"args": {
17+
"namespace": "argo",
18+
"workflow_template_name": "argo-workflow-none",
19+
"server_url": "https://argo-server.argo:2746",
20+
"allow_insecure": "true"
21+
},
22+
"ref": "",
23+
"artifacts": {
24+
"input": [
25+
{
26+
"name": "input-dicom",
27+
"value": "{{ context.input.dicom }}",
28+
"mandatory": true
29+
}
30+
],
31+
"output": [
32+
{
33+
"name": "report-dicom"
34+
}
35+
]
36+
}
37+
}
38+
]
39+
}

0 commit comments

Comments
 (0)