Skip to content

Commit 5684e44

Browse files
author
Ubuntu
committed
AWS CloudFormation auomation for MONAI Deploy Express
1 parent 0683f50 commit 5684e44

19 files changed

Lines changed: 1375 additions & 0 deletions

‎deploy/monai-deploy-express/platforms/AWS/MDE.template.json‎

Lines changed: 840 additions & 0 deletions
Large diffs are not rendered by default.

‎deploy/monai-deploy-express/platforms/AWS/README.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# MONAI Deploy Express automated deployment on AWS
2+
3+
A CDK / CloudFormation project that allows the deployment of MONAI Deploy Express on EC2 in few a command lines or a few clicks ( CloudFormation UI). MONAI Deploy Express comes with a thorough installation guide, however the number of steps and the IT knowledge required to get through its deployment might be deterrant for less advanced IT users. This project enables any users to simply install MONAI Deploy Express (MDE) via a user inteface hosted in AWS CloudFormation console. Because the Infrastructure as Code (IaC) is built via CDK, more advanced users can also modifiy the python CDK code to alter the deployment to their own needs.
4+
5+
:warning: ** When sending DICOM data to MONAI Deploy Express over the internet: Make sure to use only deidentified DICOM data. The MONAI Informatic Gateway component (MIG) does not support encrypted DICOM DIMSE transfer and therefore the solution deployed by this automation code is not proper to upload PHI over the internet**
6+
7+
## Deployment:
8+
9+
### Infrastructure as Code
10+
After the CloudFormation stack has completed you are provided with an 100% installed and functional MDE server running on AWS EC2. The server is configured for the underlaying Docker services to restart automatically upon each restart of the EC2 instance. Below is the list of actions taken by the IaC :
11+
12+
<table>
13+
<th>Action</th><th>Name of resource created</th>
14+
<tr><td>Create new VPC</td><td>MDE/mde-network/Vpc</td></tr>
15+
<tr><td>Create a new Internet Gateway</td><td>MDE/mde-network/Vpc</td></tr>
16+
<tr><td>Create a new SSH key used to securely connect on the MDE server by the end-user</td><td>/ec2/keypair/key-**************</td></tr>
17+
<tr><td>Create an Elastic IP</td><td>MONAIDeployExpress-EIP</td></tr>
18+
<tr><td>Create a public network</td><td>MDE/mde-network/Vpc/PublicSubnet1<br>MDE/mde-network/Vpc/PublicSubnet2</td></tr>
19+
<tr><td>Create a public network</td><td>MDE/mde-network/Vpc/PublicSubnet1<br>MDE/mde-network/Vpc/PublicSubnet2</td></tr>
20+
<tr><td>Create a new security group</td><td>MONAIDeployExpress-SG</td></tr>
21+
<tr><td>create a new c4d.xlarge EC2 instance running ubuntu 22.04</td><td>MONAIDeployExpress</td></tr>
22+
<tr><td>Create the default OS monaiuser account with the provided password.</td><td>--</td></tr>
23+
<tr><td>Install MONAI deploy dependencies (python, docker , nvidia drivers and docker toolkit)</td><td>--</td></tr>
24+
<tr><td>Install MONAI Informatic Gateway (MIG) CLI</td><td>--</td></tr>
25+
<tr><td>Install MONAI Deploy Express binaries</td><td>--</td></tr>
26+
<tr><td>Set the docker services to auto-restart</td><td>--</td></tr>
27+
<tr><td>Associate the Elastic IP address with he EC2 instance</td><td>--</td></tr>
28+
</table>
29+
30+
![MDE in AWS diagram](img/diagram.png)
31+
32+
### What remains to be done manually
33+
34+
#### <strong>Update the security group:</strong>
35+
For security reason the security group associated to the EC2 instance does not allow the connection from any IP on any protocol. After the deployment is complete you need to update the security group associated with the MDE server to allow SSH and DICOM access. Follow the post installation section of this document to complete these steps.
36+
37+
38+
#### <strong> Install the MDE workflows:</strong>
39+
The MDE workflow manager service is left without any configuration after the installation, so that the user can install his own MDE workflows. you can refer to the [MONAI Dpeloy Express install guide, Samples Workflows section](https://github.com/Project-MONAI/monai-deploy/tree/main/deploy/monai-deploy-express#sample-workflows) to install the default workflows.
40+
41+
## Installation
42+
43+
### Prerequisites
44+
45+
An AWS account with the sufficient ressource availability for the creation of G4 type EC2 instance, 1 VPC and 1 elastic IP.
46+
* G4 type EC2 instance quota limit increase can be requested in the `service quotas` menu of the AWS console, category `Amazon Elastic Compute Cloud (Amazon EC2)`->`All G and VT Spot Instance Requests`.
47+
* VPC count limit increase can be requested in the `service quotas` menu of the AWS console, category `Amazon Virtual Private Cloud (Amazon VPC)`->`VPCs per Region`.
48+
* Elastic IP limit increase can be requested in the `service quotas` menu of the AWS console, category `Amazon Elastic Compute Cloud (Amazon EC2)`->`EC2-VPC Elastic IPs`
49+
50+
51+
There are 2 ways to deploy this solution. Users can do the [installation via Cloudformation](#installation-via-cloudformation) template or do the [installation via CDK CLI](#installation-via-cdk-cli).
52+
53+
### Installation via CloudFormation
54+
55+
1. Copy the file `MDE.template.json` from this repository root folder to your local filesystem.
56+
2. Login to your AWS account Web console and use the top search for the "CloudFormation" service.
57+
3. In the CloudFormation service menu, click the `[ Create Stack ]` button.
58+
4. In the stack creation menu -> "Specify Template" section , select "upload a template file" and click the `[ Choose file ]` button. Browser to the `MDE.template.json` file copied locally at step 1, then click the `[ Next ]` button.
59+
5. In the "Specific Stack details" menu, fill the below information, and click the `[ Next ]` button.<br>
60+
* <strong>Stack Name:</strong> Enter "`MDE`"
61+
* <strong>BootstrapVersion:</strong> Leave the default value.
62+
* <strong>SsmParameterValueawsservicec***Parameter:</strong> Leave the default value.
63+
* <strong>monaiuserpassword:</strong> Choose a password for the monaiuser linux user. This user will be part of the root group. The password needs to be 8 chars at minimum, and contain uppercase, lowercase and special characters.
64+
6. In the "Configure Stack Options" menu, you can add tags to all the resources to be created if you would like, then press the `[ Next ]` button.
65+
7. In the "Review" menu, click the "`I acknowledge that AWS CloudFormation might create IAM resources.`" check and click the `[ Submit ]` button.
66+
67+
At this point the CloudFormation stack deployment starts and takes about 20 minutes to clomplete. You can use the refresh button in the Stacks menu at in the left side to monitor when the stack completes.
68+
69+
![MDE in AWS diagram](img/stack_deployment.png)
70+
71+
72+
Once completed, refer to the [post installation tasks](#post-installation-tasks) to access to the MDE server.
73+
74+
### Installation via CDK CLI.
75+
76+
You will need the following software packages installed locally to deploy this solution via CDK.
77+
78+
Python3/pip: The deployment automation code is written in Python.
79+
AWS CDK: Please refer to [CDK documentation](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html) to install the framework and bootstrap your AWS environment.
80+
81+
1. Bootstrap the AWS account for CDK. Refer to the CDK documentation [Bootstrapping](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#:~:text=to%20do%20that.-,Bootstrapping,-Deploying%20stacks%20with) section to know more about bootstrapping your AWS account for CDK.
82+
83+
2. Clone this repository. You can either download and unzip the package from the repository webite, or use the git clone command.
84+
85+
3. **In the cloned directory**, create a virtualenv for Python. Virtual environment is needed to deploy required versions of packages:
86+
87+
```
88+
$ python3 -m venv .venv
89+
```
90+
91+
After the init process completes and the virtualenv is created, you can use the following
92+
step to activate your virtualenv.
93+
94+
```
95+
$ source .venv/bin/activate
96+
```
97+
98+
If you are a Windows platform, you would activate the virtualenv like this:
99+
100+
```
101+
% .venv\Scripts\activate.bat
102+
```
103+
104+
Once the virtualenv is activated, you can install the required dependencies.
105+
106+
```
107+
$ pip install -r requirements.txt
108+
```
109+
110+
4. Deploy the stack via CDK:
111+
112+
```
113+
$ cdk deploy --parameters monaiuserpassword=[your monai user password]
114+
```
115+
116+
to the question `Do you wish to deploy these changes (y/n)?` press Y and Enter.
117+
118+
119+
The deployment takes approximately 20 minutes. Once completed, refer to the [post installation tasks](#post-installation-tasks) to access to the MDE server.
120+
121+
## Post deployment tasks
122+
123+
After the installation has completed, 2 steps remain to be done. 1st the security group associated with the MDE server needs to be updated with the IP address of the SSH and DICOM client(s). 2nd, Your SSH client has to be configured to use a specific SSH key to connect to the server.
124+
125+
### Configuring the Security Group for SSH and DICOM access
126+
127+
Follow the steps below to add inbound rules in the MDE security group:
128+
129+
1. Logon the AWS Web Console with the same account as the one used for the deployment of the server, and within the same region.
130+
2. In the AWS Web Console select the `EC2` service.
131+
3. In the EC2 service left menu bar, select `Security Groups`.
132+
4. In the Security Group list, select the group named `MONAIDeployExpress-SG` and click `[ Actions -> Edit inboudn rules ]`.
133+
5. In the `Edit inbound rules` menu, Update the `Source` field for each rule and select `My IP`. These rules will allow any machine going on the public network via the listed IP address to connect to the SSH port (This is not sufficient to logon the server though, the server remains only accessible to owners of the SSH key).
134+
![MDE in AWS diagram](img/security_group.png)
135+
<strong>Note:</strong> Usage of web proxies can bias the detection of the user public IP in the security group rules editing menu. If you are using a web proxy to access to the internet, ensure that the IP automatically added in the source field is the one your computer uses to communicate on the internet for non-HTTP protocols.
136+
137+
### Connecting on the server via SSH
138+
A SSH key is required to connect to the MDE server. in the EC2 console->Key pair menu, you will find that a key pair called 'MonaiDeployExpress' has been created during the deployment. However key pairs can only be downloaded 1 time upon their creation, and it is not possible to download this key pair anymore from this menu. Instead the Cloudformation/CDK deployment exports the key pair and saves it in `parameter Store`, a component of the `AWS Systems Manager` service. Follow the below steps to download the SSH key:
139+
140+
1. Logged in the AWS console with the same account and in the same region as used for the deployment, in the search field at the top of the console, search for `Parameter Store`.
141+
2. in the `Parameter Store` menu, select the parameter called `/ec2/keypair/key-*********` and click the `[ View details ]` button.
142+
3. In the details menu, click on the `show` link at the bottom left of the details pane. the Certificate key displays.
143+
4. Copy the certificate key, from the tag `-----BEGIN RSA PRIVATE KEY-----` to the tag `-----END RSA PRIVATE KEY-----`, tags included, and paste the the copied content in a new file named `mde-server.pem` on your local filesystem.
144+
145+
This pem file will be used to connect to the MDE server via SSH. Note that authentication by SSH key is the only authentication enabled by default on the MDE server.
146+
147+
5. In the AWS Web Console search field ( top menu bar ), search for the `EC2 service`. In the EC2 service left menu bar, select `Elastic IPs`. In the Elastic IPs list, note down the IP address for the `Elastic IP` named `MONAIDeployExpress-EIP`. This Elastic IP is attached to the MDE server and we will be used to connect to the server via SSH.
148+
149+
6. The methods to connect on the MDE server may change from one SSH client to another. Follow the AWS documentation about how to [Connect to your Linux instance from Windows using PuTTY](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html) to learn how to convert the `mde-server.pem` key file to a .ppk file and configure your Putty session with. Configure the EC2 instance IP address with the Elastic IP noted in the step above.
150+
151+
7. When ready to connect to the MDE server, use the Elastic IP as the hostname , and `ubuntu` as the user. Once logged in the MDE server, type the command `su - monaiuser` to switch to the monaiuser user.

‎deploy/monai-deploy-express/platforms/AWS/app.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/usr/bin/env python3
2+
import os
3+
4+
import aws_cdk as cdk
5+
6+
from mde_cloudformation.mde_cloudformation_stack import MdeCloudformationStack
7+
8+
9+
app = cdk.App()
10+
MdeCloudformationStack(app, "MDE",
11+
# If you don't specify 'env', this stack will be environment-agnostic.
12+
# Account/Region-dependent features and context lookups will not work,
13+
# but a single synthesized template can be deployed anywhere.
14+
15+
# Uncomment the next line to specialize this stack for the AWS Account
16+
# and Region that are implied by the current CLI configuration.
17+
18+
#env=cdk.Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'), region=os.getenv('CDK_DEFAULT_REGION')),
19+
20+
# Uncomment the next line if you know exactly what Account and Region you
21+
# want to deploy the stack to. */
22+
23+
#env=cdk.Environment(account='123456789012', region='us-east-1'),
24+
25+
# For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html
26+
)
27+
28+
app.synth()

‎deploy/monai-deploy-express/platforms/AWS/cdk.json‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
{
2+
"app": "python3 app.py",
3+
"watch": {
4+
"include": [
5+
"**"
6+
],
7+
"exclude": [
8+
"README.md",
9+
"cdk*.json",
10+
"requirements*.txt",
11+
"source.bat",
12+
"**/__init__.py",
13+
"python/__pycache__",
14+
"tests"
15+
]
16+
},
17+
"context": {
18+
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
19+
"@aws-cdk/core:checkSecretUsage": true,
20+
"@aws-cdk/core:target-partitions": [
21+
"aws",
22+
"aws-cn"
23+
],
24+
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
25+
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
26+
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
27+
"@aws-cdk/aws-iam:minimizePolicies": true,
28+
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
29+
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
30+
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
31+
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
32+
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
33+
"@aws-cdk/core:enablePartitionLiterals": true,
34+
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
35+
"@aws-cdk/aws-iam:standardizedServicePrincipals": true,
36+
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
37+
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
38+
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
39+
"@aws-cdk/aws-route53-patters:useCertificate": true,
40+
"@aws-cdk/customresources:installLatestAwsSdkDefault": false
41+
}
42+
}

‎deploy/monai-deploy-express/platforms/AWS/img/diagram.png‎

52.6 KB
Loading

‎deploy/monai-deploy-express/platforms/AWS/img/security_group.png‎

271 KB
Loading

‎deploy/monai-deploy-express/platforms/AWS/img/stack_deployment.png‎

199 KB
Loading

‎deploy/monai-deploy-express/platforms/AWS/mde_cloudformation/__init__.py‎

Whitespace-only changes.

‎deploy/monai-deploy-express/platforms/AWS/mde_cloudformation/cloudformationInit.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
from aws_cdk import (
2+
# Duration,
3+
CfnParameter,
4+
Duration,
5+
Stack,
6+
# aws_sqs as sqs,
7+
)
8+
import aws_cdk as cdk
9+
from aws_cdk import aws_ec2 as ec2
10+
from constructs import Construct
11+
from .network import network
12+
from .instance import instance
13+
from .userdata import userdata
14+
15+
class cloudformationInit(Construct):
16+
def __init__(self, scope: Construct, id: str, **kwargs) -> None:
17+
super().__init__(scope, id, **kwargs)
18+
self._cfnInit = ec2.CloudFormationInit.from_elements(
19+
ec2.InitCommand.shell_command("touch /tmp/Init_done")
20+
)
21+
22+
self._init_options=ec2.ApplyCloudFormationInitOptions(
23+
timeout=Duration.minutes(60)
24+
)
25+
26+
def getCfnInit(self):
27+
return self._cfnInit
28+
29+
def getCfnInitOptions(self):
30+
return self._init_options

‎deploy/monai-deploy-express/platforms/AWS/mde_cloudformation/instance.py‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
"""
2+
Generates the instance and automate the deployment.
3+
SPDX-License-Identifier: Apache 2.0
4+
"""
5+
6+
7+
from constructs import Construct
8+
import aws_cdk as cdk
9+
from aws_cdk import aws_ec2 as ec2
10+
import aws_cdk.aws_logs as logs
11+
from aws_cdk import (aws_iam as iam )
12+
13+
class instance(Construct):
14+
def __init__(self , scope: Construct , id: str , vpc: ec2.Vpc , security_group: ec2.SecurityGroup , user_data: ec2.UserData , cfn_init: ec2.CloudFormationInit , cfn_init_options: ec2.ApplyCloudFormationInitOptions , **kwargs) -> None:
15+
super().__init__(scope, id, **kwargs)
16+
machineImage = ec2.MachineImage.from_ssm_parameter(
17+
parameter_name = '/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id',
18+
os = ec2.OperatingSystemType.LINUX
19+
)
20+
21+
cfn_key_pair = ec2.CfnKeyPair( self,
22+
"MyCfnKeyPair",
23+
key_name="MONAIDeployExpress-"+cdk.Fn.ref("AWS::StackId"),
24+
# the properties below are optional
25+
key_type="rsa",
26+
tags=[cdk.CfnTag(
27+
key="Name",
28+
value="MONAIDeployExpress-KeyPair"
29+
)]
30+
)
31+
instance_role = iam.Role( self,
32+
"Role",
33+
assumed_by=iam.ServicePrincipal("ec2.amazonaws.com"),
34+
description="MONAI Deploy Express EC2 instance role."
35+
)
36+
37+
instance_role.add_managed_policy(iam.ManagedPolicy.from_aws_managed_policy_name("AmazonSSMManagedInstanceCore"))
38+
self._instance = ec2.Instance(self,
39+
'MONAIDeployExpress-instance',
40+
instance_type=ec2.InstanceType.of(ec2.InstanceClass.G4DN , ec2.InstanceSize.XLARGE),
41+
instance_name='MONAIDeployExpress',
42+
machine_image=machineImage,
43+
vpc=vpc,
44+
vpc_subnets=ec2.SubnetSelection(subnet_type=ec2.SubnetType.PUBLIC),
45+
security_group=security_group,
46+
user_data= user_data,
47+
key_name=cfn_key_pair.key_name,
48+
block_devices=[ec2.BlockDevice(
49+
device_name="/dev/sda1",
50+
volume=ec2.BlockDeviceVolume.ebs(volume_size=200, encrypted=True , volume_type=ec2.EbsDeviceVolumeType.GP3)
51+
),],
52+
role=instance_role,
53+
init=cfn_init,
54+
init_options=cfn_init_options
55+
56+
)
57+
58+
def getInstance(self) -> ec2.Instance:
59+
return self._instance

0 commit comments

Comments
 (0)