Skip to content

Commit e0b9342

Browse files
authored
Merge pull request aws-samples#1476 from sushanth-k/sushanth-k-feature-lambda-elasticache-integrationpattern
2 parents edfc7b6 + fc65897 commit e0b9342

13 files changed

Lines changed: 406 additions & 0 deletions
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
*.js
2+
!jest.config.js
3+
*.d.ts
4+
node_modules
5+
!lambda/*.js
6+
7+
# CDK asset staging directory
8+
.cdk.staging
9+
cdk.out
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
!*.d.ts
2+
3+
# CDK asset staging directory
4+
.cdk.staging
5+
cdk.out
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# AWS Lamda to Amazon ElastiCache integration pattern with CDK
2+
3+
Customers may want to connect to ElastiCache from their core lambda function(s) for use cases that involve fast read-write operations and improve latency of applications.In application like Leader board, Queue/wait-list , API rate limiting & Quota management, etc. that are built on Serverless platform there would a lambda that is computing would integrate with a either Redis or Memcache on Amazon ElastiCache. This documentation provides a quick start guide to launch Redis cluster in Amazon ElastiCache in a defined VPC and creates a lambda function that read writes from the Cache. Customer can further modify the code in lambda function as per their requirements. Customers can also configure Memcached by modifying the config and change their lambda appropriately.
4+
5+
Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/?aws-products-pricing.sort-by=item.additionalFields.productNameLowercase&aws-products-pricing.sort-order=asc&awsf.Free%20Tier%20Type=*all&awsf.tech-category=*all) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
## Requirements
10+
11+
1. Create an [AWS account](https://portal.aws.amazon.com/billing/signup?redirect_url=https%3A%2F%2Faws.amazon.com%2Fregistration-confirmation#/start/email) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
12+
2. [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed and configured
13+
3. Git Installed
14+
4. [Node and NPM](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) installed
15+
5. [AWS Cloud Development Kit (AWS CDK)](https://docs.aws.amazon.com/cdk/v2/guide/cli.html) installed
16+
17+
## Deployment Instructions
18+
1. Open choice of terminal/command line. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
19+
`git clone https://github.com/aws-samples/serverless-patterns`
20+
2. Change directory to the pattern directory:
21+
`cd lambda-elasticache-integrationpattern-cdk`
22+
3. Run below command to install required dependancies:
23+
`npm install`
24+
4. Change the VPC details to desired VPC, Redis cluster would be created in your defined VPC. Line 22-25 on the file lib/lambda-elasticache-integrationpattern-cdk-stack.ts. If you are planing to deploy cache in private subnet or a particular list of subnets then modify the line 48 appropriately.
25+
5. From the command line, run: `cdk deploy`.
26+
6. CDK would display all the changes it will make to your AWS environment, accept the changes to deploy.
27+
7. It will take 15 mins to deploy rsources and once it is complete the confirmation would be displayed on command line/terminal.
28+
8. You can go to AWS Console/Search for Cloud Formation and look for stack name LambdaElasticacheIntegrationpatternCdkStack and monitor the events during deployment.
29+
30+
31+
## Testing
32+
1. Navigate to AWS Console and search for Cloud Formation
33+
2. Click on Stacks on left and find the recently deployed pattern stack , name LambdaElasticacheIntegrationpatternCdkStack.
34+
3. Click on the stack name and click on the resources tab
35+
4. Find the lambda function , look for Type = AWS::lambda::Function and open the lambda function by clicking on the URL. The function name would be something like LambdaElasticacheIntegrat-ElasticacheRedisAccessXXXX
36+
5. Follow the steps on lamdba console testing guide [here](https://docs.aws.amazon.com/lambda/latest/dg/testing-functions.html). For this lambda the test event cam be empty or you can leave to default hello-world.
37+
`{"key1": "value1", "key2": "value2", "key3": "value3"}`
38+
39+
## Cleanup
40+
To delete the stack, run:
41+
`cdk destroy --all`
42+
43+
## Useful commands
44+
45+
* `npm run build` compile typescript to js
46+
* `npm run watch` watch for changes and compile
47+
* `npm run test` perform the jest unit tests
48+
* `cdk deploy` deploy this stack to your default AWS account/region
49+
* `cdk diff` compare deployed stack with current state
50+
* `cdk synth` emits the synthesized CloudFormation template
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
#!/usr/bin/env node
2+
import * as cdk from 'aws-cdk-lib';
3+
import { LambdaElasticacheIntegrationpatternCdkStack } from '../lib/lambda-elasticache-integrationpattern-cdk-stack';
4+
5+
const app = new cdk.App();
6+
new LambdaElasticacheIntegrationpatternCdkStack(app, 'LambdaElasticacheIntegrationpatternCdkStack', {
7+
/* If you don't specify 'env', this stack will be environment-agnostic.
8+
* Account/Region-dependent features and context lookups will not work,
9+
* but a single synthesized template can be deployed anywhere. */
10+
11+
/* Currently we are using account and region from CLI configuration. Comment if
12+
you are using other options*/
13+
env: {
14+
account: process.env.CDK_DEFAULT_ACCOUNT,
15+
region: process.env.CDK_DEFAULT_REGION
16+
}
17+
// Uncomment the next line if you know exactly what Account and Region you want to deploy the stack to.
18+
// env: { account: '123456789012', region: 'us-east-1' }
19+
// For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html
20+
21+
});
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
{
2+
"app": "npx ts-node --prefer-ts-exts bin/lambda-elasticache-integrationpattern-cdk.ts",
3+
"watch": {
4+
"include": [
5+
"**"
6+
],
7+
"exclude": [
8+
"README.md",
9+
"cdk*.json",
10+
"**/*.d.ts",
11+
"**/*.js",
12+
"tsconfig.json",
13+
"package*.json",
14+
"yarn.lock",
15+
"node_modules",
16+
"test"
17+
]
18+
},
19+
"context": {
20+
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
21+
"@aws-cdk/core:checkSecretUsage": true,
22+
"@aws-cdk/core:target-partitions": [
23+
"aws",
24+
"aws-cn"
25+
],
26+
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
27+
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
28+
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
29+
"@aws-cdk/aws-iam:minimizePolicies": true,
30+
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
31+
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
32+
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
33+
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
34+
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
35+
"@aws-cdk/core:enablePartitionLiterals": true,
36+
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
37+
"@aws-cdk/aws-iam:standardizedServicePrincipals": true,
38+
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
39+
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
40+
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
41+
"@aws-cdk/aws-route53-patters:useCertificate": true,
42+
"@aws-cdk/customresources:installLatestAwsSdkDefault": false,
43+
"@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
44+
"@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
45+
"@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
46+
"@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
47+
"@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
48+
"@aws-cdk/aws-redshift:columnId": true
49+
}
50+
}
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
{
2+
"title": "AWS Lamda to Amazon ElastiCache integration pattern",
3+
"description": "Create a Redis Cluster in Amazon ElastiCache and read write with a Lambda function.",
4+
"language": "TypeScript",
5+
"level": "200",
6+
"framework": "CDK",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"Users may want to connect to ElastiCache from their core Lambda function for use cases that involve fast read-write operations and improve latency of applications.",
11+
"In an application like a leader board, queue/wait-list , API rate limiting & quota management, etc. that are built on Serverless platform , the core computing Lambda function would integrate with a either Redis or Memcache on Amazon ElastiCache.",
12+
"This pattern provides a quick start guide to launch a Redis cluster in Amazon ElastiCache in a defined VPC and creates a Lambda function that read writes from that cache.",
13+
"Users can further modify the code in the Lambda function as per their requirements. You can configure Memcached by modifying the cdk config and change Lambda code appropriately."
14+
]
15+
},
16+
"gitHub": {
17+
"template": {
18+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-elasticache-integrationpattern-cdk",
19+
"templateURL": "serverless-patterns/lambda-elasticache-integrationpattern-cdk",
20+
"projectFolder": "lambda-elasticache-integrationpattern-cdk",
21+
"templateFile": "lambda-elasticache-integrationpattern-cdk/lib/lambda-elasticache-integrationpattern-cdk-stack.ts"
22+
}
23+
},
24+
"resources": {
25+
"bullets": [
26+
{
27+
"text": "What is AWS Lambda",
28+
"link": "https://aws.amazon.com/lambda/"
29+
},
30+
{
31+
"text": "What is Amazon ElastiCache",
32+
"link": "https://aws.amazon.com/elasticache/"
33+
},
34+
{
35+
"text": "How to test Lambda with AWS Console",
36+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/testing-functions.html"
37+
},
38+
{
39+
"text": "Further handson with ElastiCache",
40+
"link": "https://aws.amazon.com/getting-started/hands-on/boosting-mysql-database-performance-with-amazon-elasticache-for-redis/"
41+
}
42+
]
43+
},
44+
"deploy": {
45+
"text": [
46+
"cdk deploy"
47+
]
48+
},
49+
"testing": {
50+
"text": [
51+
"See the README.md file of GitHub repo for detailed testing instructions."
52+
]
53+
},
54+
"cleanup": {
55+
"text": [
56+
"Delete the stack: <code>cdk delete</code>."
57+
]
58+
},
59+
"authors": [
60+
{
61+
"name": "Sushanth Kothapally",
62+
"image": "https://avatars.githubusercontent.com/u/10820054?v=4",
63+
"bio": "Sushanth is a Solutions Architect at AWS. He is passionate about solving technical problems to meet business objectives.",
64+
"linkedin": "sushanthkothapally"
65+
}
66+
]
67+
}
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
module.exports = {
2+
testEnvironment: 'node',
3+
roots: ['<rootDir>/test'],
4+
testMatch: ['**/*.test.ts'],
5+
transform: {
6+
'^.+\\.tsx?$': 'ts-jest'
7+
}
8+
};
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
const redis = require('ioredis');
2+
3+
const client = new redis({
4+
port: process.env.REDIS_PORT,
5+
host: process.env.REDIS_HOST,
6+
maxRetriesPerRequest: 20,
7+
});
8+
9+
client.on('connect', function () {
10+
console.log('connected');
11+
});
12+
13+
exports.handler = async function (event) {
14+
await client.set('foo', 'bar');
15+
let result = await client.get('foo');
16+
return {
17+
statusCode: 200,
18+
headers: { 'Content-Type': 'text/plain' },
19+
body: result,
20+
};
21+
};
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "elasticache-redisaccess",
3+
"version": "1.0.0",
4+
"description": "pattern to integrate to elasticache-redis cluster",
5+
"main": "index.js",
6+
"scripts": {
7+
"test": "echo \"Error: no test specified\" && exit 1"
8+
},
9+
"author": "",
10+
"license": "ISC",
11+
"dependencies": {
12+
"ioredis": "^5.3.1"
13+
}
14+
}
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
import { Duration, Stack, StackProps } from 'aws-cdk-lib';
2+
import * as iam from 'aws-cdk-lib/aws-iam';
3+
import * as lambda from 'aws-cdk-lib/aws-lambda';
4+
import * as ec2 from 'aws-cdk-lib/aws-ec2';
5+
import * as redis from 'aws-cdk-lib/aws-elasticache';
6+
import { Construct } from 'constructs';
7+
8+
export class LambdaElasticacheIntegrationpatternCdkStack extends Stack {
9+
constructor(scope: Construct, id: string, props?: StackProps) {
10+
super(scope, id, props);
11+
// defines an AWS Lambda resource roles
12+
const lambdarole = new iam.Role(this,'lambda-vpc-execution-role',{
13+
assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
14+
description: 'Lambda execution role for accessing VPC',
15+
managedPolicies: [
16+
iam.ManagedPolicy.fromAwsManagedPolicyName(
17+
'service-role/AWSLambdaVPCAccessExecutionRole',
18+
),
19+
],
20+
});
21+
22+
//get default or any private vpc
23+
const defaultvpc = ec2.Vpc.fromLookup(this, 'ElastiCacheVPC', {
24+
vpcName: "Default", // can be configured where ElastiCache is deployed
25+
isDefault: true
26+
});
27+
28+
//security group for lambda vpc access
29+
const lambdasecuritygroup = new ec2.SecurityGroup(this, 'LambdaVPC-SG',{
30+
vpc:defaultvpc,
31+
allowAllOutbound: true,
32+
description: 'Security group for lambda to access Redis'
33+
});
34+
35+
//get predefined securitygroup
36+
const redissecuritygroup = new ec2.SecurityGroup(this, 'Redis-SG',{
37+
vpc:defaultvpc,
38+
allowAllOutbound: true,
39+
description: 'Security group for Redis'
40+
});
41+
redissecuritygroup.addIngressRule(
42+
ec2.Peer.securityGroupId(lambdasecuritygroup.securityGroupId),
43+
ec2.Port.tcp(6379),
44+
);
45+
46+
47+
// Get all public subnet ids, you can deploy it to privatesubnets as well
48+
const Subnets = defaultvpc.publicSubnets.map((subnet) => {
49+
return subnet.subnetId
50+
});
51+
52+
// Create redis subnet group from subnet ids
53+
const redisSubnetGroup = new redis.CfnSubnetGroup(this, 'RedisSubnetGroup', {
54+
subnetIds: Subnets,
55+
description: "Subnet group for redis"
56+
});
57+
58+
// Create Redis Cluster
59+
const redisCluster = new redis.CfnCacheCluster(this, 'RedisCluster', {
60+
autoMinorVersionUpgrade: true,
61+
cacheNodeType: 'cache.t2.small',
62+
engine: 'redis',
63+
numCacheNodes: 1,
64+
cacheSubnetGroupName: redisSubnetGroup.ref,
65+
clusterName: 'sample-redis' ,
66+
vpcSecurityGroupIds: [redissecuritygroup.securityGroupId]
67+
});
68+
69+
// Define this redis cluster is depends on redis subnet group created first
70+
redisCluster.node.addDependency(redisSubnetGroup);
71+
72+
// Lambda creation
73+
const redisaccess = new lambda.Function(this, 'Elasticache-RedisAccess', {
74+
runtime: lambda.Runtime.NODEJS_18_X, // execution environment
75+
code: lambda.Code.fromAsset('lambda'), // code loaded from "lambda" directory
76+
handler: 'index.handler', // file is "index", function is "handler"
77+
role: lambdarole,
78+
vpc:defaultvpc,
79+
allowPublicSubnet: true,
80+
securityGroups: [lambdasecuritygroup],
81+
timeout: Duration.minutes(5),
82+
environment: {
83+
REDIS_PORT: redisCluster.attrRedisEndpointPort,
84+
REDIS_HOST: redisCluster.attrRedisEndpointAddress,
85+
}
86+
});
87+
}
88+
}

0 commit comments

Comments
 (0)