Skip to content

Commit ea74212

Browse files
julianwoodjulianwood
authored
Merge pull request #2959 from dsiz/dsiz-feature-lambda-durable-functions-nodejs-calling-ecs
adding lambda-durable-functions-nodejs-calling-ecs
2 parents 8056857 + 8befcb2 commit ea74212

7 files changed

Lines changed: 783 additions & 0 deletions

File tree

‎lambda-durable-functions-nodejs-calling-ecs/.gitignore‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Node modules
2+
node_modules/
3+
package-lock.json
4+
5+
# SAM build artifacts
6+
.aws-sam/
7+
samconfig.toml
8+
9+
# IDE
10+
.vscode/
11+
.idea/
12+
*.swp
13+
*.swo
14+
*~
15+
16+
# OS
17+
.DS_Store
18+
Thumbs.db
19+
20+
# Test outputs
21+
response.json
22+
*.log
23+
24+
# Environment
25+
.env
26+
.env.local

‎lambda-durable-functions-nodejs-calling-ecs/README.md‎

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
# AWS Lambda durable functions with Amazon ECS Integration
2+
3+
This pattern demonstrates how to use AWS Lambda durable functions to orchestrate long-running Amazon ECS Fargate tasks. The Lambda function can wait up to 24 hours for ECS task completion without incurring compute charges during the wait period.
4+
5+
**Important:** Please check the [AWS documentation](https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html) for regions currently supported by AWS Lambda durable functions.
6+
7+
Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/lambda-durable-functions-nodejs-calling-ecs
8+
9+
## Architecture
10+
11+
![Architecture Diagram](architecture.png)
12+
13+
The pattern uses Lambda durable functions with the callback pattern to orchestrate ECS Fargate tasks cost-effectively.
14+
15+
### Workflow Steps
16+
17+
1. **Lambda function invoked** with task parameters (message, processing time)
18+
2. **Durable function creates callback ID** using `context.waitForCallback()`
19+
3. **ECS Fargate task started** with callback ID passed as environment variable
20+
4. **Lambda function pauses** (no compute charges during wait)
21+
5. **ECS task processes workload** and logs progress to CloudWatch
22+
6. **ECS task completes** (in production, would call `SendDurableExecutionCallbackSuccess`)
23+
7. **Lambda function resumes** and returns result
24+
25+
## Key Features
26+
27+
- ✅ **24-Hour Wait Time** - Can wait up to 24 hours for ECS task completion
28+
- ✅ **No Compute Charges During Wait** - Function suspended during wait period
29+
- ✅ **Callback Pattern** - ECS tasks call Lambda APIs directly to resume execution
30+
- ✅ **CloudWatch Logs** - Full visibility into both Lambda and ECS execution
31+
- ✅ **Generic Container** - Uses public Python image, easily replaceable
32+
- ✅ **Fargate Serverless** - No EC2 instances to manage
33+
34+
## Prerequisites
35+
36+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
37+
* [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) installed
38+
* Node.js runtime (see [supported runtimes for durable functions](https://docs.aws.amazon.com/lambda/latest/dg/durable-supported-runtimes.html))
39+
40+
## Deployment
41+
42+
1. Navigate to the pattern directory:
43+
```bash
44+
cd lambda-durable-functions-nodejs-calling-ecs
45+
```
46+
47+
2. Build the SAM application:
48+
```bash
49+
sam build
50+
```
51+
52+
3. Deploy the application:
53+
```bash
54+
sam deploy --guided
55+
```
56+
57+
During the guided deployment:
58+
- Accept default values or customize as needed
59+
- Allow SAM CLI to create IAM roles when prompted
60+
- Note the function name from the outputs
61+
62+
4. Note the `CallbackFunctionName` from the CloudFormation outputs
63+
64+
## Testing
65+
66+
### Test the Callback Pattern
67+
68+
Invoke the Lambda function with a test payload:
69+
70+
```bash
71+
aws lambda invoke \
72+
--function-name <CallbackFunctionName>:prod \
73+
--invocation-type Event \
74+
--payload '{"message":"Test ECS task","processingTime":8}' \
75+
--cli-binary-format raw-in-base64-out \
76+
response.json
77+
```
78+
79+
**Note:** A qualified ARN (version or alias) is required for durable functions. See [invoking durable functions](https://docs.aws.amazon.com/lambda/latest/dg/durable-invoking.html#durable-invoking-qualified-arns).
80+
81+
### Monitor Execution
82+
83+
Check Lambda logs:
84+
```bash
85+
aws logs tail /aws/lambda/<CallbackFunctionName> --since 2m --follow
86+
```
87+
88+
Check ECS task logs:
89+
```bash
90+
aws logs tail /ecs/lambda-ecs-durable-demo --since 2m --follow
91+
```
92+
93+
### Expected Output
94+
95+
**Lambda Logs:**
96+
```
97+
Starting Lambda durable function - Callback Pattern
98+
Callback ID created: <callback-id>
99+
Starting ECS task with callback ID...
100+
ECS task started: arn:aws:ecs:...
101+
```
102+
103+
**ECS Logs:**
104+
```
105+
=== ECS Task Started ===
106+
Callback ID: <callback-id>
107+
Message: Test ECS task
108+
Processing Time: 8 seconds
109+
Simulating work...
110+
=== Task Completed Successfully ===
111+
Result: {"status":"completed","message":"Processed: Test ECS task"}
112+
Note: In production, call Lambda SendDurableExecutionCallbackSuccess API here
113+
```
114+
115+
## How It Works
116+
117+
### Lambda durable function (Node.js)
118+
119+
The Lambda function uses the `@aws/durable-execution-sdk-js` package:
120+
121+
```javascript
122+
const { withDurableExecution } = require('@aws/durable-execution-sdk-js');
123+
124+
exports.handler = withDurableExecution(async (event, context) => {
125+
// Create callback and start ECS task
126+
const result = await context.waitForCallback(
127+
'ecs-task-callback',
128+
async (callbackId) => {
129+
// Start ECS task with callback ID
130+
const response = await ecs.send(new RunTaskCommand({
131+
// ... pass callbackId as environment variable
132+
}));
133+
},
134+
{ timeout: { hours: 1 } }
135+
);
136+
137+
return result;
138+
});
139+
```
140+
141+
### ECS Task (Python)
142+
143+
The ECS container receives the callback ID and processes the workload. In production, it would call the Lambda API:
144+
145+
```bash
146+
aws lambda send-durable-execution-callback-success \
147+
--callback-id $CALLBACK_ID \
148+
--result '{"status":"completed","data":"..."}'
149+
```
150+
151+
### Key Configuration
152+
153+
**Lambda Function:**
154+
- Runtime: `nodejs22.x` (see [supported runtimes](https://docs.aws.amazon.com/lambda/latest/dg/durable-supported-runtimes.html))
155+
- `AutoPublishAlias: prod`
156+
- `DurableConfig` with execution timeout and retention period
157+
158+
**ECS Task:**
159+
- Launch type: `FARGATE`
160+
- Public subnet with `assignPublicIp: ENABLED`
161+
- Container image: `public.ecr.aws/docker/library/python:3.12-alpine`
162+
- CloudWatch Logs enabled
163+
164+
## Customization
165+
166+
### Replace the ECS Container
167+
168+
The pattern uses a generic Python container for demonstration. To use your own container:
169+
170+
1. Update the `Image` in the `ECSTaskDefinition` resource
171+
2. Ensure your container:
172+
- Reads the `CALLBACK_ID` environment variable
173+
- Calls `aws lambda send-durable-execution-callback-success` on completion
174+
- Calls `aws lambda send-durable-execution-callback-failure` on error
175+
176+
### Adjust Timeouts
177+
178+
Modify the durable function timeout in `template.yaml`:
179+
180+
```yaml
181+
DurableConfig:
182+
ExecutionTimeout: 86400 # 24 hours in seconds
183+
RetentionPeriodInDays: 7
184+
```
185+
186+
And the callback timeout in the handler:
187+
188+
```javascript
189+
context.waitForCallback('ecs-task-callback', async (callbackId) => {
190+
// ...
191+
}, {
192+
timeout: { hours: 24 }, // Maximum wait time
193+
heartbeatTimeout: { minutes: 5 } // Optional heartbeat
194+
})
195+
```
196+
197+
## Cleanup
198+
199+
Delete the stack:
200+
201+
```bash
202+
sam delete
203+
```
204+
205+
## Additional Resources
206+
207+
- [AWS Lambda durable functions Documentation](https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html)
208+
- [Amazon ECS on AWS Fargate](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html)
209+
- [AWS SAM Documentation](https://docs.aws.amazon.com/serverless-application-model/)
210+
211+
---
212+
213+
© 2026 Amazon Web Services, Inc. or its affiliates. All Rights Reserved.

‎lambda-durable-functions-nodejs-calling-ecs/example-pattern.json‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
{
2+
"title": "AWS Lambda durable functions in NodeJS calling Amazon ECS",
3+
"description": "Orchestrate long-running ECS Fargate tasks using Lambda durable functions with callback pattern",
4+
"language": "Node.js",
5+
"level": "200",
6+
"framework": "AWS SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This pattern demonstrates how to use AWS Lambda durable functions to orchestrate Amazon ECS Fargate tasks that can run for up to 24 hours.",
11+
"The Lambda function creates a callback ID, starts an ECS task with that ID, and then automatically suspends execution without incurring compute charges during the wait period.",
12+
"When the ECS task completes, it calls the Lambda SendDurableExecutionCallbackSuccess API to resume the durable function and return results.",
13+
"This pattern is ideal for long-running batch jobs, data processing, ML training, or any workload that exceeds Lambda's 15-minute timeout."
14+
]
15+
},
16+
"gitHub": {
17+
"template": {
18+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-durable-functions-nodejs-calling-ecs",
19+
"templateURL": "serverless-patterns/lambda-durable-functions-nodejs-calling-ecs",
20+
"projectFolder": "lambda-durable-functions-nodejs-calling-ecs",
21+
"templateFile": "template.yaml"
22+
}
23+
},
24+
"resources": {
25+
"bullets": [
26+
{
27+
"text": "AWS Lambda durable functions Documentation",
28+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html"
29+
},
30+
{
31+
"text": "Amazon ECS on AWS Fargate",
32+
"link": "https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html"
33+
},
34+
{
35+
"text": "Lambda Durable Execution SDK for JavaScript",
36+
"link": "https://www.npmjs.com/package/@aws/durable-execution-sdk-js"
37+
},
38+
{
39+
"text": "SendDurableExecutionCallbackSuccess API",
40+
"link": "https://docs.aws.amazon.com/lambda/latest/api/API_SendDurableExecutionCallbackSuccess.html"
41+
}
42+
]
43+
},
44+
"deploy": {
45+
"text": [
46+
"sam build",
47+
"sam deploy --guided"
48+
]
49+
},
50+
"testing": {
51+
"text": [
52+
"See the GitHub repo for detailed testing instructions."
53+
]
54+
},
55+
"cleanup": {
56+
"text": [
57+
"<code>sam delete</code>"
58+
]
59+
},
60+
"authors": [
61+
{
62+
"name": "Surya Sai D",
63+
"image": "",
64+
"bio": "Surya works as a Technical Account Manager at AWS. He is an expert in Serverless frameworks and Event Driven Architectures. Surya is also passionate on technical writing and has contributed to AWS blogs and other Open Source Content.",
65+
"linkedin": "surya-sai-d-64920416a"
66+
}
67+
]
68+
}

‎lambda-durable-functions-nodejs-calling-ecs/lambda-durable-functions-nodejs-calling-ecs.json‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
{
2+
"title": "AWS Lambda durable functions in NodeJS calling Amazon ECS",
3+
"description": "Orchestrate long-running ECS Fargate tasks using Lambda durable functions with callback pattern",
4+
"language": "Node.js",
5+
"level": "200",
6+
"framework": "AWS SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This pattern demonstrates how to use AWS Lambda durable functions to orchestrate Amazon ECS Fargate tasks that can run for up to 24 hours.",
11+
"The Lambda function creates a callback ID, starts an ECS task with that ID, and then automatically suspends execution without incurring compute charges during the wait period.",
12+
"When the ECS task completes, it calls the Lambda SendDurableExecutionCallbackSuccess API to resume the durable function and return results.",
13+
"This pattern is ideal for long-running batch jobs, data processing, ML training, or any workload that exceeds Lambda's 15-minute timeout."
14+
]
15+
},
16+
"gitHub": {
17+
"template": {
18+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-durable-functions-nodejs-calling-ecs",
19+
"templateURL": "serverless-patterns/lambda-durable-functions-nodejs-calling-ecs",
20+
"projectFolder": "lambda-durable-functions-nodejs-calling-ecs",
21+
"templateFile": "template.yaml"
22+
}
23+
},
24+
"resources": {
25+
"bullets": [
26+
{
27+
"text": "AWS Lambda durable functions Documentation",
28+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html"
29+
},
30+
{
31+
"text": "Amazon ECS on AWS Fargate",
32+
"link": "https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html"
33+
},
34+
{
35+
"text": "Lambda Durable Execution SDK for JavaScript",
36+
"link": "https://www.npmjs.com/package/@aws/durable-execution-sdk-js"
37+
},
38+
{
39+
"text": "SendDurableExecutionCallbackSuccess API",
40+
"link": "https://docs.aws.amazon.com/lambda/latest/api/API_SendDurableExecutionCallbackSuccess.html"
41+
}
42+
]
43+
},
44+
"deploy": {
45+
"text": [
46+
"sam build",
47+
"sam deploy --guided"
48+
]
49+
},
50+
"testing": {
51+
"text": [
52+
"See the GitHub repo for detailed testing instructions."
53+
]
54+
},
55+
"cleanup": {
56+
"text": [
57+
"<code>sam delete</code>"
58+
]
59+
},
60+
"authors": [
61+
{
62+
"name": "Surya Sai D",
63+
"image": "",
64+
"bio": "Surya works as a Technical Account Manager at AWS. He is an expert in Serverless frameworks and Event Driven Architectures. Surya is also passionate on technical writing and has contributed to AWS blogs and other Open Source Content.",
65+
"linkedin": "surya-sai-d-64920416a"
66+
}
67+
],
68+
"patternArch": {
69+
"icon1": {
70+
"x": 20,
71+
"y": 50,
72+
"service": "lambda",
73+
"label": "AWS Lambda durable functions"
74+
},
75+
"icon2": {
76+
"x": 80,
77+
"y": 50,
78+
"service": "ecs",
79+
"label": "Amazon ESC task"
80+
},
81+
"line1": {
82+
"from": "icon1",
83+
"to": "icon2",
84+
"label": "wait for callback"
85+
}
86+
}
87+
}

0 commit comments

Comments
 (0)