Skip to content

Commit 2209630

Browse files
julianwoodjulianwood
authored
Merge pull request aws-samples#2931 from anuprajpara/jsnanup-feature-lambda-durable-parallel-execution-python-sam
New serverless pattern - lambda-durable-parallel-execution-python-sam
2 parents d687df9 + ac68482 commit 2209630

6 files changed

Lines changed: 409 additions & 0 deletions

File tree

‎lambda-durable-parallel-execution-python-sam/README.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# Parallel Execution of Math Operations with AWS Lambda durable functions
2+
3+
This pattern demonstrates parallel execution of mathematical operations using AWS Lambda durable functions. The workflow executes addition, subtraction, multiplication, and division operations concurrently, showcasing the power of parallel execution in durable functions.
4+
5+
## Architecture
6+
7+
The pattern uses AWS Lambda durable functions to execute multiple mathematical operations in parallel, collecting all results before returning the final response. The durable function uses `context.parallel()` to execute multiple operations concurrently.
8+
9+
```mermaid
10+
graph LR
11+
A[Lambda Invoke] --> B[Get num1 & num2]
12+
B --> C{Parallel Execution}
13+
14+
C --> D[Addition]
15+
C --> E[Subtraction]
16+
C --> F[Multiplication]
17+
C --> G[Division]
18+
19+
D --> H[Collect Results]
20+
E --> H
21+
F --> H
22+
G --> H
23+
24+
H --> I{Any Failures?}
25+
I -->|No| J[Return Results]
26+
I -->|Yes| K[Return Error]
27+
28+
style C fill:#e1f5ff
29+
style D fill:#d4edda
30+
style E fill:#d4edda
31+
style F fill:#d4edda
32+
style G fill:#fff3cd
33+
style J fill:#d4edda
34+
style K fill:#f8d7da
35+
```
36+
37+
### Workflow Steps
38+
39+
1. **User invokes Lambda function** with two numbers (num1 and num2)
40+
2. **Lambda durable functions parses input**, defaulting to 0 for missing values
41+
3. **Four operations execute in parallel**:
42+
- Addition: num1 + num2
43+
- Subtraction: num1 - num2
44+
- Multiplication: num1 * num2
45+
- Division: num1 / num2
46+
4. **Results collected** from all parallel operations
47+
5. **Response returned** with all calculation results
48+
49+
## Key Features
50+
51+
- ✅ **Parallel Execution** - All four operations run concurrently
52+
- ✅ **Error Handling** - Division by zero protection with custom retry strategy
53+
- ✅ **Batch Results** - Collects all results before returning
54+
- ✅ **Failure Detection** - Identifies if any operation failed
55+
- ✅ **Configurable Retries** - Per-step retry configuration (division has no retries)
56+
- ✅ **Structured Logging** - Detailed logging for each operation
57+
58+
## Prerequisites
59+
60+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
61+
* [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) installed
62+
63+
## Deployment
64+
65+
1. Navigate to the pattern directory:
66+
```bash
67+
cd lambda-durable-parallel-execution-python-sam
68+
```
69+
70+
2. Build the SAM application:
71+
```bash
72+
sam build
73+
```
74+
75+
3. Deploy the application:
76+
```bash
77+
sam deploy --guided
78+
```
79+
80+
During the guided deployment:
81+
- Accept default values or customize as needed
82+
- Allow SAM CLI to create IAM roles when prompted
83+
84+
4. Note the `MathFunction` name from the outputs
85+
86+
## Testing
87+
88+
### Invoke with Valid Numbers
89+
90+
```bash
91+
aws lambda invoke \
92+
--function-name <MathFunction Name>:live \
93+
--payload '{"num1": 10, "num2": 5}' \
94+
--cli-binary-format raw-in-base64-out \
95+
response.json
96+
97+
cat response.json
98+
```
99+
100+
Response:
101+
```json
102+
{
103+
"Add": 15,
104+
"Subtract": 5,
105+
"Multiply": 50,
106+
"Division": 2.0
107+
}
108+
```
109+
110+
### Test Division by Zero
111+
112+
```bash
113+
aws lambda invoke \
114+
--function-name <MathFunction Name>:live \
115+
--payload '{"num1": 10, "num2": 0}' \
116+
--cli-binary-format raw-in-base64-out \
117+
response.json
118+
119+
cat response.json
120+
```
121+
122+
Response:
123+
```json
124+
{
125+
"error_details": "One or more operations failed"
126+
}
127+
```
128+
129+
### Test with Default Values
130+
131+
```bash
132+
aws lambda invoke \
133+
--function-name <MathFunction Name>:live \
134+
--payload '{}' \
135+
--cli-binary-format raw-in-base64-out \
136+
response.json
137+
```
138+
139+
Response (both numbers default to 0):
140+
```json
141+
{
142+
"error_details": "One or more operations failed"
143+
}
144+
```
145+
146+
## Use Cases
147+
148+
- **Parallel Data Processing** - Execute multiple transformations concurrently
149+
- **Multi-Service Calls** - Call multiple APIs in parallel and aggregate results
150+
- **Batch Calculations** - Perform multiple calculations simultaneously
151+
- **Validation Workflows** - Run multiple validation checks in parallel
152+
- **Data Enrichment** - Enrich data from multiple sources concurrently
153+
154+
## Cleanup
155+
156+
```bash
157+
sam delete --stack-name <stack-name>
158+
```
159+
160+
## Learn More
161+
162+
- [AWS Lambda durable functions documentation](https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html)
163+
- [Durable Execution SDK (Python)](https://github.com/aws/aws-durable-execution-sdk-python)
164+
- [Parallel Execution](https://github.com/aws/aws-durable-execution-sdk-python/blob/main/docs/core/parallel.md)
165+
- [Retry Strategies](https://github.com/aws/aws-durable-execution-sdk-python/blob/main/docs/advanced/error-handling.md#retry-strategies)
166+
167+
---
168+
169+
Copyright 2026 Amazon.com, Inc. or its affiliates. All Rights Reserved.
170+
171+
SPDX-License-Identifier: MIT-0

‎lambda-durable-parallel-execution-python-sam/example-pattern.json‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
{
2+
"title": "Parallel execution of operations with AWS Lambda durable functions",
3+
"description": "Demonstrates parallel execution of operations using Lambda durable functions with batch result collection and error handling",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "AWS SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This pattern demonstrates parallel execution capabilities of AWS Lambda durable functions by performing four mathematical operations concurrently.",
11+
"The workflow uses context.parallel() to execute addition, subtraction, multiplication, and division operations simultaneously on two input numbers.",
12+
"Each operation is defined as a separate step with its own logging and error handling, showcasing the durable execution model.",
13+
"The division operation uses a custom retry strategy (RetryPresets.none()) to prevent retries on division by zero errors.",
14+
"Results from all parallel operations are collected using BatchResult, which tracks both successful results and failure counts.",
15+
"If any operation fails, the workflow returns an error response; otherwise, it returns all calculation results in a structured JSON format."
16+
]
17+
},
18+
"gitHub": {
19+
"template": {
20+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-durable-parallel-execution-python-sam",
21+
"templateURL": "serverless-patterns/lambda-durable-parallel-execution-python-sam",
22+
"projectFolder": "lambda-durable-parallel-execution-python-sam",
23+
"templateFile": "template.yaml"
24+
}
25+
},
26+
"resources": {
27+
"bullets": [
28+
{
29+
"text": "AWS Lambda durable functions documentation",
30+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html"
31+
},
32+
{
33+
"text": "Durable Execution SDK for Python",
34+
"link": "https://github.com/aws/aws-durable-execution-sdk-python"
35+
},
36+
{
37+
"text": "Parallel Execution",
38+
"link": "https://github.com/aws/aws-durable-execution-sdk-python/blob/main/docs/core/parallel.md"
39+
},
40+
{
41+
"text": "Retry Strategies",
42+
"link": "https://github.com/aws/aws-durable-execution-sdk-python/blob/main/docs/advanced/error-handling.md#retry-strategies"
43+
}
44+
]
45+
},
46+
"deploy": {
47+
"text": [
48+
"sam build",
49+
"sam deploy --guided"
50+
]
51+
},
52+
"testing": {
53+
"text": [
54+
"See the GitHub repo for detailed testing instructions."
55+
]
56+
},
57+
"cleanup": {
58+
"text": [
59+
"Delete the stack: <code>sam delete</code>."
60+
]
61+
},
62+
"authors": [
63+
{
64+
"name": "Anup Rajpara",
65+
"image": "https://drive.google.com/file/d/1MqpPNLCqbU4kvvtTspNXZBqD99aVIJI9/view?usp=sharing",
66+
"bio": "Anup is a Sr. Technical Account Manager at Amazon Web Services. He is passionate about serverless & event-driven architectures.",
67+
"linkedin": "anup-rajpara-developer"
68+
},
69+
{
70+
"name": "Sahil Bhimjiani",
71+
"image": "https://drive.google.com/file/d/1E2p7S5UtU36x6Sk1xPS3XnSGJyIUoqK7/view?usp=drivesdk",
72+
"bio": "Sahil Bhimjiani is a Solutions Architect at Amazon Web Services.",
73+
"linkedin": "sahil9701"
74+
}
75+
]
76+
}

‎lambda-durable-parallel-execution-python-sam/lambda-durable-parallel-execution-python-sam.json‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
{
2+
"title": "Parallel execution of operations with AWS Lambda durable functions",
3+
"description": "Demonstrates parallel execution of operations using Lambda durable functions with batch result collection and error handling",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "AWS SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This pattern demonstrates parallel execution capabilities of AWS Lambda durable functions by performing four mathematical operations concurrently.",
11+
"The workflow uses context.parallel() to execute addition, subtraction, multiplication, and division operations simultaneously on two input numbers.",
12+
"Each operation is defined as a separate step with its own logging and error handling, showcasing the durable execution model.",
13+
"The division operation uses a custom retry strategy (RetryPresets.none()) to prevent retries on division by zero errors.",
14+
"Results from all parallel operations are collected using BatchResult, which tracks both successful results and failure counts.",
15+
"If any operation fails, the workflow returns an error response; otherwise, it returns all calculation results in a structured JSON format."
16+
]
17+
},
18+
"gitHub": {
19+
"template": {
20+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-durable-parallel-execution-python-sam",
21+
"templateURL": "serverless-patterns/lambda-durable-parallel-execution-python-sam",
22+
"projectFolder": "lambda-durable-parallel-execution-python-sam",
23+
"templateFile": "template.yaml"
24+
}
25+
},
26+
"resources": {
27+
"bullets": [
28+
{
29+
"text": "AWS Lambda durable functions documentation",
30+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html"
31+
},
32+
{
33+
"text": "Durable Execution SDK for Python",
34+
"link": "https://github.com/aws/aws-durable-execution-sdk-python"
35+
},
36+
{
37+
"text": "Parallel Execution",
38+
"link": "https://github.com/aws/aws-durable-execution-sdk-python/blob/main/docs/core/parallel.md"
39+
},
40+
{
41+
"text": "Retry Strategies",
42+
"link": "https://github.com/aws/aws-durable-execution-sdk-python/blob/main/docs/advanced/error-handling.md#retry-strategies"
43+
}
44+
]
45+
},
46+
"deploy": {
47+
"text": [
48+
"sam build",
49+
"sam deploy --guided"
50+
]
51+
},
52+
"testing": {
53+
"text": [
54+
"See the GitHub repo for detailed testing instructions."
55+
]
56+
},
57+
"cleanup": {
58+
"text": [
59+
"Delete the stack: <code>sam delete</code>."
60+
]
61+
},
62+
"authors": [
63+
{
64+
"name": "Anup Rajpara",
65+
"image": "https://drive.google.com/file/d/1MqpPNLCqbU4kvvtTspNXZBqD99aVIJI9/view?usp=sharing",
66+
"bio": "Anup is a Sr. Technical Account Manager at Amazon Web Services. He is passionate about serverless & event-driven architectures.",
67+
"linkedin": "anup-rajpara-developer"
68+
},
69+
{
70+
"name": "Sahil Bhimjiani",
71+
"image": "https://drive.google.com/file/d/1E2p7S5UtU36x6Sk1xPS3XnSGJyIUoqK7/view?usp=drivesdk",
72+
"bio": "Sahil Bhimjiani is a Solutions Architect at Amazon Web Services.",
73+
"linkedin": "sahil9701"
74+
}
75+
],
76+
"patternArch": {
77+
"icon1": {
78+
"x": 50,
79+
"y": 50,
80+
"service": "lambda",
81+
"label": "AWS Lambda durable functions"
82+
}
83+
}
84+
}

‎lambda-durable-parallel-execution-python-sam/src/lambda_function.py‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
from aws_durable_execution_sdk_python import BatchResult, DurableContext, durable_execution, durable_step
2+
from aws_durable_execution_sdk_python.retries import RetryPresets
3+
from aws_durable_execution_sdk_python.config import StepConfig
4+
5+
@durable_execution
6+
def lambda_handler(event: dict, context: DurableContext) -> dict:
7+
context.logger.info(event)
8+
num1 = event.get('num1', 0)
9+
num2 = event.get('num2', 0)
10+
11+
# No retries
12+
step_config = StepConfig(retry_strategy=RetryPresets.none())
13+
14+
# Define the functions for each step
15+
def add_nums():
16+
context.logger.info(f"Adding {num1} and {num2}")
17+
return num1 + num2
18+
19+
def sub_nums():
20+
context.logger.info(f"Subtracting {num2} from {num1}")
21+
return num1 - num2
22+
23+
def mul_nums():
24+
context.logger.info(f"Multiplying {num1} and {num2}")
25+
return num1 * num2
26+
27+
def div_nums():
28+
context.logger.info(f"Dividing {num1} by {num2}")
29+
if num2 == 0:
30+
raise ValueError("Division by zero is not allowed")
31+
return num1 / num2
32+
33+
try:
34+
# context.parallel takes a list of step definitions and executes them in parallel.
35+
# It returns a list of results in the same order as the input list.
36+
results = context.parallel([
37+
lambda ctx: ctx.step(lambda _: add_nums(), name="addition"),
38+
lambda ctx: ctx.step(lambda _: sub_nums(), name="subtraction"),
39+
lambda ctx: ctx.step(lambda _: mul_nums(), name="multiplication"),
40+
lambda ctx: ctx.step(lambda _: div_nums(), name="division", config=step_config)
41+
])
42+
43+
if results.failure_count != 0:
44+
raise Exception(f"One or more operations failed")
45+
46+
add, sub, mul, div = results.get_results()
47+
return {
48+
"Add": add,
49+
"Subtract": sub,
50+
"Multiply": mul,
51+
"Division": div
52+
}
53+
54+
except Exception as e:
55+
return {
56+
"error_details": str(e)
57+
}

‎lambda-durable-parallel-execution-python-sam/src/requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
aws-durable-execution-sdk-python

0 commit comments

Comments
 (0)