From cea65036f885fbaec73b617d0b884b901b04b6f0 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 16:51:03 -0400 Subject: [PATCH 01/17] [TON-388] feat(aws_quickstart): port InstrumentationResourceTypes to main_extended and main_extended_workflow v4.10.0 (PR #306) added the InstrumentationResourceTypes parameter only to main_v2.yaml. main_extended.yaml and main_extended_workflow.yaml are the templates UI launches actually use going forward, so the parameter and the DatadogSite + InstrumentationResourceTypes passthrough to the role stack belong there too. Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/main_extended.yaml | 10 ++++++++++ aws_quickstart/main_extended_workflow.yaml | 10 ++++++++++ 2 files changed, 20 insertions(+) diff --git a/aws_quickstart/main_extended.yaml b/aws_quickstart/main_extended.yaml index 7c8c8434..8c7b9098 100644 --- a/aws_quickstart/main_extended.yaml +++ b/aws_quickstart/main_extended.yaml @@ -116,6 +116,13 @@ Parameters: For cross-account Agentless Scanning: the ARNs of Agentless Scanner roles authorized to scan this account. AllowedPattern: "|arn:aws:iam::[0-9]{12}:role/.+" Default: "" + InstrumentationResourceTypes: + Type: CommaDelimitedList + Description: >- + Comma-separated list of AWS resource types (UDM form, e.g. aws:ec2:instance, aws:ecs:cluster, aws:eks:cluster) + that the Datadog integration role should be granted the IAM permissions required to instrument with the Datadog + Agent. Leave blank to skip granting any extra instrumentation permissions. + Default: "" Rules: CSPMResourceCollectionValidState: Assertions: @@ -269,6 +276,8 @@ Resources: ExternalId: !GetAtt DatadogAPICall.Outputs.ExternalId IAMRoleName: !Ref IAMRoleName ResourceCollectionPermissions: !If [ResourceCollectionPermissions, true, false] + InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] + DatadogSite: !Ref DatadogSite DdAWSAccountId: !If - IsGov - !If @@ -327,6 +336,7 @@ Metadata: - ScannerDelegateRoleName - ScannerInstanceRoleARN - DisableMetricCollection + - InstrumentationResourceTypes ParameterLabels: APIKey: default: "DatadogApiKey *" diff --git a/aws_quickstart/main_extended_workflow.yaml b/aws_quickstart/main_extended_workflow.yaml index 5b2e19a4..6e29c866 100644 --- a/aws_quickstart/main_extended_workflow.yaml +++ b/aws_quickstart/main_extended_workflow.yaml @@ -126,6 +126,13 @@ Parameters: For cross-account Agentless Scanning: the ARNs of Agentless Scanner roles authorized to scan this account. AllowedPattern: "|arn:aws:iam::[0-9]{12}:role/.+" Default: "" + InstrumentationResourceTypes: + Type: CommaDelimitedList + Description: >- + Comma-separated list of AWS resource types (UDM form, e.g. aws:ec2:instance, aws:ecs:cluster, aws:eks:cluster) + that the Datadog integration role should be granted the IAM permissions required to instrument with the Datadog + Agent. Leave blank to skip granting any extra instrumentation permissions. + Default: "" WorkflowId: Description: >- Unique identifier for this workflow instance. This is generated by the Datadog UI and used to track @@ -474,6 +481,8 @@ Resources: ExternalId: !Ref ExternalId IAMRoleName: !Ref IAMRoleName ResourceCollectionPermissions: !If [ResourceCollectionPermissions, true, false] + InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] + DatadogSite: !Ref DatadogSite DdAWSAccountId: !If - IsGov - !If @@ -756,6 +765,7 @@ Metadata: - ScannerDelegateRoleName - ScannerInstanceRoleARN - DisableMetricCollection + - InstrumentationResourceTypes ParameterLabels: APIKey: default: "DatadogApiKey *" From 445939dee1b1002bababdb19a98a70230375422f Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 16:55:46 -0400 Subject: [PATCH 02/17] [TON-388] chore(aws_quickstart): bump to v4.11.0 with changelog entry Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 4 ++++ aws_quickstart/version.txt | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index 2edc93e9..3ce577ac 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,3 +1,7 @@ +# 4.11.0 (May 19, 2026) + +- Make the `InstrumentationResourceTypes` parameter (added in 4.10.0) available in the extended templates as well, so customers launching from the workflow path can also grant Agent management instrumentation permissions on their integration role. + # 4.10.0 (May 13, 2026) - Add `InstrumentationResourceTypes` parameter to `main_v2.yaml`. When set to a comma-separated list of UDM resource types (e.g. `aws:ec2:instance,aws:ecs:cluster,aws:eks:cluster`), the integration role's permission-attach Lambda calls `GET /api/unstable/instrumenter/aws/iam_permissions?resource_type=...&chunked=true` and attaches the returned IAM permissions as additional managed policies on the integration role, so customers can install the Datadog Agent on those resources without extra IAM setup. Failure to fetch or attach these extra permissions is non-blocking — the integration install proceeds with a warning. Affects `main_v2.yaml`, `datadog_integration_role.yaml`, `attach_integration_permissions.py` diff --git a/aws_quickstart/version.txt b/aws_quickstart/version.txt index d3d9bf7a..bf721f57 100644 --- a/aws_quickstart/version.txt +++ b/aws_quickstart/version.txt @@ -1 +1 @@ -v4.10.0 +v4.11.0 From caf18eb6e51e064d8f83d798d35ef1bb5a91c8ab Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Wed, 20 May 2026 14:12:44 -0400 Subject: [PATCH 03/17] [TON-388] docs(aws_quickstart): simplify 4.11.0 changelog entry Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index 3ce577ac..044a03b4 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,6 +1,6 @@ # 4.11.0 (May 19, 2026) -- Make the `InstrumentationResourceTypes` parameter (added in 4.10.0) available in the extended templates as well, so customers launching from the workflow path can also grant Agent management instrumentation permissions on their integration role. +- Make the `InstrumentationResourceTypes` parameter (added in 4.10.0) available in `main_extended.yaml` and `main_extended_workflow.yaml` as well # 4.10.0 (May 13, 2026) From 678c7af0acdbd531d768eb8c33e74e42a767ff1c Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Wed, 20 May 2026 16:10:52 -0400 Subject: [PATCH 04/17] [TON-388] fix(aws_quickstart): shift InstrumentationResourceTypes port to main_workflow (drop main_extended) main_extended.yaml isn't on the UI launch path; revert there and apply to main_workflow.yaml instead. Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 2 +- aws_quickstart/main_extended.yaml | 10 ---------- aws_quickstart/main_workflow.yaml | 10 ++++++++++ 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index 044a03b4..57ccb090 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,6 +1,6 @@ # 4.11.0 (May 19, 2026) -- Make the `InstrumentationResourceTypes` parameter (added in 4.10.0) available in `main_extended.yaml` and `main_extended_workflow.yaml` as well +- Make the `InstrumentationResourceTypes` parameter (added in 4.10.0) available in `main_workflow.yaml` and `main_extended_workflow.yaml` as well # 4.10.0 (May 13, 2026) diff --git a/aws_quickstart/main_extended.yaml b/aws_quickstart/main_extended.yaml index 8c7b9098..7c8c8434 100644 --- a/aws_quickstart/main_extended.yaml +++ b/aws_quickstart/main_extended.yaml @@ -116,13 +116,6 @@ Parameters: For cross-account Agentless Scanning: the ARNs of Agentless Scanner roles authorized to scan this account. AllowedPattern: "|arn:aws:iam::[0-9]{12}:role/.+" Default: "" - InstrumentationResourceTypes: - Type: CommaDelimitedList - Description: >- - Comma-separated list of AWS resource types (UDM form, e.g. aws:ec2:instance, aws:ecs:cluster, aws:eks:cluster) - that the Datadog integration role should be granted the IAM permissions required to instrument with the Datadog - Agent. Leave blank to skip granting any extra instrumentation permissions. - Default: "" Rules: CSPMResourceCollectionValidState: Assertions: @@ -276,8 +269,6 @@ Resources: ExternalId: !GetAtt DatadogAPICall.Outputs.ExternalId IAMRoleName: !Ref IAMRoleName ResourceCollectionPermissions: !If [ResourceCollectionPermissions, true, false] - InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] - DatadogSite: !Ref DatadogSite DdAWSAccountId: !If - IsGov - !If @@ -336,7 +327,6 @@ Metadata: - ScannerDelegateRoleName - ScannerInstanceRoleARN - DisableMetricCollection - - InstrumentationResourceTypes ParameterLabels: APIKey: default: "DatadogApiKey *" diff --git a/aws_quickstart/main_workflow.yaml b/aws_quickstart/main_workflow.yaml index d361e220..3aa6038b 100644 --- a/aws_quickstart/main_workflow.yaml +++ b/aws_quickstart/main_workflow.yaml @@ -68,6 +68,13 @@ Parameters: External ID for the IAM role trust policy. This is generated by the Datadog UI and ensures that only your Datadog organization can assume this role. Type: String + InstrumentationResourceTypes: + Type: CommaDelimitedList + Description: >- + Comma-separated list of AWS resource types (UDM form, e.g. aws:ec2:instance, aws:ecs:cluster, aws:eks:cluster) + that the Datadog integration role should be granted the IAM permissions required to instrument with the Datadog + Agent. Leave blank to skip granting any extra instrumentation permissions. + Default: "" Mappings: # AccountIdGovCloud is only read on GovCloud sites (ddog-gov.com, us2.ddog-gov.com). # Commercial sites carry "NOT_APPLICABLE" so Fn::FindInMap can resolve the key at @@ -330,6 +337,8 @@ Resources: ExternalId: !Ref ExternalId IAMRoleName: !Ref IAMRoleName ResourceCollectionPermissions: !If [ResourceCollectionPermissions, true, false] + InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] + DatadogSite: !Ref DatadogSite DdAWSAccountId: !If - IsGov - !If @@ -490,6 +499,7 @@ Metadata: Parameters: - IAMRoleName - DisableResourceCollection + - InstrumentationResourceTypes ParameterLabels: APIKey: default: "DatadogApiKey *" From 015f548a52efb9cf085cf3f385888bde09925135 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 15:25:54 -0400 Subject: [PATCH 05/17] [TON-466] feat(aws_quickstart): forward CloudTrail events to Datadog instrumenter-events intake Adds an EventBridge connection, API destination, invocation role, and EC2 CloudTrail rule as a new nested stack, conditionally deployed when InstrumentationResourceTypes is set. Single-region by design. Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/README.md | 3 + .../datadog_agent_install_forwarding.yaml | 93 +++++++++++++++++++ aws_quickstart/main_v2.yaml | 17 ++++ 3 files changed, 113 insertions(+) create mode 100644 aws_quickstart/datadog_agent_install_forwarding.yaml diff --git a/aws_quickstart/README.md b/aws_quickstart/README.md index 43f39ad0..46b1f0c2 100644 --- a/aws_quickstart/README.md +++ b/aws_quickstart/README.md @@ -22,6 +22,9 @@ This template creates the following AWS resources required by the Datadog AWS in - The [Datadog Forwarder Lambda function](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring) to ship logs from S3 and CloudWatch, custom metrics and traces from Lambda functions to Datadog - The Datadog Forwarder only deploys to the AWS region where the AWS integration CloudFormation stack is launched. If you operate in multiple AWS regions, you can deploy the Forwarder stack (without the rest of the AWS integration stack) directly to other regions as needed. - The Datadog Forwarder is installed with default settings as a nested stack, edit the nested stack directly to update the forwarder specific settings. +- An EventBridge pipeline (connection, API destination, invocation role, and a CloudTrail rule) that forwards resource lifecycle events to Datadog so the Agent install feature can react in real time. + - Deployed only when `InstrumentationResourceTypes` is set. Leaving it blank skips this pipeline entirely. + - Single-region by design: the pipeline covers the AWS region the stack is deployed in. Deploy the stack in each region you want covered. ## Updating your CloudFormation Stack diff --git a/aws_quickstart/datadog_agent_install_forwarding.yaml b/aws_quickstart/datadog_agent_install_forwarding.yaml new file mode 100644 index 00000000..f1e5a3b5 --- /dev/null +++ b/aws_quickstart/datadog_agent_install_forwarding.yaml @@ -0,0 +1,93 @@ +# version: +AWSTemplateFormatVersion: 2010-09-09 +Description: Datadog AWS Integration - EventBridge forwarding for agent install +Parameters: + APIKey: + Description: >- + API key for the Datadog account (find at https://app.datadoghq.com/organization-settings/api-keys). + Type: String + AllowedPattern: "([0-9a-f]{32})?" + NoEcho: true + APPKey: + Description: >- + APP key for the Datadog account (find at https://app.datadoghq.com/organization-settings/application-keys). + Type: String + AllowedPattern: "(([0-9a-f]{40})|(ddapp_[a-zA-Z0-9]{34}))?" + NoEcho: true + DatadogSite: + Type: String + Default: datadoghq.com + Description: >- + Datadog site events are forwarded to. +Resources: + # EventBridge connection storing the Datadog API + APP key credentials used by + # the API destination below. Both headers are routed through Secrets Manager: + # DD-API-KEY via ApiKeyAuthParameters, DD-APPLICATION-KEY via + # InvocationHttpParameters.HeaderParameters with IsValueSecret=true. + DDIntakeConnection: + Type: AWS::Events::Connection + Properties: + Name: datadog-agent-resource-update-intake-connection + Description: Datadog instrumenter-events intake credentials + AuthorizationType: API_KEY + AuthParameters: + ApiKeyAuthParameters: + ApiKeyName: DD-API-KEY + ApiKeyValue: !Ref APIKey + InvocationHttpParameters: + HeaderParameters: + - Key: DD-APPLICATION-KEY + Value: !Ref APPKey + IsValueSecret: true + DDIntakeApiDestination: + Type: AWS::Events::ApiDestination + Properties: + Name: datadog-agent-resource-update-intake-destination + ConnectionArn: !GetAtt DDIntakeConnection.Arn + InvocationEndpoint: !Sub "https://api.${DatadogSite}/api/unstable/instrumenter/events" + HttpMethod: POST + InvocationRateLimitPerSecond: 10 + DDEventBridgeInvocationRole: + Type: AWS::IAM::Role + Properties: + RoleName: datadog-agent-resource-update-eventbridge-invocation-role + AssumeRolePolicyDocument: + Version: 2012-10-17 + Statement: + - Effect: Allow + Principal: + Service: events.amazonaws.com + Action: sts:AssumeRole + Condition: + StringEquals: + aws:SourceAccount: !Ref AWS::AccountId + Policies: + - PolicyName: datadog-eventbridge-invoke + PolicyDocument: + Version: 2012-10-17 + Statement: + - Effect: Allow + Action: events:InvokeApiDestination + Resource: !Sub "arn:${AWS::Partition}:events:*:${AWS::AccountId}:api-destination/*" + DDEventForwardingRuleEC2: + Type: AWS::Events::Rule + Properties: + Name: datadog-agent-resource-update-rule-ec2 + Description: Forward EC2 CloudTrail events to the Datadog instrumenter-events intake + State: ENABLED + EventPattern: + source: + - aws.ec2 + detail-type: + - "AWS API Call via CloudTrail" + detail: + eventName: + - RunInstances + - CreateTags + - DeleteTags + errorCode: + - exists: false + Targets: + - Id: datadog-intake + Arn: !GetAtt DDIntakeApiDestination.Arn + RoleArn: !GetAtt DDEventBridgeInvocationRole.Arn diff --git a/aws_quickstart/main_v2.yaml b/aws_quickstart/main_v2.yaml index 3e86464c..3ccfd113 100644 --- a/aws_quickstart/main_v2.yaml +++ b/aws_quickstart/main_v2.yaml @@ -130,6 +130,11 @@ Conditions: Fn::Equals: - !Ref DisableResourceCollection - false + ShouldForwardEvents: + Fn::Not: + - Fn::Equals: + - !Join ["", !Ref InstrumentationResourceTypes] + - "" IsGov: Fn::Or: - Fn::Equals: @@ -184,6 +189,18 @@ Resources: Parameters: DdApiKey: !Ref APIKey DdSite: !Ref DatadogSite + # EventBridge pipeline forwarding CloudTrail events to Datadog's instrumenter-events + # intake. Deployed only when at least one InstrumentationResourceTypes value is set; + # single-region (covers the region this stack is deployed in). + DatadogAgentInstallForwardingStack: + Type: AWS::CloudFormation::Stack + Condition: ShouldForwardEvents + Properties: + TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_install_forwarding.yaml" + Parameters: + APIKey: !Ref APIKey + APPKey: !Ref APPKey + DatadogSite: !Ref DatadogSite Outputs: IAMRoleName: Description: AWS IAM Role named to be used with the DataDog AWS Integration From 6144df95df13e1406b562068f37d932f9567a89a Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 15:45:57 -0400 Subject: [PATCH 06/17] [TON-466] feat(aws_quickstart): gate forwarding rules per InstrumentationResourceTypes; add EKS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add an EKS CloudTrail rule (CreateCluster, TagResource, UntagResource) and gate each rule on whether its UDM type appears in InstrumentationResourceTypes. Substring check is via Fn::Split / Fn::Join — CFN has no Conditions-level substring intrinsic. Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/README.md | 4 +- .../datadog_agent_install_forwarding.yaml | 46 +++++++++++++++++++ aws_quickstart/main_v2.yaml | 1 + 3 files changed, 49 insertions(+), 2 deletions(-) diff --git a/aws_quickstart/README.md b/aws_quickstart/README.md index 46b1f0c2..d957bf7f 100644 --- a/aws_quickstart/README.md +++ b/aws_quickstart/README.md @@ -22,8 +22,8 @@ This template creates the following AWS resources required by the Datadog AWS in - The [Datadog Forwarder Lambda function](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring) to ship logs from S3 and CloudWatch, custom metrics and traces from Lambda functions to Datadog - The Datadog Forwarder only deploys to the AWS region where the AWS integration CloudFormation stack is launched. If you operate in multiple AWS regions, you can deploy the Forwarder stack (without the rest of the AWS integration stack) directly to other regions as needed. - The Datadog Forwarder is installed with default settings as a nested stack, edit the nested stack directly to update the forwarder specific settings. -- An EventBridge pipeline (connection, API destination, invocation role, and a CloudTrail rule) that forwards resource lifecycle events to Datadog so the Agent install feature can react in real time. - - Deployed only when `InstrumentationResourceTypes` is set. Leaving it blank skips this pipeline entirely. +- An EventBridge pipeline (connection, API destination, invocation role, and per-service CloudTrail rules) that forwards resource lifecycle events to Datadog so the Agent install feature can react in real time. + - Deployed only when `InstrumentationResourceTypes` is set. A rule is created for each listed resource type — listing `aws:ec2:instance` deploys the EC2 rule, listing `aws:eks:cluster` deploys the EKS rule. - Single-region by design: the pipeline covers the AWS region the stack is deployed in. Deploy the stack in each region you want covered. ## Updating your CloudFormation Stack diff --git a/aws_quickstart/datadog_agent_install_forwarding.yaml b/aws_quickstart/datadog_agent_install_forwarding.yaml index f1e5a3b5..7f64d751 100644 --- a/aws_quickstart/datadog_agent_install_forwarding.yaml +++ b/aws_quickstart/datadog_agent_install_forwarding.yaml @@ -19,6 +19,28 @@ Parameters: Default: datadoghq.com Description: >- Datadog site events are forwarded to. + InstrumentationResourceTypes: + Type: String + Default: "" + Description: >- + Comma-separated list of AWS resource types (UDM form, e.g. aws:ec2:instance, + aws:eks:cluster) to forward CloudTrail events for. Only rules for the listed + types are deployed. +Conditions: + # CFN has no substring intrinsic in Conditions. Pad the joined string with the + # delimiter on both ends, split on the padded token, rejoin without the delimiter, + # and check whether the result differs from the original — if it does, the token + # was present. + IncludeEC2: + Fn::Not: + - Fn::Equals: + - !Join ["", !Split [",aws:ec2:instance,", !Sub ",${InstrumentationResourceTypes},"]] + - !Sub ",${InstrumentationResourceTypes}," + IncludeEKS: + Fn::Not: + - Fn::Equals: + - !Join ["", !Split [",aws:eks:cluster,", !Sub ",${InstrumentationResourceTypes},"]] + - !Sub ",${InstrumentationResourceTypes}," Resources: # EventBridge connection storing the Datadog API + APP key credentials used by # the API destination below. Both headers are routed through Secrets Manager: @@ -71,6 +93,7 @@ Resources: Resource: !Sub "arn:${AWS::Partition}:events:*:${AWS::AccountId}:api-destination/*" DDEventForwardingRuleEC2: Type: AWS::Events::Rule + Condition: IncludeEC2 Properties: Name: datadog-agent-resource-update-rule-ec2 Description: Forward EC2 CloudTrail events to the Datadog instrumenter-events intake @@ -91,3 +114,26 @@ Resources: - Id: datadog-intake Arn: !GetAtt DDIntakeApiDestination.Arn RoleArn: !GetAtt DDEventBridgeInvocationRole.Arn + DDEventForwardingRuleEKS: + Type: AWS::Events::Rule + Condition: IncludeEKS + Properties: + Name: datadog-agent-resource-update-rule-eks + Description: Forward EKS CloudTrail events to the Datadog instrumenter-events intake + State: ENABLED + EventPattern: + source: + - aws.eks + detail-type: + - "AWS API Call via CloudTrail" + detail: + eventName: + - CreateCluster + - TagResource + - UntagResource + errorCode: + - exists: false + Targets: + - Id: datadog-intake + Arn: !GetAtt DDIntakeApiDestination.Arn + RoleArn: !GetAtt DDEventBridgeInvocationRole.Arn diff --git a/aws_quickstart/main_v2.yaml b/aws_quickstart/main_v2.yaml index 3ccfd113..ad2cd370 100644 --- a/aws_quickstart/main_v2.yaml +++ b/aws_quickstart/main_v2.yaml @@ -201,6 +201,7 @@ Resources: APIKey: !Ref APIKey APPKey: !Ref APPKey DatadogSite: !Ref DatadogSite + InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] Outputs: IAMRoleName: Description: AWS IAM Role named to be used with the DataDog AWS Integration From fad780b234a08bd5dc73b102911f3a9fed328e8f Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 15:50:59 -0400 Subject: [PATCH 07/17] [TON-466] feat(aws_quickstart): filter tag events to target resource type EC2 CreateTags/DeleteTags are scoped to instances via resourcesSet item resourceId prefix "i-"; EKS TagResource/UntagResource are scoped to cluster ARNs via wildcard match. Creation events (RunInstances, CreateCluster) bypass the filter through EventBridge $or because their request payloads don't carry the filter field. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../datadog_agent_install_forwarding.yaml | 31 ++++++++++++++----- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/aws_quickstart/datadog_agent_install_forwarding.yaml b/aws_quickstart/datadog_agent_install_forwarding.yaml index 7f64d751..a0d15533 100644 --- a/aws_quickstart/datadog_agent_install_forwarding.yaml +++ b/aws_quickstart/datadog_agent_install_forwarding.yaml @@ -91,6 +91,9 @@ Resources: - Effect: Allow Action: events:InvokeApiDestination Resource: !Sub "arn:${AWS::Partition}:events:*:${AWS::AccountId}:api-destination/*" + # $or splits each rule into a creation branch (no resource filter — the request + # doesn't carry the field we'd filter on) and a tag branch (filtered to the + # target resource type so we don't spam the intake with sibling resources). DDEventForwardingRuleEC2: Type: AWS::Events::Rule Condition: IncludeEC2 @@ -104,12 +107,19 @@ Resources: detail-type: - "AWS API Call via CloudTrail" detail: - eventName: - - RunInstances - - CreateTags - - DeleteTags errorCode: - exists: false + $or: + - eventName: + - RunInstances + - eventName: + - CreateTags + - DeleteTags + requestParameters: + resourcesSet: + items: + resourceId: + - prefix: "i-" Targets: - Id: datadog-intake Arn: !GetAtt DDIntakeApiDestination.Arn @@ -127,12 +137,17 @@ Resources: detail-type: - "AWS API Call via CloudTrail" detail: - eventName: - - CreateCluster - - TagResource - - UntagResource errorCode: - exists: false + $or: + - eventName: + - CreateCluster + - eventName: + - TagResource + - UntagResource + requestParameters: + resourceArn: + - wildcard: "*:cluster/*" Targets: - Id: datadog-intake Arn: !GetAtt DDIntakeApiDestination.Arn From 8657ccd99fb2cbe546823fd037f16da6dbf42b9d Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 15:56:58 -0400 Subject: [PATCH 08/17] [TON-466] feat(aws_quickstart): rename forwarding template, bump to v4.11.0, changelog MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Rename datadog_agent_install_forwarding.yaml to datadog_agent_resource_update_forwarding.yaml (the pipeline forwards resource update events; agent install is one consumer) - DatadogAgentInstallForwardingStack -> DatadogAgentResourceUpdateForwardingStack in main_v2.yaml - Bump version.txt to v4.11.0 + add 4.11.0 changelog entry - Revert README — leave matching current prod Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 4 ++++ aws_quickstart/README.md | 3 --- ...ing.yaml => datadog_agent_resource_update_forwarding.yaml} | 2 +- aws_quickstart/main_v2.yaml | 4 ++-- aws_quickstart/version.txt | 2 +- 5 files changed, 8 insertions(+), 7 deletions(-) rename aws_quickstart/{datadog_agent_install_forwarding.yaml => datadog_agent_resource_update_forwarding.yaml} (98%) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index 57ccb090..549f97b7 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,3 +1,7 @@ +# 4.12.0 (May 19, 2026) + +- Add an EventBridge → Datadog API destination pipeline that forwards CloudTrail events for the resource types listed in `InstrumentationResourceTypes`, enabling the Datadog Agent install feature to react to lifecycle changes in real time. When `InstrumentationResourceTypes` is non-empty, a nested stack provisions an `AWS::Events::Connection` (DD-API-KEY + DD-APPLICATION-KEY, both routed through Secrets Manager), an `AWS::Events::ApiDestination` pointing at `https://api.${DatadogSite}/api/unstable/instrumenter/events`, an invocation role, and one `AWS::Events::Rule` per UDM type in the list. v1 covers `aws:ec2:instance` (`RunInstances`, `CreateTags`/`DeleteTags` filtered to instance resource IDs) and `aws:eks:cluster` (`CreateCluster`, `TagResource`/`UntagResource` filtered to cluster ARNs). Single-region by design — deploy the stack in each region you want covered. Affects `main_v2.yaml`, `datadog_agent_resource_update_forwarding.yaml` (new) + # 4.11.0 (May 19, 2026) - Make the `InstrumentationResourceTypes` parameter (added in 4.10.0) available in `main_workflow.yaml` and `main_extended_workflow.yaml` as well diff --git a/aws_quickstart/README.md b/aws_quickstart/README.md index d957bf7f..43f39ad0 100644 --- a/aws_quickstart/README.md +++ b/aws_quickstart/README.md @@ -22,9 +22,6 @@ This template creates the following AWS resources required by the Datadog AWS in - The [Datadog Forwarder Lambda function](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring) to ship logs from S3 and CloudWatch, custom metrics and traces from Lambda functions to Datadog - The Datadog Forwarder only deploys to the AWS region where the AWS integration CloudFormation stack is launched. If you operate in multiple AWS regions, you can deploy the Forwarder stack (without the rest of the AWS integration stack) directly to other regions as needed. - The Datadog Forwarder is installed with default settings as a nested stack, edit the nested stack directly to update the forwarder specific settings. -- An EventBridge pipeline (connection, API destination, invocation role, and per-service CloudTrail rules) that forwards resource lifecycle events to Datadog so the Agent install feature can react in real time. - - Deployed only when `InstrumentationResourceTypes` is set. A rule is created for each listed resource type — listing `aws:ec2:instance` deploys the EC2 rule, listing `aws:eks:cluster` deploys the EKS rule. - - Single-region by design: the pipeline covers the AWS region the stack is deployed in. Deploy the stack in each region you want covered. ## Updating your CloudFormation Stack diff --git a/aws_quickstart/datadog_agent_install_forwarding.yaml b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml similarity index 98% rename from aws_quickstart/datadog_agent_install_forwarding.yaml rename to aws_quickstart/datadog_agent_resource_update_forwarding.yaml index a0d15533..7adc4a48 100644 --- a/aws_quickstart/datadog_agent_install_forwarding.yaml +++ b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml @@ -1,6 +1,6 @@ # version: AWSTemplateFormatVersion: 2010-09-09 -Description: Datadog AWS Integration - EventBridge forwarding for agent install +Description: Datadog AWS Integration - EventBridge forwarding of resource update events Parameters: APIKey: Description: >- diff --git a/aws_quickstart/main_v2.yaml b/aws_quickstart/main_v2.yaml index ad2cd370..b5723ae7 100644 --- a/aws_quickstart/main_v2.yaml +++ b/aws_quickstart/main_v2.yaml @@ -192,11 +192,11 @@ Resources: # EventBridge pipeline forwarding CloudTrail events to Datadog's instrumenter-events # intake. Deployed only when at least one InstrumentationResourceTypes value is set; # single-region (covers the region this stack is deployed in). - DatadogAgentInstallForwardingStack: + DatadogAgentResourceUpdateForwardingStack: Type: AWS::CloudFormation::Stack Condition: ShouldForwardEvents Properties: - TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_install_forwarding.yaml" + TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_resource_update_forwarding.yaml" Parameters: APIKey: !Ref APIKey APPKey: !Ref APPKey diff --git a/aws_quickstart/version.txt b/aws_quickstart/version.txt index bf721f57..63a54c41 100644 --- a/aws_quickstart/version.txt +++ b/aws_quickstart/version.txt @@ -1 +1 @@ -v4.11.0 +v4.12.0 From 3c39a7c6d78b57846d102c76b2e0b644c23222c2 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 16:03:04 -0400 Subject: [PATCH 09/17] [TON-466] chore(aws_quickstart): scrub prose references and drop authored comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drop "instrumenter-events" from connection and rule descriptions, the main_v2 comment, and the changelog entry. Remove the explanatory comments I added under Conditions and Resources (substring-trick and $or rationale). The intake URL itself stays — it's the actual ApiDestination endpoint. Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 2 +- ...atadog_agent_resource_update_forwarding.yaml | 17 +++-------------- aws_quickstart/main_v2.yaml | 2 +- 3 files changed, 5 insertions(+), 16 deletions(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index 549f97b7..76bee829 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,6 +1,6 @@ # 4.12.0 (May 19, 2026) -- Add an EventBridge → Datadog API destination pipeline that forwards CloudTrail events for the resource types listed in `InstrumentationResourceTypes`, enabling the Datadog Agent install feature to react to lifecycle changes in real time. When `InstrumentationResourceTypes` is non-empty, a nested stack provisions an `AWS::Events::Connection` (DD-API-KEY + DD-APPLICATION-KEY, both routed through Secrets Manager), an `AWS::Events::ApiDestination` pointing at `https://api.${DatadogSite}/api/unstable/instrumenter/events`, an invocation role, and one `AWS::Events::Rule` per UDM type in the list. v1 covers `aws:ec2:instance` (`RunInstances`, `CreateTags`/`DeleteTags` filtered to instance resource IDs) and `aws:eks:cluster` (`CreateCluster`, `TagResource`/`UntagResource` filtered to cluster ARNs). Single-region by design — deploy the stack in each region you want covered. Affects `main_v2.yaml`, `datadog_agent_resource_update_forwarding.yaml` (new) +- Add an EventBridge → Datadog API destination pipeline that forwards CloudTrail events for the resource types listed in `InstrumentationResourceTypes`, enabling the Datadog Agent install feature to react to lifecycle changes in real time. When `InstrumentationResourceTypes` is non-empty, a nested stack provisions an `AWS::Events::Connection` (DD-API-KEY + DD-APPLICATION-KEY, both routed through Secrets Manager), an `AWS::Events::ApiDestination` pointing at the per-site Datadog intake URL, an invocation role, and one `AWS::Events::Rule` per UDM type in the list. v1 covers `aws:ec2:instance` (`RunInstances`, `CreateTags`/`DeleteTags` filtered to instance resource IDs) and `aws:eks:cluster` (`CreateCluster`, `TagResource`/`UntagResource` filtered to cluster ARNs). Single-region by design — deploy the stack in each region you want covered. Affects `main_v2.yaml`, `datadog_agent_resource_update_forwarding.yaml` (new) # 4.11.0 (May 19, 2026) diff --git a/aws_quickstart/datadog_agent_resource_update_forwarding.yaml b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml index 7adc4a48..f6dba303 100644 --- a/aws_quickstart/datadog_agent_resource_update_forwarding.yaml +++ b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml @@ -27,10 +27,6 @@ Parameters: aws:eks:cluster) to forward CloudTrail events for. Only rules for the listed types are deployed. Conditions: - # CFN has no substring intrinsic in Conditions. Pad the joined string with the - # delimiter on both ends, split on the padded token, rejoin without the delimiter, - # and check whether the result differs from the original — if it does, the token - # was present. IncludeEC2: Fn::Not: - Fn::Equals: @@ -42,15 +38,11 @@ Conditions: - !Join ["", !Split [",aws:eks:cluster,", !Sub ",${InstrumentationResourceTypes},"]] - !Sub ",${InstrumentationResourceTypes}," Resources: - # EventBridge connection storing the Datadog API + APP key credentials used by - # the API destination below. Both headers are routed through Secrets Manager: - # DD-API-KEY via ApiKeyAuthParameters, DD-APPLICATION-KEY via - # InvocationHttpParameters.HeaderParameters with IsValueSecret=true. DDIntakeConnection: Type: AWS::Events::Connection Properties: Name: datadog-agent-resource-update-intake-connection - Description: Datadog instrumenter-events intake credentials + Description: Credentials for the Datadog resource update intake AuthorizationType: API_KEY AuthParameters: ApiKeyAuthParameters: @@ -91,15 +83,12 @@ Resources: - Effect: Allow Action: events:InvokeApiDestination Resource: !Sub "arn:${AWS::Partition}:events:*:${AWS::AccountId}:api-destination/*" - # $or splits each rule into a creation branch (no resource filter — the request - # doesn't carry the field we'd filter on) and a tag branch (filtered to the - # target resource type so we don't spam the intake with sibling resources). DDEventForwardingRuleEC2: Type: AWS::Events::Rule Condition: IncludeEC2 Properties: Name: datadog-agent-resource-update-rule-ec2 - Description: Forward EC2 CloudTrail events to the Datadog instrumenter-events intake + Description: Forward EC2 CloudTrail events to the Datadog resource update intake State: ENABLED EventPattern: source: @@ -129,7 +118,7 @@ Resources: Condition: IncludeEKS Properties: Name: datadog-agent-resource-update-rule-eks - Description: Forward EKS CloudTrail events to the Datadog instrumenter-events intake + Description: Forward EKS CloudTrail events to the Datadog resource update intake State: ENABLED EventPattern: source: diff --git a/aws_quickstart/main_v2.yaml b/aws_quickstart/main_v2.yaml index b5723ae7..213026ad 100644 --- a/aws_quickstart/main_v2.yaml +++ b/aws_quickstart/main_v2.yaml @@ -189,7 +189,7 @@ Resources: Parameters: DdApiKey: !Ref APIKey DdSite: !Ref DatadogSite - # EventBridge pipeline forwarding CloudTrail events to Datadog's instrumenter-events + # EventBridge pipeline forwarding CloudTrail events to the Datadog resource update # intake. Deployed only when at least one InstrumentationResourceTypes value is set; # single-region (covers the region this stack is deployed in). DatadogAgentResourceUpdateForwardingStack: From a0a0276ef1130c40a1310117c7b56aad9944bac9 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 16:07:50 -0400 Subject: [PATCH 10/17] [TON-466] docs(aws_quickstart): rewrite changelog entry at product level Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index 76bee829..aa8e5e2c 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,6 +1,6 @@ # 4.12.0 (May 19, 2026) -- Add an EventBridge → Datadog API destination pipeline that forwards CloudTrail events for the resource types listed in `InstrumentationResourceTypes`, enabling the Datadog Agent install feature to react to lifecycle changes in real time. When `InstrumentationResourceTypes` is non-empty, a nested stack provisions an `AWS::Events::Connection` (DD-API-KEY + DD-APPLICATION-KEY, both routed through Secrets Manager), an `AWS::Events::ApiDestination` pointing at the per-site Datadog intake URL, an invocation role, and one `AWS::Events::Rule` per UDM type in the list. v1 covers `aws:ec2:instance` (`RunInstances`, `CreateTags`/`DeleteTags` filtered to instance resource IDs) and `aws:eks:cluster` (`CreateCluster`, `TagResource`/`UntagResource` filtered to cluster ARNs). Single-region by design — deploy the stack in each region you want covered. Affects `main_v2.yaml`, `datadog_agent_resource_update_forwarding.yaml` (new) +- Add an EventBridge pipeline that forwards EC2 and EKS lifecycle CloudTrail events to Datadog, enabling the Datadog Agent install feature to react to resource changes in real time. Customers enable forwarding per resource type when configuring Agent install. Single-region by design — deploy the stack in each region you want covered. Affects `main_v2.yaml`, `datadog_agent_resource_update_forwarding.yaml` (new) # 4.11.0 (May 19, 2026) From 5ba9c4a3ea0801e0f8e1d90f2cd83b9ea3fc6c25 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 16:11:45 -0400 Subject: [PATCH 11/17] [TON-466] docs(aws_quickstart): trim changelog entry Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index aa8e5e2c..25b9d8fa 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,6 +1,6 @@ # 4.12.0 (May 19, 2026) -- Add an EventBridge pipeline that forwards EC2 and EKS lifecycle CloudTrail events to Datadog, enabling the Datadog Agent install feature to react to resource changes in real time. Customers enable forwarding per resource type when configuring Agent install. Single-region by design — deploy the stack in each region you want covered. Affects `main_v2.yaml`, `datadog_agent_resource_update_forwarding.yaml` (new) +- Add an EventBridge pipeline that forwards EC2 and EKS lifecycle CloudTrail events to Datadog, enabling the Datadog Agent install feature to react to resource changes in real time. Customers enable forwarding per resource type when configuring Agent install. # 4.11.0 (May 19, 2026) From 385eb108ac5ed5c9c5c5f5bd340738739dfc9bdd Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 16:25:30 -0400 Subject: [PATCH 12/17] [TON-466] docs(aws_quickstart): rename "Agent install" to "Agent management" Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index 25b9d8fa..bb25b78e 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,6 +1,6 @@ # 4.12.0 (May 19, 2026) -- Add an EventBridge pipeline that forwards EC2 and EKS lifecycle CloudTrail events to Datadog, enabling the Datadog Agent install feature to react to resource changes in real time. Customers enable forwarding per resource type when configuring Agent install. +- Add an EventBridge pipeline that forwards EC2 and EKS lifecycle CloudTrail events to Datadog, enabling the Datadog Agent management feature to react to resource changes in real time. Customers enable forwarding per resource type when configuring Agent management. # 4.11.0 (May 19, 2026) From 925d1a97650fa2634168f4480859b1053b473ce8 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Tue, 19 May 2026 16:45:18 -0400 Subject: [PATCH 13/17] [TON-466] feat(aws_quickstart): wire forwarding stack into main_extended and main_extended_workflow These two templates also need the InstrumentationResourceTypes parameter (originally added only to main_v2 in v4.10.0) plus the same gating, role-stack wiring, and conditional forwarding stack as main_v2. Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/main_extended.yaml | 18 ++++++++++++++++++ aws_quickstart/main_extended_workflow.yaml | 19 +++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/aws_quickstart/main_extended.yaml b/aws_quickstart/main_extended.yaml index 7c8c8434..bf89f36d 100644 --- a/aws_quickstart/main_extended.yaml +++ b/aws_quickstart/main_extended.yaml @@ -189,6 +189,11 @@ Conditions: Fn::Equals: - !Ref DisableResourceCollection - false + ShouldForwardEvents: + Fn::Not: + - Fn::Equals: + - !Join ["", !Ref InstrumentationResourceTypes] + - "" EnableAgentlessScanning: Fn::And: - Fn::Not: @@ -286,6 +291,19 @@ Resources: Parameters: DdApiKey: !Ref APIKey DdSite: !Ref DatadogSite + # EventBridge pipeline forwarding CloudTrail events to the Datadog resource update + # intake. Deployed only when at least one InstrumentationResourceTypes value is set; + # single-region (covers the region this stack is deployed in). + DatadogAgentResourceUpdateForwardingStack: + Type: AWS::CloudFormation::Stack + Condition: ShouldForwardEvents + Properties: + TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_resource_update_forwarding.yaml" + Parameters: + APIKey: !Ref APIKey + APPKey: !Ref APPKey + DatadogSite: !Ref DatadogSite + InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] Outputs: IAMRoleName: Description: AWS IAM Role named to be used with the DataDog AWS Integration diff --git a/aws_quickstart/main_extended_workflow.yaml b/aws_quickstart/main_extended_workflow.yaml index 6e29c866..1eb62956 100644 --- a/aws_quickstart/main_extended_workflow.yaml +++ b/aws_quickstart/main_extended_workflow.yaml @@ -219,6 +219,11 @@ Conditions: Fn::Equals: - !Ref DisableResourceCollection - false + ShouldForwardEvents: + Fn::Not: + - Fn::Equals: + - !Join ["", !Ref InstrumentationResourceTypes] + - "" EnableAgentlessScanning: Fn::And: - Fn::Not: @@ -556,6 +561,20 @@ Resources: stack_id: !Ref AWS::StackId forwarder_arn: !GetAtt ForwarderStack.Outputs.DatadogForwarderArn + # EventBridge pipeline forwarding CloudTrail events to the Datadog resource update + # intake. Deployed only when at least one InstrumentationResourceTypes value is set; + # single-region (covers the region this stack is deployed in). + DatadogAgentResourceUpdateForwardingStack: + Type: AWS::CloudFormation::Stack + Condition: ShouldForwardEvents + Properties: + TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_resource_update_forwarding.yaml" + Parameters: + APIKey: !Ref APIKey + APPKey: !Ref APPKey + DatadogSite: !Ref DatadogSite + InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] + # Step 6: Notify starting agentless scanning setup (if enabling) NotifyAgentlessScanningStarted: Type: Custom::WorkflowStatus From c464a1bec87232590421ce379e23f63de8b91155 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Wed, 20 May 2026 16:12:01 -0400 Subject: [PATCH 14/17] [TON-466] fix(aws_quickstart): shift forwarding stack to main_workflow (drop main_extended) main_extended.yaml isn't on the UI launch path; revert there and add the forwarding wiring to main_workflow.yaml alongside main_v2 and main_extended_workflow. Co-Authored-By: Claude Opus 4.7 (1M context) --- aws_quickstart/main_extended.yaml | 18 ------------------ aws_quickstart/main_workflow.yaml | 19 +++++++++++++++++++ 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/aws_quickstart/main_extended.yaml b/aws_quickstart/main_extended.yaml index bf89f36d..7c8c8434 100644 --- a/aws_quickstart/main_extended.yaml +++ b/aws_quickstart/main_extended.yaml @@ -189,11 +189,6 @@ Conditions: Fn::Equals: - !Ref DisableResourceCollection - false - ShouldForwardEvents: - Fn::Not: - - Fn::Equals: - - !Join ["", !Ref InstrumentationResourceTypes] - - "" EnableAgentlessScanning: Fn::And: - Fn::Not: @@ -291,19 +286,6 @@ Resources: Parameters: DdApiKey: !Ref APIKey DdSite: !Ref DatadogSite - # EventBridge pipeline forwarding CloudTrail events to the Datadog resource update - # intake. Deployed only when at least one InstrumentationResourceTypes value is set; - # single-region (covers the region this stack is deployed in). - DatadogAgentResourceUpdateForwardingStack: - Type: AWS::CloudFormation::Stack - Condition: ShouldForwardEvents - Properties: - TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_resource_update_forwarding.yaml" - Parameters: - APIKey: !Ref APIKey - APPKey: !Ref APPKey - DatadogSite: !Ref DatadogSite - InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] Outputs: IAMRoleName: Description: AWS IAM Role named to be used with the DataDog AWS Integration diff --git a/aws_quickstart/main_workflow.yaml b/aws_quickstart/main_workflow.yaml index 3aa6038b..f2721843 100644 --- a/aws_quickstart/main_workflow.yaml +++ b/aws_quickstart/main_workflow.yaml @@ -119,6 +119,11 @@ Conditions: Fn::Equals: - !Ref DisableResourceCollection - false + ShouldForwardEvents: + Fn::Not: + - Fn::Equals: + - !Join ["", !Ref InstrumentationResourceTypes] + - "" IsGov: Fn::Or: - Fn::Equals: @@ -412,6 +417,20 @@ Resources: stack_id: !Ref AWS::StackId forwarder_arn: !GetAtt ForwarderStack.Outputs.DatadogForwarderArn + # EventBridge pipeline forwarding CloudTrail events to the Datadog resource update + # intake. Deployed only when at least one InstrumentationResourceTypes value is set; + # single-region (covers the region this stack is deployed in). + DatadogAgentResourceUpdateForwardingStack: + Type: AWS::CloudFormation::Stack + Condition: ShouldForwardEvents + Properties: + TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_resource_update_forwarding.yaml" + Parameters: + APIKey: !Ref APIKey + APPKey: !Ref APPKey + DatadogSite: !Ref DatadogSite + InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] + # Step 6: Final notification - stack complete (with forwarder) NotifyStackCompleteWithForwarder: Type: Custom::WorkflowStatus From 5a88ea27887837ddb6aa63b28a5146296d1ffe56 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Wed, 27 May 2026 10:34:12 -0400 Subject: [PATCH 15/17] [TON-466][TON-473] feat(aws_quickstart): forward EC2/EKS non-tag update events Extends the forwarding pipeline (4.12.0) to also forward EC2 ModifyInstanceAttribute and EKS UpdateClusterConfig / UpdateClusterVersion CloudTrail events. These represent queryable-field changes that affect Agent management rule evaluation but previously only reached Datadog via the hourly reconciler. Bumps to v4.13.0. Co-Authored-By: Claude Opus 4.7 --- aws_quickstart/CHANGELOG.md | 4 ++++ aws_quickstart/datadog_agent_resource_update_forwarding.yaml | 3 +++ aws_quickstart/version.txt | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index bb25b78e..0b417621 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,3 +1,7 @@ +# 4.13.0 (May 27, 2026) + +- Extend the EventBridge pipeline (added in 4.12.0) to forward additional EC2 instance and EKS cluster update events, so the Datadog Agent management feature reacts to more types of resource changes in real time. + # 4.12.0 (May 19, 2026) - Add an EventBridge pipeline that forwards EC2 and EKS lifecycle CloudTrail events to Datadog, enabling the Datadog Agent management feature to react to resource changes in real time. Customers enable forwarding per resource type when configuring Agent management. diff --git a/aws_quickstart/datadog_agent_resource_update_forwarding.yaml b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml index f6dba303..a47f0425 100644 --- a/aws_quickstart/datadog_agent_resource_update_forwarding.yaml +++ b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml @@ -101,6 +101,7 @@ Resources: $or: - eventName: - RunInstances + - ModifyInstanceAttribute - eventName: - CreateTags - DeleteTags @@ -131,6 +132,8 @@ Resources: $or: - eventName: - CreateCluster + - UpdateClusterConfig + - UpdateClusterVersion - eventName: - TagResource - UntagResource diff --git a/aws_quickstart/version.txt b/aws_quickstart/version.txt index 63a54c41..c4475d31 100644 --- a/aws_quickstart/version.txt +++ b/aws_quickstart/version.txt @@ -1 +1 @@ -v4.12.0 +v4.13.0 From 864284b7f3bdd4d22a71cba340781d663ae870d5 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Wed, 27 May 2026 10:46:50 -0400 Subject: [PATCH 16/17] [TON-466] chore(aws_quickstart): drop main_v2 wiring, collapse changelog to single entry - Remove ShouldForwardEvents condition and DatadogAgentResourceUpdateForwardingStack resource from main_v2.yaml. The template is deprecated and no longer the UI launch path; forwarding ships via main_workflow.yaml and main_extended_workflow.yaml. - Collapse the two staged CHANGELOG entries (forwarding pipeline + non-tag update events) into a single v4.13.0 entry; revert version.txt to v4.13.0. Co-Authored-By: Claude Opus 4.7 --- aws_quickstart/CHANGELOG.md | 6 +----- aws_quickstart/main_v2.yaml | 18 ------------------ aws_quickstart/version.txt | 2 +- 3 files changed, 2 insertions(+), 24 deletions(-) diff --git a/aws_quickstart/CHANGELOG.md b/aws_quickstart/CHANGELOG.md index b8a93c21..66a1995f 100644 --- a/aws_quickstart/CHANGELOG.md +++ b/aws_quickstart/CHANGELOG.md @@ -1,10 +1,6 @@ -# 4.14.0 (May 27, 2026) - -- Extend the EventBridge pipeline (added in 4.13.0) to forward additional EC2 instance and EKS cluster update events, so the Datadog Agent management feature reacts to more types of resource changes in real time. - # 4.13.0 (May 27, 2026) -- Add an EventBridge pipeline that forwards EC2 and EKS lifecycle CloudTrail events to Datadog, enabling the Datadog Agent management feature to react to resource changes in real time. Customers enable forwarding per resource type when configuring Agent management. +- Add an EventBridge pipeline that forwards EC2 and EKS resource lifecycle and configuration update events to Datadog, enabling the Datadog Agent management feature to react to changes in real time. Customers enable forwarding per resource type when configuring Agent management. # 4.12.0 (May 27, 2026) diff --git a/aws_quickstart/main_v2.yaml b/aws_quickstart/main_v2.yaml index 213026ad..3e86464c 100644 --- a/aws_quickstart/main_v2.yaml +++ b/aws_quickstart/main_v2.yaml @@ -130,11 +130,6 @@ Conditions: Fn::Equals: - !Ref DisableResourceCollection - false - ShouldForwardEvents: - Fn::Not: - - Fn::Equals: - - !Join ["", !Ref InstrumentationResourceTypes] - - "" IsGov: Fn::Or: - Fn::Equals: @@ -189,19 +184,6 @@ Resources: Parameters: DdApiKey: !Ref APIKey DdSite: !Ref DatadogSite - # EventBridge pipeline forwarding CloudTrail events to the Datadog resource update - # intake. Deployed only when at least one InstrumentationResourceTypes value is set; - # single-region (covers the region this stack is deployed in). - DatadogAgentResourceUpdateForwardingStack: - Type: AWS::CloudFormation::Stack - Condition: ShouldForwardEvents - Properties: - TemplateURL: "https://.s3.amazonaws.com/aws//datadog_agent_resource_update_forwarding.yaml" - Parameters: - APIKey: !Ref APIKey - APPKey: !Ref APPKey - DatadogSite: !Ref DatadogSite - InstrumentationResourceTypes: !Join [",", !Ref InstrumentationResourceTypes] Outputs: IAMRoleName: Description: AWS IAM Role named to be used with the DataDog AWS Integration diff --git a/aws_quickstart/version.txt b/aws_quickstart/version.txt index cabad0ce..c4475d31 100644 --- a/aws_quickstart/version.txt +++ b/aws_quickstart/version.txt @@ -1 +1 @@ -v4.14.0 +v4.13.0 From 52369c8399dec3d34a5418698dabe847cf46d519 Mon Sep 17 00:00:00 2001 From: Raymond Eah Date: Wed, 27 May 2026 10:56:44 -0400 Subject: [PATCH 17/17] [TON-466] fix(aws_quickstart): drop fixed RoleName so stack deploys in multiple regions IAM role names are account-global. With an explicit RoleName the second-region deploy of the same template fails with EntityAlreadyExists. Letting CloudFormation auto-generate the name lets customers deploy the integration in every region they want covered. Co-Authored-By: Claude Opus 4.7 --- aws_quickstart/datadog_agent_resource_update_forwarding.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/aws_quickstart/datadog_agent_resource_update_forwarding.yaml b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml index a47f0425..59123d71 100644 --- a/aws_quickstart/datadog_agent_resource_update_forwarding.yaml +++ b/aws_quickstart/datadog_agent_resource_update_forwarding.yaml @@ -64,7 +64,6 @@ Resources: DDEventBridgeInvocationRole: Type: AWS::IAM::Role Properties: - RoleName: datadog-agent-resource-update-eventbridge-invocation-role AssumeRolePolicyDocument: Version: 2012-10-17 Statement: