Infrastructure Drift Detector

🚨 Automated Infrastructure Drift Detection with CloudTrail Attribution

A serverless AWS solution that automatically detects infrastructure drift by comparing Terraform state files with actual AWS resources, identifies who made changes via CloudTrail, and sends detailed email notifications.

✨ Features

• 🔍 Automated Drift Detection - Scheduled scans compare Terraform state with actual AWS resources
• 📊 Real-Time Dashboard - Beautiful web UI with live metrics, charts, and detailed drift analysis
• 👤 Change Attribution - Enhanced CloudTrail integration identifies who made changes (including ad-hoc creators)
• 🧹 Intelligent Filtering - Zero-noise comparison ignores data sources and properties not captured by scans
• 📧 Premium Notifications - Stunning "Cyber-Premium" HTML emails and ultra-compact "Console-style" text reports
• 💥 Impact Analysis - Automatic assessment of drift impact and risk levels
• 🛠️ Remediation Guidance - Actionable recommendations and step-by-step fixes
• 📈 Historical Tracking - DynamoDB-backed persistence with trend analysis and compliance scoring
• 📊 Multi-State Support - Monitor multiple Terraform state files simultaneously
• ⚡ Serverless Architecture - Built on AWS Lambda for cost-effective operation
• 🎯 Resource Coverage - Supports EC2, S3, RDS, Lambda, IAM, VPC, and more
• ☁️ Pure CloudFormation - Deploy entirely via CloudFormation, no shell scripts

🏗️ Architecture

Components

Drift Detection Engine:

• Lambda function orchestrates the entire drift detection process
• Scheduled Trigger: EventBridge rule triggers scans every 6 hours by default (configurable)
• Event-Based Trigger: (Optional) EventBridge rule triggers scans on specific AWS resource changes (EC2, S3, RDS, etc.)
• CloudWatch captures all logs and metrics

State Management:

• Terraform state files stored in S3
• Deployment artifacts stored in separate S3 bucket
• Lambda layer contains Python dependencies

Resource Monitoring:

• Scans EC2 instances, S3 buckets, IAM roles, and more
• Compares actual AWS state with Terraform state
• Identifies configuration drift

Change Attribution:

• CloudTrail provides event history
• Identifies who made changes and when
• Includes IP address, user agent, and request details

Notifications:

• SNS topic sends email alerts
• Beautiful HTML emails with drift details
• Includes severity, impact, and remediation steps

CI/CD Pipeline

Automated Deployment:

• GitHub Actions builds and tests code
• Security scanning (Bandit, Safety)
• Multi-environment deployment (dev → staging → prod)
• Automatic rollback on failures

┌─────────────────┐
│  EventBridge    │  Triggers every 6 hours by default (configurable)
│   Schedule      │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Lambda         │  Drift Detector Function
│  Function       │  - Reads Terraform state from S3
└────────┬────────┘  - Scans actual AWS resources
         │           - Compares and identifies drift
         │           - Queries CloudTrail for attribution
         ▼
┌─────────────────┐
│  CloudTrail     │  Identifies who made changes
│  Analyzer       │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  SNS Topic      │  Sends email notifications
│  (Email)        │
└─────────────────┘

📋 Prerequisites

• Terraform state files in S3 (see docs/PREREQUISITES.md)
• AWS CLI configured with appropriate credentials
• IAM permissions for Lambda, SNS, CloudTrail, etc.
• Python 3.11+ (for packaging dependencies)

Note: You do NOT need Terraform installed! Only the state files.

🚀 Quick Start

Option 1: GitHub Actions CI/CD (Recommended)

1. Fork this repository
2. Configure GitHub Secrets (see CI/CD Setup)
3. Push to develop branch → Auto-deploys to Development
4. Push to main branch → Auto-deploys to Staging
5. Manual workflow dispatch → Deploy to Production

See .github/workflows/README.md for detailed CI/CD documentation.

Option 2: Automated Script Deployment

# Configure (edit these or set as environment variables)
export TERRAFORM_STATE_BUCKET="your-terraform-state-bucket"
export TERRAFORM_STATE_KEYS="prod/terraform.tfstate"
export EMAIL_ADDRESSES="your-email@example.com"
export SCAN_SCHEDULE="rate(30 minutes)"
export ENABLE_EVENT_BASED_SCANNING="true"

# Deploy
chmod +x scripts/deploy.sh
./scripts/deploy.sh

Option 3: Manual CloudFormation Deployment

See docs/DEPLOY-CLOUDFORMATION.md for step-by-step manual deployment.

Cleanup

# Delete all resources
chmod +x scripts/cleanup.sh
./scripts/cleanup.sh

📧 Email Notification Example

🚨 DRIFT ALERT: CRITICAL [3 Items]
──────────────────────────────────────────────────────────────────────
📅 2026-01-27 14:19:48 UTC | ⏱️ 3.89s | 📦 57 Scanned
📈 🔴:1 🟠:0 🟡:0 🟢:2

1. ✏️ aws_instance [MODIFIED]
🆔 ID    : i-0f949261ac6ac8d6e
👤 Actor : ashish @ 2026-01-27 13:00:45 UTC
⚡ Risk  : CRITICAL / SECURITY
🔧 Changes:
  ↳ Property 'instance_type' changed from 't2.micro' to 't2.nano'
  ↳ Property 'public_ip' changed from '98.89.27.141' to '54.235.239.53'
🛠️ Remediation:
  1. Review the detected changes in detail
  2. Update Terraform configuration to match desired state
  3. Run: terraform plan -out=tfplan

[... more items ...]
──────────────────────────────────────────────────────────────────────
🤖 Infra Drift Detector | Visit dashboard for full trace analysis.
──────────────────────────────────────────────────────────────────────

📊 Supported AWS Resources

The drift detector currently monitors 8 AWS resource types:

• Compute: EC2 Instances, Lambda Functions
• Storage: S3 Buckets
• Database: RDS Instances
• Networking: VPCs, Subnets, Security Groups
• IAM: IAM Roles

Each resource type includes comprehensive attribute monitoring and CloudTrail event tracking.

For complete details, see docs/MONITORING-CAPABILITIES.md

📁 Project Structure

infrastructure-drift-detector/
├── scripts/
│   ├── deploy.sh                    # Automated deployment
│   ├── cleanup.sh                   # Automated cleanup
│   └── test-deployment.sh           # Deployment validation
│
├── cloudformation/
│   ├── drift-detector-stack.yaml    # CloudFormation template
│   └── iam-policy.json              # IAM permissions reference
│
├── src/                             # Lambda function source
│   ├── drift_detector.py            # Core drift detection engine
│   ├── terraform_state_reader.py    # State file parser (S3 + local)
│   ├── aws_scanner.py               # Resource scanner (EC2/S3/RDS/IAM/VPC...)
│   ├── cloudtrail_analyzer.py       # Change attribution
│   ├── impact_analyzer.py           # Impact assessment & remediation
│   ├── notification_service.py      # Email & Slack notifications
│   ├── persistence_service.py       # DynamoDB scan history
│   ├── dashboard_api.py             # API Gateway handler for dashboard
│   ├── utils/
│   │   └── logger.py                # Structured logging
│   └── templates/
│       └── drift_alert_email.html   # Jinja2 email template
│
├── frontend/
│   └── index.html                   # Dashboard SPA (served via Lambda)
│
├── config/
│   └── settings.py                  # Pydantic settings (env vars)
│
├── docs/                            # Documentation
│   ├── QUICK-START.md               # 5-minute quick start
│   ├── DEPLOY-CLOUDFORMATION.md     # CloudFormation guide
│   ├── PREREQUISITES.md             # Prerequisites
│   ├── DEPLOYMENT.md                # Detailed deployment
│   ├── ARCHITECTURE.md              # Architecture deep dive
│   ├── MONITORING-CAPABILITIES.md   # Monitored resources & events
│   ├── TRIGGER-METHODS.md           # How to trigger scans
│   ├── CI-CD-SETUP.md               # CI/CD pipeline setup
│   └── PROJECT-STRUCTURE.md        # Project structure overview
│
├── tests/
│   └── test_fixes.py                # Pytest test suite
│
├── lambda_handler.py                # Lambda entry point
├── requirements.txt                 # Python dependencies
└── README.md                        # This file

💰 Cost Estimate

Approximate monthly costs (us-east-1):

Scan Interval	Lambda	CloudWatch	SNS	S3	Total
6 hours (default)	~$0.05	~$0.50	~$0.50	~$0.10	~$1.15/month
30 minutes	~$0.20	~$0.50	~$0.50	~$0.10	~$1.30/month

🗑️ Cleanup

# Automated cleanup
./scripts/cleanup.sh

# Or manual cleanup
aws cloudformation delete-stack --stack-name infrastructure-drift-detector
aws cloudformation wait stack-delete-complete --stack-name infrastructure-drift-detector

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
aws s3 rb "s3://drift-detector-deployment-${ACCOUNT_ID}" --force

🔐 Security

• ✅ Least privilege IAM roles
• ✅ Encrypted environment variables
• ✅ No hardcoded credentials
• ✅ CloudWatch logging enabled
• ✅ S3 bucket encryption recommended
• ✅ Automated security scanning (Bandit, Safety)
• ✅ Code quality checks (Black, Flake8, MyPy)

🔄 CI/CD Setup

GitHub Actions Workflows

This project includes comprehensive CI/CD pipelines:

• Deploy - Automated deployment to dev/staging/prod
• PR Validation - Code quality, security, and build checks
• Cleanup - Safe infrastructure teardown
• Scheduled Tests - Weekly drift detection testing

Required GitHub Secrets

Configure these in Settings → Secrets and variables → Actions:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

# Development
DEV_TERRAFORM_STATE_BUCKET
DEV_TERRAFORM_STATE_KEYS
DEV_EMAIL_ADDRESSES

# Staging
STAGING_TERRAFORM_STATE_BUCKET
STAGING_TERRAFORM_STATE_KEYS
STAGING_EMAIL_ADDRESSES

# Production
PROD_TERRAFORM_STATE_BUCKET
PROD_TERRAFORM_STATE_KEYS
PROD_EMAIL_ADDRESSES

Deployment Workflow

develop branch → Development environment (auto)
main branch    → Staging environment (auto)
Manual trigger → Production environment (requires approval)

See .github/workflows/README.md for complete CI/CD documentation.

📚 Documentation

• docs/MONITORING-CAPABILITIES.md - What's monitored, CloudTrail history, notification behavior
• docs/TRIGGER-METHODS.md - How to trigger drift detection (scheduled, manual, CLI)
• docs/ARCHITECTURE.md - Architecture diagrams and detailed design
• docs/QUICK-START.md - 5-minute quick start guide
• docs/DEPLOY-CLOUDFORMATION.md - Pure CloudFormation deployment
• docs/CI-CD-SETUP.md - CI/CD pipeline setup
• docs/PREREQUISITES.md - Prerequisites and setup guide
• docs/DEPLOYMENT.md - Detailed deployment instructions
• docs/CLOUDFORMATION-READY.md - Template validation results
• docs/PROJECT-STRUCTURE.md - Project structure overview

🧪 Testing

# Run tests
pytest tests/test_fixes.py -v

# Run with coverage
pytest tests/test_fixes.py --cov=src --cov-report=html

📝 Recent Code Quality Improvements

This project recently underwent code quality improvements:

• ✅ Zero-Noise Comparison - Only alerts on properties present in both expected and actual states
• ✅ Data Source Protection - Automatically filters out "ghost drift" from non-managed resources
• ✅ Deep Attribution - Scans CloudTrail response elements to identify ad-hoc resource creators
• ✅ Premium Reporting - "Cyber-Premium" dark-mode HTML templates and compact terminal alerts
• ✅ Template-Based Emails - Jinja2 templates for better separation of concerns

See CODE_QUALITY_IMPROVEMENTS.md for details.

🤝 Contributing

This is a production-ready solution for infrastructure drift detection. Feel free to customize for your specific needs.

---

Built with ❤️ for DevOps teams who care about infrastructure drift

Deployed with ☁️ CloudFormation - No shell scripts, no hacks!