Self-Managed Kubernetes Cluster on AWS

This repository contains Terraform Infrastructure as Code (IaC) for deploying a production-ready, self-managed Kubernetes cluster on AWS with GitOps capabilities.

Note: This infrastructure is explicitly developed for the G11 Blog application and includes all necessary Kubernetes operators and components required to run it.

Features

• Self-Managed Kubernetes Cluster: Multi-master, multi-worker architecture with auto-scaling
• Service Mesh: Istio v1.28.1 for traffic management and observability
• GitOps & Progressive Delivery: ArgoCD and Argo Rollouts for continuous deployment
• TLS & Security: Cert-Manager, Let's Encrypt, and Sealed Secrets
• Storage & Database: AWS EFS CSI Driver and PostgreSQL Operator (Crunchy Data PGO)
• Monitoring: Metrics Server for Horizontal Pod Autoscaling
• Pre-configured Environments: Production (g11-prod) and staging (g11-stg) namespaces ready to use

Prerequisites

• Terraform >= 1.0
• AWS CLI configured with appropriate credentials
• An S3 bucket for Terraform state storage
• A registered domain name with Route53 hosted zone

Quick Start

1. Generate SSH Keys

Create an SSH key pair for accessing the EC2 instances:

ssh-keygen -t ed25519 -f key.pem -C "k8s-cluster-key" -N ""

2. Generate Sealed Secrets Certificates (Optional)

Generate custom certificates for Sealed Secrets. If skipped, the controller will auto-generate its own.

# Generate private key
openssl genrsa -out sealed-secrets-key.pem 4096

# Generate public certificate
openssl req -new -x509 -key sealed-secrets-key.pem \
  -out sealed-secrets-cert.pem \
  -days 3650 \
  -subj "/CN=sealed-secret/O=sealed-secret"

Important: Keep sealed-secrets-key.pem secure and add it to .gitignore. The certificate is automatically uploaded to S3 during cluster initialization.

For more details, see Sealed Secrets Setup.

3. Configure Variables

Copy the example variables file and customize it for your environment:

cp .tfvars.example my-environment.tfvars

Edit my-environment.tfvars with your specific configuration values.

4. Configure Backend (State Storage)

Create a backend configuration file for remote state storage in S3.

Example: state.config Create state-prod.config:

bucket = "your-terraform-state-bucket"
key    = "your-project-prod.tfstate"
region = "us-east-1"

5. Initialize Terraform

# For demo environment
terraform init -reconfigure -backend-config=state-demo.config

# For production environment
terraform init -reconfigure -backend-config=state-prod.config

6. Deploy Infrastructure

# For demo
terraform apply -var-file="demo.tfvars"

# For production
terraform apply -var-file="prod.tfvars"

Post-Deployment

Access the Cluster

1. List running instances:

   aws ec2 describe-instances \
     --region us-east-1 \
     --filters "Name=instance-state-name,Values=running" \
     --query 'Reservations[*].Instances[*].[Tags[?Key==`Name`].Value | [0], PublicDnsName, PublicIpAddress, InstanceId]' \
     --output table

2. SSH into the master node:

   ssh -i key.pem admin@<public-dns>

3. Verify cluster status:

   export KUBECONFIG=/etc/kubernetes/admin.conf
   kubectl get nodes -o wide

Get ArgoCD Admin Password

kubectl get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" -n argocd | base64 -d

Access ArgoCD at https://argocd.<your-domain>

Installed Components

This infrastructure automatically installs and configures all operators and components required for the G11 Blog application.

Core System Components

• kube-system: Kubernetes core, AWS EFS CSI Driver, Metrics Server, Sealed Secrets
• istio-system: Istio v1.28.1 service mesh (control plane, CNI, ingress gateway)
• argocd: ArgoCD for GitOps with Argo Rollouts for progressive delivery
• cluster-autoscaler: Pod-aware node auto-scaling
• cert-manager: TLS certificate management with Let's Encrypt
• postgres-operator: Crunchy Data PostgreSQL Operator

Application Namespaces

• g11-prod: Production environment with PostgreSQL (hippo) and Istio injection
• g11-stg: Staging environment with PostgreSQL (hippo) and Istio injection

Additional Tools

• Calico v3.31.0 for pod networking
• Helm v3.x for package management

Cleanup

To destroy the infrastructure:

# For demo
terraform destroy -var-file="demo.tfvars"

# For production
terraform destroy -var-file="prod.tfvars"

Additional Documentation

• AWS Cloud Provider Setup
• Cluster Autoscaler Verification
• Sealed Secrets Setup

Related Repositories

• G11 Helm Charts - Application deployment manifests

Support

For issues and questions, please open an issue in the repository.