Skip to content

feat(aws): add AWS Marketplace target to s3-cloud #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
pcfreak30 wants to merge 1 commit into
base: develop
Choose a base branch
from
Draft

feat(aws): add AWS Marketplace target to s3-cloud #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
237 changes: 237 additions & 0 deletions .github/workflows/packer-ci-aws.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
# CI workflow for AWS builds.
# Triggered by packer-ci.yml dispatcher via workflow_dispatch (gh workflow run --ref).
# Builds an AMI, validates it, cleans up on failure.
# AMIs are kept until PR close (dispatcher sends cleanup-mode=close) or
# push-to-develop (ephemeral, cleaned inline).

name: CI AWS

on:
workflow_call:
inputs:
snapshot-prefix:
description: 'Snapshot prefix (e.g. pinner-s3-aws-pr-123- or pinner-s3-aws-ci-)'
required: true
type: string
cleanup-mode:
description: 'Cleanup mode: none, inline (delete after validate), or close (delete all PR AMIs)'
required: false
default: 'inline'
type: string
pr-number:
description: 'PR number (for close cleanup)'
required: false
type: string
rescue-password:
description: 'Set root password for serial console rescue access (debug only, never for release)'
required: false
type: string
workflow_dispatch:
inputs:
snapshot-prefix:
description: 'Snapshot prefix (e.g. pinner-s3-aws-pr-123- or pinner-s3-aws-ci-)'
required: true
type: string
cleanup-mode:
description: 'Cleanup mode: none, inline (delete after validate), or close (delete all PR AMIs)'
required: false
default: 'inline'
type: choice
options:
- 'inline'
- 'none'
- 'close'
pr-number:
description: 'PR number (for close cleanup)'
required: false
type: string
rescue-password:
description: 'Set root password for serial console rescue access (debug only, never for release)'
required: false
type: string

concurrency:
group: ci-aws-${{ github.ref }}
cancel-in-progress: false

jobs:
build-and-validate:
if: inputs.cleanup-mode != 'close'
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
env:
VENDOR: aws
SNAPSHOT_PREFIX: ${{ inputs.snapshot-prefix }}
CLEANUP_MODE: ${{ inputs.cleanup-mode }}
PR_NUMBER: ${{ inputs.pr-number }}
steps:
- name: Checkout repo
uses: actions/checkout@v5
with:
fetch-depth: 0

- name: Setup SSH key
env:
CI_BUILD_SSH_KEY: ${{ secrets.CI_BUILD_SSH_KEY }}
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh
if echo "$CI_BUILD_SSH_KEY" | base64 -d > ~/.ssh/id_ed25519 2>/dev/null; then
: # base64 decoded
else
echo "$CI_BUILD_SSH_KEY" > ~/.ssh/id_ed25519
fi
chmod 600 ~/.ssh/id_ed25519

- name: Setup Packer
uses: hashicorp/setup-packer@v3
with:
version: latest

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Discover VPC and Subnet
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
run: |
echo "==> Discovering default VPC..."
VPC_ID=$(aws ec2 describe-vpcs --filters "Name=isDefault,Values=true" --query 'Vpcs[0].VpcId' --output text)
if [ "$VPC_ID" = "None" ] || [ -z "$VPC_ID" ]; then
echo "Error: No default VPC found. Please create a default VPC or specify VPC_ID explicitly."
exit 1
fi
echo "VPC_ID=$VPC_ID"

echo "==> Discovering public subnet in AZ that supports t3.small..."
# Query subnets and their AZs, then filter out us-east-1e (no t3 support)
SUBNET_INFO=$(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=map-public-ip-on-launch,Values=true" \
--query 'Subnets[*].[SubnetId,AvailabilityZone]' --output text)

SUBNET_ID=""
while read -r sub az; do
if [ "$az" != "us-east-1e" ]; then
SUBNET_ID="$sub"
echo "Found subnet $SUBNET_ID in AZ $az"
break
fi
done <<< "$SUBNET_INFO"

if [ -z "$SUBNET_ID" ]; then
echo "Error: No suitable public subnet found outside us-east-1e."
exit 1
fi

echo "VPC_ID=$VPC_ID" >> "$GITHUB_ENV"
echo "SUBNET_ID=$SUBNET_ID" >> "$GITHUB_ENV"

- name: Build AMI
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
PKR_VAR_snapshot_prefix: ${{ steps.config.outputs.snapshot-prefix }}
PKR_VAR_vpc_id: ${{ steps.vpc.outputs.vpc_id }}
PKR_VAR_subnet_id: ${{ steps.vpc.outputs.subnet_id }}
run: |
cd deploy/"$VENDOR"
make build SNAPSHOT_PREFIX="$SNAPSHOT_PREFIX" VPC_ID="$VPC_ID" SUBNET_ID="$SUBNET_ID"
Comment thread
kody-ai[bot] marked this conversation as resolved.

- name: Validate AMI
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
VPC_ID: ${{ env.VPC_ID }}
SUBNET_ID: ${{ env.SUBNET_ID }}
RESCUE_PASSWORD: ${{ github.event.inputs.rescue-password }}

@kody-ai kody-ai Bot Jul 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Bug high

The github.event.inputs.rescue-password context is empty for workflow_call triggers, silently dropping the operator-supplied rescue password and making the cleanup-on-failure guard always true, which deregisters a failed AMI even when the operator intended to keep it for debugging. Replace github.event.inputs.rescue-password with inputs.rescue-password on both lines.

          RESCUE_PASSWORD: ${{ inputs.rescue-password }}
...
        if: failure() && inputs.rescue-password == ''
Prompt for LLM 

File .github/workflows/packer-ci-aws.yml:

Line 156:

The `github.event.inputs.rescue-password` context is empty for `workflow_call` triggers, silently dropping the operator-supplied rescue password and making the `cleanup-on-failure` guard always true, which deregisters a failed AMI even when the operator intended to keep it for debugging. Replace `github.event.inputs.rescue-password` with `inputs.rescue-password` on both lines.

Suggested Code:

          RESCUE_PASSWORD: ${{ inputs.rescue-password }}
...
        if: failure() && inputs.rescue-password == ''

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

@kody-ai kody-ai Bot Jul 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Bug high

The github.event.inputs.rescue-password context is not populated under workflow_call triggers, leaving RESCUE_PASSWORD empty and causing the cleanup-on-failure guard to always deregister the instance even when a rescue password was supplied. Use inputs.rescue-password instead of github.event.inputs.rescue-password on both lines, matching the job's existing use of inputs.cleanup-mode and inputs.pr-number.

          RESCUE_PASSWORD: ${{ inputs.rescue-password }}
...
        if: failure() && inputs.rescue-password == ''
Prompt for LLM 

File .github/workflows/packer-ci-aws.yml:

Line 156:

The `github.event.inputs.rescue-password` context is not populated under `workflow_call` triggers, leaving `RESCUE_PASSWORD` empty and causing the `cleanup-on-failure` guard to always deregister the instance even when a rescue password was supplied. Use `inputs.rescue-password` instead of `github.event.inputs.rescue-password` on both lines, matching the job's existing use of `inputs.cleanup-mode` and `inputs.pr-number`.

Suggested Code:

          RESCUE_PASSWORD: ${{ inputs.rescue-password }}
...
        if: failure() && inputs.rescue-password == ''

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

run: |
cd deploy/"$VENDOR"
make validate VPC_ID="$VPC_ID" SUBNET_ID="$SUBNET_ID"

- name: Upload manifest
if: always()
uses: actions/upload-artifact@v4
with:
name: manifest-aws
path: deploy/aws/manifest.json
if-no-files-found: ignore
retention-days: 7

- name: Cleanup on success (inline mode)
if: success() && inputs.cleanup-mode == 'inline'
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
cd deploy/"$VENDOR"
if [ -f manifest.json ]; then
echo "==> Cleaning up AMI (inline mode)..."
make cleanup-snapshot || echo "Warning: manifest cleanup failed, daily prune will catch it"
Comment thread
kody-ai[bot] marked this conversation as resolved.
fi

- name: Cleanup on failure
if: failure() && github.event.inputs.skip-cleanup == 'false'
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
cd deploy/"$VENDOR"
if [ -f manifest.json ]; then
echo "==> Cleaning up AMI from failed build/validate..."
make cleanup-snapshot || echo "Warning: manifest cleanup failed, daily prune will catch it"
fi
# Prune any orphaned AMIs with this prefix
python3 scripts/prune-snapshots.py --max-age-hours 0 --prefix "$SNAPSHOT_PREFIX" 2>/dev/null \
|| echo "Warning: prune fallback failed, daily prune will catch it"

cleanup:
if: inputs.cleanup-mode == 'close'
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
env:
VENDOR: aws
PR_NUMBER: ${{ inputs.pr-number }}
steps:
- name: Checkout repo
uses: actions/checkout@v5
with:
fetch-depth: 1

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Delete PR AMIs
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_DEFAULT_REGION: us-east-1
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
if [ -z "$PR_NUMBER" ]; then
echo "No PR number, skipping cleanup."
exit 0
fi
PREFIX="pinner-s3-aws-pr-${PR_NUMBER}-"
echo "==> Cleaning up AMIs with prefix: $PREFIX"
cd deploy/aws
python3 scripts/prune-snapshots.py --max-age-hours 0 --prefix "$PREFIX"
36 changes: 33 additions & 3 deletions .github/workflows/packer-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,15 @@ on:
workflow_dispatch:
inputs:
vendor:
description: 'Vendor to build (digitalocean, vultr, or all)'
description: 'Vendor to build (digitalocean, vultr, aws, or all)'
required: false
default: 'all'
type: choice
options:
- 'all'
- 'digitalocean'
- 'vultr'
- 'aws'
skip-cleanup:
description: 'Skip cleanup after build (keep snapshot for manual testing)'
required: false
Expand All @@ -56,6 +57,10 @@ on:
options:
- 'true'
- 'false'
rescue-password:
description: 'Set root password for serial console rescue access (AWS debug only)'
required: false
type: string

concurrency:
group: packer-ci-${{ github.ref }}
Expand All @@ -82,7 +87,7 @@ jobs:
run: |
# PR close: cleanup all vendors
if [ "$GITHUB_EVENT_NAME" = "pull_request" ] && [ "$GITHUB_EVENT_ACTION" = "closed" ]; then
echo 'vendors=["digitalocean","vultr"]' >> "$GITHUB_OUTPUT"
echo 'vendors=["digitalocean","vultr","aws"]' >> "$GITHUB_OUTPUT"
echo "is-closed=true" >> "$GITHUB_OUTPUT"
echo "Detected vendors (PR close): all"
exit 0
Expand All @@ -93,7 +98,7 @@ jobs:
if [ "$GITHUB_EVENT_NAME" = "workflow_dispatch" ]; then
INPUT="${{ github.event.inputs.vendor }}"
if [ "$INPUT" = "all" ]; then
echo 'vendors=["digitalocean","vultr"]' >> "$GITHUB_OUTPUT"
echo 'vendors=["digitalocean","vultr","aws"]' >> "$GITHUB_OUTPUT"
else
echo "vendors=[\"$INPUT\"]" >> "$GITHUB_OUTPUT"
fi
Expand All @@ -116,6 +121,9 @@ jobs:
if echo "$CHANGED" | grep -q '^deploy/vultr/'; then
VENDORS="${VENDORS}vultr "
fi
if echo "$CHANGED" | grep -q '^deploy/aws/'; then
VENDORS="${VENDORS}aws "
fi

# Shared files affect ALL vendors — add them additively.
if echo "$CHANGED" | grep -qE '^(packer/scripts/|deploy/shared/|docker-compose\.yml|sidecar/)'; then
Expand All @@ -125,6 +133,9 @@ jobs:
if ! echo "$VENDORS" | grep -q 'vultr'; then
VENDORS="${VENDORS}vultr "
fi
if ! echo "$VENDORS" | grep -q 'aws'; then
VENDORS="${VENDORS}aws "
fi
fi

# Build JSON array from space-separated list
Expand Down Expand Up @@ -210,6 +221,7 @@ jobs:
additional_files: >-
deploy/${{ matrix.vendor }}/scripts/900-cleanup.sh
deploy/${{ matrix.vendor }}/scripts/validate.sh
deploy/${{ matrix.vendor }}/scripts/001_provision.sh
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.

Comment thread
kody-ai[bot] marked this conversation as resolved.
- name: Shellcheck (DO application-tag, digitalocean only)
if: matrix.vendor == 'digitalocean'
Expand All @@ -228,6 +240,12 @@ jobs:
deploy/shared/files/var/lib/cloud/scripts/per-instance/001_onboot
deploy/shared/files/etc/update-motd.d/99-one-click

- name: Shellcheck (repo-level AWS validator)
if: matrix.vendor == 'aws'
uses: ludeeus/action-shellcheck@master
with:
additional_files: 'scripts/validate-aws.sh'

- name: Python compile check
run: python3 -m py_compile deploy/${{ matrix.vendor }}/submit.py deploy/${{ matrix.vendor }}/scripts/prune-snapshots.py deploy/${{ matrix.vendor }}/scripts/prune-instances.py

Expand Down Expand Up @@ -330,3 +348,15 @@ jobs:
-f cleanup-mode="$CLEANUP_MODE" \
-f pr-number="$PR_NUMBER"
echo "Dispatched CI Vultr on ref $REF"

dispatch-aws:
Comment thread
kody-ai[bot] marked this conversation as resolved.
needs: [detect-vendors, check-permissions]
if: always() && !cancelled() && needs.check-permissions.outputs.authorized == 'true' && contains(needs.detect-vendors.outputs.vendors, 'aws') && needs.detect-vendors.outputs.is-closed != 'true'
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
uses: ./.github/workflows/packer-ci-aws.yml
secrets: inherit
with:
snapshot-prefix: pinner-s3-aws-pr-${{ github.event.pull_request.number }}-
cleanup-mode: none
pr-number: ${{ github.event.pull_request.number }}
Comment thread
kody-ai[bot] marked this conversation as resolved.
rescue-password: ${{ github.event.inputs.rescue-password }}
Comment thread
kody-ai[bot] marked this conversation as resolved.

Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Comment thread
kody-ai[bot] marked this conversation as resolved.
Loading
Loading