Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,65 @@ jobs:
path: dist/modules/check-*/progress.json
if-no-files-found: ignore
retention-days: 7

runtime:
name: Runtime simulation
needs: test
runs-on: ubuntu-24.04
timeout-minutes: 45
env:
POWERTOOLS_ACCEPTANCE_DIR: ${{ github.workspace }}/dist/runtime-acceptance
GOMAXPROCS: '2'
GOFLAGS: '-p=1'
GOMEMLIMIT: 128MiB
GOGC: '20'
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: go.mod
cache-dependency-path: '**/go.sum'
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
with:
node-version: '22.21.1'
- uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
python-version: '3.14'
enable-cache: false
- name: Test runtime evidence safeguards
run: uv run --no-project python tools/test_runtime_ci.py
- name: Install pinned TypeScript reference dependencies
run: npm ci --prefix tools/reference
- name: Build both architectures and run Lambda, streaming, and Valkey simulations
run: uv run --no-project python integration/local/run.py --skip-module-checks
- name: Verify Batch logs and spans from this run
run: uv run --no-project python integration/local/batch_report.py
- name: Build KMS interoperability executable
run: go build -trimpath -o dist/local/amd64/kmsinterop ./integration/kmsinterop
env:
GOOS: linux
GOARCH: amd64
- name: Verify Go ciphertext with the pinned TypeScript provider
working-directory: tools/reference
run: node generate-datamasking-kms.mjs --verify-go ../../dist/local/amd64/kmsinterop
- name: Validate and summarize current-commit runtime evidence
if: always()
run: uv run --no-project python integration/local/ci_report.py
- name: Ensure simulations preserve checked-in acceptance history
run: git diff --exit-code
- name: Upload runtime acceptance and diagnostics
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: runtime-simulation-${{ github.sha }}-${{ github.run_id }}-${{ github.run_attempt }}
path: |
dist/runtime-acceptance/*.json
dist/runtime-acceptance/*.md
dist/local/*.json
dist/local/*.log
dist/local/streaming/*.json
dist/local/streaming/*.log
if-no-files-found: warn
retention-days: 7
1 change: 1 addition & 0 deletions docs/CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,3 +278,4 @@ Race-detector execution is intentionally excluded because CGO must remain disabl
- [x] SIGNER-BODY-HANDOFF: Preserve asynchronous base-transport body ownership for custom signers returning the input request or a clone sharing its body. Four deterministic success/error regressions reproduced premature closure before the fix; packaged Signer tests, vet, tidy verification and its independent consumer build pass with CGO disabled. Distinct/empty bodies and nil signing results retain cleanup coverage. See issue #25 and [Signer ownership](SIGNER.md) (2026-10-05).
- [x] MODULE-REVIEW-20261005: Review the main entry points and critical state/ownership/error/decoding paths of the 26 maintained public modules excluding Logger. Twenty-five native Windows packaged checks and standalone consumers passed with CGO disabled; the exact-baseline hosted Linux run passed all 31 repository modules, including KMS and frozen X-Ray. Register issues #25–#28 before production changes; merge the verified Signer fix before starting the shared decoding fix. Local Docker runtime acceptance was unavailable. See [the review scope and evidence](MODULE_REVIEW_2026_10_05.md).
- [x] UTF8-DECODING: Reuse Commons.DecodeUTF8 in Parser, Parameters and JMESPath while preserving each reference path's BOM and gzip fallback rules. Generate 56 expected cases from the pinned TypeScript 2.35.0 packages: 20 Parser, 16 Parameters and 20 JMESPath cases. New regressions failed before the fix; all three affected packaged modules passed tests, vet, tidy and standalone consumer builds afterward, with CGO disabled and GOWORK=off. Parameters additionally verifies Auto transforms and untransformed raw bytes. See issues #26–#28 and [the review report](MODULE_REVIEW_2026_10_05.md) (2026-10-05).
- [x] CI-RUNTIME: Add native PR/main runtime simulation after packaged checks, fresh amd64/arm64 integration builds, Docker amd64 RIE/streaming/Valkey execution, saved Batch checks and Go-to-TypeScript KMS interoperability. PR #32's Go CI run 37233122495 passed all 31 modules / 28 consumers and 904 + 95 + 14 + 108 runtime assertions with CGO disabled, no cleanup errors, and preserved checked-in acceptance history. Ten offline evidence safeguards, 37 release and 22 automation regressions, actionlint and licenses pass. Require runtime success on preparation PR/main release gates and add the observed GitHub Actions check to the existing active main ruleset, preserving the other rules. No cloud AWS or arm64 execution is claimed. See issue #31 and [the hosted acceptance summary](CI_RUNTIME_ACCEPTANCE.json) (2026-10-05).
46 changes: 46 additions & 0 deletions docs/CI_RUNTIME_ACCEPTANCE.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"execution": {
"source_sha": "bf024a84d1317262d80da2f94d664d6a5d048a91",
"run_id": "37233122495",
"run_attempt": "1"
},
"runtime_architecture": "amd64",
"binary_architectures": [
"amd64",
"arm64"
],
"cgo_enabled": false,
"module_checks": "reused from prerequisite Modules and Lambda artifacts job",
"suites": {
"Lambda RIE and Valkey": {
"passed": true,
"assertions": 904
},
"Streaming": {
"passed": true,
"assertions": 95
},
"Batch logs and spans": {
"passed": true,
"assertions": 14
},
"Go to TypeScript KMS": {
"passed": true,
"assertions": 108
}
},
"binary_sha256": {
"amd64/bootstrap": "165758ad2d24e7e4288cbf34ec173ea8ba7da4a033634c319041836af9893c90",
"amd64/stream-bootstrap": "ab01acbad6032dcdeeb7d6932fc95482dfd66fa0202c3c9eaf0841be451ec53a",
"arm64/bootstrap": "752fbe75ae5ad7b0427a458f158b07cb78000ec446943bd65ab6e5a4fef78eb3",
"arm64/stream-bootstrap": "6e7a7f5f19ea98a64dc21c89821ecd2bb50b6f55bc708aab1abf2a4bfc4bcc01",
"amd64/capture": "474b5493d9b79476cfa66396c0935a7447e3a68937821981fe18bf88cd98d50d",
"amd64/kmsinterop": "eb184304aa6f626ad6971a4c34ae88220d01f7d74c5b28c7d6a773fd9c117ac4"
},
"errors": [],
"passed": true,
"recorded_at": "2026-10-05",
"workflow_url": "https://github.com/rambow-cloud/powertools-lambda-go/actions/runs/37233122495",
"pull_request": 32,
"pull_request_head_sha": "e1b7b21c04c055eba4f430f98742eea6c69002d2"
}
39 changes: 39 additions & 0 deletions docs/LOCAL_INTEGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,45 @@

Latest recorded runtime acceptance (2026-10-04, Logger TextMarshaler follow-up): 904/904 Lambda RIE assertions, 95/95 streaming checks and 14/14 saved Batch checks, including six new assertions for empty-string TextMarshaler values/errors. All 31 packaged modules and 28 public consumers passed one complete check; the same flow rebuilt normal and streaming handlers for Linux amd64/arm64 with CGO disabled. Docker executed amd64. Docker Desktop was restarted before validation after an engine HTTP 500; a private temporary container kept the engine active during the complete run and was removed afterward. Runtime metadata records that module checks and builds both executed. See [packaged evidence](MODULE_ACCEPTANCE_LOGGER.json), [acceptance history](LOCAL_VALIDATION.md) and [runtime evidence](LOCAL_ACCEPTANCE.json). No AWS resources were used; exhaustive parity and browser review remain separate gates.

## GitHub Actions

First hosted runtime acceptance (2026-10-05): PR #32's native
[Go CI run](https://github.com/rambow-cloud/powertools-lambda-go/actions/runs/37233122495)
passed all 31 packaged modules, 28 public consumers, 904 RIE/Valkey assertions,
95 streaming checks, 14 Batch checks, and 108 KMS reverse-interop checks.
The runtime rebuilt both architectures, executed amd64, preserved checked-in
reports, and cleaned up without errors. The [acceptance summary](CI_RUNTIME_ACCEPTANCE.json)
records the tested PR merge SHA, run/attempt, suite counts, and six binary hashes;
it is distinct from the historical local acceptance above.

Go CI runs **Runtime simulation** on every pull request, main push, and manual
dispatch after **Modules and Lambda artifacts** passes. It installs Node.js
22.21.1 and reference dependencies with `npm ci`, then uses
`integration/local/run.py --skip-module-checks`: module tests are reused from
the prerequisite job while normal and streaming fixtures are freshly built for
Linux amd64 and arm64. Docker executes amd64 Lambda RIE, streaming, and the live
TypeScript/Go Valkey bridge. Batch checks consume that same run's logs/spans;
the separate KMS executable emits Go ciphertext for the pinned TypeScript
provider's reverse interoperability checks. All endpoints and credentials are
synthetic; no AWS secrets or cloud resources are required.

CI sets `POWERTOOLS_ACCEPTANCE_DIR` to the ignored `dist/runtime-acceptance`
directory so Streaming, Batch, and KMS reports preserve the checked-in local
acceptance history. Each report identifies its commit, workflow run, and attempt.
`integration/local/ci_report.py` rejects missing/empty/failed suites, stale
identity, reused builds, missing binaries, Batch artifact hash mismatches, and
cleanup errors. It writes a job summary and SHA-256 hashes for all six binaries.
The `runtime-simulation-SHA-RUN-ATTEMPT` artifact retains reports and diagnostic
logs for seven days, including on failure; build caches and binaries are excluded.

Both merge protection and release publication must require **Runtime simulation**.
See [maintainer setup](MAINTAINING.md) for rollout. Checked-in Go reference
corpora run in the module job; the full Node fixture regeneration command remains
a deliberate development operation when reference versions or generators change.
Runtime simulation does not establish arm64 execution or live AWS acceptance.

## Local execution

Run from the repository root with local Go, Python 3.10+ (or `uv`), Node.js with the pinned reference dependencies, and Docker Desktop using Linux containers:

```powershell
Expand Down
16 changes: 12 additions & 4 deletions docs/MAINTAINING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ merge requirements. This guide does not claim those settings are already active.
that issue, Summary, and Testing filled in. There is no special owner,
`no-issue`, label, or bootstrap bypass in the policy check.
3. Verify the first **PR contribution policy**, **Modules and Lambda artifacts**,
and **Build documentation** runs on that PR's latest revision. Inspect the
**Runtime simulation**, and **Build documentation** runs on that PR's latest revision. Inspect the
workflow diff before approving a fork workflow run.
4. Configure the protection below **before merging the bootstrap PR**, selecting
the three checks from their observed successful runs. Do not require a check
the four checks from their observed successful runs. Do not require a check
that has never run or was renamed. Confirm the PR's merge box applies the rules.
5. Merge the reviewed bootstrap PR through the protected PR interface. Forms
become available once they are on the default branch; verify the New issue
Expand All @@ -38,8 +38,8 @@ needed for basic PR and status-check protection.
only one active maintainer. Do not enable code-owner or last-push approval
requirements until another eligible reviewer is available
- Require conversation resolution and these exact, stable status-check job names:
**PR contribution policy**, **Modules and Lambda artifacts**, and
**Build documentation**. Select GitHub Actions as the expected source where
**PR contribution policy**, **Modules and Lambda artifacts**,
**Runtime simulation**, and **Build documentation**. Select GitHub Actions as the expected source where
available, and require the branch to be up to date before merging
- Block force pushes and deletion of `main`
- Leave the bypass list empty, including repository administrators. For classic
Expand All @@ -56,6 +56,14 @@ and [author review limitation](https://docs.github.com/en/pull-requests/how-tos/

## Triage and safe CI

When introducing **Runtime simulation** to an existing protected repository,
first verify its native PR run and commit-associated artifacts. Then add that
exact job name to the existing main ruleset's required status checks, selecting
GitHub Actions as the expected source. Preserve every existing rule and bypass
setting. Enable the check before merging the CI rollout PR. The publication
tool also requires runtime success on both the preparation PR and its exact
merged main commit; missing, pending, failed, or skipped checks cannot publish.

Use issue comments to record accepted scope and remaining questions. Optional
labels are organizational only; no custom label creation, project board,
assignment, CLA, or signing service is required to start contributing. If a
Expand Down
3 changes: 2 additions & 1 deletion docs/RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ recorded source must still be its first parent; if main advances first, rerun
preparation against current main and use the newly generated PR.

With automatic publication enabled, merging this preparation PR authorizes its
frozen release batch. After both required main checks pass, GoReleaser publishes
frozen release batch. After all three required main checks pass (Modules and Lambda
artifacts, Runtime simulation, and Build documentation), GoReleaser publishes
in dependency order and the tracking issue closes after public consumer checks.
Ordinary feature/bug PR merges do not publish modules.

Expand Down
5 changes: 4 additions & 1 deletion integration/local/batch_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
import json
from pathlib import Path

from run import acceptance_path, execution_identity

root = Path(__file__).resolve().parents[2]
directory = root / "dist/local"
report = json.loads((directory / "report.json").read_text(encoding="utf-8"))
Expand Down Expand Up @@ -48,7 +50,8 @@ def check(name, value):

check("three successful invocation probes", sum("batch" in value["response"] for value in report["invocations"]) == 3)
summary = {"source": "existing local Docker artifacts; no additional invocations", "passed": all(item["passed"] for item in checks), "checks": checks, "sha256": {name: hashlib.sha256((directory / name).read_bytes()).hexdigest() for name in ("report.json", "otlp.json", "lambda.log")}}
(root / "docs/BATCH_ACCEPTANCE.json").write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")
summary["execution"] = execution_identity()
acceptance_path("BATCH_ACCEPTANCE.json").write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")
failed = [item["name"] for item in checks if not item["passed"]]
print(f"Batch composition: {len(checks) - len(failed)}/{len(checks)} checks passed")
if failed:
Expand Down
Loading
Loading