Skip to content

fix: update acceptance tests for staging #1208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
itslenny merged 6 commits into from
Jul 16, 2026
Merged

fix: update acceptance tests for staging #1208

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8e45f63
fix: update acceptance tests for staging
itslenny Jul 6, 2026
d03a4fa
Add acceptance test coverage for remote CDN
itslenny Jul 11, 2026
433f455
Update acceptance/README.md
itslenny Jul 14, 2026
a5d45ec
Update acceptance/specs/cdn.test.ts
itslenny Jul 14, 2026
32a47a7
test signed url cache invalidation, code cleanup
itslenny Jul 14, 2026
a5a0cf3
check ACCEPTANCE_ALLOW_DESTRUCTIVE for reset-projects script
itslenny Jul 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion acceptance/API_COVERAGE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,21 @@ These run in the `wire` profile in addition to smoke coverage:
| ---------- | ----------------------------------------------------------------------- |
| Wire/SigV4 | raw `aws-chunked` PutObject and UploadPart, trailer-signature rejection |

## CDN Edge Coverage

These assert real Cloudflare edge cache behavior (`cf-cache-status` MISS/HIT, ETag/304, purge
propagation) rather than the storage API's own request handling. There's no local equivalent -
`cf-cache-status` is only ever set on responses that pass through the deployed edge - so this
coverage only runs against a real remote target with CDN enabled (the `CDN Edge` capability below).

| Area | Covered APIs / behavior |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Cache MISS/HIT | MISS then HIT across public, authenticated, and signed access; signed invalid-token and authenticated anon-key denial still bypass the cache correctly |
| Conditional GET | ETag `If-None-Match` returns 304 on a cache HIT |
| Object write ops | UPSERT, MOVE, and COPY create fresh cache entries without disturbing unrelated entries (old MOVE path, COPY source); DELETE invalidates the cache entry |
| Transformations | distinct transform dimensions cache independently of each other and of the base object |
| Cache Purge | object, bucket, and tenant purge only invalidate their own scope; object/bucket/tenant transformation purge invalidates the transform but leaves the base object cached |

## Capability-Gated Coverage

These require target-specific capabilities. They run when the capability is enabled or derived from
Expand All @@ -38,7 +53,8 @@ the configured target, and the selected profile includes the spec:
| Capability | Enable with | Covered APIs / behavior |
| ---------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Admin | `ACCEPTANCE_ENABLE_ADMIN=true` plus admin URL/API key | admin status, API key protection, tenant read/create/patch/upsert/delete/health, tenant migration run/reset-current/jobs, queue migration enabled/disabled contract, queue/migration validation, JWKS validation/add/deactivate/status, orphan scan/sync validation, S3 credential create/authenticate/list/delete |
| CDN | `ACCEPTANCE_ENABLE_CDN=true` | `/cdn/:bucket/*` cache purge for existing objects and documented missing-object error |
| CDN | `ACCEPTANCE_ENABLE_CDN=true` | `/cdn/:bucket/*` and `/cdn` cache purge for objects, buckets, the tenant, and their transformations; purge succeeds even when the target object no longer exists |
| CDN Edge | Derived from `ACCEPTANCE_ENABLE_CDN=true` and `ACCEPTANCE_TARGET=remote` | Real Cloudflare edge cache behavior - see [CDN Edge Coverage](#cdn-edge-coverage) above |
| Render | `ACCEPTANCE_ENABLE_RENDER=true` | public, authenticated, and signed image transformation routes, `webp` output format, non-image input errors, invalid transformation validation |
| RLS | `ACCEPTANCE_ENABLE_RLS_SETUP=true` plus anon/authenticated keys and RLS resource config | authenticated allow and anon deny for read/write on configured policies |
| Path edges | Derived from `ACCEPTANCE_TARGET` and `STORAGE_BACKEND` | list-v2 preservation for object names with empty path segments; local S3/MinIO backends skip this case directly |
Expand Down
68 changes: 43 additions & 25 deletions acceptance/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ The file backend also works because imgproxy mounts the local `./data` directory

```bash
mkdir -p data
STORAGE_BACKEND=file ACCEPTANCE_ENABLE_RENDER=true npm run acceptance -- --profile full acceptance/specs/cdn-render.test.ts
STORAGE_BACKEND=file ACCEPTANCE_ENABLE_RENDER=true npm run acceptance -- --profile full acceptance/specs/render.test.ts
```

Local CI enables render tests for both S3 and file backend runs. The S3 matrix sets
Expand Down Expand Up @@ -121,30 +121,32 @@ secrets as environment secrets.

## Useful Configuration

| Variable | Meaning |
| --------------------------------- | ------------------------------------------------------------------------------------------------- |
| `ACCEPTANCE_BASE_URL` | REST base URL. Defaults to `http://127.0.0.1:5000`. |
| `ACCEPTANCE_PROFILE` | Acceptance profile to run. Sample env and local CI default to `full`. |
| `ACCEPTANCE_S3_ENDPOINT` | S3 endpoint. Defaults to `$ACCEPTANCE_BASE_URL/s3`. |
| `ACCEPTANCE_TUS_ENDPOINT` | TUS endpoint. Defaults to `$ACCEPTANCE_BASE_URL/upload/resumable`. |
| `ACCEPTANCE_X_FORWARDED_HOST` | Optional tenant-routing host header for multitenant targets. |
| `ACCEPTANCE_ADMIN_URL` | Admin API base URL for admin tests. |
| `ACCEPTANCE_SERVICE_KEY` | Service role JWT for REST tests. |
| `ACCEPTANCE_S3_ACCESS_KEY_ID` | S3 protocol access key. |
| `ACCEPTANCE_S3_SECRET_ACCESS_KEY` | S3 protocol secret. |
| `ACCEPTANCE_REGION` | S3 signing region. Defaults to `us-east-1`. |
| `ACCEPTANCE_RESOURCE_PREFIX` | Prefix for all resources created by this run. |
| `ACCEPTANCE_ENABLE_ADMIN` | Enables admin route tests. Requires admin URL and API key. |
| `ACCEPTANCE_ENABLE_CDN` | Enables CDN purge tests. Managed local runs provide a purge stub by default. |
| `ACCEPTANCE_ENABLE_RENDER` | Enables image transformation tests. |
| `ACCEPTANCE_ENABLE_RLS_SETUP` | Enables RLS tests; requires service, anon, authenticated keys and bucket/prefix policy resources. |
| `ACCEPTANCE_ENABLE_VECTOR` | Enables vector bucket API tests. Requires local pgvector or a configured S3 Vectors target. |
| `ACCEPTANCE_ENABLE_ICEBERG` | Enables Iceberg catalog API tests. |
| `ACCEPTANCE_ENABLE_WIRE` | Enables wire-level tests outside the `wire` / `full` profiles. |
| `ACCEPTANCE_RLS_BUCKET` | Bucket used by opt-in RLS tests. Defaults to local dummy `bucket2`. |
| `ACCEPTANCE_RLS_READ_OBJECT` | Existing object used for RLS read checks; unset self-provisions one under the write prefix. |
| `ACCEPTANCE_RLS_WRITE_PREFIX` | Prefix where authenticated role may upload and anon may not. |
| `ACCEPTANCE_ALLOW_DESTRUCTIVE` | Required for destructive tests when `ACCEPTANCE_TARGET=remote`. |
| Variable | Meaning |
| ----------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| `ACCEPTANCE_BASE_URL` | REST base URL. Defaults to `http://127.0.0.1:5000`. |
| `ACCEPTANCE_PROFILE` | Acceptance profile to run. Sample env and local CI default to `full`. |
| `ACCEPTANCE_S3_ENDPOINT` | S3 endpoint. Defaults to `$ACCEPTANCE_BASE_URL/s3`. |
| `ACCEPTANCE_TUS_ENDPOINT` | TUS endpoint. Defaults to `$ACCEPTANCE_BASE_URL/upload/resumable`. |
| `ACCEPTANCE_X_FORWARDED_HOST` | Optional tenant-routing host header for multitenant targets. |
| `ACCEPTANCE_ADMIN_URL` | Admin API base URL for admin tests. |
| `ACCEPTANCE_SERVICE_KEY` | Service role JWT for REST tests. |
| `ACCEPTANCE_S3_ACCESS_KEY_ID` | S3 protocol access key. |
| `ACCEPTANCE_S3_SECRET_ACCESS_KEY` | S3 protocol secret. |
| `ACCEPTANCE_REGION` | S3 signing region. Defaults to `us-east-1`. |
| `ACCEPTANCE_RESOURCE_PREFIX` | Prefix for all resources created by this run. |
| `ACCEPTANCE_ENABLE_ADMIN` | Enables admin route tests. Requires admin URL and API key. |
| `ACCEPTANCE_ADMIN_RETURN_TENANT_SENSITIVE_DATA` | Whether the admin API returns sensitive tenant fields (keys, database URL). Defaults to `true`. |
| `ACCEPTANCE_ADMIN_DATABASE_URL_OVERRIDE` | Database URL to provision tenants with when sensitive tenant data isn't returned (see above). |
| `ACCEPTANCE_ENABLE_CDN` | Enables CDN purge tests. Managed local runs provide a purge stub by default. |
| `ACCEPTANCE_ENABLE_RENDER` | Enables image transformation tests. |
| `ACCEPTANCE_ENABLE_RLS_SETUP` | Enables RLS tests; requires service, anon, authenticated keys and bucket/prefix policy resources. |
| `ACCEPTANCE_ENABLE_VECTOR` | Enables vector bucket API tests. Requires local pgvector or a configured S3 Vectors target. |
| `ACCEPTANCE_ENABLE_ICEBERG` | Enables Iceberg catalog API tests. |
| `ACCEPTANCE_ENABLE_WIRE` | Enables wire-level tests outside the `wire` / `full` profiles. |
| `ACCEPTANCE_RLS_BUCKET` | Bucket used by opt-in RLS tests. Defaults to local dummy `bucket2`. |
| `ACCEPTANCE_RLS_READ_OBJECT` | Existing object used for RLS read checks; unset self-provisions one under the write prefix. |
| `ACCEPTANCE_RLS_WRITE_PREFIX` | Prefix where authenticated role may upload and anon may not. |
| `ACCEPTANCE_ALLOW_DESTRUCTIVE` | Required for destructive tests when `ACCEPTANCE_TARGET=remote`. |

## HTTPS And Wire Tests

Expand All @@ -161,3 +163,19 @@ npm run acceptance:run -- --profile wire
`ACCEPTANCE_TLS_REJECT_UNAUTHORIZED=false` sets `NODE_TLS_REJECT_UNAUTHORIZED=0` in the runner
for local self-signed certificates. Do not use it for remote runs unless the target is explicitly
provisioned for that.

## Reset A Target Project

Repeated destructive runs against a shared target can leave behind buckets from crashed or interrupted runs.
The script `acceptance/scripts/reset-project.ts` wipes every storage, analytics (Iceberg), and vector bucket
in the target project so the next run starts clean. It preserves `ACCEPTANCE_RLS_BUCKET` (emptying its contents
but not deleting the bucket, since it carries hand-configured RLS policies the suite can't recreate).

The script reads the same `ACCEPTANCE_*` variables as the rest of the suite and does not load an
env file itself, so run it with `tsx --env-file`:

```bash
tsx --env-file=.env.acceptance-staging acceptance/scripts/reset-project.ts --yes
```

Omit `--yes` to preview what would be deleted without changing anything.
Loading
Loading