feat(storage): S3/MinIO provider + config-driven selection (DEV-48)#12
Merged
Conversation
…tion - Implement s3Storage (aws-sdk-go-v2) for AWS S3 and MinIO (DEV-48) - Add storagex.New factory; select provider from BUCKET_PROVIDER config - Neutralise GetObjectAttrs to a provider-agnostic storagex.ObjectAttrs - Add S3Config + BUCKET_NAME/S3_* env wiring; drop hardcoded "mpiper" bucket - MinIO support via S3_ENDPOINT_URL + path-style addressing - Add MinIO round-trip integration test (skips without S3_TEST_ENDPOINT) - Ignore local e2e artifacts (.env.local, .secrets, compose override)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First child of the S3 epic (DEV-41). Adds a Go S3/MinIO
StorageXimplementation and makes provider selection config-driven.Changes
s3.go—s3Storageviaaws-sdk-go-v2implementing the fullStorageXinterface; presigned URLs vias3.PresignClient; MinIO support throughS3_ENDPOINT_URL+ path-style addressing.factory.go—storagex.New(...)selects GCS vs S3 from the configured provider.GetObjectAttrsnow returns a provider-agnosticstoragex.ObjectAttrsinstead of the GCS-specific*storage.ObjectAttrs(the single caller only checks existence). Unblocks any non-GCS impl.S3Config+S3_*/BUCKET_NAMEenv vars; service now picks provider fromBUCKET_PROVIDER(was hardcodedGCPProvider) and uses the configured bucket (was hardcoded"mpiper").Spec divergences (deliberate)
The DEV-48 issue's interface signature was stale — the real
StorageXtakesbucketper call, uses option structs, and had the GCS-typedGetObjectAttrs. Implemented against the actual interface; neutralised the GCS leak.Verification
go build ./...+go vetclean.s3_test.go): put → head → get (body match) → presign (signed, contains key) → delete → post-delete miss. Verified green against a realminio/miniocontainer; skips whenS3_TEST_ENDPOINTis unset (so CI stays green).Next: DEV-49 (Python worker S3) + add MinIO to compose → fully-local credential-free e2e.
🤖 Generated with Claude Code