Skip to content

Commit ec93bbb

Browse files
committed
Fix method to read
1 parent 2543bde commit ec93bbb

5 files changed

Lines changed: 87 additions & 26 deletions

File tree

README.md

Lines changed: 47 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,12 @@ go install github.com/richardartoul/gobuildcache@latest
3333

3434
## Usage
3535

36+
You can configure `gobuildcache` by passing arguments directly to the program via `GOCACHEPROG`, or by using environment variables. The CLI argument approach is recommended for CI/CD workflows.
37+
38+
### Using CLI Arguments (Recommended)
39+
3640
```bash
37-
export GOCACHEPROG=gobuildcache
41+
export GOCACHEPROG="gobuildcache --backend=disk"
3842
go build ./...
3943
go test ./...
4044
```
@@ -43,63 +47,93 @@ By default, `gobuildcache` uses an on-disk cache stored in the OS default tempor
4347

4448
For "production" use-cases in CI, you'll want to configure `gobuildcache` to use S3 Express One Zone, Google Cloud Storage, or a similarly low latency distributed backend.
4549

50+
### Using Environment Variables (Alternative)
51+
52+
You can also configure `gobuildcache` using environment variables:
53+
54+
```bash
55+
export GOCACHEPROG=gobuildcache
56+
export GOBUILDCACHE_BACKEND_TYPE=disk
57+
go build ./...
58+
go test ./...
59+
```
60+
61+
**Note:** CLI arguments take precedence over environment variables when both are provided.
62+
4663
### Using S3
4764

65+
**Using CLI arguments (recommended):**
66+
4867
```bash
49-
export GOBUILDCACHE_BACKEND_TYPE=s3
50-
export GOBUILDCACHE_S3_BUCKET=$BUCKET_NAME
68+
export GOCACHEPROG="gobuildcache --backend=s3 --s3-bucket=$BUCKET_NAME --s3-prefix=go-build-cache/"
69+
export AWS_REGION=$BUCKET_REGION
70+
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY
71+
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
72+
go build ./...
73+
go test ./...
5174
```
5275

53-
You'll also have to provide AWS credentials. `gobuildcache` embeds the AWS V2 S3 SDK so any method of providing credentials to that library will work, but the simplest is to use environment variables as demonstrated below.
76+
**Using environment variables:**
5477

5578
```bash
5679
export GOCACHEPROG=gobuildcache
5780
export GOBUILDCACHE_BACKEND_TYPE=s3
5881
export GOBUILDCACHE_S3_BUCKET=$BUCKET_NAME
82+
export GOBUILDCACHE_S3_PREFIX=go-build-cache/
5983
export AWS_REGION=$BUCKET_REGION
6084
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY
6185
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
6286
go build ./...
6387
go test ./...
6488
```
6589

90+
You'll need to provide AWS credentials. `gobuildcache` embeds the AWS V2 S3 SDK so any method of providing credentials to that library will work (environment variables, IAM roles, etc.).
91+
6692
> **Note**: All configuration environment variables support both `GOBUILDCACHE_<KEY>` and `<KEY>` forms (e.g., both `GOBUILDCACHE_S3_BUCKET` and `S3_BUCKET` work). The prefixed version takes precedence if both are set. The prefixed form is recommended to avoid conflicts with other tools. If the prefixed variable is set to an empty string, it falls through to the unprefixed version (or default).
6793
6894
### Using Google Cloud Storage (GCS)
6995

96+
**Using CLI arguments (recommended):**
97+
98+
```bash
99+
export GOCACHEPROG="gobuildcache --backend=gcs --gcs-bucket=$BUCKET_NAME --gcs-prefix=go-build-cache/"
100+
go build ./...
101+
go test ./...
102+
```
103+
104+
**Using environment variables:**
105+
70106
```bash
107+
export GOCACHEPROG=gobuildcache
71108
export GOBUILDCACHE_BACKEND_TYPE=gcs
72109
export GOBUILDCACHE_GCS_BUCKET=$BUCKET_NAME
110+
export GOBUILDCACHE_GCS_PREFIX=go-build-cache/
111+
go build ./...
112+
go test ./...
73113
```
74114

75115
GCS authentication uses Application Default Credentials. You can provide credentials in one of the following ways:
76116

77117
1. **Service Account JSON file** (recommended for CI):
78118
```bash
79119
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json
80-
export GOCACHEPROG=gobuildcache
81-
export GOBUILDCACHE_BACKEND_TYPE=gcs
82-
export GOBUILDCACHE_GCS_BUCKET=$BUCKET_NAME
120+
export GOCACHEPROG="gobuildcache --backend=gcs --gcs-bucket=$BUCKET_NAME"
83121
go build ./...
84122
go test ./...
85123
```
86124

87125
2. **Metadata service** (when running on GCP):
88126
```bash
89127
# No credentials file needed - uses metadata service automatically
90-
export GOCACHEPROG=gobuildcache
91-
export GOBUILDCACHE_BACKEND_TYPE=gcs
92-
export GOBUILDCACHE_GCS_BUCKET=$BUCKET_NAME
128+
export GOCACHEPROG="gobuildcache --backend=gcs --gcs-bucket=$BUCKET_NAME"
93129
go build ./...
94130
go test ./...
95131
```
96132

97133
3. **gcloud CLI credentials** (for local development):
98134
```bash
99135
gcloud auth application-default login
100-
export GOCACHEPROG=gobuildcache
101-
export GOBUILDCACHE_BACKEND_TYPE=gcs
102-
export GOBUILDCACHE_GCS_BUCKET=$BUCKET_NAME
136+
export GOCACHEPROG="gobuildcache --backend=gcs --gcs-bucket=$BUCKET_NAME"
103137
go build ./...
104138
go test ./...
105139
```

examples/github_actions_gcs.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
workflow_dispatch:
9+
10+
jobs:
11+
test:
12+
name: Run Tests
13+
runs-on: ubuntu-latest
14+
steps:
15+
- name: Checkout code
16+
uses: actions/checkout@v4
17+
- name: Set up Go
18+
uses: actions/setup-go@v5
19+
with:
20+
go-version: '1.25'
21+
# Use our own cache, obviously.
22+
cache: false
23+
- name: Install gobuildcache
24+
run: go install github.com/richardartoul/gobuildcache@latest
25+
- name: Authenticate to Google Cloud
26+
uses: google-github-actions/auth@v2
27+
with:
28+
credentials_json: ${{ secrets.GCP_SA_KEY }}
29+
- name: Run short tests
30+
env:
31+
GOCACHEPROG: "gobuildcache --backend=gcs --gcs-bucket=${{ secrets.GCS_BUCKET }} --gcs-prefix=go-build-cache/${{ github.repository }}"
32+
run: go test ./...

examples/github_actions_s3.yml

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,13 +24,10 @@ jobs:
2424
run: go install github.com/richardartoul/gobuildcache@latest
2525
- name: Run short tests
2626
env:
27-
GOBUILDCACHE_BACKEND_TYPE: s3
28-
GOBUILDCACHE_S3_BUCKET: EXAMPLE_BUCKET
29-
GOBUILDCACHE_S3_PREFIX: EXAMPLE_PREFIX
3027
AWS_REGION: EXAMPLE_REGION
3128
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
3229
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
33-
GOCACHEPROG: gobuildcache
30+
GOCACHEPROG: "gobuildcache --backend=s3 --s3-bucket=EXAMPLE_BUCKET --s3-prefix=EXAMPLE_PREFIX"
3431
run: go test ./...
3532

3633

examples/github_actions_tigris.yml

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,13 +24,10 @@ jobs:
2424
run: go install github.com/richardartoul/gobuildcache@latest
2525
- name: Run short tests
2626
env:
27-
GOBUILDCACHE_BACKEND_TYPE: s3
28-
GOBUILDCACHE_S3_BUCKET: EXAMPLE_BUCKET
29-
GOBUILDCACHE_S3_PREFIX: EXAMPLE_PREFIX
27+
AWS_ENDPOINT_URL_S3: https://t3.storage.dev
28+
AWS_ENDPOINT_URL_IAM: https://iam.storage.dev
3029
AWS_REGION: auto
3130
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
3231
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
33-
AWS_ENDPOINT_URL_S3: https://t3.storage.dev
34-
AWS_ENDPOINT_URL_IAM: https://iam.storage.dev
35-
GOCACHEPROG: gobuildcache
32+
GOCACHEPROG: "gobuildcache --backend=s3 --s3-bucket=EXAMPLE_BUCKET --s3-prefix=EXAMPLE_PREFIX"
3633
run: go test ./...

pkg/backends/gcs.go

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,10 @@ type GCS struct {
2828
func NewGCS(bucket, prefix string) (*GCS, error) {
2929
ctx := context.Background()
3030

31-
// Create GCS client using Application Default Credentials
32-
// This will use GOOGLE_APPLICATION_CREDENTIALS env var or metadata service
33-
client, err := storage.NewClient(ctx)
31+
// Create GCS client using Application Default Credentials.
32+
// WithJSONReads forces the JSON API for downloads (default is XML).
33+
// This is required for GCS Anywhere Cache compatibility.
34+
client, err := storage.NewClient(ctx, storage.WithJSONReads())
3435
if err != nil {
3536
return nil, fmt.Errorf("failed to create GCS client: %w", err)
3637
}

0 commit comments

Comments
 (0)