Skip to content

Commit a2d061f

Browse files
ServerSideHannesServerSideHannes
committed
docs: update README and add chart configuration reference
- Rewrite README to be more concise with streamlined diagrams - Add chart/README.md with complete helm values reference - Link to chart/README.md from main README
1 parent bff9e72 commit a2d061f

2 files changed

Lines changed: 91 additions & 77 deletions

File tree

‎README.md‎

Lines changed: 28 additions & 77 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
<h1 align="center">S3Proxy</h1>
88

99
<p align="center">
10-
<strong>Transparent encryption for S3. Zero code changes.</strong>
10+
<strong>Transparent client-side encryption for S3. Zero code changes.</strong>
1111
</p>
1212

1313
<p align="center">
@@ -21,23 +21,16 @@
2121

2222
## Overview
2323

24-
S3's server-side encryption is great, but your cloud provider holds the keys. With S3Proxy, **you** control encryption.
24+
S3's server-side encryption is great, but your cloud provider holds the keys. S3Proxy sits between your app and S3, encrypting everything **before** it leaves your infrastructure.
2525

2626
```
27-
┌─────────────────────────────────────────────────────────────────────────┐
28-
│ │
29-
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
30-
│ │ │ plain │ │ AES │ │ │
31-
│ │ Your App │ ──────▶ │ S3Proxy │ ──────▶ │ S3 │ │
32-
│ │ │ data │ │ 256 │ │ │
33-
│ └──────────┘ └──────────┘ └──────────┘ │
34-
│ │ │
35-
│ ┌─────┴─────┐ │
36-
│ │ You own │ │
37-
│ │ the keys │ │
38-
│ └───────────┘ │
39-
│ │
40-
└─────────────────────────────────────────────────────────────────────────┘
27+
┌──────────┐ ┌──────────┐ ┌──────────┐
28+
│ │ plain │ │ AES │ │
29+
│ Your App │ ──────▶ │ S3Proxy │ ──────▶ │ S3 │
30+
│ │ data │ │ 256 │ │
31+
└──────────┘ └──────────┘ └──────────┘
32+
│
33+
You own the keys.
4134
```
4235

4336
<p align="center">
@@ -63,11 +56,11 @@ helm install s3proxy oci://ghcr.io/serversidehannes/s3proxy-python/charts/s3prox
6356
aws s3 --endpoint-url http://s3proxy-python:4433 cp file.txt s3://bucket/
6457
```
6558

66-
> Clients use the **same credentials** configured on the proxy. See [How It Works](#how-it-works).
59+
That's it. Point any S3 client at the proxy, use the **same credentials** you configured above.
6760

6861
---
6962

70-
## Tested Integrations
63+
## Battle-Tested
7164

7265
<p align="center">
7366
<img src="https://img.shields.io/badge/PostgreSQL_17-336791?style=flat-square&logo=postgresql&logoColor=white" alt="PostgreSQL">
@@ -83,58 +76,27 @@ aws s3 --endpoint-url http://s3proxy-python:4433 cp file.txt s3://bucket/
8376
| ScyllaDB 6.x | Scylla Operator 1.19 | Scylla Manager |
8477
| ClickHouse 24.x | Altinity Operator | clickhouse-backup |
8578

86-
All verified: **backup → cluster delete → restore → data integrity check**
79+
All verified: **backup, cluster delete, restore, data integrity check.**
8780

8881
---
8982

9083
## How It Works
9184

92-
### Credential Flow
85+
**Credential flow** — S3 clients sign requests with their secret key. When S3Proxy encrypts the payload, the body changes and the original signature is invalidated. The proxy re-signs with the same key. Configure credentials once on the proxy, all clients use them.
9386

94-
```
95-
┌────────────┐ signs request ┌────────────┐ re-signs ┌────────────┐
96-
│ │ ─────────────────▶ │ │ ──────────────▶ │ │
97-
│ Client │ (your creds) │ S3Proxy │ (same creds) │ S3 │
98-
│ │ ◀───────────────── │ │ ◀────────────── │ │
99-
└────────────┘ decrypted └────────────┘ encrypted └────────────┘
100-
```
101-
102-
S3 clients sign requests with their secret key. When S3Proxy encrypts the payload, it changes the body—invalidating the original signature. The proxy must re-sign, which requires the secret key.
103-
104-
**→ Configure credentials once on the proxy. All clients use those same credentials.**
105-
106-
### Encryption Architecture
87+
**Envelope encryption** — Your master key derives a KEK (Key Encryption Key). Each object gets a random DEK (Data Encryption Key), encrypted with AES-256-GCM. The DEK is wrapped by the KEK and stored as object metadata. Your master key never touches S3.
10788

10889
```
109-
┌─────────────────────────────────────────────────────────────┐
110-
│ │
111-
│ Your Master Key │
112-
│ │ │
113-
│ ▼ │
114-
│ ┌───────────┐ │
115-
│ │ KEK │ Key Encryption Key (derived) │
116-
│ └─────┬─────┘ │
117-
│ │ wraps │
118-
│ ▼ │
119-
│ ┌───────────┐ │
120-
│ │ DEK │ Data Encryption Key (random per object) │
121-
│ └─────┬─────┘ │
122-
│ │ encrypts │
123-
│ ▼ │
124-
│ ┌───────────┐ │
125-
│ │ Data │ AES-256-GCM │
126-
│ └───────────┘ │
127-
│ │
128-
└─────────────────────────────────────────────────────────────┘
90+
Master Key → KEK (derived via SHA-256)
91+
└→ wraps DEK (random per object)
92+
└→ encrypts data (AES-256-GCM)
12993
```
13094

131-
Your master key never touches S3. DEKs are wrapped and stored as object metadata.
132-
13395
---
13496

13597
## Production Deployment
13698

137-
### With External Secrets
99+
### External Secrets (recommended)
138100

139101
```bash
140102
kubectl create secret generic s3proxy-secrets \
@@ -155,9 +117,7 @@ helm install s3proxy oci://ghcr.io/serversidehannes/s3proxy-python/charts/s3prox
155117
| Gateway | `http://s3-gateway.<ns>` |
156118
| Ingress | `https://s3proxy.example.com` |
157119

158-
### Health Checks
159-
160-
`GET /healthz` · `GET /readyz`
120+
Health: `GET /healthz` · `GET /readyz` · Metrics: `GET /metrics`
161121

162122
---
163123

@@ -168,37 +128,28 @@ helm install s3proxy oci://ghcr.io/serversidehannes/s3proxy-python/charts/s3prox
168128
| `replicaCount` | `3` | Pod replicas |
169129
| `s3.host` | `s3.amazonaws.com` | S3 endpoint (AWS, MinIO, R2, etc.) |
170130
| `s3.region` | `us-east-1` | AWS region |
171-
| `secrets.encryptKey` | — | AES-256 key (32 bytes) |
172-
| `secrets.awsAccessKeyId` | — | AWS access key |
173-
| `secrets.awsSecretAccessKey` | — | AWS secret key |
131+
| `secrets.encryptKey` | — | Encryption key |
174132
| `secrets.existingSecrets.enabled` | `false` | Use existing K8s secret |
175133
| `redis-ha.enabled` | `true` | Deploy embedded Redis HA |
176-
| `gateway.enabled` | `false` | Create `s3-gateway` service |
134+
| `gateway.enabled` | `false` | Create gateway service |
177135
| `ingress.enabled` | `false` | Enable ingress |
178-
| `performance.memoryLimitMb` | `64` | Memory budget for concurrency (Linux only) |
179-
180-
> **Note:** Memory-based concurrency limiting requires Linux. The `malloc_trim` syscall used to release memory back to the OS is not available on macOS.
136+
| `performance.memoryLimitMb` | `64` | Memory budget for streaming concurrency |
181137

182-
→ [chart/values.yaml](chart/values.yaml) for all options.
138+
See [chart/README.md](chart/README.md) for all options.
183139

184140
---
185141

186142
## FAQ
187143

188-
**Can I use existing unencrypted data?**
189-
Yes. S3Proxy detects unencrypted objects and returns them as-is. To migrate, copy through the proxy.
144+
**Can I use existing unencrypted data?** Yes. S3Proxy detects unencrypted objects and returns them as-is. Migrate by copying through the proxy.
190145

191-
**What if I lose my encryption key?**
192-
Data is unrecoverable. Back up your key.
146+
**What if I lose my encryption key?** Data is unrecoverable. Back up your key.
193147

194-
**What if Redis fails during multipart upload?**
195-
Upload fails and must restart. Use `redis-ha.enabled=true` with persistence.
148+
**What if Redis fails mid-upload?** Upload fails and must restart. Use `redis-ha.enabled=true` with persistence.
196149

197-
**Does it work with MinIO/R2/Spaces?**
198-
Yes. Set `s3.host` to your endpoint.
150+
**MinIO / R2 / Spaces?** Yes. Set `s3.host` to your endpoint.
199151

200-
**Presigned URLs?**
201-
GET works. PUT/POST don't (signature computed over plaintext, but proxy encrypts).
152+
**Presigned URLs?** GET works. PUT/POST don't — the proxy encrypts the body which invalidates the pre-signed signature.
202153

203154
---
204155

‎chart/README.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# S3Proxy Helm Chart
2+
3+
## Install
4+
5+
```bash
6+
helm install s3proxy oci://ghcr.io/serversidehannes/s3proxy-python/charts/s3proxy-python \
7+
--set secrets.encryptKey="your-key" \
8+
--set secrets.awsAccessKeyId="AKIA..." \
9+
--set secrets.awsSecretAccessKey="wJalr..."
10+
```
11+
12+
## Values
13+
14+
| Value | Default | Description |
15+
|-------|---------|-------------|
16+
| `replicaCount` | `3` | Pod replicas |
17+
| `image.repository` | `ghcr.io/ServerSideHannes/s3proxy-python` | Container image |
18+
| `image.tag` | `latest` | Image tag |
19+
| `image.pullPolicy` | `IfNotPresent` | Pull policy |
20+
| `s3.host` | `s3.amazonaws.com` | S3 endpoint |
21+
| `s3.region` | `us-east-1` | AWS region |
22+
| `server.port` | `4433` | Proxy listen port |
23+
| `server.noTls` | `true` | Disable TLS (in-cluster only) |
24+
| `performance.memoryLimitMb` | `64` | Memory budget for streaming |
25+
| `logLevel` | `DEBUG` | Log level |
26+
| `secrets.encryptKey` | `""` | AES-256 encryption key |
27+
| `secrets.awsAccessKeyId` | `""` | AWS access key |
28+
| `secrets.awsSecretAccessKey` | `""` | AWS secret key |
29+
| `secrets.existingSecrets.enabled` | `false` | Use pre-created K8s secret |
30+
| `secrets.existingSecrets.name` | `""` | Existing secret name |
31+
| `secrets.existingSecrets.keys.encryptKey` | `S3PROXY_ENCRYPT_KEY` | Key name in existing secret |
32+
| `secrets.existingSecrets.keys.awsAccessKeyId` | `AWS_ACCESS_KEY_ID` | Key name in existing secret |
33+
| `secrets.existingSecrets.keys.awsSecretAccessKey` | `AWS_SECRET_ACCESS_KEY` | Key name in existing secret |
34+
| `redis-ha.enabled` | `true` | Deploy embedded Redis HA |
35+
| `redis-ha.replicas` | `1` | Redis replicas |
36+
| `redis-ha.auth` | `false` | Enable Redis auth |
37+
| `redis-ha.haproxy.enabled` | `true` | Deploy HAProxy for Redis |
38+
| `redis-ha.persistentVolume.enabled` | `true` | Persistent storage |
39+
| `redis-ha.persistentVolume.size` | `10Gi` | Volume size |
40+
| `externalRedis.url` | `""` | External Redis URL |
41+
| `externalRedis.uploadTtlHours` | `24` | Upload state TTL |
42+
| `externalRedis.existingSecret` | `""` | K8s secret with Redis password |
43+
| `externalRedis.passwordKey` | `redis-password` | Key name in Redis secret |
44+
| `service.type` | `ClusterIP` | Service type |
45+
| `service.port` | `4433` | Service port |
46+
| `ingress.enabled` | `false` | Enable ingress |
47+
| `ingress.className` | `nginx` | Ingress class |
48+
| `ingress.annotations` | nginx streaming defaults | Ingress annotations |
49+
| `ingress.hosts` | `[]` | Ingress hostnames |
50+
| `ingress.tls` | `[]` | Ingress TLS config |
51+
| `gateway.enabled` | `false` | ExternalName gateway service |
52+
| `gateway.serviceName` | `s3-gateway` | Gateway service name |
53+
| `gateway.ingressService` | `ingress-nginx-controller...` | Target ingress service |
54+
| `resources.requests.cpu` | `100m` | CPU request |
55+
| `resources.requests.memory` | `512Mi` | Memory request |
56+
| `resources.limits.cpu` | `500m` | CPU limit |
57+
| `resources.limits.memory` | `512Mi` | Memory limit |
58+
| `nodeSelector` | `{}` | Node selector |
59+
| `tolerations` | `[]` | Tolerations |
60+
| `affinity` | `{}` | Affinity rules |
61+
| `topologySpreadConstraints` | `[]` | Topology spread |
62+
| `podDisruptionBudget.enabled` | `true` | Enable PDB |
63+
| `podDisruptionBudget.minAvailable` | `1` | Min available pods |

0 commit comments

Comments
 (0)