Skip to content

Commit a535f93

Browse files
authored
docs(air-gapped): improve registry mirrors for tenant Kubernetes (#461)
## Summary - Clarify that tenant control plane (Kamaji) uses management cluster's registry mirrors automatically, and only worker nodes need separate configuration - Add Option A showing how to configure registry mirrors via the platform Package CR - Add collapsible section with `kubectl patch` alternative for existing deployments - Add authentication example for containerd TOML configuration - Add "How it works" section explaining the secret distribution mechanism - Fix per-cluster secret naming to match actual code (`kubernetes-<cluster-name>-patch-containerd`) ## Test plan - [x] Verify the page renders correctly with Hugo - [x] Verify the collapsible `<details>` section renders properly with code block inside - [x] Verify all internal links work (#2-configure-container-registry-mirrors anchor) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified air-gapped Kubernetes install flow: control planes inherit registry mirrors automatically; worker nodes require separate mirror configuration. * Added two supported approaches for configuring worker node mirrors, including an alternate method to patch an existing platform deployment. * Documented authenticated mirror registry options and use of image pull secrets. * Added per-cluster mirror override support and precedence rules when both global and per-cluster configs exist. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 0f35846 + dc96ecf commit a535f93

1 file changed

Lines changed: 162 additions & 7 deletions

File tree

content/en/docs/v1/install/kubernetes/air-gapped.md

Lines changed: 162 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -150,8 +150,12 @@ Read the [`talosctl` configuration guide]({{% ref "/docs/v1/install/kubernetes/t
150150

151151
## 5. Configure Container Registry Mirrors for Tenant Kubernetes
152152

153-
To use registry mirrors, tenant Kubernetes clusters need to be configured separately,
154-
although in the same way as the Talos nodes.
153+
Tenant Kubernetes clusters in Cozystack use [Kamaji](https://kamaji.clastix.io/) for the control plane.
154+
The control plane components run as pods on the management cluster nodes,
155+
so they automatically use the registry mirrors configured in [step 2](#2-configure-container-registry-mirrors) for Talos.
156+
157+
However, tenant **worker nodes** run as separate virtual machines with their own containerd instance.
158+
These worker nodes need a separate registry mirror configuration.
155159

156160
To perform this configuration, you first need to deploy a Cozystack cluster of
157161
(or upgrade your cluster to) version v0.32.0 or later.
@@ -161,7 +165,124 @@ Check your current cluster version with:
161165
kubectl get deploy -n cozy-system cozystack -oyaml | grep installer
162166
```
163167

164-
Generate a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) named `patch-containerd` using the following example:
168+
### Option A: Configure via platform package
169+
170+
The platform package can automatically generate the `patch-containerd` secret
171+
from the `registries` section in the platform values.
172+
173+
Add the `registries` section to your **cozystack-platform.yaml**:
174+
175+
```yaml
176+
apiVersion: cozystack.io/v1alpha1
177+
kind: Package
178+
metadata:
179+
name: cozystack.cozystack-platform
180+
spec:
181+
variant: isp-full
182+
components:
183+
platform:
184+
values:
185+
# ... your existing publishing, networking, etc. ...
186+
registries:
187+
mirrors:
188+
docker.io:
189+
endpoints:
190+
- http://10.0.0.1:8082
191+
ghcr.io:
192+
endpoints:
193+
- http://10.0.0.1:8083
194+
gcr.io:
195+
endpoints:
196+
- http://10.0.0.1:8084
197+
registry.k8s.io:
198+
endpoints:
199+
- http://10.0.0.1:8085
200+
quay.io:
201+
endpoints:
202+
- http://10.0.0.1:8086
203+
cr.fluentbit.io:
204+
endpoints:
205+
- http://10.0.0.1:8087
206+
docker-registry3.mariadb.com:
207+
endpoints:
208+
- http://10.0.0.1:8088
209+
config:
210+
"10.0.0.1:8082":
211+
tls:
212+
insecureSkipVerify: true
213+
auth:
214+
username: myuser
215+
password: mypass
216+
```
217+
218+
Then apply it:
219+
220+
```bash
221+
kubectl apply -f cozystack-platform.yaml
222+
```
223+
224+
This will create a `patch-containerd` secret in the `cozy-system` namespace,
225+
which is automatically copied to every tenant Kubernetes cluster.
226+
227+
<details class="alert alert-info p-3 mb-4">
228+
<summary><strong>Alternatively, patch an existing platform package</strong></summary>
229+
230+
If the platform package is already deployed, you can add registry mirrors with a patch:
231+
232+
```bash
233+
kubectl patch packages.cozystack.io cozystack.cozystack-platform --type=merge -p '{
234+
"spec": {
235+
"components": {
236+
"platform": {
237+
"values": {
238+
"registries": {
239+
"mirrors": {
240+
"docker.io": {
241+
"endpoints": ["http://10.0.0.1:8082"]
242+
},
243+
"ghcr.io": {
244+
"endpoints": ["http://10.0.0.1:8083"]
245+
},
246+
"gcr.io": {
247+
"endpoints": ["http://10.0.0.1:8084"]
248+
},
249+
"registry.k8s.io": {
250+
"endpoints": ["http://10.0.0.1:8085"]
251+
},
252+
"quay.io": {
253+
"endpoints": ["http://10.0.0.1:8086"]
254+
},
255+
"cr.fluentbit.io": {
256+
"endpoints": ["http://10.0.0.1:8087"]
257+
},
258+
"docker-registry3.mariadb.com": {
259+
"endpoints": ["http://10.0.0.1:8088"]
260+
}
261+
},
262+
"config": {
263+
"10.0.0.1:8082": {
264+
"tls": {
265+
"insecureSkipVerify": true
266+
},
267+
"auth": {
268+
"username": "myuser",
269+
"password": "mypass"
270+
}
271+
}
272+
}
273+
}
274+
}
275+
}
276+
}
277+
}
278+
}'
279+
```
280+
281+
</details>
282+
283+
### Option B: Create the secret manually
284+
285+
Alternatively, create a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) named `patch-containerd` directly:
165286

166287
```yaml
167288
apiVersion: v1
@@ -208,12 +329,46 @@ stringData:
208329
skip_verify = true
209330
```
210331

211-
This secret will be copied for every tenant Kubernetes cluster deployed in Cozystack.
332+
If your registry mirrors require authentication, add a custom `Authorization` header
333+
with Base64-encoded credentials:
334+
335+
```toml
336+
server = "https://registry-1.docker.io"
337+
[host."http://10.0.0.1:8082"]
338+
capabilities = ["pull", "resolve"]
339+
skip_verify = true
340+
[host."http://10.0.0.1:8082".header]
341+
Authorization = "Basic bXl1c2VyOm15cGFzcw=="
342+
```
343+
344+
To generate the Base64-encoded value, run:
345+
346+
```bash
347+
echo -n 'myuser:mypass' | base64
348+
```
349+
350+
For dynamic or token-based authentication (e.g., Docker Hub), use
351+
[Kubernetes image pull secrets](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/)
352+
instead of plaintext credentials.
353+
354+
### How it works
212355

213-
It's possible to configure registry mirrors for a particular tenant Kubernetes cluster:
356+
The `patch-containerd` secret from the `cozy-system` namespace is automatically copied
357+
to every tenant Kubernetes cluster namespace during deployment.
358+
The secret data is mounted into worker node VMs as containerd registry configuration files
359+
at `/etc/containerd/certs.d/<registry>/hosts.toml`.
214360

215-
- The tenant cluster must be deployed with a Kubernetes package version 0.23.1 or later, which is available since Cozystack 0.32.1.
216-
- Before deploying the tenant cluster, create a Kubernetes Secret named `kubernetes-<cluster name>` with the same contents as shown above.
361+
### Per-cluster configuration
362+
363+
It is possible to configure registry mirrors for a particular tenant Kubernetes cluster
364+
instead of using the global `patch-containerd` secret:
365+
366+
- The tenant cluster must be deployed with a Kubernetes package version 0.23.1 or later, which is available since Cozystack 0.32.1.
367+
- Before deploying the tenant cluster, create a Kubernetes Secret named `kubernetes-<cluster-name>-patch-containerd` in the tenant cluster namespace, using the same format as the examples above.
368+
369+
{{% alert color="warning" %}}
370+
**Important:** If both the global `patch-containerd` secret and a per-cluster secret exist, the global secret takes precedence and the per-cluster secret is ignored. To use a per-cluster configuration, ensure that the global `patch-containerd` secret in the `cozy-system` namespace is not present.
371+
{{% /alert %}}
217372

218373
To learn more about registry configuration values, read the [CRI Plugin configuration guide](
219374
https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration)

0 commit comments

Comments
 (0)