Add a default-off Headlamp user-namespace gate

[devantler]

🤖 Generated by the Daily AI Engineer (Codex instance)

Summary

Add a production-only, default-off inline Kustomize patch gate that can later opt Headlamp into Kubernetes user namespaces without changing the current deployment. This is the first independently shippable step for the next stateful pilot under parent #1807.

Evidence and motivation

• k8s 1.36 readiness: clean on removals; pilot user namespaces, watch VolumeGroupSnapshot #1807 names Headlamp as the next separately gated stateful pilot after the completed Longhorn idmapped-mount proof and cleanup in Restore userns Longhorn smoke harness to default-off #2619.
• The pinned Headlamp chart 0.43.0 explicitly renders Deployment.spec.template.spec.hostUsers: true by default. The existing Kyverno rule uses a conditional +(hostUsers): false anchor, so a namespace label cannot override that authored chart value.
• Setting the chart value hostUsers: false renders the required pod setting without changing the existing 256Mi ReadWriteOnce plugins PVC, post-rendered plugins-dir volume, or chart-owned UID/GID/fsGroup values.
• An unreferenced partial HelmRelease patch file is unsafe in this repository because KSail can validate standalone YAML outside active Kustomize patch references. The default-off gate therefore belongs as a fully commented JSON6902 block inside the existing Hetzner apps kustomization.
• The active fix(deploy): reboot stale nodes so containerd adopts refreshed GHCR auth #2635 rollout can reboot nodes and refresh containerd authentication. Activation must wait until that rollout is complete and node/Longhorn/Headlamp health is stable, so storage remount evidence is not confounded.

Audience and hypothesis

Platform operators need a reversible, reviewable switch before changing a stateful dashboard. If an exact-target JSON6902 block is documented but fully commented, current local and production renders remain unchanged; a synthetic enabled render should set only Headlamp spec.values.hostUsers to false and preserve its plugin/PVC wiring.

Proposed solution

• Add one fully commented JSON6902 patch block beside the existing user-namespace rollout patches in k8s/providers/hetzner/apps/kustomization.yaml.
• Target only helm.toolkit.fluxcd.io/v2 HelmRelease headlamp in namespace headlamp and add /spec/values/hostUsers = false.
• Document the activation/recovery lifecycle and why a namespace-label patch cannot work. Do not activate the block in this delivery.
• Do not add an orphan patch file or change the base namespace, Kyverno policy, local overlay, PVC, plugin manager, or post-renderer.
• Track activation and live verification as a separate child after this gate merges and fix(deploy): reboot stale nodes so containerd adopts refreshed GHCR auth #2635 is fully rolled out.

Acceptance criteria

• Current local and production renders are byte-for-behavior unchanged because the new block remains fully commented.
• A synthetic enabled render sets Headlamp HelmRelease spec.values.hostUsers to false while local remains unchanged.
• Helm template validation for chart 0.43.0 proves default true versus enabled false at Deployment.spec.template.spec.hostUsers.
• Headlamp pluginsManager remains enabled; the main and plugin containers retain plugins-dir mounts; the post-renderer still maps plugins-dir to PVC claim headlamp; the PVC remains 256Mi ReadWriteOnce.
• Naming, embedded JSON, both Kustomize overlays, KSail local/prod validation, the configured NSA security scan, and diff checks remain green.
• The comment names the lifecycle: uncomment exactly one block to enable; re-comment/revert it to recover; never delete the PVC.

Success signal and guardrail

Success is a validated default-off switch whose synthetic enabled state changes only the intended Headlamp user-namespace value. The guardrail is zero runtime change in this PR and no mutation of the local provider, shared base, plugins, security context, or persistent storage.

Part of #1807.

[coderabbitai]

🔗 Related PRs

#2336 - feat(homepage): run inside a user namespace (hostUsers: false) [merged]
#2370 - fix(autoscaling): break the DaemonSet-VPA node ratchet; keep Longhorn storage off autoscale nodes [merged]
#2569 - feat(policy): opt proven stateless apps into user namespaces [merged]
#2608 - feat(cilium): stage bandwidth manager with BBR [merged]
#2620 - test(storage): restore userns smoke harness default-off [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!