Activate and verify Headlamp user namespaces

[devantler]

🤖 Generated by the Daily AI Engineer (Codex instance)

Evidence

Platform issue #1807 requires each stateful workload to graduate through an explicit user-namespace pilot. Headlamp is the next pilot after Longhorn, but activation must stay separate from the default-off gate in #2650 so the rollout can happen only after the concurrent node-auth changes in #2635 have settled and live storage health is known-good.

Headlamp persists plugins on the headlamp-plugins PVC. Its chart renders hostUsers: true by default, so activation must explicitly set spec.values.hostUsers: false and preserve the existing volume identity throughout the experiment.

Impact

This is the live verification step for Headlamp's two-container pod. A failed rollout could prevent the plugin sidecar from writing files, prevent the main container from reading those files, or disturb the Longhorn-backed plugin volume. Recovery must therefore revert only the user-namespace value and must never delete the PVC.

Acceptance criteria

• Add a default-off Headlamp user-namespace gate #2650 is merged, fix(deploy): reboot stale nodes so containerd adopts refreshed GHCR auth #2635 has settled, every production node is Ready, Longhorn is healthy, and Headlamp is stable before activation.
• Record the existing Headlamp PVC, PV, and Longhorn volume identities before changing the gate.
• Enable the exact-target Headlamp values patch so the rendered Deployment has spec.template.spec.hostUsers: false.
• Verify the Deployment is Available 1/1 and the pod is 2/2 Ready.
• Verify both containers have non-zero subordinate UID/GID mappings and the plugin installer logs contain no EACCES or EPERM errors.
• Verify installed plugin files are visible from the main Headlamp container, and the route, OIDC login, and plugins still work.
• Verify the same PVC/PV/Longhorn volume remains bound and healthy, with no mount or warning events.
• Document the observation window and evidence on the issue.
• If recovery is needed, re-disable the gate, reconcile to healthy, and retain the existing PVC.

Part of #1807.

[coderabbitai]

🔗 Related PRs

#2336 - feat(homepage): run inside a user namespace (hostUsers: false) [merged]
#2498 - fix(crossview): grant the OIDC user port-forward in the crossview namespace [merged]
#2569 - feat(policy): opt proven stateless apps into user namespaces [merged]
#2578 - fix(policy-reporter): run all components at the HA replica floor [merged]
#2620 - test(storage): restore userns smoke harness default-off [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!