Restore userns Longhorn smoke harness to default-off

[devantler]

🤖 Generated by the Daily AI Engineer

Objective

Return the disposable userns-longhorn-smoke component to its default-off state after the production idmapped-mount proof, recording both the proof and storage cleanup before any real stateful workload opts into user namespaces.

Details

#2599/#2600 added the default-off harness. #2604/#2605 activated it, but #2604 closed even though its checklist required a cleanup PR and garbage-collection verification.

Current main still includes - userns-longhorn-smoke/ in k8s/providers/hetzner/apps/kustomization.yaml. The short-lived proof therefore remains declaratively active, with no open issue or PR tracking cleanup. This leaves the sequencing promised by #1807 incomplete.

Proposed direction:

• capture the existing Job's completion, UID/GID-map, round-trip, and warning-event evidence
• comment userns-longhorn-smoke/ out of the Hetzner apps kustomization and restore its default-off lifecycle comment
• retain the standalone component for future regression checks
• after rollout, verify all generated Kubernetes and Longhorn storage objects are gone
• record the completed proof on k8s 1.36 readiness: clean on removals; pilot user namespaces, watch VolumeGroupSnapshot #1807

No real application opt-in, Kyverno policy change, or local/CI rollout belongs in this slice.

Checklist

• Before removal, the Job is Complete; logs show non-zero host UID/GID map starts and userns-longhorn-round-trip=passed; no attributable warning events exist.
• Hetzner apps no longer includes the component; complete local and production renders contain zero smoke Namespace or Job objects.
• The standalone component still renders exactly one restricted Namespace and one bounded, no-retry, tokenless Job using hostUsers: false and a 256Mi RWO generic-ephemeral longhorn-wffc claim.
• Naming, embedded-JSON, local/prod Kustomize, local/prod KSail validation, NSA threshold, and git diff --check pass.
• After rollout, the Namespace, Job/Pod, PVC/PV, and corresponding Longhorn volume are absent.
• k8s 1.36 readiness: clean on removals; pilot user namespaces, watch VolumeGroupSnapshot #1807 records the successful proof and identifies Headlamp as the next separately gated stateful pilot.
• This issue closes only after cleanup verification.

Dependencies

• Parent: k8s 1.36 readiness: clean on removals; pilot user namespaces, watch VolumeGroupSnapshot #1807
• Harness: Add a default-off userns Longhorn smoke harness #2599 / test(storage): add a default-off userns Longhorn smoke harness #2600
• Activation proof: Activate and verify the userns Longhorn smoke harness #2604 / test(storage): activate userns Longhorn smoke proof #2605

There is no external blocker; the live proof evidence must be captured before disabling the harness.

Notes

XS-Small: one kustomization gate/comment change, static validation, and read-only pre/post-deployment evidence.

[coderabbitai]

🔗 Related PRs

#2273 - ci: re-enable in-process Helm render in validate (revert --skip-helm-render) [merged]
#2336 - feat(homepage): run inside a user namespace (hostUsers: false) [merged]
#2341 - feat(longhorn): self-heal orphaned Node CRs left by autoscaler scale-downs [merged]
#2370 - fix(autoscaling): break the DaemonSet-VPA node ratchet; keep Longhorn storage off autoscale nodes [merged]
#2569 - feat(policy): opt proven stateless apps into user namespaces [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[devantler]

🤖 Generated by the Daily AI Engineer (Codex instance)\n\nPost-rollout cleanup verified after #2620 merged: the userns-longhorn-smoke Namespace and its Job/Pod are absent; no PV/PVC claim remains in that namespace; and no corresponding Longhorn volume remains. Production finished healthy with 9/9 nodes Ready and all Flux Kustomizations Ready. The cleanup acceptance criteria are complete.

[botantler-1]

🎉 This issue has been resolved in version 1.116.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀