feat: plan-centric reconcile foundations (Steps 1-4) (#87)

* feat: plan-centric reconcile foundations (Steps 1-4)

Lay the groundwork for making Plan the central reconciliation abstraction
across all SeiNode phases. This is the non-behavioral half of the refactor
— all new code is additive and existing behavior is unchanged.

Step 1: Add TargetPhase/FailedPhase to TaskPlan CRD
- Executor sets node phase atomically with plan completion/failure
- Both fields are optional; empty means no transition (backward compatible)

Step 2: Extract resource generators to internal/noderesource/
- Moves StatefulSet, Service, PVC generation out of the controller package
- Breaks the import cycle that would exist between task and controller
- Controller now calls noderesource.* directly (no delegate wrappers)

Step 3: New controller-side tasks (ensure-data-pvc, apply-statefulset, apply-service)
- Fire-and-forget tasks that complete synchronously in Execute
- Registered in the task registry, not yet wired into any plan

Step 4: Executor handles TargetPhase and convergence plan cleanup
- On plan completion: applies TargetPhase to node status in same patch
- On plan failure: applies FailedPhase to node status in same patch
- Convergence plans (target == current phase): nils the plan on completion
- OnPlanComplete/OnPlanFailed callbacks for metrics/events

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove callbacks from executor, use Conditions for failure

Remove PhaseTransition struct and OnPlanComplete/OnPlanFailed callbacks.
The executor directly owns phase transitions via TargetPhase/FailedPhase
without needing callback indirection.

On plan failure, set a PlanFailed Condition on the node with the failed
task type as the Reason and a message describing what went wrong. This
follows the standard K8s Conditions pattern already used by monitor tasks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: Condition Reason to CamelCase, preserve init plan on completion

Two fixes from expert review:

1. PlanFailed Condition Reason now uses "TaskFailed" (CamelCase) instead
   of the kebab-case task type string. K8s convention requires Reasons to
   be CamelCase machine-readable tokens. The task type is already in the
   Message field.

2. nilPlanIfConvergence now captures the pre-mutation phase and compares
   against it, so only convergence plans (same phase before and after) get
   nilled. Init plans (Initializing → Running) keep their completed plan
   visible in status for observability.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: bdchatham

api/v1alpha1/seinode_types.go

@@ -159,8 +159,8 @@ type FailedTaskInfo struct {
 	MaxRetries int `json:"maxRetries"`
 }
 
-// TaskPlan tracks an ordered sequence of sidecar tasks that the controller
-// executes to initialize a node.
+// TaskPlan tracks an ordered sequence of tasks that the controller
+// executes to drive a node toward a target state.
 type TaskPlan struct {
 	// ID is a unique identifier for this plan instance.
 	// +optional
@@ -172,6 +172,18 @@ type TaskPlan struct {
 	// Tasks is the ordered list of tasks to execute.
 	Tasks []PlannedTask `json:"tasks"`
 
+	// TargetPhase is the SeiNodePhase the executor sets on the owning
+	// resource when the plan completes successfully. When empty, the
+	// executor does not perform a phase transition.
+	// +optional
+	TargetPhase SeiNodePhase `json:"targetPhase,omitempty"`
+
+	// FailedPhase is the SeiNodePhase the executor sets on the owning
+	// resource when the plan fails terminally. When empty, the executor
+	// does not perform a phase transition on failure.
+	// +optional
+	FailedPhase SeiNodePhase `json:"failedPhase,omitempty"`
+
 	// FailedTaskIndex is the index of the task that caused the plan to fail.
 	// +optional
 	FailedTaskIndex *int `json:"failedTaskIndex,omitempty"`

config/crd/sei.io_seinodedeployments.yaml

@@ -864,6 +864,18 @@ spec:
                   Plan tracks the active group-level task plan (genesis assembly,
                   deployment, etc.). Nil when no plan is in progress.
                 properties:
+                  failedPhase:
+                    description: |-
+                      FailedPhase is the SeiNodePhase the executor sets on the owning
+                      resource when the plan fails terminally. When empty, the executor
+                      does not perform a phase transition on failure.
+                    enum:
+                    - Pending
+                    - Initializing
+                    - Running
+                    - Failed
+                    - Terminating
+                    type: string
                   failedTaskDetail:
                     description: FailedTaskDetail records diagnostics about the task
                       that caused the plan to fail.
@@ -905,6 +917,18 @@ spec:
                     - Complete
                     - Failed
                     type: string
+                  targetPhase:
+                    description: |-
+                      TargetPhase is the SeiNodePhase the executor sets on the owning
+                      resource when the plan completes successfully. When empty, the
+                      executor does not perform a phase transition.
+                    enum:
+                    - Pending
+                    - Initializing
+                    - Running
+                    - Failed
+                    - Terminating
+                    type: string
                   tasks:
                     description: Tasks is the ordered list of tasks to execute.
                     items:

config/crd/sei.io_seinodes.yaml

@@ -628,6 +628,18 @@ spec:
                   Plan tracks the active task sequence for this node. A planner generates
                   the plan based on the node's current state and conditions.
                 properties:
+                  failedPhase:
+                    description: |-
+                      FailedPhase is the SeiNodePhase the executor sets on the owning
+                      resource when the plan fails terminally. When empty, the executor
+                      does not perform a phase transition on failure.
+                    enum:
+                    - Pending
+                    - Initializing
+                    - Running
+                    - Failed
+                    - Terminating
+                    type: string
                   failedTaskDetail:
                     description: FailedTaskDetail records diagnostics about the task
                       that caused the plan to fail.
@@ -669,6 +681,18 @@ spec:
                     - Complete
                     - Failed
                     type: string
+                  targetPhase:
+                    description: |-
+                      TargetPhase is the SeiNodePhase the executor sets on the owning
+                      resource when the plan completes successfully. When empty, the
+                      executor does not perform a phase transition.
+                    enum:
+                    - Pending
+                    - Initializing
+                    - Running
+                    - Failed
+                    - Terminating
+                    type: string
                   tasks:
                     description: Tasks is the ordered list of tasks to execute.
                     items:

internal/controller/node/controller.go

@@ -20,6 +20,7 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/predicate"
 
 	seiv1alpha1 "github.com/sei-protocol/sei-k8s-controller/api/v1alpha1"
+	"github.com/sei-protocol/sei-k8s-controller/internal/noderesource"
 	"github.com/sei-protocol/sei-k8s-controller/internal/planner"
 	"github.com/sei-protocol/sei-k8s-controller/internal/platform"
 	"github.com/sei-protocol/sei-k8s-controller/internal/task"
@@ -85,14 +86,18 @@ func (r *SeiNodeReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ct
 		return ctrl.Result{}, fmt.Errorf("validating spec: %w", err)
 	}
 
+	// TODO: reconcile peers should become a part of a plan if it needs to be performed.
 	if err := r.reconcilePeers(ctx, node); err != nil {
 		return ctrl.Result{}, fmt.Errorf("reconciling peers: %w", err)
 	}
 
+	// TODO: this should be a part of a plan and not part of the main reconciliation flow.
 	if err := r.ensureNodeDataPVC(ctx, node); err != nil {
 		return ctrl.Result{}, fmt.Errorf("ensuring data PVC: %w", err)
 	}
 
+	// TODO: if the plan becomes the abstraction it should be, then p from planner.ForNode should just take an existing
+	// plan off the node if one exists, otherwise, create one based on the state it sees.
 	switch node.Status.Phase {
 	case "", seiv1alpha1.PhasePending:
 		return r.reconcilePending(ctx, node, p)
@@ -143,6 +148,8 @@ func (r *SeiNodeReconciler) reconcilePending(ctx context.Context, node *seiv1alp
 func (r *SeiNodeReconciler) reconcileInitializing(ctx context.Context, node *seiv1alpha1.SeiNode) (ctrl.Result, error) {
 	plan := node.Status.Plan
 
+	// TODO: This should be a part of the plan, not a special case we need to filter out for the reconciliation before we
+	// go ahead with the actual plan.
 	if !planner.NeedsBootstrap(node) || planner.IsBootstrapComplete(plan) {
 		if err := r.reconcileNodeStatefulSet(ctx, node); err != nil {
 			return ctrl.Result{}, fmt.Errorf("reconciling statefulset: %w", err)
@@ -157,6 +164,9 @@ func (r *SeiNodeReconciler) reconcileInitializing(ctx context.Context, node *sei
 		return result, err
 	}
 
+	// TODO: transitioning should be a part of ExecutePlan, I would think. This way, when a plan is done, it has materialized
+	// the proper state and does some final patch to the node, basically leaving it in a state that other logic can observe and
+	// act on as well. For instance, if a plan succeeds, the plan knows what state to put it in. Same for if it fails.
 	if plan.Phase == seiv1alpha1.TaskPlanComplete {
 		return r.transitionPhase(ctx, node, seiv1alpha1.PhaseRunning)
 	}
@@ -168,9 +178,12 @@ func (r *SeiNodeReconciler) reconcileInitializing(ctx context.Context, node *sei
 
 // reconcileRunning converges owned resources and handles runtime tasks.
 func (r *SeiNodeReconciler) reconcileRunning(ctx context.Context, node *seiv1alpha1.SeiNode) (ctrl.Result, error) {
+
+	// TODO: ideally this becomes a task in a plan
 	if err := r.reconcileNodeStatefulSet(ctx, node); err != nil {
 		return ctrl.Result{}, fmt.Errorf("reconciling statefulset: %w", err)
 	}
+	// TODO: ideally this becomes a task in a plan
 	if err := r.reconcileNodeService(ctx, node); err != nil {
 		return ctrl.Result{}, fmt.Errorf("reconciling service: %w", err)
 	}
@@ -286,7 +299,7 @@ func (r *SeiNodeReconciler) handleNodeDeletion(ctx context.Context, node *seiv1a
 
 func (r *SeiNodeReconciler) deleteNodeDataPVC(ctx context.Context, node *seiv1alpha1.SeiNode) error {
 	pvc := &corev1.PersistentVolumeClaim{}
-	err := r.Get(ctx, types.NamespacedName{Name: nodeDataPVCName(node), Namespace: node.Namespace}, pvc)
+	err := r.Get(ctx, types.NamespacedName{Name: noderesource.DataPVCName(node), Namespace: node.Namespace}, pvc)
 	if apierrors.IsNotFound(err) {
 		return nil
 	}
@@ -297,7 +310,7 @@ func (r *SeiNodeReconciler) deleteNodeDataPVC(ctx context.Context, node *seiv1al
 }
 
 func (r *SeiNodeReconciler) ensureNodeDataPVC(ctx context.Context, node *seiv1alpha1.SeiNode) error {
-	desired := generateNodeDataPVC(node, r.Platform)
+	desired := noderesource.GenerateDataPVC(node, r.Platform)
 	if err := ctrl.SetControllerReference(node, desired, r.Scheme); err != nil {
 		return fmt.Errorf("setting owner reference: %w", err)
 	}
@@ -311,7 +324,7 @@ func (r *SeiNodeReconciler) ensureNodeDataPVC(ctx context.Context, node *seiv1al
 }
 
 func (r *SeiNodeReconciler) reconcileNodeStatefulSet(ctx context.Context, node *seiv1alpha1.SeiNode) error {
-	desired := generateNodeStatefulSet(node, r.Platform)
+	desired := noderesource.GenerateStatefulSet(node, r.Platform)
 	desired.SetGroupVersionKind(appsv1.SchemeGroupVersion.WithKind("StatefulSet"))
 	if err := ctrl.SetControllerReference(node, desired, r.Scheme); err != nil {
 		return fmt.Errorf("setting owner reference: %w", err)
@@ -321,7 +334,7 @@ func (r *SeiNodeReconciler) reconcileNodeStatefulSet(ctx context.Context, node *
 }
 
 func (r *SeiNodeReconciler) reconcileNodeService(ctx context.Context, node *seiv1alpha1.SeiNode) error {
-	desired := generateNodeHeadlessService(node)
+	desired := noderesource.GenerateHeadlessService(node)
 	desired.SetGroupVersionKind(corev1.SchemeGroupVersion.WithKind("Service"))
 	if err := ctrl.SetControllerReference(node, desired, r.Scheme); err != nil {
 		return fmt.Errorf("setting owner reference: %w", err)

internal/controller/node/labels.go

@@ -9,6 +9,7 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/log"
 
 	seiv1alpha1 "github.com/sei-protocol/sei-k8s-controller/api/v1alpha1"
+	"github.com/sei-protocol/sei-k8s-controller/internal/noderesource"
 	"github.com/sei-protocol/sei-k8s-controller/internal/planner"
 	"github.com/sei-protocol/sei-k8s-controller/internal/task"
 )
@@ -18,7 +19,7 @@ func (r *SeiNodeReconciler) buildSidecarClient(node *seiv1alpha1.SeiNode) task.S
 	if r.BuildSidecarClientFn != nil {
 		return r.BuildSidecarClientFn(node)
 	}
-	c, err := sidecar.NewSidecarClientFromPodDNS(node.Name, node.Namespace, sidecarPort(node))
+	c, err := sidecar.NewSidecarClientFromPodDNS(node.Name, node.Namespace, noderesource.SidecarPort(node))
 	if err != nil {
 		log.Log.Info("failed to build sidecar client", "node", node.Name, "error", err)
 		return nil

internal/controller/node/plan_execution.go

@@ -17,6 +17,7 @@ import (
 	"sigs.k8s.io/controller-runtime/pkg/client/fake"
 
 	seiv1alpha1 "github.com/sei-protocol/sei-k8s-controller/api/v1alpha1"
+	"github.com/sei-protocol/sei-k8s-controller/internal/noderesource"
 	"github.com/sei-protocol/sei-k8s-controller/internal/planner"
 	"github.com/sei-protocol/sei-k8s-controller/internal/platform/platformtest"
 	"github.com/sei-protocol/sei-k8s-controller/internal/task"
@@ -194,7 +195,7 @@ func TestNodeReconcile_RunningPhase_UpdatesStatefulSetImage(t *testing.T) {
 	node.Status.Phase = seiv1alpha1.PhaseRunning
 
 	// Pre-create a StatefulSet with the old image.
-	oldSts := generateNodeStatefulSet(node, platformtest.Config())
+	oldSts := noderesource.GenerateStatefulSet(node, platformtest.Config())
 	oldSts.SetGroupVersionKind(appsv1.SchemeGroupVersion.WithKind("StatefulSet"))
 
 	r, c := newNodeReconciler(t, node, oldSts)
@@ -500,7 +501,7 @@ func TestNodeDeletion_SnapshotNode_WithoutRetain_DeletesPVC(t *testing.T) {
 		ObjectMeta: metav1.ObjectMeta{
 			Name:      "data-snap-0",
 			Namespace: "default",
-			Labels:    resourceLabelsForNode(node),
+			Labels:    noderesource.ResourceLabels(node),
 		},
 	}
 

internal/controller/node/reconciler_test.go

@@ -0,0 +1,52 @@
+package node
+
+import (
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+
+	seiv1alpha1 "github.com/sei-protocol/sei-k8s-controller/api/v1alpha1"
+)
+
+func newGenesisNode(name, namespace string) *seiv1alpha1.SeiNode { //nolint:unparam // test helper designed for reuse
+	return &seiv1alpha1.SeiNode{
+		ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: namespace},
+		Spec: seiv1alpha1.SeiNodeSpec{
+			ChainID: "sei-test",
+			Image:   "ghcr.io/sei-protocol/seid:latest",
+			Entrypoint: &seiv1alpha1.EntrypointConfig{
+				Command: []string{"seid"},
+				Args:    []string{"start"},
+			},
+			Validator: &seiv1alpha1.ValidatorSpec{},
+			Sidecar:   &seiv1alpha1.SidecarConfig{Port: 7777},
+		},
+	}
+}
+
+func newSnapshotNode(name, namespace string) *seiv1alpha1.SeiNode { //nolint:unparam // test helper designed for reuse
+	return &seiv1alpha1.SeiNode{
+		ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: namespace},
+		Spec: seiv1alpha1.SeiNodeSpec{
+			ChainID: "sei-test",
+			Image:   "ghcr.io/sei-protocol/seid:latest",
+			FullNode: &seiv1alpha1.FullNodeSpec{
+				Snapshot: &seiv1alpha1.SnapshotSource{
+					S3: &seiv1alpha1.S3SnapshotSource{
+						TargetHeight: 100000000,
+					},
+					TrustPeriod: "9999h0m0s",
+				},
+			},
+			Sidecar: &seiv1alpha1.SidecarConfig{Port: 7777},
+		},
+	}
+}
+
+func findContainer(containers []corev1.Container, name string) *corev1.Container { //nolint:unparam // test helper designed for reuse
+	for i := range containers {
+		if containers[i].Name == name {
+			return &containers[i]
+		}
+	}
+	return nil
+}