[RayService] Add skipGateway option for NewClusterWithIncrementalUpgrade

[savitagm]

Summary

Adds skipGateway: true support to NewClusterWithIncrementalUpgrade, enabling zero-downtime rolling upgrades for deployments that use a client-side load balancer instead of a Gateway/HTTPRoute for traffic splitting.

When skipGateway: true is set:

• The operator skips Gateway and HTTPRoute reconciliation entirely
• Traffic split is communicated via targetCapacity in activeServiceStatus / pendingServiceStatus — the client-side LB reads these values to route traffic
• TrafficRoutedPercent reflects the actual split only after the pending cluster's Serve deployments are HEALTHY and worker replica parity is met (capacity-ready gate)
• Serve config blocking during upgrade is consistent with the gateway path: active cluster's serve config is frozen while UpgradeInProgress=true; the updated config goes to the pending cluster

Changes

Commit	Description
6cbc10a0	Add skipGateway field to ClusterUpgradeOptions CRD; skip Gateway/HTTPRoute reconciliation when set
d85d6397	Replace timer-based TC step with serve deployment health gate (isPendingClusterCapacityReady)
3ee51aa4	Skip gateway-specific field validations (HTTPRoute, Gateway names) when skipGateway=true
dc97d621	Capacity-parity check: TC stalls until pending has ceil(TC% × activeWorkers) available workers; add upgradeTimeoutSeconds support
88b0001c / 951e4fbd	Exempt skipGateway services from in-tree autoscaler requirement (gateway path requires it; skipGateway does not)
d68528ba	Fix spurious "pending cluster not yet capacity-ready" log firing every reconcile when no upgrade is in progress

Key behaviors

Capacity-ready gate — TC advances only when:

1. All pending cluster Serve deployments are HEALTHY
2. pendingAvailableWorkers >= ceil(pendingTC% × activeAvailableWorkers) — or pending maxReplicas < activeAvailable (capacity reduction path, always passes)

TC stepping — With maxSurgePercent=20 and 2 active workers:

• TC steps 0→20→40→60, then stalls at 60 (ceil(60%×2)=2 workers needed)
• Unblocks once pending cluster has ≥2 available workers, then advances to 80→100

Serve config consistency — Both gateway and skipGateway paths use the same blocking logic: while UpgradeInProgress=true, serve config changes to the active cluster are held; they go to the pending cluster. Escape hatches:

• Delete the pending cluster → exits upgrade mode, serve config propagates to active on next reconcile
• Trigger a new spec change → old pending replaced, new pending picks up updated serve config

upgradeTimeoutSeconds — Acts as a per-attempt circuit breaker: after timeout, the pending cluster is deleted and a Warning event is emitted. The operator will retry (create a new pending cluster) on the next reconcile since the spec still differs. To permanently stop a timed-out upgrade, either revert the spec or delete the pending cluster manually.

Future work

Permanent-abort on timeout (upgradeTimeoutSeconds as a hard stop): currently the timeout is a per-attempt circuit breaker — after deleting the pending cluster, the operator immediately retries because shouldPrepareNewCluster detects the spec hash still differs. A future enhancement would annotate the RayService with the timed-out spec hash (ray.io/upgrade-timeout-hash) and check it in shouldPrepareNewCluster to skip retrying for the same hash. The annotation would be cleared when a new (different) spec change arrives. This was deferred to keep this PR focused.

Testing

Manually tested all 7 scenarios via the /skipgw-rollout-test skill against a live RayService (llm-experiments/aipts-1466-skip-gw-test) with A100 GPUs:

#	Test	Result
1	Steady-state: no spurious logs	PASS
2	maxSurgePercent=100 (TC jumps 0→100)	PASS
3	Re-trigger mid-upgrade	PASS
4	Serve config blocked on active	PASS
5	upgradeTimeoutSeconds	PASS
6	TC stepping with multi-worker active cluster	PASS
7	minReplicas=0	PASS

Unit tests added for isPendingClusterCapacityReady covering: nil apps, nil clusters, capacity reduction bypass, insufficient/sufficient workers, and TC=0.