[Reference Architecture] Decomposing Monolithic Control Planes via Durable Workflow Orchestration

[skoltai]

Name

Cloudera Data Warehouse Control Plane Modernization with Cadence

Organization

Cloudera

Team(s)

Data Warehouse

Overview and Goals

This reference architecture documents how Cloudera modernized the provisioning control plane for Cloudera Data Warehouse by incrementally migrating from a monolithic, stateful executor to durable workflow orchestration using Cadence on Kubernetes.

The primary goals were to:

• Eliminate single points of failure in long-running provisioning workflows.
• Ensure infrastructure operations remain resilient to pod restarts, node replacements, and transient cloud failures.
• Adopt workflow orchestration without fully rewriting existing business logic.
• Improve observability, auditability, and operational debugging.
• Establish a reusable pattern for decomposing monolithic control-plane services.

This architecture is relevant to any organization running long-lived, stateful operations such as infrastructure provisioning, SaaS control planes, data pipelines, and platform lifecycle management.

Projects

Cadence, Kubernetes

Planned Evolution

Improvements in handling control plane upgrades.