Cluster API and Declarative Cluster Lifecycle

Managing Kubernetes clusters as Kubernetes objects: the Cluster, Machine, MachineSet, MachineDeployment and MachineHealthCheck CRDs, the three-provider model, ClusterClass topologies, rolling upgrades, the bootstrap paradox, and when the complexity actually pays for itself.

Overview

Clusters as Kubernetes resources

Every organization that runs more than a few Kubernetes clusters confronts the same asymmetry. Inside a cluster everything is declarative, reconciled and self-healing; outside it, cluster creation is a Terraform run or a wiki page.

Cluster API removes that asymmetry by making the cluster itself a Kubernetes resource. You describe the desired cluster in a Cluster object, controllers reconcile toward it, and drift is corrected the same way a Deployment corrects a deleted Pod.

Mirroring workload semantics

The design decision that makes it learnable is the deliberate mirroring of core workload semantics. An engineer who understands rolling updates understands node-pool upgrades immediately.

Machine — Pod, as Machine is to Node what Pod is to container
MachineSet — ReplicaSet
MachineDeployment — Deployment, with maxSurge and maxUnavailable
MachineHealthCheck — A probe whose remediation is replacing the machine
KubeadmControlPlane — Genuinely new: control-plane replacement involves etcd membership and cannot be a naive rolling update

The provider model

Cluster API defines contracts, not implementations. That is how the same API describes an EKS cluster, a vSphere cluster and a rack of bare metal.

Infrastructure — Compute and networking
Bootstrap — The cloud-init that runs kubeadm
Control plane — Control-plane lifecycle

> [!TIP] Swapping AWS for vSphere changes the referenced infrastructure objects, not the Cluster, MachineDeployment or MachineHealthCheck semantics above them.

Composing with the ecosystem

Because clusters are just objects, they compose with everything else in Kubernetes.

GitOps applying a Cluster manifest to the management cluster — A new cluster is a pull request
RBAC on the management cluster — Cluster-provisioning authorization
Policy engines — Validate cluster specs like Pod specs
ClusterClass — Fleet-wide templating: a hundred clusters share one definition with per-cluster variables

None of this is free

You now operate a management cluster whose availability gates every lifecycle operation across the fleet, plus provider controllers with their own upgrade cadence and alpha-to-beta API churn.

Many clusters, several infrastructures, or on-prem and bare metal where nothing managed exists — Three EKS clusters that Terraform already handles

Where interviewers push

CRD mapping — The exact mapping to core analogues
Control-plane upgrades — What makes them safe when etcd quorum is at stake
MachineHealthCheck — How it avoids reprovisioning the fleet during a network partition
Bootstrap — The bootstrap paradox and the pivot that resolves it
Alternatives — The honest comparison with Terraform and managed services

> [!IMPORTANT] A candidate who cannot say when Cluster API is overkill has probably not run it.

Key concepts

Management cluster: An ordinary Kubernetes cluster hosting the Cluster API core controllers and the provider controllers, and holding the Cluster, Machine, MachineDeployment and MachineHealthCheck objects for every managed cluster. All lifecycle operations are kubectl against it. It is a single point of failure for fleet operations, though not for the running workload clusters themselves.
Machine: The declarative spec for the infrastructure hosting one Kubernetes node. It references a provider-specific infrastructure object and a bootstrap config, and once the node registers, status.nodeRef links it to its Node. Machines are immutable in practice: changing a machine means replacing it. Deleting a Machine deletes the underlying instance.
MachineSet and MachineDeployment: MachineSet keeps a stable number of Machines running, exactly as a ReplicaSet does for Pods. MachineDeployment owns MachineSets and performs rolling updates governed by maxSurge and maxUnavailable, so a node-pool version bump or instance-type change is structurally identical to a Deployment rollout.
KubeadmControlPlane: The control-plane provider object. It owns control-plane Machines as a group, manages etcd membership when scaling or replacing them, exposes spec.replicas and spec.version, and refuses operations that would break quorum. Replicas should be odd. Upgrades scale up, join the new member, verify health, then remove an old one.
Provider triad: Infrastructure providers create compute and network resources (CAPA, CAPZ, CAPG, CAPV, Metal3, CAPD). Bootstrap providers generate BootstrapData, typically cloud-init that runs kubeadm init or kubeadm join. Control-plane providers manage the control plane as a unit. Cluster API defines the contracts; providers implement them, which is why one API describes many substrates.
MachineHealthCheck: Selects Machines and declares them unhealthy when a node condition such as Ready being False or Unknown persists past a timeout, or when a node never appears within the startup timeout. The owning MachineSet or KubeadmControlPlane then replaces the machine. Remediation short-circuits when too many selected machines are unhealthy at once, and control-plane remediation additionally protects etcd quorum.
ClusterClass and managed topology: A ClusterClass bundles infrastructure, control-plane and worker templates plus variables and patches. A Cluster then declares spec.topology naming the class, version and worker deployments, and the topology controller expands it into the full object graph. It removes per-cluster boilerplate and makes fleet-wide changes a single edit — which is also its danger, because one edit rolls every referencing cluster.
Pivot (clusterctl move): Transferring Cluster API objects from one management cluster to another. clusterctl move pauses reconciliation by setting Cluster.spec.paused, waits for controllers to clear their block-move annotation, recreates the objects on the target, and lets the target reconcile. Status subresources are not restored — controllers rebuild them. It is explicitly not a backup and restore mechanism.

When to use it

Operating tens or hundreds of clusters where per-cluster imperative provisioning has become the bottleneck and every cluster has drifted slightly from every other
Running across more than one infrastructure — cloud plus vSphere, or cloud plus bare metal — and wanting one lifecycle API instead of one tool per substrate
On-prem and bare metal, where Metal3 with Ironic gives you the managed-service experience that no vendor sells you
Self-managed control planes with strict requirements — specific etcd tuning, custom API server flags, an air-gapped registry — that a managed service will not expose
GitOps-driven cluster provisioning, where creating a cluster should be a reviewed pull request against a repository rather than a privileged human running a tool

Interview questions

Why manage clusters as Kubernetes objects at all? What does Cluster API give you that Terraform or a managed service does not, and when is it the wrong choice?
Walk me through exactly what happens from applying a Cluster manifest to a working workload cluster.
How does a control-plane upgrade work in Cluster API, and what makes it safe?
Explain MachineHealthCheck and its remediation safeguards. What goes wrong without them?
What is the bootstrap paradox in Cluster API, and how do you resolve it? What happens to your fleet if the management cluster dies?
What problem does ClusterClass solve, and what are the risks of adopting it?