Kubernetes Custom Resources

CRDs: once a CRD is applied, kubectl get databases works like kubectl get pods. The API server validates instances against the CRD's OpenAPI v3 schema and stores them in etcd. CRDs are the foundation of the operator pattern — a controller watches these custom resources and reconciles the desired state with the actual state of external systems.

Operator pattern: a database operator watches Database custom resources. When one is created, it provisions the database instance, creates a Service and Secret, and sets up backups. On deletion, it runs cleanup. On version change, it performs a rolling upgrade — encoding what a DBA would do manually into automated controller logic.

Reconciliation loop: the controller-runtime framework calls Reconcile(ctx, req) whenever a watched resource changes. The function is idempotent — it reads the current state, determines what actions are needed, and applies them. Errors cause a requeue with backoff. The loop does not rely on event ordering — it always acts on current state, making it resilient to missed events.

Admission webhooks: mutating webhooks run first — used to inject default values (sidecar containers, annotations, resource limits). Validating webhooks run after mutation — used to enforce policies (required labels, disallowed images, resource limit maximums). OPA/Gatekeeper and Kyverno implement policy enforcement as validating admission webhooks.

Finalizers: without finalizers, deleting a custom resource leaves the external resource (e.g. a cloud database) orphaned. The controller adds finalizers: ["myapp.io/cleanup"] when the resource is created. On deletion, the API server sets deletionTimestamp. The controller detects this, deletes the external resource, then removes the finalizer — allowing the Kubernetes resource to be purged from etcd.