A platform engineer explains GitOps to a team adopting Kubernetes: "GitOps means Git is your single source of truth for everything — application configs, Kubernetes manifests, Helm chart values, infrastructure definitions. You never apply changes directly to the cluster. Every change is a Git commit. An operator running in the cluster — ArgoCD in our case — continuously watches the Git repo. When it detects a difference between what Git says should exist and what actually exists in the cluster, it reconciles: it brings the cluster to match Git. Humans push to Git; the operator pushes to the cluster." What is the reconciliation loop in GitOps?
Reconciliation loop: the heart of GitOps. The operator (ArgoCD, Flux) runs an infinite control loop: (1) Read desired state from Git, (2) Read observed state from the cluster via Kubernetes API, (3) Compare — compute the diff, (4) If different, apply changes to make observed = desired. This runs on a configurable interval (e.g., every 3 minutes) and also on Git push events via webhooks. GitOps principles (from Weaveworks, who coined the term): Declarative: describe what you want, not how to get there (Kubernetes YAML, Helm charts, Kustomize). Versioned and immutable: all desired state in Git with full history. Pulled automatically: the operator pulls from Git (not push-based CI) — the cluster initiates the connection, which is more secure for cloud environments. Continuously reconciled: the system self-heals to match desired state, correcting manual changes (drift). Key vocabulary: Desired state: what Git says should exist. Observed state: what the Kubernetes API currently shows. Drift: any difference between desired and observed. Self-healing: automatic correction of drift. In conversation: 'With GitOps, if someone kubectl-applies a hotfix directly to production, ArgoCD will revert it within minutes. The Git repo is law.'
2 / 5
A DevOps engineer explains ArgoCD to a developer unfamiliar with GitOps tooling: "ArgoCD is a declarative GitOps controller for Kubernetes. You define an Application custom resource that tells ArgoCD: watch this Git repo, at this path, for this target cluster and namespace. ArgoCD then continuously syncs. The Application has a health status — Healthy, Progressing, Degraded — and a sync status — Synced or OutOfSync. OutOfSync means Git and the cluster diverge. You can configure auto-sync to automatically apply changes, or leave it manual so a human reviews the diff and clicks sync." In ArgoCD, what does OutOfSync mean and when is it expected?
ArgoCD sync status: Synced: observed cluster state matches desired Git state. OutOfSync: a difference exists — could be a new Git commit not yet applied, a manual change made to the cluster, or a resource that drifted. Unknown: ArgoCD cannot determine the state. Health status: Healthy: all resources are running as expected. Progressing: deployment or rollout is in progress. Degraded: a pod has crash-looped or a Deployment failed to roll out. Suspended: a CronJob or similar is paused. ArgoCD Application vocabulary: Application CR: the ArgoCD custom resource defining a deployment target. App of Apps: a pattern where one ArgoCD Application manages a Git directory full of other Application CRs — enables bootstrapping an entire cluster from a single sync. Sync policy: automated = ArgoCD auto-applies on OutOfSync. Manual = human triggers sync. Sync waves: ordering mechanism — resources with lower wave numbers sync first (e.g., CRDs before operators, namespaces before deployments). Sync options: CreateNamespace=true (create ns if missing), PruneLast=true (delete old resources after creating new), Replace=true (force replace instead of patch). Resource hooks: PreSync, Sync, PostSync Jobs run at specified phases. In conversation: 'We use manual sync for production and auto-sync for dev/staging. Anything going to prod requires a human to review the diff in ArgoCD before clicking sync.'
3 / 5
A senior engineer explains why they chose Flux over ArgoCD at architecture review: "Flux is a CNCF graduated project — it's GitOps for Kubernetes implemented as a set of controllers. Each controller is a focused CRD: GitRepository watches a Git repo, Kustomization applies manifests, HelmRelease manages Helm chart deployments. The image automation controllers watch container registries and can automatically update image tags in Git when new images are pushed. We liked Flux's composability and the fact that all state lives in the cluster as CRDs — there's no separate Flux UI state, it's pure Kubernetes." What is a HelmRelease in Flux and why is it more GitOps-native than running Helm directly?
Flux HelmRelease: a Custom Resource that declaratively describes a Helm chart release — the chart source (HelmRepository, GitRepository, or OCI), version constraint, and values. The Helm controller continuously reconciles it, meaning: if someone runs helm upgrade directly and changes values, Flux will revert them on the next reconciliation cycle. This enforces GitOps: Helm charts are no longer deployed imperatively by CI pipelines. Flux CRDs: GitRepository: defines a Git source (URL, branch, credentials, interval). OCIRepository: pulls from an OCI (container) registry — e.g., Helm charts stored in ECR or GHCR. HelmRepository: points to a Helm chart repository. Kustomization: applies a kustomize directory from a GitRepository source, with health assessment and dependency ordering. HelmRelease: manages a Helm chart release with reconciliation. ImageRepository: watches a container registry for new image tags. ImagePolicy: defines tag selection policy (semver, alphabetical). ImageUpdateAutomation: automatically commits updated image tags to Git when new images match a policy. Flux vs ArgoCD: both implement GitOps for Kubernetes. Flux: CRD-native, composable controllers, better image automation. ArgoCD: richer UI, App of Apps pattern, sync waves, RBAC built-in. Both are CNCF graduated. In conversation: 'With Flux image automation, pushing a new image to ECR automatically opens a PR updating the image tag in our config repo. Humans review and merge; Flux deploys.'
4 / 5
An SRE explains a GitOps incident to the postmortem meeting: "Someone made an emergency hotfix directly in the cluster — kubectl edited the Deployment to bump the image tag. ArgoCD detected drift within three minutes and auto-synced, reverting the hotfix. The fix disappeared silently. In hindsight, the right process was: commit the fix to the Git repo, push, let ArgoCD sync. For true emergencies we have a 'break-glass' procedure: pause ArgoCD sync on that Application, apply manually, then immediately commit to Git and re-enable sync. This preserves the audit trail." What is drift in a GitOps context and why is it problematic?
Drift in GitOps: any difference between what Git declares (desired state) and what exists in the cluster (observed state). Sources of drift: Manual kubectl changes: the most common. Someone edits a Deployment, ConfigMap, or Secret directly. Cluster autoscaler or VPA changes: the platform modifies resource requests automatically. Admission webhooks: mutating webhooks inject or modify fields that aren't in the Git manifest. Operators: Kubernetes operators may update resources they manage, which might conflict with GitOps-managed manifests. In-place secret rotation: external secrets operators update secrets from vaults. Why problematic: in a GitOps model, the cluster should be fully reproducible from Git. If someone makes a kubectl change without committing to Git, that knowledge lives only in the cluster. If the cluster is rebuilt or ArgoCD auto-syncs, the change is lost. The team has no audit trail of who made the change or why. Break-glass procedures: formal process for emergency cluster changes outside normal GitOps flow. Best practice: (1) Pause GitOps reconciliation for the affected resource, (2) Apply emergency fix, (3) Immediately commit the same change to Git, (4) Re-enable reconciliation. All break-glass events should be logged and followed up in postmortem. In conversation: 'Drift is an organizational risk, not just a technical one. It means your Git repo no longer describes your production system — that's a knowledge problem as much as a tooling problem.'
5 / 5
A platform team lead presents their GitOps repository structure: "We use a mono-repo for all cluster configuration. The structure follows the environment-per-directory pattern: clusters/dev/, clusters/staging/, clusters/production/. Each directory has its own ArgoCD Application pointing to it. We use Kustomize for environment-specific overrides — base/ has the shared manifests, overlays/dev/ patches resource limits down and disables PodDisruptionBudgets, overlays/production/ bumps replicas and enables HPA. This way we have one set of base manifests and environment-specific differences tracked in Git." What is Kustomize and how does it support multi-environment GitOps?
Kustomize: built into kubectl (since 1.14) and all GitOps tools. Philosophy: plain YAML everywhere, no templating language. Instead of Go templates (Helm), Kustomize uses patches applied on top of base manifests. Structure: base/: common manifests (Deployment, Service, ConfigMap) valid for all environments. overlays/dev/: kustomization.yaml that references base and applies patches (reduce replicas, lower resource limits). overlays/production/: different patches (increase replicas, add HPA, enable PodDisruptionBudget). Kustomize features: namePrefix/nameSuffix: add environment prefix to all resource names. commonLabels/commonAnnotations: add labels across all resources. images: override image tags without editing manifests. configMapGenerator/secretGenerator: auto-generate ConfigMaps/Secrets with content hash appended to name (triggers rolling update on change). patches: strategic merge patches or JSON6902 patches for fine-grained overrides. Helm vs Kustomize: Helm uses Go templates and values.yaml (more complex, better for distributable charts). Kustomize uses overlays (simpler, better for managing your own apps across environments). GitOps repo patterns: Mono-repo: all environments in one repo. Poly-repo: separate repos per team or environment. Config repo separate from app repo: CI builds images, writes image tag to config repo, GitOps syncs config repo. In conversation: 'With Kustomize, dev and prod manifests share 90% of their YAML. The overlay only contains the diffs. That's much easier to audit than separate manifest copies per environment.'