← All Posts

GitOps with ArgoCD — A Practical Guide to Declarative Deployments

Matthias Bruns · · 8 min read
gitops argocd kubernetes devops engineering

The Deployment Problem Nobody Talks About

Your CI pipeline builds the image. Then what? Someone runs kubectl apply. Or a Helm command buried in a deploy step. Or a script that SSHes into a bastion host. Three months later, nobody knows what’s actually running in production — because the cluster state drifted from what’s in Git.

GitOps fixes this by making Git the single source of truth for your cluster state. ArgoCD is the most widely adopted GitOps operator for Kubernetes, and for good reason: it’s declarative, it auto-syncs, and it shows you exactly where reality diverges from intent.

This guide covers how to set it up properly — not the “hello world” version, but the version that survives real workloads.

What GitOps Actually Means

GitOps is a deployment pattern with four principles, formalized by the OpenGitOps project:

  1. Declarative — the entire desired system state is described declaratively (YAML, Helm charts, Kustomize overlays)
  2. Versioned and immutable — the desired state is stored in Git, giving you audit trail and rollback for free
  3. Pulled automatically — an agent (ArgoCD) continuously reconciles cluster state to match Git
  4. Continuously reconciled — drift is detected and corrected without manual intervention

The key shift: your CI pipeline no longer touches the cluster. It builds and pushes an image, then updates a manifest in Git. ArgoCD handles the rest.

Installing ArgoCD

The recommended installation uses the official Helm chart. For production, use the HA manifest:

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml

Or via Helm for more control:

helm repo add argo https://argoproj.github.io/argo-helm
helm install argocd argo/argo-cd \
  --namespace argocd \
  --create-namespace \
  --set server.extraArgs={--insecure} \
  --set configs.params."server\.insecure"=true

The --insecure flag disables TLS on the ArgoCD server itself — you’ll terminate TLS at your ingress controller, which is the standard pattern. Don’t expose ArgoCD without TLS termination somewhere in the chain.

Get the initial admin password:

kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath="{.data.password}" | base64 -d

Change it immediately. Better yet, configure SSO via OIDC and disable the admin account entirely.

Repository Structure That Scales

The most common mistake: putting application manifests in the same repo as the application code. This creates circular dependencies — a code change triggers CI, which updates manifests, which triggers CI again.

Use a dedicated config repository:

infra-gitops/
├── apps/
│   ├── api/
│   │   ├── base/
│   │   │   ├── deployment.yaml
│   │   │   ├── service.yaml
│   │   │   └── kustomization.yaml
│   │   └── overlays/
│   │       ├── staging/
│   │       │   ├── kustomization.yaml
│   │       │   └── replicas-patch.yaml
│   │       └── production/
│   │           ├── kustomization.yaml
│   │           └── replicas-patch.yaml
│   └── frontend/
│       ├── base/
│       └── overlays/
├── platform/
│   ├── cert-manager/
│   ├── ingress-nginx/
│   └── monitoring/
└── argocd/
    ├── projects.yaml
    └── applicationsets.yaml

Key decisions:

  • Kustomize over Helm for app manifests — Helm is great for third-party charts. For your own services, Kustomize overlays are simpler to review in PRs and easier to diff.
  • Separate apps/ from platform/ — platform components (cert-manager, ingress, monitoring) have different change cadences and approval requirements.
  • One directory per environment overlay — makes promotion explicit and auditable.

Application Manifests

An ArgoCD Application resource tells the controller what to deploy and where:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: api-staging
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/infra-gitops.git
    targetRevision: main
    path: apps/api/overlays/staging
  destination:
    server: https://kubernetes.default.svc
    namespace: api
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

The critical settings:

  • automated.prune: true — removes resources that no longer exist in Git. Without this, deleted manifests leave orphaned resources in the cluster.
  • automated.selfHeal: true — reverts manual kubectl changes. This is the whole point of GitOps — if someone patches something by hand, ArgoCD corrects it.
  • retry — transient failures happen (API server overload, webhook timeouts). Exponential backoff prevents cascading retries.

ApplicationSets for Multi-Environment

Managing individual Application resources per service per environment doesn’t scale. ApplicationSets generate Applications from templates:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: apps
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - git:
        repoURL: https://github.com/your-org/infra-gitops.git
        revision: main
        directories:
          - path: apps/*/overlays/*
  template:
    metadata:
      name: '{{ index .path.segments 1 }}-{{ index .path.segments 3 }}'
    spec:
      project: default
      source:
        repoURL: https://github.com/your-org/infra-gitops.git
        targetRevision: main
        path: '{{ .path.path }}'
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{ index .path.segments 1 }}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

This auto-discovers every apps/<service>/overlays/<env> directory and creates an Application for it. The goTemplate: true flag enables Go template syntax, which is the recommended mode for new ApplicationSets. Add a new service? Create the directory structure, push, done.

Secrets Management

The one thing you cannot put in Git as-is: secrets. Three viable approaches:

Sealed Secrets

Bitnami Sealed Secrets encrypts secrets client-side with a cluster-specific public key. Only the controller in the cluster can decrypt them.

kubeseal --format yaml < secret.yaml > sealed-secret.yaml

The sealed secret is safe to commit. Simple, but key rotation requires re-encrypting all secrets.

External Secrets Operator

External Secrets Operator syncs secrets from external stores (AWS Secrets Manager, Vault, GCP Secret Manager) into Kubernetes secrets:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: api-secrets
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets
    kind: ClusterSecretStore
  target:
    name: api-secrets
  data:
    - secretKey: DATABASE_URL
      remoteRef:
        key: /production/api/database-url

This is the recommended approach for production. Secrets live in a dedicated secrets manager with its own access policies, audit logging, and rotation. The Kubernetes secret is derived, not source-of-truth.

SOPS

Mozilla SOPS encrypts specific values in YAML files. ArgoCD has native SOPS support via a plugin. Good middle ground if you don’t want to run an external secrets manager.

Environment Promotion

The promotion flow in GitOps:

  1. CI builds image api:sha-abc123 and pushes to registry
  2. CI opens a PR against the config repo updating the staging overlay’s image tag
  3. ArgoCD syncs staging automatically
  4. After validation, a second PR (or manual merge) updates the production overlay
  5. ArgoCD syncs production

Automate step 2 with image updater or a simple CI job:

# In your app repo's CI pipeline (separate step after cloning infra-gitops)
- name: Update staging manifest
  run: |
    kustomize edit set image api=$IMAGE_TAG
    git commit -am "chore: update api to $IMAGE_TAG"
    git push
  working-directory: infra-gitops/apps/api/overlays/staging

For production promotion, require a PR with approval. This gives you:

  • Audit trail — who approved the production deploy and when
  • Rollback — git revert the merge commit
  • Diff review — the PR shows exactly what changed between environments

Sync Windows and Waves

Not everything should sync immediately. ArgoCD supports sync windows to restrict when syncs happen:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: production
spec:
  syncWindows:
    - kind: allow
      schedule: '0 8-17 * * 1-5'
      duration: 9h
      applications: ['*']
      timeZone: Europe/Berlin

This restricts production syncs to business hours on weekdays — when someone is around to respond if things break.

For ordering dependencies (database migration before app deployment), use sync waves:

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-1"  # Runs before wave 0

Negative waves run first. Use wave -1 for migrations, wave 0 for the app, wave 1 for post-deploy checks.

Monitoring and Alerts

ArgoCD exposes Prometheus metrics on :8082/metrics. The essential alerts:

  • App sync failed — argocd_app_info{sync_status="OutOfSync"} for more than 10 minutes
  • App health degraded — argocd_app_info{health_status!="Healthy"} for more than 5 minutes
  • Sync operation errors — argocd_app_sync_total{phase="Error"} rate increase

ArgoCD also supports notifications via Slack, Teams, webhooks, or email. Configure at minimum: sync failed and health degraded notifications for production apps.

Common Pitfalls

Putting secrets in Git unencrypted. Sounds obvious, but it happens. Use pre-commit hooks with gitleaks or detect-secrets to catch this before it lands.

Not enabling pruning. Without prune: true, deleting a manifest from Git leaves the resource running. You end up with ghost services that nobody maintains.

Ignoring resource hooks. ArgoCD respects resource hooks for pre-sync and post-sync jobs. Use PreSync hooks for database migrations instead of init containers — they’re visible in the ArgoCD UI and have proper error handling.

Syncing everything to one namespace. Use the destination namespace in your Application to enforce namespace boundaries. Combine with ArgoCD projects to restrict which namespaces a team can deploy to.

The Bottom Line

GitOps with ArgoCD removes the “what’s running in production?” guessing game. Every deployment is a Git commit. Every rollback is a revert. Every change is reviewable.

The setup investment is front-loaded — repository structure, ApplicationSets, secrets management, sync policies. Once it’s running, deployments become boring. And boring deployments are exactly what you want.

Start with one service in staging. Get the feedback loop right. Then expand to production and add services incrementally. Don’t try to migrate everything at once.

Reader settings

Font size