Zum Inhalt

Central ArgoCD GitOps Repository

Einordnung ins Overtime-Projekt

Für Overtime wurde bewusst zwischen Anwendungscode und Deployment-Konfiguration getrennt. Dieses ADR beschreibt, warum alle Kubernetes-Manifeste in einem zentralen GitOps-Repository statt verteilt über die einzelnen Service-Repos verwaltet werden.

1. Kontext & Problemstellung (Deutsch)

Kubernetes-Manifeste für deployte Applikationen müssen versioniert und verwaltet werden. In der Praxis sind zwei Muster verbreitet: Die Manifeste liegen direkt im Quellcode-Repository der jeweiligen Anwendung ("Single Repo" Ansatz), oder sämtliche Deployment-Manifeste werden in einem dedizierten Repository, strikt getrennt vom Anwendungscode, gebündelt ("GitOps Repo" Ansatz).

Diese Architektur-Entscheidung hat weitreichende Konsequenzen für die Zugriffssteuerung (Access Control), die Nachvollziehbarkeit (Auditability) und die konzeptionelle Trennung zwischen "Was macht der Code?" (Applikation) und "Wo und wie läuft der Code?" (Infrastruktur). Da sich in Overtime mehrere unabhängige Applikationen und Infrastruktur-Komponenten dieselben Ziel-Cluster teilen, muss eine skalierbare und sichere Struktur gefunden werden.

2. Architecture Decision Record (English)

ADR-009: Central ArgoCD GitOps Repository

Status

Accepted

Context and Problem Statement

Kubernetes manifests for deployed applications need a version-controlled home. Two patterns are common: colocating manifests in each application's source repo or keeping all deployment manifests in a dedicated repository separate from application code. The choice directly impacts access control, auditability, and the clean separation between application logic and deployment state.

Decision Drivers

  • Independent Auditability: The deployment history must be independently auditable from the application code history.
  • Least Privilege CI: CI pipelines must be able to update image tags programmatically without requiring write access to the application source repositories.
  • Single Source of Truth: ArgoCD must be able to watch a single, trusted source of truth for the entire cluster state.
  • Co-Management: Multiple unrelated applications and infrastructure components (Django backend, MySQL, MinIO, mysql-backup) share the same clusters — their manifests should be co-managed efficiently.

Considered Options

  • Manifests in each app repo — Simpler to start. However, CI pipelines need write access to the app repos to bump versions, deployment history gets mixed with code history, and ArgoCD has to be configured to watch dozens of disparate repositories.
  • Central GitOps repo — All Kubernetes manifests live in one central repository. Provides clean access control, a single ArgoCD source of truth, and CI runners only need a scoped deploy token for this specific repository.

Decision Outcome

Chosen: Central overtime-gitops repository.

By centralizing the state, we achieve a strict separation of concerns. Developers work in the overtime-backend repository. When a release is cut, the CI pipeline pushes solely the new image tag to the overtime-gitops repository. ArgoCD monitors only this GitOps repository and synchronizes the state to the clusters.

Consequences & Trade-offs

Positive Consequences
  • Absolute Source of Truth: ArgoCD relies on one central repository. Cluster drift is always detectable by comparing the live cluster state directly to the overtime-gitops/main branch.
  • Clean Audit Log: The deployment history is a clean git log consisting of deploy: update X to <sha> commits. This can be reviewed and audited entirely separately from application feature commits.
  • Granular Security: Access to production manifests can be restricted via GitLab repository permissions independently of the application source code repositories.
  • Frictionless Onboarding: Adding a new service simply requires adding an ApplicationSet and an apps/{service}/ directory in the GitOps repo — zero changes to CI templates or the app repositories are necessary.
Negative Consequences / Trade-offs
  • Single Point of Failure: overtime-gitops becomes a strict dependency for all deployments. It must remain highly available and accessible from the management cluster's ArgoCD controller.

3. Implementation & Repository Structure (English)

The GitOps repository utilizes Kustomize and ArgoCD's App-of-Apps / ApplicationSet pattern to dynamically map folders to target clusters.

Directory Layout

(Note: In the structure below, test and production overlays are consistently mapped to overtime-staging and overtime-production across both apps and infrastructure for clarity).

overtime-gitops/
├── apps/
│   ├── django-backend/
│   │   ├── base/                     # Shared manifest set
│   │   │   ├── deployment.yaml
│   │   │   └── kustomization.yaml
│   │   └── overlays/
│   │       ├── overtime-production/  # Prod environment patches
│   │       └── overtime-staging/     # Staging environment patches
│   └── example-app/
├── infra/
│   ├── minio/
│   │   ├── base/
│   │   └── overlays/
│   │       ├── overtime-production/
│   │       └── overtime-staging/
│   ├── mysql/
│   │   ├── base/
│   │   └── overlays/
│   │       ├── overtime-production/
│   │       └── overtime-staging/
│   └── mysql-backup/
│       ├── base/
│       └── overlays/
│           ├── overtime-production/
│           └── overtime-staging/
└── root-app/                         # ArgoCD ApplicationSets
    ├── appproject-apps.yaml          # AppProject: application workloads
    ├── appproject-infrastructure.yaml# AppProject: infra components
    ├── appset-django-backend.yaml    # ApplicationSet → edge clusters
    ├── appset-minio.yaml
    ├── appset-mysql.yaml
    ├── appset-mysql-backup.yaml
    └── appset-sealed-secrets.yaml    # ApplicationSet → all clusters

Here is an example of the Django backend ApplicationSet: 

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: django-backend
  namespace: argocd
spec:
  generators:
    # Targets only clusters with the label cluster-role=edge (set by ansible edge-register.yml).
    # Cluster names must match overlay directory names: overtime-staging, overtime-production
    - clusters:
        selector:
          matchLabels:
            cluster-role: edge
  template:
    metadata:
      name: 'django-backend-{{name}}' # e.g. django-backend-overtime-staging
      labels:
        category: apps
        target-cluster: '{{name}}'
      annotations:
        # Only refresh this app when files in its own overlay path change.
        # Prevents a single monorepo commit from fanning out to all apps.
        argocd.argoproj.io/manifest-generate-paths: "."
    spec:
      project: apps
      source:
        repoURL: 'https://gitlab.com/alexb-overtime/overtime-argocd.git'
        targetRevision: HEAD
        path: 'apps/django-backend/overlays/{{name}}'
      destination:
        server: '{{server}}'
        namespace: django-backend
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

The App-of-Apps Pattern

ArgoCD is bootstrapped exactly once to watch the root-app/ directory. Each ApplicationSet defined in root-app/ acts as a generator. It creates one ArgoCD Application per matching target cluster using Kubernetes cluster label selectors.

Adding a new cluster to the corresponding label group automatically provisions all assigned applications to it — no manual ArgoCD UI configuration is required.

Write Access Model

Only one credential requires write access to overtime-gitops: the GITOPS_DEPLOY_TOKEN. This is a GitLab deploy token used exclusively by the .update-gitops CI job. Application source repos (e.g., overtime-backend) have absolutely no write access to overtime-gitops.

[overtime-backend CI]  →  writes to  →  [overtime-gitops] (Image tag update only)
                                                ↓ (ArgoCD polls git)
                                         [Target Cluster State]

Rollback Strategy

Rolling back a deployment is entirely Git-native. It simply requires a git revert of the specific automated image tag commit within overtime-gitops. ArgoCD syncs the previous tag automatically within its standard polling interval. No kubectl rollout undo or Helm rollback mechanics are needed or permitted.

4. Compliance Einordnung (Deutsch)

Das detaillierte Mapping auf BSI IT-Grundschutz, CRA und CIS Benchmarks wird später ergänzt.