Semgrep for Static Application Security Testing (SAST)

Einordnung ins Overtime-Projekt

Damit sicherheitskritische Muster, etwa das versehentliche Loggen von personenbezogenen Daten aus Overtime-Chats oder Passwörter, nicht erst im Review, sondern schon in der Pipeline auffallen, wird Static Application Security Testing (SAST) eingesetzt.

---

1. Kontext & Problemstellung (Deutsch)

Für das Overtime-Backend müssen sicherheitskritische Code-Muster abgefangen werden, bevor sie in einen Build oder gar in die Produktion gelangen. Neben klassischen Schwachstellen aus den OWASP Top Ten (wie SQL-Injection, Cross-Site Scripting (XSS) oder unsichere Deserialisierung) gibt es in einer Django-Umgebung spezifische Fallstricke, wie z.B. fehlerhaft implementierte CSRF-Bypasses oder die unvorsichtige Nutzung von mark_safe und raw()-SQL-Queries.

Ein weiteres Risiko für die Overtime-Plattform ist das versehentliche Loggen von personenbezogenen Daten (PII). Wenn Entwickler:innen String-Interpolationen (f-Strings) oder %-Formatierungen direkt in Logging-Aufrufen verwenden, werden sensible Werte fest in den Log-String gebacken, ohne dass PII-Scrubber sinnvoll eingreifen können.

---

2. Architecture Decision Record (English)

ADR-013: Semgrep for SAST

Status

Accepted

Context and Problem Statement

Static Application Security Testing (SAST) must catch security anti-patterns before they reach a build: SQL injection, XSS, insecure deserialization, use of unsafe Django APIs, OWASP Top Ten patterns, and project-specific rules such as PII leaking through string interpolation in log calls. The tool must run in CI without a paid SaaS dependency and must be highly extensible with custom structural rules.

Decision Drivers

• Framework Coverage: Must cover Django-specific patterns (CSRF bypass, mark_safe, raw() SQL, unsafe deserialization).
• Extensibility: Project-specific rules (e.g., PII logging anti-patterns) must be easily expressible and testable within the same tool.
• No Vendor Lock-in: No SaaS token or paid tier required for utilizing registry rulesets in open-source CI mode.
• Pipeline Integration: Must support returning an exit-code 1 on critical findings to actively block failing CI pipelines.
• Compliance: Align with standard security requirements (e.g., automated security checks in the delivery pipeline).

Considered Options

• Bandit — A well-known Python-only tool. However, it lacks deep Django-specific rules, cannot be easily extended to catch custom structural patterns (like our specific structlog issues), and has been largely unmaintained since 2022.
• SonarQube — Highly comprehensive, but requires maintaining a heavy self-hosted server or paying for a cloud tier.
• Semgrep — Offers rich registry rulesets for Django, Python, OWASP, and Docker. Custom rules can be written natively in YAML. The OSS mode requires no token and runs as a single static binary, easily extensible via local .semgrep/ rules.

Decision Outcome

Chosen: Semgrep. We will utilize the registry rulesets p/django p/python p/owasp-top-ten p/docker, actively extended by project-local rules stored in .semgrep/.

Rule Sources & Custom Configurations

semgrep ci evaluates two sources of rules in parallel. No CI pipeline configuration changes are required to add new local rules:

Source	How it is loaded	What it covers
p/django p/python p/owasp-top-ten p/docker	Via SEMGREP_RULES environment variable	Django security, Python anti-patterns, OWASP Top Ten, Dockerfile best practices
.semgrep/*.yml (in the repo root)	Auto-discovered by semgrep ci	Project-specific rules (e.g., custom PII logging bans)

Consequences & Trade-offs

Positive Consequences

• Version-Controlled Security: Custom rules ship directly in the application repository (.semgrep/) and are versioned alongside the code they protect.
• Zero-Friction Adoption: No separate pipeline job or SaaS token is needed for custom rules — semgrep ci discovers them automatically.
• Strict Gating: Rule findings block the pipeline directly (semgrep ci exits non-zero on errors).
• Agility: New PII patterns or framework anti-patterns can be banned instantly by adding a simple YAML rule, rather than relying on slow process changes or human reviews.

Negative Consequences / Trade-offs

• Network Dependency: Registry rulesets are pulled from the internet at scan time. An air-gapped setup would require maintaining a local Semgrep registry mirror.
• No Central Dashboard: semgrep ci in OSS mode does not post findings to a central SaaS dashboard. Results are surfaced purely via artifacts: reports: sast (SARIF output) in the GitLab Security Dashboard and Merge Request security widgets.
• Learning Curve: Writing custom rules requires developers to learn and maintain knowledge of Semgrep's specific pattern syntax (or use LLMs).

---

3. Pipeline Implementation & Code Snippet (English)

The following GitLab CI template (overtime-ci-templates/sast.yml) executes Semgrep as a fast, localized CI job. Downstream repositories simply include this template and override the SEMGREP_RULES variable if different tech stacks (like Ansible or Kubernetes) require other registry rules.

.semgrep-sast:
  stage: test
  image: semgrep/semgrep:1.166.0@sha256:c180f0c93a17b420c0af5006214a29d3c747c5459c732b740191adf657dd0068
  variables:
    SEMGREP_VERSION: "1.166.0"
    # Offizielle und von der Community gepflegte Regelsätze für Django, Python und Docker
    SEMGREP_RULES: "p/django p/python p/owasp-top-ten p/docker"
  script:
    # --sarif writes ALL findings (new and pre-existing) to the artifact so they
    # are visible in the MR security widget regardless of diff-mode filtering.
    # The exit code is preserved: non-zero on blocking findings, zero otherwise.
    - semgrep ci --sarif --output semgrep.sarif
  after_script:
    # Ensure the artifact exists even when the job is cancelled or errors early.
    - '[ -f semgrep.sarif ] || echo "{}" > semgrep.sarif'
  artifacts:
    reports:
      sast: semgrep.sarif
    paths:
      - semgrep.sarif
    expose_as: "Semgrep SAST Report"
    when: always

Example Custom Rules Implementation & PII Anti-Patterns

String interpolation in log calls bakes values into the event string before the application's PII processor chain runs, making programmatic scrubbing difficult or sometimes impossible. We ban these patterns via .semgrep/pii-logging.yml.

Code Violations and Correct Patterns

1. f-string as log event: The value is interpolated by Python before structlog sees it.

# ❌ Blocked by Semgrep
log.info(f"customer {name} called from {phone}")

# ✅ Correct: name is a named field, drop_pii_keys_processor removes it and phone gets anonymized to +49XXXXX for example.
log.info("customer called", phone=phone)

2. %-formatted string as log event: Same underlying problem via old-style formatting.

# ❌ Blocked by Semgrep
log.warning("failed for %s" % user_email)

# ✅ Correct
log.warning("request failed", user_email=user_email)

Example Concrete Semgrep Rule Definition (pii-logging.yml)

The following definition implements the exact abstract AST logic required to look into the logging arguments and parse string literals safely without colliding with Semgrep's brace syntax:

rules:
  - id: structlog-no-fstring-event
    message: >
      f-string passed as the log event bypasses structlog's PII processor chain.
      Python evaluates the f-string before structlog sees any arguments, so
      drop_pii_keys_processor and pii_scrub_processor cannot redact embedded values.
      Use kwargs instead: log.info("event description", field=value)
    severity: ERROR
    languages: [python]
    patterns:
      - pattern: $LOG.$METHOD(f"...", ...)
      - metavariable-regex:
          metavariable: $METHOD
          regex: ^(debug|info|warning|warn|error|critical|exception|msg)$

---

4. Compliance Einordnung (Deutsch)

Das detaillierte Mapping auf BSI IT-Grundschutz, CRA und CIS Benchmarks wird später ergänzt.