Unnecessary Harbor Pod Restarts Triggered on every PR merge

[amar-opsverse]

Expected behavior and actual behavior:

Expected behavior:
Harbor pods should remain stable without unnecessary restarts when no functional changes are made to Harbor's configuration or resources. ArgoCD should only trigger syncs for actual changes in Harbor's deployment or configuration.

Actual behavior:
Every time a PR is merged into the GitOps repository, ArgoCD detects changes in Harbor-related resources (like secrets or ConfigMaps) and triggers a sync. This results in Harbor pods restarting, even when there are no functional changes. This disrupts availability, impacts builds, and causes unnecessary delays in deployments.
Despite adding the ignoreDifferences rule for Harbor-related secrets in the ArgoCD App YAML, the pod restarts continue due to the checksum in Helm templates for the Deployment and StatefulSet, which detects changes in the associated resources.

Steps to reproduce the problem:

Set up Harbor with ArgoCD for GitOps-based deployment.
Add Harbor to ArgoCD with ignoreDifferences set for secrets and ConfigMaps in the ArgoCD App YAML.
Merge any PR to the GitOps repository that isn't related to Harbor.
Observe that ArgoCD triggers a sync, resulting in Harbor pods restarting, even if no changes were made to Harbor-related resources.
Versions:

Harbor version: v2.5.5
Kubernetes version: 1.31
ArgoCD version: v2.3.3
Helm version: v3.15.4

harbor:
  expose:
    type: ingress
    tls:
      enabled: true
      certSource: secret
      secretName: infra-harbor-ingress
      notarySecretName: infra-harbor-ingress-notary
    ingress:
      hosts:
        core: registry.devopsnow.io
        notary: notary-registry.devopsnow.io
      controller: default
      annotations:
        ingress.kubernetes.io/ssl-redirect: "true"
        ingress.kubernetes.io/proxy-body-size: "0"
        nginx.ingress.kubernetes.io/ssl-redirect: "true"
        nginx.ingress.kubernetes.io/proxy-body-size: "0"
        cert-manager.io/cluster-issuer: cluster-letsencrypt-prod
    clusterIP:
      name: harbor
      annotations: {}
      ports:
        httpPort: 80
        httpsPort: 443
        notaryPort: 4443

  externalURL: https://registry.devopsnow.io

  persistence:
    enabled: true
    resourcePolicy: "keep"
    imageChartStorage:
      type: s3
      s3:
        region: us-east-1
        bucket: "<BUCKET_NAME>"
        accesskey: "<ACCESS_KEY>"
        secretkey: "<SECRET_KEY>"

  updateStrategy:
    type: Recreate

  harborAdminPassword: "<HARBOR_ADMIN_PASSWORD>"

  core:
    replicas: 2

  registry:
    replicas: 2
    relativeurls: true

  chartmuseum:
    replicas: 1

  trivy:
    enabled: true
    image:
      tag: v2.9.1
    gitHubToken: "<GITHUB_TOKEN>"
    skipUpdate: false

  notary:
    server:
      replicas: 2
    signer:
      replicas: 2

[MinerYang]

Hi @amar-opsverse ,

Could you help us to understand more about the situation? Since you mentioned ArgoCD detects changes in Harbor-related resources (like secrets or ConfigMaps) and triggers a sync, so what cause this ca.crt in the secret changed exactly?

[amar-opsverse]

We are also uncertain about the origin of these changes, but we suspect they may be related to the dynamic nature of the secret used for internal communication in the Helm values file.

core:
  image:
    repository: goharbor/harbor-core
    tag: v2.5.5
  # Secret is used when core server communicates with other components.
  # If a secret key is not specified, Helm will generate one.
  # Must be a string of 16 chars.
  secret: ""

[MinerYang]

Could you provide your expose.tls configuration in the values.yaml?

[amar-opsverse]

Could you provide your expose.tls configuration in the values.yaml?

  expose:
    # Set the way how to expose the service. Set the type as "ingress",
    # "clusterIP", "nodePort" or "loadBalancer" and fill the information
    # in the corresponding section
    type: ingress
    tls:
      enabled: true
      # The source of the tls certificate. Set it as "auto", "secret"
      # or "none" and fill the information in the corresponding section
      # 1) auto: generate the tls certificate automatically
      # 2) secret: read the tls certificate from the specified secret.
      # The tls certificate can be generated manually or by cert manager
      # 3) none: configure no tls certificate for the ingress. If the default
      # tls certificate is configured in the ingress controller, choose this option
      certSource: secret
      secret:
        # The name of secret which contains keys named:
        # "tls.crt" - the certificate
        # "tls.key" - the private key
        secretName: infra-harbor-ingress
        # The name of secret which contains keys named:
        # "tls.crt" - the certificate
        # "tls.key" - the private key
        # Only needed when the "expose.type" is "ingress".
        notarySecretName: infra-harbor-ingress-notary

[amar-opsverse]

https://medium.com/@imacq/ignoring-differences-in-argocd-e531807bfea0

This doc explains a similar issue