[ENHANCE] Remove / allow removing timestamp from pod template reloader.stakater.com/last-reloaded-from annotation

[michalrysavy-ext95730]

Is your feature request related to a problem? Please describe.
I would like to guarantee deployments restart also when the config change occurs during reloader outage. It is possible to use reloadOnCreate+syncAfterRestart, but it's also problematic for us.
Using annotations reloadStrategy, in addition to hash, the annotation also contains timestamp (observedAt field in embedded json in annotation). And this causes that every monitored deployment is restarted (even without configuration changes) each time the reloader is restarted or it's leader is elected (which is also described in documentation).
If the timestamp was removed from the annotation, annotation value would be the same and the deployment wouldn't be restarted (the annotation would be completely same, of course).
This works well using env-vars strategy because generated environment variables values contain hashes only (and no timestamp or other information). But I personally prefer annotation-base strategy and it would be better to work in the same way.

Describe the solution you'd like
I'd like to have possibility (even using annotation strategy) to restart deployment even when the config change occurs during reloader outage (after reloader recovery, of course) but avoid all deployments restart when reloader is restarted.
My suggestion is to have annotation value strictly same when the configuration is same (i.e. no timestamp in value and a predictable way to order fields in embedded json in annotation).

Describe alternatives you've considered
Timestamp doesn't have to be removed, but it could be moved to another annotation on the deployment level (instead of pod level).

Additional context
We are using ArgoCD

[MuneebAijaz]

@michalrysavy-ext95730 feel free to open a PR for this change

[michalrysavy-ext95730]

I tried to change behavior, but I realized that it is not so trivial. Problem is that the checksum in the annotation is not whole-config checksum, but only last reconciled configMap/secret resource checksum. So removing timestamp from annotation works perfect only when there is single config resource for the deployment.

Having more config sources it doesn't work and during controller (Reloader) restart (having syncAfterRestart) deployment is restarted as many times as it has config sources. For example having three configMaps in envFrom, first checksum is calculated for first config and stored in the annotation and it invokes restart, then checksum for second configMap is calculated and stored in the annotation and it also invokes restart (because annotation is changed due to different checksum for different configMaps) and finally the same for the third configMap.

So I'm afraid that the annotation strategy is bad, because it has single common annotation that is always used for different piece of configuration. It should be changed in the one of the following ways:

• single annotation with single checksum, but for the whole ordered config
• single annotation with multiple checksums (each for every configMap/secret), but the final json has to be predictable and always same when the config is unchanged
• multiple annotations, each for every configMap/secret (like in the envs strategy)