feat: print diff resource as yaml #2542

10000-ki · 2024-09-29T07:09:21Z

resolved: #2492

AS-IS

Output diff data in text format

{data={key1=val1}, metadata={ownerReferences=[{name=kube-root-ca.crt, uid=1ef74cb4-dbbd-45ef-9caf-aa76186594ea, apiVersion=v1, kind=ConfigMap}]}}

{metadata={ownerReferences=[{apiVersion=v1, kind=ConfigMap, name=kube-root-ca.crt, uid=1ef74cb4-dbbd-45ef-9caf-aa76186594ea}]}, data={key1=different value}}

problem : It's difficult to compare

TO-BE

Output diff data in yml format

data:
  key1: "val1"
metadata:
  ownerReferences:
  - name: "kube-root-ca.crt"
    uid: "1ef74cb4-dbbd-45ef-9caf-aa76186594ea"
    apiVersion: "v1"
    kind: "ConfigMap"
 
 ---
metadata:
  ownerReferences:
  - apiVersion: "v1"
    kind: "ConfigMap"
    name: "kube-root-ca.crt"
    uid: "1ef74cb4-dbbd-45ef-9caf-aa76186594ea"
data:
  key1: "different value"

If the user sorts fields through Jackson settings .enable(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS)
it will look much better because the keys will be displayed in sorted order.

Signed-off-by: 10000-ki <[email protected]>

10000-ki · 2024-09-29T07:10:52Z

...orsdk/operator/processing/dependent/kubernetes/SSABasedGenericKubernetesResourceMatcher.java

-      log.debug("Pruned actual: \n {} \n desired: \n {} ", prunedActual, desiredMap);
+      var actualYml = objectMapper.asYaml(prunedActual);
+      var desiredYml = objectMapper.asYaml(desiredMap);
+      log.debug("Pruned actual: \n {} \n desired: \n {} ", actualYml, desiredYml);


It is possible to use a library or write code to compare YAML files. However, as the size of the YAML file increases, the cost of processing grows. Therefore, I have not added this functionality here and simply focused on outputting the data.

If deemed necessary, I can consider adding it in the future.

I'm not sure that the sorting would make that much of a difference compared to serializing to YAML in the first place…

Signed-off-by: 10000-ki <[email protected]>

10000-ki · 2024-09-29T07:12:59Z

As mentioned above, users can configure Jackson to sort field keys when logging YAML files by enabling the SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS setting
This ensures that the YAML fields are ordered alphabetically, making the log output more readable and consistent

However, I'm not sure if this should be supported directly within josdk. Since sorting itself incurs a performance cost, it might be better to allow users to opt-in selectively if they need it, rather than enabling it by default.

metacosm · 2024-09-30T07:56:26Z

...orsdk/operator/processing/dependent/kubernetes/SSABasedGenericKubernetesResourceMatcher.java

@@ -104,7 +104,9 @@ public boolean matches(R actual, R desired, Context<?> context) {
    removeIrrelevantValues(desiredMap);

    if (LoggingUtils.isNotSensitiveResource(desired)) {
-      log.debug("Pruned actual: \n {} \n desired: \n {} ", prunedActual, desiredMap);
+      var actualYaml = objectMapper.asYaml(prunedActual);


I would protect these calls with a check to whether debug is enabled so as to not incur the YAML serialization cost in that situation.

I would protect these calls with a check to whether debug is enabled so as to not incur the YAML serialization cost in that situation.

yes it would be better thank you, i will fix it

@metacosm
72911d6

fixed it in this commit

thank you!!

Signed-off-by: 10000-ki <[email protected]>

metacosm

Thank you!

Signed-off-by: 10000-ki <[email protected]>

csviri · 2024-09-30T08:23:30Z

...orsdk/operator/processing/dependent/kubernetes/SSABasedGenericKubernetesResourceMatcher.java

+      var desiredYaml = serialization.asYaml(desiredMap);
+      log.debug("Pruned actual yaml: \n {} \n desired yaml: \n {} ", actualYaml, desiredYaml);
+    } else {
+      log.debug("Pruned actual map: \n {} \n desired map: \n {} ", prunedActualMap, desiredMap);


Not sure if I follow this. "If debug is not enabled we log something on debug" ?

Ah yes, missed that, though there should actually be no output since debug is checked internally (iirc) but it does make sense to remove the else branch altogether, indeed.

https://github.com/operator-framework/java-operator-sdk/pull/2542/files#diff-7568f2dc58ce3a6a5d7ce2d5227aeb49402136b091fb19a4d8cadb28351ce709L107

we remain the debug logging regardless of whether the debug mode was enabled.
Would it be better to remove the else block?

Although the cost of the else block itself doesn't seem significant, I'd like to hear your thoughts.

What I mean is that the else block never logs any message, since only called if no debug is enabled but logs on debug level.

@csviri

Ah understand

96920ab

I thought it would be better to just remove this part, so I modified it as shown above. What do you think?

Signed-off-by: 10000-ki <[email protected]>

csviri

LGTM

csviri · 2024-09-30T20:57:46Z

Thank you @10000-ki !

bachmanity1 · 2024-10-02T05:38:23Z

Hi @10000-ki. Thanks for the PR!

I think printing the whole actual and desired resource is a bit inconvenient since we still need to copy and paste it into other tools to get the real diff. How about improving the logging logic to directly print the actual diff, like shown below?

2024-10-02 14:30:29,489 1 i.j.o.p.d.k.SSABasedGenericKubernetesResourceMatcher [DEBUG] Diff between actual and desired state for resource: Deployment with name: test in namespace: default is: 
--- 
+++ 
@@ -15,1 +15,1 @@
-      - image: "nginx:1.17.0"
+      - image: "nginx:1.17.1"

I can work on this if you think the change would be helpful.

metacosm · 2024-10-02T07:17:32Z

Hi @bachmanity1,

I was kind of thinking the same thing but seeing the resource in YAML format is already helpful. We'd happily consider a PR to output the actual diffs! Keep in mind, though, that we want to keep this output as efficient as possible so it should probably be only activated via a flag.

10000-ki · 2024-10-02T08:40:28Z

@bachmanity1

I think printing the whole actual and desired resource is a bit inconvenient since we still need to copy and paste it into other tools to get the real diff. How about improving the logging logic to directly print the actual diff, like shown below?

oh sure great!!

feat: print diff resource as yaml

ea02163

Signed-off-by: 10000-ki <[email protected]>

openshift-ci bot requested review from adam-sandor and csviri September 29, 2024 07:09

10000-ki commented Sep 29, 2024

View reviewed changes

feat: fix naming

2633cfd

Signed-off-by: 10000-ki <[email protected]>

metacosm requested changes Sep 30, 2024

View reviewed changes

feat: logging in yaml format only in debug mode

72911d6

Signed-off-by: 10000-ki <[email protected]>

metacosm approved these changes Sep 30, 2024

View reviewed changes

10000-ki added 2 commits September 30, 2024 17:11

feat: fix log msg

200983b

Signed-off-by: 10000-ki <[email protected]>

feat: fix lint error

c09abc2

Signed-off-by: 10000-ki <[email protected]>

csviri requested changes Sep 30, 2024

View reviewed changes

feat: remove useless code

96920ab

Signed-off-by: 10000-ki <[email protected]>

csviri approved these changes Sep 30, 2024

View reviewed changes

metacosm approved these changes Sep 30, 2024

View reviewed changes

csviri merged commit 7ba6f8a into operator-framework:next Sep 30, 2024
20 checks passed

bachmanity1 mentioned this pull request Oct 2, 2024

feat: improve diff logging #2544

Merged

metacosm pushed a commit that referenced this pull request Oct 10, 2024

feat: print diff resource as yaml (#2542)

d3390c9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: print diff resource as yaml #2542

feat: print diff resource as yaml #2542

10000-ki commented Sep 29, 2024 •

edited

Loading

10000-ki Sep 29, 2024

metacosm Sep 30, 2024

10000-ki commented Sep 29, 2024 •

edited

Loading

metacosm Sep 30, 2024 •

edited

Loading

10000-ki Sep 30, 2024 •

edited by metacosm

Loading

10000-ki Sep 30, 2024

metacosm left a comment

csviri Sep 30, 2024

metacosm Sep 30, 2024

10000-ki Sep 30, 2024

csviri Sep 30, 2024

10000-ki Sep 30, 2024

csviri left a comment

csviri commented Sep 30, 2024

bachmanity1 commented Oct 2, 2024 •

edited

Loading

metacosm commented Oct 2, 2024

10000-ki commented Oct 2, 2024

feat: print diff resource as yaml #2542

feat: print diff resource as yaml #2542

Conversation

10000-ki commented Sep 29, 2024 • edited Loading

AS-IS

TO-BE

Choose a reason for hiding this comment

Choose a reason for hiding this comment

10000-ki commented Sep 29, 2024 • edited Loading

metacosm Sep 30, 2024 • edited Loading

Choose a reason for hiding this comment

10000-ki Sep 30, 2024 • edited by metacosm Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

metacosm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

csviri left a comment

Choose a reason for hiding this comment

csviri commented Sep 30, 2024

bachmanity1 commented Oct 2, 2024 • edited Loading

metacosm commented Oct 2, 2024

10000-ki commented Oct 2, 2024

10000-ki commented Sep 29, 2024 •

edited

Loading

10000-ki commented Sep 29, 2024 •

edited

Loading

metacosm Sep 30, 2024 •

edited

Loading

10000-ki Sep 30, 2024 •

edited by metacosm

Loading

bachmanity1 commented Oct 2, 2024 •

edited

Loading