[RHELC-1736] Fix different exit codes #1392

hosekadam · 2024-09-30T10:02:33Z

The convert2rhel finished with different exit codes when inhibitor was found and different subcommands were used.

In the code was missing part for handling the inhibitors during analysis exit.

Jira Issues:

RHELC-1736

Checklist

PR has been tested manually in a VM (either author or reviewer)
Jira issue has been made public if possible
[RHELC-] or [HMS-] is part of the PR title
GitHub label has been added to help with Release notes
PR title explains the change from the user's point of view
Code and tests are documented properly
The commits are squashed to as few commits as possible (without losing data)
When merged: Jira issue has been updated to Release Pending if relevant

codecov · 2024-09-30T10:03:34Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.11%. Comparing base (261d533) to head (efd4c76).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1392      +/-   ##
==========================================
+ Coverage   96.09%   96.11%   +0.02%     
==========================================
  Files          72       72              
  Lines        5174     5178       +4     
  Branches      896      897       +1     
==========================================
+ Hits         4972     4977       +5     
  Misses        119      119              
+ Partials       83       82       -1

Flag	Coverage Δ
centos-linux-7	`91.63% <100.00%> (+0.02%)`	⬆️
centos-linux-8	`92.49% <100.00%> (+0.02%)`	⬆️
centos-linux-9	`92.61% <100.00%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

has-bot · 2024-09-30T10:28:44Z

/packit test --labels sanity

Comment generated by an automation.

Log | Bot Usage

hosekadam · 2024-10-01T09:58:45Z

/packit test --labels sanity

Venefilyn

I feel like there isn't a need for a new unit test, but rather modifying the existing ones to fail if it is the old style

Especially given how many mocks the new test creates it seems unnecessary to have a whole new test function for it

hosekadam · 2024-10-10T13:53:14Z

@Venefilyn I remember I tried it, but modifying some existing one would result in a complex test. Also I remember finding the right one was a bit problematic - many of the tests aren't commented, so you need to figure out which one is the best from the code.

So I chose the way where I create single test, easy to read and simply testing the issue.

Venefilyn · 2024-10-10T14:10:36Z

@Venefilyn I remember I tried it, but modifying some existing one would result in a complex test. Also I remember finding the right one was a bit problematic - many of the tests aren't commented, so you need to figure out which one is the best from the code.

So I chose the way where I create single test, easy to read and simply testing the issue.

@hosekadam Ack sounds good. Though with this implementation you suggested we'll not get everything we wanted I feel. What if we do something like this for the implementation? Means we get the right returns but the same output. Your current implementation I feel doesn't handle analyze stage correctly as it doesn't handle anything in the analyze stage

diff --git a/convert2rhel/main.py b/convert2rhel/main.py
index c62a2669..4053fb47 100644
--- a/convert2rhel/main.py
+++ b/convert2rhel/main.py
@@ -65,6 +65,7 @@ _REPORT_MAPPING = {
     ),
 }
 
+
 # Track the exit codes for different scenarios during the conversion.
 class ConversionExitCodes:
     # No errors detected during the conversion
@@ -196,28 +197,36 @@ def main_locked():
         rollback_changes()
         provide_status_after_rollback(pre_conversion_results, include_all_reports=True)
 
-        if backup.backup_control.rollback_failed:
-            return ConversionExitCodes.FAILURE
+        if _get_failed_actions(pre_conversion_results):
+            return ConversionExitCodes.SUCCESSFUL
 
         return ConversionExitCodes.SUCCESSFUL
     except _InhibitorsFound as err:
         loggerinst.critical_no_exit(str(err))
-        results = _pick_conversion_results(process_phase, pre_conversion_results, post_conversion_results)
+        results = _pick_conversion_results(
+            process_phase, pre_conversion_results, post_conversion_results
+        )
         _handle_main_exceptions(process_phase, results)
 
         return _handle_inhibitors_found_exception()
     except exceptions.CriticalError as err:
         loggerinst.critical_no_exit(err.diagnosis)
-        results = _pick_conversion_results(process_phase, pre_conversion_results, post_conversion_results)
+        results = _pick_conversion_results(
+            process_phase, pre_conversion_results, post_conversion_results
+        )
         return _handle_main_exceptions(process_phase, results)
     except (Exception, SystemExit, KeyboardInterrupt) as err:
-        results = _pick_conversion_results(process_phase, pre_conversion_results, post_conversion_results)
+        results = _pick_conversion_results(
+            process_phase, pre_conversion_results, post_conversion_results
+        )
         return _handle_main_exceptions(process_phase, results)
     finally:
         if not backup.backup_control.rollback_failed:
             # Write the assessment to a file as json data so that other tools can
             # parse and act upon it.
-            results = _pick_conversion_results(process_phase, pre_conversion_results, post_conversion_results)
+            results = _pick_conversion_results(
+                process_phase, pre_conversion_results, post_conversion_results
+            )
 
             if results and process_phase in _REPORT_MAPPING:
                 json_report, txt_report = _REPORT_MAPPING[process_phase]
@@ -228,6 +237,10 @@ def main_locked():
     return ConversionExitCodes.SUCCESSFUL
 
 
+def _get_failed_actions(results):
+    return actions.find_actions_of_severity(results, "SKIP", level_for_raw_action_data)
+
+
 def _raise_for_skipped_failures(results):
     """Analyze the action results for failures
 
@@ -236,7 +249,7 @@ def _raise_for_skipped_failures(results):
     :raises SystemExit: In case we detect any actions that has level of `SKIP`
         or above.
     """
-    failures = actions.find_actions_of_severity(results, "SKIP", level_for_raw_action_data)
+    failures = _get_failed_actions(results)
     if failures:
         # The report will be handled in the error handler, after rollback.
         message = (
@@ -400,7 +413,9 @@ def provide_status_after_rollback(pre_conversion_results, include_all_reports):
         return
 
     if not pre_conversion_results:
-        loggerinst.info("\nConversion interrupted before analysis of system completed. Report not generated.\n")
+        loggerinst.info(
+            "\nConversion interrupted before analysis of system completed. Report not generated.\n"
+        )
 
         return

hosekadam · 2024-10-14T12:20:19Z

@Venefilyn Thanks for the tip! You are right, this is a better solution than I prepared.

r0x0d · 2024-10-14T13:10:58Z

/packit test --labels sanity

Venefilyn · 2024-10-16T12:54:32Z

@danmyway should we update the integration tests to match?

danmyway · 2024-10-17T12:01:03Z

@Venefilyn @hosekadam I've addressed the changes in the failing tests and some more. I'll trigger the whole suite to see, where we need to make other adjustments (if any)

danmyway · 2024-10-17T12:19:28Z

/packit test

Venefilyn · 2024-10-18T11:56:27Z

@hosekadam can you change the other tests that are failing as well? They should also look for exit code 2 instead of 0

hosekadam · 2024-10-22T12:59:56Z

/packit test

hosekadam · 2024-10-22T15:56:58Z

/packit retest-failed

kokesak

You will also need to fix the exit status to "2" in these tests:

tests/integration/tier0/non-destructive/basic-sanity-checks/main.fmf

kokesak · 2024-10-23T10:42:23Z

Also this test is causing issues - https://artifacts.osci.redhat.com/testing-farm/4d3cdcfd-3e04-4b04-aa7c-c394c9b4b07c/work-non-destructiveeqcz6bcj/plans/tier0/core/non-destructive/execute/data/guest/default-0/tests/integration/tier0/non-destructive/custom-repository/custom_invalid_repo_provided-14/output.txt.

Did something changed? The test is expecting the CUSTOM_REPOSITORIES_ARE_VALID::UNABLE_TO_ACCESS_REPOSITORIES message to appear but it doesn't appear in the logs

hosekadam · 2024-10-23T11:42:17Z

Also this test is causing issues - https://artifacts.osci.redhat.com/testing-farm/4d3cdcfd-3e04-4b04-aa7c-c394c9b4b07c/work-non-destructiveeqcz6bcj/plans/tier0/core/non-destructive/execute/data/guest/default-0/tests/integration/tier0/non-destructive/custom-repository/custom_invalid_repo_provided-14/output.txt.

I think this got changed in #1288 and since this got merged, the CUSTOM_REPOSITORIES_ARE_VALID has dependency in SUBSCRIBE_SYSTEM. Is the test still relevant? @kokesak

kokesak · 2024-10-23T11:58:38Z

Also this test is causing issues - https://artifacts.osci.redhat.com/testing-farm/4d3cdcfd-3e04-4b04-aa7c-c394c9b4b07c/work-non-destructiveeqcz6bcj/plans/tier0/core/non-destructive/execute/data/guest/default-0/tests/integration/tier0/non-destructive/custom-repository/custom_invalid_repo_provided-14/output.txt.

I think this got changed in #1288 and since this got merged, the CUSTOM_REPOSITORIES_ARE_VALID has dependency in SUBSCRIBE_SYSTEM. Is the test still relevant? @kokesak

I see. We can fix it here if that is ok with you. I suggest setting up the CONVERT2RHEL_INCOMPLETE_ROLLBACK env var and lets see if that help. You only need to add these two lines to the https://github.com/oamg/convert2rhel/blob/main/tests/integration/tier0/non-destructive/custom-repository/test_custom_repository.py#L40C1-L40C64

@pytest.mark.parametrize("environment_variables", ["CONVERT2RHEL_INCOMPLETE_ROLLBACK"], indirect=True)
def test_custom_invalid_repo_without_rhsm(shell, convert2rhel, environment_variables):
...

hosekadam · 2024-10-23T13:13:13Z

/packit test

kokesak · 2024-10-24T08:50:51Z

tests/integration/tier0/non-destructive/custom-repository/test_custom_repository.py

+@pytest.mark.parametrize("environment_variables", ["CONVERT2RHEL_INCOMPLETE_ROLLBACK"], indirect=True)
+def test_custom_invalid_repo_without_rhsm(shell, convert2rhel, environment_variables):


it seems that we have fixed the test but since we introduced the INCOMPLETE_ROLLBACK env var and the rollback actually fails then the exit code for this test case is actually 1 and not 2

Please, revert this.
It actually discovered a regression introduced in #1288 as Adam mentioned.
It's reported here https://issues.redhat.com/browse/RHELC-1753.
I don't really like that we're trying to make the test pass just for the sake of making it green :(

Done. Should I trigger the whole suite again when the rpms are built? @danmyway

Sorry @hosekadam, I missed the message, triggered the suite instead

The convert2rhel finished with different exit codes when inhibitor was found and different subcommands were used. In the code was missing part for handling the inhibitors during analysis exit.

Signed-off-by: Daniel Diblik <[email protected]>

Fix another exit codes based on the test run which should be 2.

danmyway · 2024-10-25T09:09:58Z

/packit test

Venefilyn

Just leaving the dev ack approval

hosekadam requested a review from a team as a code owner September 30, 2024 10:02

hosekadam added the kind/bug-fix A bug has been fixed label Sep 30, 2024

hosekadam force-pushed the fix-exit-codes branch from 529e911 to 85263cd Compare September 30, 2024 10:03

hosekadam added the tests/sanity PR ready to run the sanity test suit. Equivalent to `/packit test --labels sanity`. label Sep 30, 2024

hosekadam force-pushed the fix-exit-codes branch from 85263cd to 3581e81 Compare October 9, 2024 14:37

Venefilyn force-pushed the fix-exit-codes branch from 52e5874 to 896a3aa Compare October 9, 2024 18:57

Venefilyn reviewed Oct 10, 2024

View reviewed changes

hosekadam force-pushed the fix-exit-codes branch from 896a3aa to 58a8f3c Compare October 14, 2024 12:18

hosekadam force-pushed the fix-exit-codes branch from 58a8f3c to b741564 Compare October 14, 2024 12:29

danmyway requested a review from a team as a code owner October 17, 2024 11:59

danmyway requested review from danmyway and kokesak and removed request for danmyway October 17, 2024 11:59

hosekadam force-pushed the fix-exit-codes branch from 5ad2b9a to b982f80 Compare October 22, 2024 12:50

kokesak requested changes Oct 23, 2024

View reviewed changes

tests/integration/tier0/non-destructive/basic-sanity-checks/main.fmf Show resolved Hide resolved

hosekadam force-pushed the fix-exit-codes branch 2 times, most recently from 80065ea to 0e62884 Compare October 23, 2024 11:44

hosekadam force-pushed the fix-exit-codes branch from 0e62884 to 16e8a5d Compare October 23, 2024 13:02

kokesak reviewed Oct 24, 2024

View reviewed changes

hosekadam force-pushed the fix-exit-codes branch from 16e8a5d to fe0ef9e Compare October 24, 2024 13:39

hosekadam and others added 2 commits October 24, 2024 15:40

[RHELC-1736] Fix different exit codes

00b99d7

The convert2rhel finished with different exit codes when inhibitor was found and different subcommands were used. In the code was missing part for handling the inhibitors during analysis exit.

Address changes in the tests

fea4d38

Signed-off-by: Daniel Diblik <[email protected]>

hosekadam force-pushed the fix-exit-codes branch from fe0ef9e to c49b967 Compare October 24, 2024 13:40

Fix exit codes in integration tests

efd4c76

Fix another exit codes based on the test run which should be 2.

hosekadam force-pushed the fix-exit-codes branch from c49b967 to efd4c76 Compare October 24, 2024 13:48

Venefilyn approved these changes Oct 28, 2024

View reviewed changes

Venefilyn added the merge-after-tests-ok label Oct 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RHELC-1736] Fix different exit codes #1392

[RHELC-1736] Fix different exit codes #1392

hosekadam commented Sep 30, 2024 •

edited

Loading

codecov bot commented Sep 30, 2024 •

edited

Loading

has-bot commented Sep 30, 2024

hosekadam commented Oct 1, 2024

Venefilyn left a comment

hosekadam commented Oct 10, 2024

Venefilyn commented Oct 10, 2024

hosekadam commented Oct 14, 2024

r0x0d commented Oct 14, 2024

Venefilyn commented Oct 16, 2024

danmyway commented Oct 17, 2024

danmyway commented Oct 17, 2024

Venefilyn commented Oct 18, 2024

hosekadam commented Oct 22, 2024

hosekadam commented Oct 22, 2024

kokesak left a comment

kokesak commented Oct 23, 2024 •

edited

Loading

hosekadam commented Oct 23, 2024

kokesak commented Oct 23, 2024

hosekadam commented Oct 23, 2024

kokesak Oct 24, 2024

danmyway Oct 24, 2024

hosekadam Oct 24, 2024

danmyway Oct 25, 2024

danmyway commented Oct 25, 2024

Venefilyn left a comment

		@pytest.mark.parametrize("environment_variables", ["CONVERT2RHEL_INCOMPLETE_ROLLBACK"], indirect=True)
		def test_custom_invalid_repo_without_rhsm(shell, convert2rhel, environment_variables):

[RHELC-1736] Fix different exit codes #1392

Are you sure you want to change the base?

[RHELC-1736] Fix different exit codes #1392

Conversation

hosekadam commented Sep 30, 2024 • edited Loading

codecov bot commented Sep 30, 2024 • edited Loading

Codecov Report

has-bot commented Sep 30, 2024

hosekadam commented Oct 1, 2024

Venefilyn left a comment

Choose a reason for hiding this comment

hosekadam commented Oct 10, 2024

Venefilyn commented Oct 10, 2024

hosekadam commented Oct 14, 2024

r0x0d commented Oct 14, 2024

Venefilyn commented Oct 16, 2024

danmyway commented Oct 17, 2024

danmyway commented Oct 17, 2024

Venefilyn commented Oct 18, 2024

hosekadam commented Oct 22, 2024

hosekadam commented Oct 22, 2024

kokesak left a comment

Choose a reason for hiding this comment

kokesak commented Oct 23, 2024 • edited Loading

hosekadam commented Oct 23, 2024

kokesak commented Oct 23, 2024

hosekadam commented Oct 23, 2024

kokesak Oct 24, 2024

Choose a reason for hiding this comment

danmyway Oct 24, 2024

Choose a reason for hiding this comment

hosekadam Oct 24, 2024

Choose a reason for hiding this comment

danmyway Oct 25, 2024

Choose a reason for hiding this comment

danmyway commented Oct 25, 2024

Venefilyn left a comment

Choose a reason for hiding this comment

hosekadam commented Sep 30, 2024 •

edited

Loading

codecov bot commented Sep 30, 2024 •

edited

Loading

kokesak commented Oct 23, 2024 •

edited

Loading