Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-18636 Fix how we handle Gradle exits in CI #18681

Open
wants to merge 3 commits into
base: trunk
Choose a base branch
from

Conversation

mumrah
Copy link
Member

@mumrah mumrah commented Jan 23, 2025

This patch removes usages of ignoreFailures in our CI and changes the XML copy task to a finalizer task instead of doLast closure. Also fixes a fall-through case in junit.py where we did not recognize an error prior to running the tests (such as the javadoc task).


Since we are running our CI Gradle commands with --continue, we should not normally need to explicitly ignoreFailures in the test tasks. After all, that's what --continue does (ignore failures from tasks). The reason we included it was to ensure our doLast closure was always run. When a test task fails, the main task action will fail and prevent any doLast or other later parts of the task action from running.

Instead of copying out the test XML in a doLast, we can use a finalizer task. A finalizer task will run regardless of task outcome. This is a bit nicer than what we were doing.

The one case where we need non-standard exit behavior is when running the main test suite on trunk. We may have flaky re-runs of tests which allow the ":test" task to exit normally. However, we do not want to cache the test results if there were flaky failures as this may potentially "hide" flaky tests from our PRs. So far, the only way we have determined to prevent a task from being cached is for it to fail.

Here are the possible outcomes of our ":test" task:

Test results Task result Gradle exit junit.py exit
No failures 0 0 0
Flaky failures 1 1 0
Hard failures 1 1 1
Error (no tests run) N/A 1 1

For our ":quarantinedTest" task:

Test results Task result Gradle exit junit.py exit
No failures 0 0 0
Flaky failures 0 0 0
Hard failures 1 1 1
Error (no tests run) N/A 1 1

The only difference is in the "Flaky failures" behavior.

@github-actions github-actions bot added the build Gradle build or GitHub Actions label Jan 23, 2025
@mumrah
Copy link
Member Author

mumrah commented Jan 24, 2025

With the javadoc error reintroduced, the build now shows

There was an error during the test or quarantinedTest task. Please check the logs
Error: Process completed with exit code 1.

https://github.com/apache/kafka/actions/runs/12938661800/job/36089513936?pr=18681

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Gradle build or GitHub Actions clients
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant