-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HIVE-28538: Post JDK11-Fix locale-related discrepancies in test cases between local and Jenkins environments #5476
base: master
Are you sure you want to change the base?
Conversation
… between local and Jenkins environments
Quality Gate passedIssues Measures |
@@ -92,7 +92,7 @@ | |||
<test.excludes.additional/> | |||
<!-- Plugin and Plugin Dependency Versions --> | |||
<ant.contrib.version>1.0b3</ant.contrib.version> | |||
<maven.test.jvm.args>-Xmx2048m -DJETTY_AVAILABLE_PROCESSORS=4</maven.test.jvm.args> | |||
<maven.test.jvm.args>-Xmx2048m -DJETTY_AVAILABLE_PROCESSORS=4 -Duser.country=US</maven.test.jvm.args> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that in the configuration of the maven-surefile-plugin we have the following:
<environmentVariables>
<TZ>US/Pacific</TZ>
<LANG>en_US.UTF-8</LANG>
...
</environmentVariables>
Shouldn't this be enough to guarantee that there are no discrepancies. Why do we need to set also the JVM properties?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The existing configuration in the maven-surefire-plugin indeed sets the LANG environment variable to en_US.UTF-8, which helps ensure that environment variables are set correctly within the Jenkins build environment. However, this setting does not fully address the locale discrepancies that can occur when tests are run locally on different developer machines with varying locale settings.
Environment variables can influence the locale settings for the system environment but might not comprehensively cover all locale-specific behaviors within the JVM. Passing the locale explicitly via <maven.test.jvm.args> ensures that the JVM itself runs with the specified locale, aligning both local and Jenkins environments more reliably.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't the JVM picking up the default values from the respective environment variables?
The Java Virtual Machine sets the default locale during startup based on the host environment.
Are you saying that locally the environmentVariables
do not work as expected?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
JVM is picking up the timezone but not the locale as in the environment we are setting the TZ
Locale influences how data is formatted and presented, such as dates and times, but does not determine the actual time.
Timezone determines the current time based on the region's offset from UTC but does not affect how dates and times are formatted
Thus the test are failing with the formatting problem of am/AM or pm/PM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zabetak Can you help to review
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to the comments above it seems that setting the LANG
environment variable is not something that works on every system and that's why you opted to use the user.country
property. For future reference, can you please give an example of a system where setting the LANG
variable does not work.
Other than that the changes LGTM, and we can merge this PR.
@zabetak Thanks for the review, I have addressed the comment |
What changes were proposed in this pull request?
Passing the locale in the pom.xml as <maven.test.jvm.args> to align both environments of local and jenkins
Why are the changes needed?
Post jdk11: In JDK11, changes were made to the Java locale handling as documented in the following issues:
(https://bugs.openjdk.org/browse/JDK-8145136)
https://bugs.openjdk.org/browse/JDK-8211985
These changes result in different locale behaviors. This issue does not occur in Jenkins because the test cases are written according to the en_US locale. However, when run locally, the locale may differ (e.g., en_IN), causing discrepancies. For instance, en_IN would expect am/pm while the tests expect AM/PM, leading to failures.
The changes are needed to address test failures that occur locally but not in Jenkins. This discrepancy arises due to different locale settings between the two environments.
Various tests will start failing in local without alarming in the jenkins for example TestMiniLlapLocalCliDriver.udf_date_format.q as the locale in jenkins is en_US whereas varying from the user locale. ( for example en_IN ), the test will start failing
There may be other test failures not yet captured due to different locales in local environments. Ensure all tests run with the en_US locale to identify and resolve any further issues.
Does this PR introduce any user-facing change?
NO