Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubernetes service discovery breaking change in version 29.0.0 #17529

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

m17kea
Copy link

@m17kea m17kea commented Dec 2, 2024

Fixes #15942

Description

This PR updates the Apache Druid documentation to highlight a breaking change in Kubernetes service discovery behavior introduced in version 29. Specifically, the format of the timestamps used in the <cluster-identifier>-leaderelection-coordinator and <cluster-identifier>-leaderelection-overlord ConfigMaps was updated to a new format that was not documented. This change caused errors during leader election and service discovery due to mismatched date parsing expectations.

The goal of this PR is to ensure that users upgrading to version 29 are aware of this breaking change and can make the necessary adjustments to avoid issues in their Druid clusters.

Documentation Changes
Added a note in the Upgrade notes documentation for version 29 about the updated date format in ConfigMaps.
Provided an example of the command required to help users update their clusters accordingly.

Release Note

This PR updates the documentation to warn users about a breaking change in Kubernetes service discovery introduced in version 29. The date format in leader election ConfigMaps (<cluster-identifier>-leaderelection-coordinator and <cluster-identifier>-leaderelection-overlord) was updated to a stricter ISO-8601-like format. Users should ensure they delete the old config mapss once upgrade to version 29.0.0 and above so they are recreated with the correct date format.


Key changed/added classes in this PR Documentation updates only, no code changes.

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@m17kea
Copy link
Author

m17kea commented Dec 2, 2024

Likely to have been introduced by #15449

@georgew5656
Copy link
Contributor

georgew5656 commented Jan 7, 2025

Likely to have been introduced by #15449

this looks good to me, @m17kea can you add a note that the operator might need to delete the configmap multiple times? for example if they have two coordinators or overlords running and are doing a rolling update, and one of the old coordinators/overlords regains leadership after the configmap is deleted

@georgew5656 georgew5656 self-requested a review January 7, 2025 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Coordinator crashes after upgrading to 29.0.0
2 participants