Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(iceberg): Handling options like "s3.region" #16404

Merged
merged 1 commit into from
Sep 6, 2024

Conversation

Xuanwo
Copy link
Member

@Xuanwo Xuanwo commented Sep 5, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Our parser doesn't allow users to write s3.region in options. Instead, users must use
"s3.region", but it's stored as is which makes the behavior in wrong. We need to remove the quotes here.

We only do this while building catalog so this won't affect existing catalogs.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Sep 5, 2024
@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Sep 5, 2024
Copy link

what-the-diff bot commented Sep 5, 2024

PR Summary

  • Removal of Quotation Marks in Catalog Configurations
    The update introduces a change to the Iceberg Catalog system. In existing setups, certain options within the properties (props) field of Hive Metastore Catalog Configuration (HmsCatalogConfig) and REST Catalog Configuration (RestCatalogConfig) are surrounded by quotes. This change eliminates these quotation marks to simplify configurations and reduce potential parsing errors.

  • Conversion of props field values into Map
    The code update also transforms the values in the props field into a Map data structure. This means the system discards unnecessary quotation marks, both around the keys and values, resulting in a cleaner, easier-to-use data format. This can improve workflow efficiency and reduce potential misunderstandings caused by redundant symbols.

@dosubot dosubot bot added A-storage Area: databend storage C-bug Category: something isn't working labels Sep 5, 2024
@Xuanwo Xuanwo requested a review from BohuTANG September 6, 2024 04:11
@Xuanwo Xuanwo added this pull request to the merge queue Sep 6, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Sep 6, 2024
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Sep 6, 2024
@BohuTANG BohuTANG merged commit 125cb42 into databendlabs:main Sep 6, 2024
105 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-storage Area: databend storage C-bug Category: something isn't working lgtm This PR has been approved by a maintainer pr-bugfix this PR patches a bug in codebase size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants