Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement basic support for rest.sigv4_enabled for the Iceberg REST catalog #917

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

phillipleblanc
Copy link
Contributor

@phillipleblanc phillipleblanc commented Jan 27, 2025

Implements basic support for connecting to the AWS Glue Iceberg REST catalog by supporting the parameters rest.sigv4-enabled, rest.signing-name and rest.signing-region. See the corresponding parameters in the PyIceberg docs: https://py.iceberg.apache.org/configuration/#rest-catalog

I've added a new feature to the iceberg-catalog-rest called sigv4 which when enabled, will recognize the above configuration parameters. I've left this disabled by default.

I added the crate request_middleware to allow running middleware on requests, and if the rest.sigv4-enabled configuration is enabled, the middleware to sign the request and add the correct Authorization header is enabled.

I've tested this implementation in my project, and it works great for connecting to the AWS Glue REST catalog, i.e. https://glue.<region>.amazonaws.com/iceberg

I based this logic on the equivalent logic in PyIceberg: https://github.com/apache/iceberg-python/blob/6fffb644518bb64e8f33883d850edbe18c12bd07/pyiceberg/catalog/rest.py#L475

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant