New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for reading tables with temporal versions in Delta Lake connector #21024

Open

ebyhr opened this issue Mar 12, 2024 · 3 comments · May be fixed by #24801

Labels

delta-lake enhancement

Member

ebyhr commented Mar 12, 2024

Follow-up of #15894

We are going to support only version-id in the initial implementation.

ebyhr added enhancement delta-lake labels

Contributor

findinpath commented Mar 15, 2024

Take into account the delta.io RFC https://github.com/delta-io/delta/blob/master/protocol_rfcs/in-commit-timestamps.md

Member Author

ebyhr commented Oct 9, 2024

The in-commit-timestamps.md file has been moved to https://github.com/delta-io/delta/blob/master/protocol_rfcs/accepted/in-commit-timestamps.md

Contributor

findinpath commented Oct 10, 2024

Here seems to be the delta.io algorithm for translating a given timestamp to the actual transaction https://github.com/delta-io/delta/blob/7b9c4ed34dcf102ff3dca0abed3f668ebc9f3061/spark/src/main/scala/org/apache/spark/sql/delta/DeltaHistoryManager.scala#L126-L136

chenjian2664 linked a pull request

that will close this issue

Support time travel using temporal version in Delta Lake #24801

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment