Skip to content

Commit

Permalink
feat: Add the in-commit timestamp field to CommitInfo (#581)
Browse files Browse the repository at this point in the history
## What changes are proposed in this pull request?
This PR adds the inCommitTimestamp filed to CommitInfo. This lays the
groundwork for supporting ICT for table properties and for writes.

We update transaction write tests to ignore the in-commit timestamp
field since the write path does not currently support in-commit
timestamps.

## How was this change tested?
All existing tests pass.
  • Loading branch information
OussamaSaoudi-db authored Jan 15, 2025
1 parent 606db20 commit 616e9ac
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 1 deletion.
8 changes: 7 additions & 1 deletion kernel/src/actions/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -331,8 +331,13 @@ where
struct CommitInfo {
/// The time this logical file was created, as milliseconds since the epoch.
/// Read: optional, write: required (that is, kernel always writes).
/// If in-commit timestamps are enabled, this is always required.
pub(crate) timestamp: Option<i64>,
/// The time this logical file was created, as milliseconds since the epoch. Unlike
/// `timestamp`, this field is guaranteed to be monotonically increase with each commit.
/// Note: If in-commit timestamps are enabled, both the following must be true:
/// - The `inCommitTimestamp` field must always be present in CommitInfo.
/// - The CommitInfo action must always be the first one in a commit.
pub(crate) in_commit_timestamp: Option<i64>,
/// An arbitrary string that identifies the operation associated with this commit. This is
/// specified by the engine. Read: optional, write: required (that is, kernel alwarys writes).
pub(crate) operation: Option<String>,
Expand Down Expand Up @@ -694,6 +699,7 @@ mod tests {
"commitInfo",
StructType::new(vec![
StructField::new("timestamp", DataType::LONG, true),
StructField::new("inCommitTimestamp", DataType::LONG, true),
StructField::new("operation", DataType::STRING, true),
StructField::new(
"operationParameters",
Expand Down
6 changes: 6 additions & 0 deletions kernel/src/transaction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,12 @@ fn generate_commit_info(
.get_mut("operationParameters")
.ok_or_else(|| Error::missing_column("operationParameters"))?
.data_type = hack_data_type;

// Since writing in-commit timestamps is not supported, we remove the field so it is not
// written to the log
commit_info_data_type
.fields
.shift_remove("inCommitTimestamp");
commit_info_field.data_type = DataType::Struct(commit_info_data_type);

let commit_info_evaluator = engine.get_expression_handler().get_evaluator(
Expand Down

0 comments on commit 616e9ac

Please sign in to comment.