Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate LexOrderingRef and LexRequirementRef #13233

Conversation

jatin510
Copy link
Contributor

@jatin510 jatin510 commented Nov 2, 2024

Which issue does this PR close?

Closes #13220.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added physical-expr Physical Expressions core Core DataFusion crate proto Related to proto crate functions labels Nov 2, 2024
@jatin510
Copy link
Contributor Author

jatin510 commented Nov 2, 2024

@alamb am i going in right directions ?
as this PR changes lots of files ? 😅

@jatin510 jatin510 force-pushed the feature/13220-deprecate-LexOrderingRef-and-LexRequirementRef branch from 0c2e077 to a9023bf Compare November 2, 2024 20:36
@jatin510 jatin510 marked this pull request as ready for review November 3, 2024 04:10
@jatin510 jatin510 changed the title converted LexOrderingRef to &LexOrdering Deprecate LexOrderingRef and LexRequirementRef Nov 3, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb am i going in right directions ?
as this PR changes lots of files ? 😅

Yes, THANK YOU @jatin510 -- I think this is looking really nice. I left a few comments about how to improve this further, but overall it is definitely in the right direction

I agree it is a large change, but this PR (along with the previous one) removes one of the major "rough edges" in the LexOrdering API in my opinon

FYI @akurmustafa and @berkaysynnada

If you are willing to polish this a little more that would be great. I also think we could do the polishing as a follow on PR if you prefer

benchmarks/src/sort.rs Outdated Show resolved Hide resolved
pub fn capacity(&self) -> usize {
self.inner.capacity()
}

pub fn from_ref(lex_ordering_ref: &LexOrdering) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method could be removed as it is the same as clone()

self
}

/// Converts a `LexRequirement` into a `LexOrdering`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also included this change in #13222 (mostly as a note that this will be a logical conflict)

@@ -545,6 +596,16 @@ impl IntoIterator for LexRequirement {
}
}

impl<'a> IntoIterator for &'a LexOrdering {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Self {
streams: vec![],
schema: None,
expressions: EMPTY_ORDER.get_or_init(LexOrdering::default),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

What do you think about moving this EMPTY_ORDER to LexOrdering?

Something like

impl LexOrdering {
...
  /// Return an empty LexOrdering (no expressions)
  pub empty() -> &'static LexOrdering { 
    EMPTY_ORDER.get_or_init(LexOrdering::default),
  }
}

That would avoid needing to copy LexOrderings quite as much (I'll comment inline

@@ -1553,7 +1552,7 @@ mod tests {
Arc::new(BuiltInWindowExpr::new(
last_value_func,
&[],
LexOrderingRef::default(),
LexOrdering::default().as_ref(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the as_ref() is needed here (it just makes another copy)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using

&LexOrdering::default()

Comment on lines 397 to 399
.output_ordering()
.cloned()
.unwrap_or_default(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we had LexOrdering::empty() as explained below, we could avoid this clone like this

            &sort_exec
                .properties()
                .output_ordering()
                .unwrap_or(LexOrdering::empty()),

There are similar things below too

@jatin510
Copy link
Contributor Author

jatin510 commented Nov 3, 2024

Thanks !
made the suggested changes.
We can come up with a follow-up PR for more refinements!
@alamb

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me -- thank you @jatin510

@alamb
Copy link
Contributor

alamb commented Nov 4, 2024

FYI @berkaysynnada

Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, very nice refactor. Thank you @jatin510

@berkaysynnada
Copy link
Contributor

I have merged with main to resolve conflicts. I will merge this once the tests pass

@berkaysynnada berkaysynnada merged commit 9005585 into apache:main Nov 5, 2024
26 checks passed
@alamb
Copy link
Contributor

alamb commented Nov 5, 2024

Thanks again @berkaysynnada and @jatin510

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate functions physical-expr Physical Expressions proto Related to proto crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deprecate LexOrderingRef and LexRequirementRef
3 participants