chore(deps): Update arrow and datafusion to 49.0.0 #24605
Conversation
This commit copies in our dependency code from influxdb_iox in order for us to be able to upgrade from a forked version of 46.0.0 to 49.0.0 of both arrow and datafusion. Most of the important changes were around how we consumed the crates in influxdb3(_server/_write). Those diffs are particularly worth looking at as the rest was a straight copy and we don't touch those crates in our development currently for influxdb3 edge.
|
@pauldix I removed the protobuf lint as this requires us to be in the IOx repo to work. I commented it out with a todo if we wanted to come back to it, but since we won't make changes to these crates or protobuf files here, I think we can safely comment it out for now. This lint still exists in the IOx repo and so I think that's okay, but I wanted to get your input on it. |
| @@ -141,7 +141,7 @@ pub struct TableDefinition { | |||
| pub name: String, | |||
| #[serde(skip_serializing, skip_deserializing)] | |||
| pub schema: Option<Schema>, | |||
| columns: BTreeMap<String, ColumnType>, | |||
| columns: BTreeMap<String, i16>, | |||
There was a problem hiding this comment.
Why this change? Is ColumnType no longer public?
There was a problem hiding this comment.
I'm going to merge in for now so I can build on this, but let me know the motivation on this change.
There was a problem hiding this comment.
ColumnType does not implement Serialize anymore and can only be constructed via try_from(i16) and it can be turned into an i16. So we can serialize an i16 but not a ColumnType and thus this change.
|
The Co-authored-by trailers weren't added in the merge. We can add them to the commit and do a force push, but I don't have those kinds of perms. |
This commit copies in our dependency code from influxdb_iox in order for us to be able to upgrade from a forked version of 46.0.0 to 49.0.0 of both arrow and datafusion. Most of the important changes were around how we consumed the crates in influxdb3(_server/_write). Those diffs are particularly worth looking at as the rest was a straight copy and we don't touch those crates in our development currently for influxdb3 edge.
@pauldix In order to make sure credit is given where it's do I'm going through git logs to make sure everyone who committed in the iox_repo gets credit here with a coauthored by trailer for the commit as this was mostly not my code beyond the influxdb3 portions. I just wanted to open up the PR so you could look at it sooner rather than later.Here's the command I used to grab all of the authors in the IOx repo for all of the folders changed in influxdb since the last common commit between these two repos till now when I'm copying in all of the code.
I had to hand remove a few Author doubles with different emails used and opted for the InfluxData email if there was one. With this we can give proper credit for most of these changes sans the influxdb3 crate ones which I did.
Co-authored-by: Adam Curtis [email protected]
Co-authored-by: Andrew Lamb [email protected]
Co-authored-by: Carol (Nichols || Goulding) [email protected]
Co-authored-by: Chunchun Ye [email protected]
Co-authored-by: Curtis Lee Fulton [email protected]
Co-authored-by: Dom Dwyer [email protected]
Co-authored-by: Fraser Savage [email protected]
Co-authored-by: Geoffrey Wossum [email protected]
Co-authored-by: Jack [email protected]
Co-authored-by: Jeffrey Smith II [email protected]
Co-authored-by: Joe-Blount [email protected]
Co-authored-by: Marco Neumann [email protected]
Co-authored-by: Marko Mikulicic [email protected]
Co-authored-by: Martin Hilton [email protected]
Co-authored-by: Nga Tran [email protected]
Co-authored-by: opeyemi [email protected]
Co-authored-by: Paul Dix [email protected]
Co-authored-by: Raphael Taylor-Davies [email protected]
Co-authored-by: wiedld [email protected]
Co-authored-by: yeame [email protected]