Consider using Jiff instead of Chrono

[v1gnesh]

Description

I'm not sure how feasible or desirable this is, so just opening this for hearing what you think -
@ariebovenberg's whenever

Here's how it benches -

Docs - https://whenever.readthedocs.io/en/stable/

[MarcoGorelli]

Polars uses a Rust crate (chrono) for temporal features, as well as just plain integer arithmetic where possible. datetime is only returned to the user when they ask for a Python scalar from Polars

I don't think there's desire to take on extra required Python dependencies, but you could always just convert to integer and then parse with whenever (or whatever else you like) yourself outside of Polars

Is there any other usage of whenever you had in mind?

[ariebovenberg]

Thought I’d chime in since I was tagged.

As suggested, whenever should be easy to interop through the standard library. Although—I wouldn’t say no to whenever being associated with such a popular lib as polars 😉

@MarcoGorelli did you have a look at jiff yet? It’s a relatively new Rust datetime lib that takes inspiration from well-designed libraries in other languages.

[v1gnesh]

Is there any other usage of whenever you had in mind?

Not particularly. Just thought that for a product where speed is a key feature, associating with other projects aiming for the same has got to be beneficial. +1 for considering jiff as well.

[MarcoGorelli]

thanks both!

there is some desire to have a faster version of Chrono in Rust, but it looks like Jiff may be slower than Chrono? https://github.com/BurntSushi/jiff/blob/master/COMPARE.md#chrono-is-faster-than-jiff-in-some-cases

[ariebovenberg]

I can't speak for wanting faster datetime handling in polars. My preference for jiff is due to its IMHO superior datamodel, in line with what's proven to work in other languages. Chrono's datamodel is more idiosyncratic. You can see in their issue tracker that this gives problems. A modest improvement is in the works (version 0.5) but progress is very slow (understandable, but still)

edit: grammar

[MarcoGorelli]

could you elaborate please? is there anywhere in polars where chrono's datamodel leads to issues, that could be addressed by jiff?

[ariebovenberg]

could you elaborate please?

My concerns overlap mostly with the ones mentioned by jiff, but these stand out to me:

• Chrono doesn’t support handling local datetimes that land in a “gap”. Most modern libraries and standards have agreed on ways to handle this. The design of chrono however makes this impossible (see chrono issue 1448)
• Most modern date time libraries and standards (UNIX time) choose to ignore leap seconds in a consistent way. Chrono chose to support them partially, which results in an API that requires a lot of explaining.

is there anywhere in polars where chrono's datamodel leads to issues, that could be addressed by Jiff?

😅 I’m not a polars user myself, so I can’t say whether this affects it. Being tagged by OP, I just wanted to add my 2 cents about datetime libraries. Feel free to make your own assessment about the right choice for polars—Chrono is definitely the more established library and a safe choice at the moment.

[MarcoGorelli]

thanks, interesting! ok for non-existent times this may be interesting, as the only strategies Polars currently supports for those are to raise or to coerce to None

[BurntSushi]

Jiff author here.

thanks both!

there is some desire to have a faster version of Chrono in Rust, but it looks like Jiff may be slower than Chrono? https://github.com/BurntSushi/jiff/blob/master/COMPARE.md#chrono-is-faster-than-jiff-in-some-cases

It really depends on what you're doing. Jiff's strptime routine is quite a bit faster than Chrono's for example. So if you need to do a lot of ad hoc datetime parsing, that could help.

What operations in chrono are performance critical for polars? I haven't put a ton of effort into optimizing everything. There are definitely more opportunities there. When I last looked at polars internals, my very uninformed take-away was that y'all had solved some of the problems already, where the problems had to do with converting back-and-forth between Unix epoch days (a single integer) and a more expanded representation (year/month/day). That's less of a Chrono issue (kinda), since some operations are more efficient on Unix epoch days and some are more efficient on the expanded representation.

Aside from performance, the Jiff docs include a comparison with Chrono: https://docs.rs/jiff/latest/jiff/_documentation/comparison/index.html#chrono-v0438

Many of the benefits come from better time zone handling. I don't know how much of that polars does.