Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timestamp.parse_rfc3339 function #12

Merged
merged 19 commits into from
Jan 20, 2025
Merged

Conversation

mooreryan
Copy link
Contributor

@mooreryan mooreryan commented Jan 10, 2025

See #9

Grammar for parsing and for property test generators taken from section 5.6 - Internet Date/Time Format of RFC 3339.

Most of the testing is done by comparing this to test oracles with property tests. I have FFI for Erlang and JS versions to compare to as well as a test file of 1000 timestamps parsed by the OCaml ptime library. That is included because the Erlang and JS versions have slight limitations compared to the the implementation in this module (mainly regarding fractional second precision), but if you don't want it in there I can take it out.

Couple of issues with the tests that will need to be resolved at some point:

  • The Erlang oracle used in the property tests will need to be updated once this is fixed upstream. (The Erlang oracle code adjusts the time to correct for that issue.)
  • The int_uniform used in timestamp_rfc3339_timestamp_roundtrip_property_test doesn't get to the boundaries of 0000 to 9999 years. It's out of the range of the prng int generator. (I fixed this in a subsequent commit.)
  • The round trip test from string -> timestamp -> string will need to be updated once to_rfc3339 handles fractional seconds.

@mooreryan
Copy link
Contributor Author

The only part of the interface that I needed to change (other than the parse function) was exposing a duration.normalised function so that I could convert a timestamp into a duration.

This fix gets around the integer range limits for generating decent random values, allowing property tests to hit the full range of RFC 3339 timestamps.
@mooreryan mooreryan marked this pull request as draft January 11, 2025 06:46
@mooreryan mooreryan marked this pull request as ready for review January 11, 2025 06:48
@mooreryan
Copy link
Contributor Author

mooreryan commented Jan 11, 2025

After fixing the range of the timestamp generator, I had to cherry pick the commit from this other PR: #10.

Copy link
Member

@lpil lpil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow! Super nice work!! I've left a bunch of questions and notes inline.

Thank you so much!!

src/gleam/time/duration.gleam Outdated Show resolved Hide resolved
test/gleam/time/rfc3339_generator.gleam Outdated Show resolved Hide resolved
test/gleam/time/timestamp_test.gleam Outdated Show resolved Hide resolved
test/gleam/time/timestamp_test.gleam Outdated Show resolved Hide resolved
parse_rfc3339_should_match_oracle_for("1970-01-01T23:59:60Z")
}

fn parse_rfc3339_should_match_oracle_for(date_time) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete this function and inline the body into all the places it was used please. Tests should be as simple as possible

src/gleam/time/timestamp.gleam Show resolved Hide resolved
src/gleam/time/timestamp.gleam Outdated Show resolved Hide resolved
src/gleam/time/timestamp.gleam Outdated Show resolved Hide resolved
src/gleam/time/timestamp.gleam Outdated Show resolved Hide resolved
from bytes: BitArray,
count count: Int,
acc acc: Int,
k k: Int,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's k?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k is just the name of the loop counter. I could change the name to something more descriptive?

@mooreryan
Copy link
Contributor Author

Thanks for looking at this and the comments!

Most of your comments I was able to address without additional clarification. I left a couple of follow up comments.

Would you like me to squash all the tiny commits I created as I was addressing the issues?

@mooreryan
Copy link
Contributor Author

Oh two other questions for you:


I had thought to make some tests like this:

pub fn parse_rfc3339_returns_error_for_bad_timestamp_0_test() {
  timestamp.parse_rfc3339("12349-01-01T00:00:00Z")
  |> should.equal(Error(Nil))
}

pub fn parse_rfc3339_returns_error_for_bad_timestamp_1_test() {
  timestamp.parse_rfc3339("1234-019-01T00:00:00Z")
  |> should.equal(Error(Nil))
}

with specific examples that break the grammar in some way, but then I thought it might not be worth the effort. What do you think? (Ideally, I could just generate non-conforming datetime strings, but that would be a bit of work.)


The other thing, do you want me to make a couple of parsing tests that don't use oracles at all? Some stuff like

pub fn blah_test() {
  let assert Ok(ts) = timestamp.parse_rfc3339(...)
  timestamp.to_unix_seconds_and_nanoseconds(ts)
  |> should.equal(#(..., ...))

There are quite a few specific examples that are tested (in the parse_rfc3339_matches_oracle_example_N_test functions), but they use the oracles, and so you can't glance at the tests to see the expected values. I don't think this is necessarily an issue, what do you think?

@lpil
Copy link
Member

lpil commented Jan 16, 2025

I don't mind RE tests that show it not parsing. Your call.

The other thing, do you want me to make a couple of parsing tests that don't use oracles at all

Maybe a couple would be nice! Thank you.

@mooreryan
Copy link
Contributor Author

I updated the tests, and made a few more tweaks to address the last unresolved comments.

The last unresolved issue is regarding how the doc comment of the function should be--whether it should detail the specifications of the parsing (comment is here: #12 (comment)). I'm not really sure what is best regarding that comment.

Copy link
Member

@lpil lpil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!!! Fantastic work!

Let's leave it as you've got it there 👍

@lpil lpil merged commit a910cf2 into gleam-lang:main Jan 20, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants