Fails on timestamp comparison

[andsel]

Build scan:
Issue seen locally.

Repro line:
Set the local timezone to something different than UTC.

• this fails

TZ="Europe/Rome" bundle exec rspec -fd spec/inputs/jdbc_spec.rb:1469

• this successed

TZ=UTC bundle exec rspec -fd spec/inputs/jdbc_spec.rb:1469

bundle exec rspec spec/inputs/jdbc_spec.rb

or

bundle exec rspec -fd spec/inputs/jdbc_spec.rb:1469

Reproduces locally?:
yes

Applicable branches:
main commit 6a75566

Failure history:

Failure excerpt:

Failures:

  1) LogStash::Inputs::Jdbc when fetching Various Typed data should convert all columns to valid Event acceptable data types
     Failure/Error: expect(event.get("started_at")).to be_a_logstash_timestamp_equivalent_to("1999-12-31T00:00:00.000Z")
       expected 1999-12-30T23:00:00Z to be a LogStash::Timestamp equivalent to 1999-12-31T00:00:00Z
     # ./spec/inputs/jdbc_spec.rb:1429:in `block in <main>'
     # /home/andrea/.rvm/gems/jruby-9.2.19.0/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:46:in `block in <main>'
     # /home/andrea/.rvm/gems/jruby-9.2.19.0/gems/logstash-devutils-2.4.0-java/lib/logstash/devutils/rspec/spec_helper.rb:61:in `block in <main>'

  2) LogStash::Inputs::Jdbc when debug logging and a count query raises a count related error if the count query raises an error should create an event normally
     Failure/Error: expect(event.get("started_at")).to be_a_logstash_timestamp_equivalent_to("1999-12-31T00:00:00.000Z")
       expected 1999-12-30T23:00:00Z to be a LogStash::Timestamp equivalent to 1999-12-31T00:00:00Z
     # ./spec/inputs/jdbc_spec.rb:1476:in `block in <main>'
     # /home/andrea/.rvm/gems/jruby-9.2.19.0/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:46:in `block in <main>'
     # /home/andrea/.rvm/gems/jruby-9.2.19.0/gems/logstash-devutils-2.4.0-java/lib/logstash/devutils/rspec/spec_helper.rb:61:in `block in <main>'

  3) LogStash::Inputs::Jdbc when iteratively running plugin#run with timestamp tracking column with column value should successfully update sql_last_value
     Failure/Error: expect(plugin.instance_variable_get("@value_tracker").value).to eq(Time.parse(times[1]))
     
       expected: 2015-05-07 13:14:15.000000000 +0200
            got: 2015-05-07 13:14:15.000000000 +0000
     
       (compared using ==)
     
       Diff:
       @@ -1 +1 @@
       -2015-05-07 13:14:15 +0200
       +2015-05-07 13:14:15 +0000
       
     # ./spec/inputs/jdbc_spec.rb:776:in `block in <main>'
     # /home/andrea/.rvm/gems/jruby-9.2.19.0/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:46:in `block in <main>'
     # /home/andrea/.rvm/gems/jruby-9.2.19.0/gems/logstash-devutils-2.4.0-java/lib/logstash/devutils/rspec/spec_helper.rb:61:in `block in <main>'

Finished in 19.45 seconds (files took 2.93 seconds to load)
66 examples, 3 failures

Failed examples:

rspec ./spec/inputs/jdbc_spec.rb:1424 # LogStash::Inputs::Jdbc when fetching Various Typed data should convert all columns to valid Event acceptable data types
rspec ./spec/inputs/jdbc_spec.rb:1469 # LogStash::Inputs::Jdbc when debug logging and a count query raises a count related error if the count query raises an error should create an event normally
rspec ./spec/inputs/jdbc_spec.rb:767 # LogStash::Inputs::Jdbc when iteratively running plugin#run with timestamp tracking column with column value should successfully update sql_last_value

Considerations

Looking into, seems that the slippery rise in

logstash-integration-jdbc/lib/logstash/plugin_mixins/jdbc/jdbc.rb

Line 264 in 6a75566

LogStash::Timestamp.new(value.to_time)

, the Date#to_time adds the local timezone.
In general:

require "date"
d = Date.parse("1999/12/31")
d.to_time != d.to_datetime.to_time

This issue is correlated to #53 and it's here to discuss if let the plugin work with Logstash::Timestamp or use Date and DateTime ruby classes, in general how to solve this problem, keeping in mind that the user can customize the timezone setting the option jdbc_default_timezone.

cc @kares