add unit tests for Kalman filter library

[murphyk]

Verify that the JSL kalman filter / smoother code returns the same marginal means, covariances and log marginal likelihood as when using tfd.LinearGaussianModel,
similar to the 1d test used in kalman_sampler_demo.ipynb.

[maheswarantp]

I would like to work on this one if its okay. Sorry for the late response, I had exams scheduled during this time

[murphyk]

Sure

On Sun, May 1, 2022 at 7:26 AM Maheswaran Parameswaran < ***@***.***> wrote: I would like to work on this one if its okay :) — Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDK6EGE62VXTK3P6WGFYADVH2IBRANCNFSM5UDW3H6A> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Sent from Gmail Mobile

[gerdm]

Hi @maheswarantp,

should I assign this issue to you?

[maheswarantp]

Sure, I am working on this

[maheswarantp]

Minor changes: @murphyk I am just wrapping this, you can see my code here will send a PR in a day or two.

TODO

• Usage of different covariance comparison techniques other than Sum of Squared Distances
• Comparisons of Log Likelihoods
  It would be a great help if you could suggest any changes if needed

[gerdm]

@maheswarantp, can you also create a script for this? We want to add automatic unit-tests for future releases of JSL.

[murphyk]

Unit tests should be scripts which have statements like assert np.allclose(A,B,tol=1e-2) that can be checked automatically, without needing to inspect plots by eye.

You define the model twice, once using TFP notation, and once using JAX. This introduces the possibility of inconsistency.
You should define the parmaeters once (eg as numpy arrays) and then convert to TFP or JSL objects.

Most importantly, from your plot it looks like JSL and TFP ave slighlty different posteriors for time step 1,
presumably due to the edge case of how the prior and/ or initial observation is used. Please resolve this discrepancy.

[murphyk]

Please use https://docs.python.org/3/library/unittest.html

[maheswarantp]

Okay, I will work on that, thanks :)

[maheswarantp]

Building on @gileshd kalman_filter_test.py, I have added a few more unit tests which could be useful. They are

• Test to ensure proper behavior of the kalman filter as comapred to TFP when random noise in measurements is induced
• Test to ensure proper behavior of the kalman filter as compared to TFP when initial prior measurements are randomized

I have also extended the unit tests to include the kalman smoother code, testing the posterior sampling of JSL and TFP here

Also @murphyk , regarding the unit tests for log marginal likelihoods asked, do you mean the conditional likelihood computed in kalman smoother code or something different?

[murphyk]

@maheswarantp Please make a proper PR. Also please use pytest framework.

[gileshd]

@maheswarantp no need for playing around with paths with:
sys.path.insert(0,os.path.abspath(os.path.dirname(__file__)))
(e.g. here).

You can just use absolute imports as is done in #54 (and discussed in issue #53).

[gileshd]

@maheswarantp re kalman_filter_initial_prior_test(), I imagine having unseeded random numbers (i.e. the calls to np.random) in a test might make it quite hard to debug because it will be hard to reproduce the failure case.

In fact this test should probably fail because the current implementation of tfp_filter does not include an argument to change the prior covariance, it is set as the identity matrix:

    prior = tfd.MultivariateNormalDiag(mu0, tf.ones([state_size]))

Therefore the random numbers in kalman_filter_initial_prior_test only affect the JSL kalman filter not the tfp one.

I think in general this particular test probably isn't necessary or suitable for as part of a suite of automated tests.