Console Live Data different between Application level and Device level

[jpmeijers]

Summary

Per-device Live Data has uplinks and join at correct times:

Per-application Live Data has joins from yesterday, but join from today and uplinks from today are missing:

Comparing these two Live Data view, things do not tie up. The per-device view shows three uplinks - which I also see in the storage integration. The per-application Live Data shows two joins and an uplink after each join. The joins are from yesterday around 16h. The three uplinks were all three today around 13h (according to the storage integration too).

The per-application Live Data has the uplinks from today missing - and I think a join from today is missing too.

Steps to Reproduce

1. Not sure

What do you see now?

Live Data views do not agree.

What do you want to see instead?

They should show the same data

Environment

Ubuntu 20.04
Chrome Version 100.0.4896.88 (Official Build) (64-bit)
TTS 3.19.0

How do you propose to implement this?

...

How do you propose to test this?

...

Can you do this yourself and submit a Pull Request?

No

[kschiffer]

The reason for this mismatch is a difference in event retention per entity type. This means that in some cases the number of historical events retained is different for the application stream and the end device stream.
For data coming in live, the messages should match as expected. However, we also recently changed which event types are coming through for applications:

EndDevice CRUD:

end_device.*
ns.end_device.*
as.end_device.*
js.end_device.*

AS Forwarded Uplinks/Downlinks:

as.*.forward

JS Joins:

js.join.accept

Errors and warnings:

*.fail
*.warning
*.drop
*.reject

NS MAC layer:

ns.class.switch.*

The end device stream will give a couple of other event types in addition to that.

I will think about a way to communicate this info also in the Console UI.

[kschiffer]

So I have reviewed the problem again and it became apparent that there are indeed gaps between the application and end device layer events that should not be there. The issue originates from the backend and we are currently verifying a fix.

Thanks for your help in identifying this.