Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter out verbose events in the Console #4150

Merged
merged 11 commits into from
May 12, 2021

Conversation

kschiffer
Copy link
Contributor

@kschiffer kschiffer commented May 10, 2021

Summary

This PR will apply a toggle-able verbosity filter to the event stream which will by default filter out all events that are not typically of direct interest.

References #2231

With verbose mode deactivated (default) only the following events will be ingested:

export const EVENT_VERBOSE_FILTERS = [
  'as.*.drop',
  'as.down.data.forward',
  'as.up.data.forward',
  'gs.down.send',
  'gs.gateway.connect',
  'gs.gateway.disconnect',
  'gs.status.receive',
  'gs.up.receive',
  'js.join.accept',
  'js.join.reject',
  'ns.mac.*.answer.reject',
  '*.warning',
  '*.fail',
  'organization.*',
  'user.*',
  'gateway.*',
  'application.*',
  'end_device.*',
  'client.*',
  'oauth.*',
]

Changes

  • Add a constant module for event filters
  • Update store logic for applying filters to event streams
  • Update the <Events /> component to enable filtering out verbose events
  • Update entity event containers accordingly

Testing

Manual testing in the staging environment

Regressions

This could affect event ingestion in the Console

Notes for Reviewers

  • I did not add the filter for organization events, since the filter would have no effect. For completeness I added the redux primitives for organizations though.
  • I have implemented this filter at the ingestion level within the reducer, meaning that filtered events will not be added to the store. This means that historical verbose events can not be toggled, since they were already discarded. I've decided to go this route to also lift some weight on the whole event store logic (see also Data views slow down and freeze after receiving many events #2887). With this implementation it will be possible to examine events for entities with a higher event frequency.
    • Later on, other filter could be applied on top of this but filtered out on the component level so that the visibility can be toggled for past events as well.

Checklist

  • Scope: The referenced issue is addressed, there are no unrelated changes.
  • Compatibility: The changes are backwards compatible with existing API, storage, configuration and CLI, according to the compatibility commitments in README.md for the chosen target branch.
  • Documentation: Relevant documentation is added or updated.
  • Changelog: Significant features, behavior changes, deprecations and fixes are added to CHANGELOG.md.
  • Commits: Commit messages follow guidelines in CONTRIBUTING.md, there are no fixup commits left.

@kschiffer kschiffer added c/console This is related to the Console technical debt Not necessarily broken, but could be done better/cleaner ui/web This is related to a web interface labels May 10, 2021
@kschiffer kschiffer added this to the v3.13.0 milestone May 10, 2021
@kschiffer kschiffer requested a review from bafonins May 10, 2021 17:14
@kschiffer kschiffer requested a review from asmulko as a code owner May 10, 2021 17:14
@kschiffer kschiffer self-assigned this May 10, 2021
Copy link
Contributor

@bafonins bafonins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, is this a temporary solution for #2231? What is the plan there?

I experience significant delay when opening event details for both development and production builds. This does not happen to me on the latest v3.12 branch as well as on staging Console

Also, Im getting the following warning when opening the events tab:

Could not load worker TypeError: name.split is not a function
    at AppConfig.exports.moduleUrl (ace.js:3722)
    at WorkerClient.$createWorkerFromOldConfig (ace.js:19056)
    at new WorkerClient (ace.js:19034)
    at Mode.createWorker (json.js?818b:300)
    at EditSession.$startWorker (ace.js:9735)
    at EditSession.$onChangeMode (ace.js:9691)
    at EditSession.setMode (ace.js:9655)
    at ReactAce.libs_0dc51107365406cd4f90../node_modules/react-ace/lib/ace.js.ReactAce.componentDidMount (ace.js:73)
    at ReactAce.eval (react-hot-loader.development.js?c2cb:707)
    at commitLifeCycles (react-dom.development.js?7f13:20664) 
    at ReactAce (eval at ES6ProxyComponentFactory (webpack-internal:///./node_modules/react-hot-loader/dist/react-hot-loader.development.js), <anonymous>:14:7)
    at div
    at CodeEditor (eval at ES6ProxyComponentFactory (webpack-internal:///./node_modules/react-hot-loader/dist/react-hot-loader.development.js), <anonymous>:14:7)
    at eval (webpack-internal:///./pkg/webui/console/components/events/details/raw.js:168:24)
    at div
    at EventDetails (webpack-internal:///./pkg/webui/console/components/events/details/index.js:306:24)

@kschiffer
Copy link
Contributor Author

So, is this a temporary solution for #2231? What is the plan there?

Yes this is a temporary fix for the event stream being too verbose. This is to improve the UX already until the new event UX is designed and implemented.

I experience significant delay when opening event details for both development and production builds. This does not happen to me on the latest v3.12 branch as well as on staging Console

Also, Im getting the following warning when opening the events tab:

I think this was introduced via #4104, see also securingsincity/react-ace#732

I downgraded react-ace back. Can you see if this resolves the issue? I didn't have a big delay either way but feel like the downgraded version is quicker indeed.

Copy link
Contributor

@bafonins bafonins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, downgrading react-ace solves perf issues

@kschiffer kschiffer force-pushed the feature/2231-event-verbosity-filter branch from fecfd64 to fb0fab5 Compare May 12, 2021 10:44
@github-actions github-actions bot added the dependencies Pull requests that update a dependency file label May 12, 2021
@kschiffer kschiffer force-pushed the feature/2231-event-verbosity-filter branch from fb0fab5 to 9f1aa8c Compare May 12, 2021 11:34
@johanstokking johanstokking merged commit 646c4dd into v3.13 May 12, 2021
@johanstokking johanstokking deleted the feature/2231-event-verbosity-filter branch May 12, 2021 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c/console This is related to the Console dependencies Pull requests that update a dependency file technical debt Not necessarily broken, but could be done better/cleaner ui/web This is related to a web interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants