Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor handle_raw_in to avoid data loss under high usage scenarios #31

Closed
wants to merge 12 commits into from

Conversation

italovalcy
Copy link

Fixes #25
Fixes #27

Note: This PR was built on top of PR #28

Description of the change

Added mechanism to prevent race conditions for the variable connection.remaining_data. This variable can be updated from two or more different threads handling kytos/core.openflow.raw.in KytosEvents. Thus, I've added a threading Lock mechanism per connection which will guarantee only one thread can update at the same time.

Additionally, the handle_raw_in method was creating one KytosEvent for each individual multipart OpenFlow message, which could lead to data loss as described in issue #25. To deal with such scenarios, I've changed the way handle_raw_in works for multipart messages: instead of creating one KytosEvent as soon as the message was unpacked, we group them together and emit the Kytos event at the end; before emitting the Kytos events for all the messages, we save the number of messages that are expected to be received. Thus, the handle_multipart_replies() is now able to know how many parts should be received and it will wait a couple of seconds before assembling them all into the switch's FlowStats. The maximum wait interval is based on the existing setting STATS_INTERVAL (half of it).

It is worth mentioning that other improvements can be done in the future, such as: grouping all multiple parts of a reply into a single KytosEvent and having handle_multipart_reply() to loop through the messages; having a dedicated queue to handle prioritized OpenFlow messages (such as echo req/reply).

Release notes

  • Added mechanism to prevent race conditions for connection remaining data (OpenFlow messages received buffer)
  • Improved the handling of OpenFlow multipart messages, to avoid data loss between different event handlers (threads)

@italovalcy italovalcy closed this Nov 10, 2021
@italovalcy italovalcy deleted the issue_25+issue_27 branch November 10, 2021 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant