#576 Drop dependencies to Primus and Primus plugins

[snowteamer]

Closes #576

Summary of changes:

• Drops Primus.js and the primus-rooms and primus-responder plugins in favor of a pubsub implementation using only the ws socket library in Node and the WebSocket API in the browser.
• Increased the initial WebSocket connection timeout from 3000 to 5000 ms (Primus defaults to 10000 ms).
• Correction of a bug that often caused redundant sub/unsub requests.
• Less verbose backend console logging (from 19 k lines of build output to ~10 500 lines).
• New 'backend/server/broadcastEntry' SBP selector.

Additional information:

The core of this proposed pubsub framework consists of only two files:

• ~/backend/utils/pubsub.js which exports a createServer() function.
• ~/shared/pubsub.js which exports a createClient() factory suitable to create either ordinary pubsub clients in the browser or server-side clients in Node.
  Such server-side clients can be useful e.g. in testing or to talk to other servers.

Currently, it comes with a set of so-called "default" event handlers, designed to handle only basic pubsub protocol stuff and ENTRY messages as required by the application.

However this protocol could be extended by providing appropriate "custom" message handlers to the factories.

In progress:

Primus has automatic reconnection features which are not all implemented in this PR yet:

• Reconnect upon initial connection timeout
• Reconnect when the server restarts
• Reconnect when the online event occurs, including in case the web frontend was started in offline mode. It seems to be working now, but again tests are still missing.
• Client-side connection loss detection independently of the offline/online events using ping/pong messages (https://github.com/primus/primus#heartbeats-and-latency).

All these features are options so they can be disabled if we do not need them.

Again, testing these features might be tricky, so any help or advice will be much appreciated.

[joepie91]

Initial review; it's end of day for me now, I will likely post further review comments later :)

[taoeffect]

@snowteamer this is a really fantastic PR you've sent in! Because of its size, it will take me a bit longer to review fully, however I've gone ahead and left some feedback for now so that you can work on it / respond to it while I continue to review.

[taoeffect]

@snowteamer OK this is my final review I think. Only minor things remaining, posted comments, plus a request to use okTurtles.events where possible unless there's a reason not to. You're already using it in some parts of the code, I'm not sure why we need to re-implement event emission, and if there isn't a strong reason to, let's not do that.

[taoeffect]

Here we lost the timeout value of 3000, could that be added back in please (especially since the default appears to be 5000)?

[snowteamer]

That was on purpose, 3000 is likely too short:

• We have seen that timing issues can cause the Travis build to fail.
• All web socket libraries that I have checked use a higher default timeout or do not seem to specify any at all:
  • reconnecting-websocket: 4000
  • Primus: 10000
  • socket.io: 20000

[taoeffect]

ok, it's still useful to specify it here though to remind us that's an option and what its current value is without diving into the implementation. To your first point, we can also extend it for travis, e.g. something like this:

timeout: process.env.NODE_ENV === 'development' ? 10000 : 5000 // extend for Travis

[snowteamer]

Unfortunately crypto.randomBytes() doesn't seem to exist in the browser, so I will likely have to revert tocrypto.getRandomValues() :(

[taoeffect]

Unfortunately crypto.randomBytes() doesn't seem to exist in the browser, so I will likely have to revert tocrypto.getRandomValues() :(

Ok, I replied with a suggestion on Slack

[taoeffect]

Approved! Excellent work @snowteamer! 😄 🥇 🎉

Resolved, afaik?