Thoughts on plugin interface options

[alilleybrinker]

With Hipcheck's planned plugin system, one of the open questions is how to have plugins interact with the hc binary. Currently, we've identified three high-level candidates:

• Inter-Process Communication: Each plugin runs as its own process, with some IPC mechanism between each and the main hc process.
• WebAssembly: The hc binary runs a WebAssembly runtime internally which executes plugins within that runtime, passing data through the runtime boundary.
• C Foreign Function Interface: The hc binary dynamically links with dynamic libraries, communicating using a C FFI.

Each of these has serious trade-offs and open questions!

For IPC, there's a question of what IPC mechanism we'd use, especially given Hipcheck's goal of being cross-platform (at least Linux, macOS, and Windows, and ideally any non-Linux POSIX system like the BSDs and Solaris-derivatives). There are many IPC mechanisms with different performance characteristics.

For WebAssembly, there are questions about what WebAssembly runtime to choose, and general lack of familiarity with WebAssembly.

For C FFI, we have to deal with the limitations of C, making sure users compile things that are ABI-compatible, and generally linking is kinda messy / not a great user experience.

There's also the possibility that we could support more than one of these, allowing plugins to indicate in the plugin manifest which of these they are. The internal plugin API could be agnostic to the linking mechanism, and then use a specific mode depending on the plugin.

[j-lanson]

If we go with the Wasm or C FFI options for loading plugins, we will probably want to perform hash checks on the in-memory plugin binary data (in addition to or instead of the plugin file) to avoid TOCTOU issues. The theoretical attack of an adversary changing a plugin's data between a file-based hash check and the loading of that plugin's binary data seems pretty farfetched, but we might as well be clean about it.

[j-lanson]

According to the 2021 link Venus provided in chat yesterday (https://nullderef.com/blog/plugin-tech/), libsodium did a benchmark of WASM and found it to be 3x slower than native Rust code. I'm much in favor of the WASM plugin approach, but that would be a hard overhead to swallow given our discussion yesterday of wanting to drive down execution time.

[alilleybrinker]

Yeah, performance is definitely the main concern with the possibility of using WebAssembly. We’ll want to benchmark it ourselves, both because the article is a few years old and WebAssembly libraries have seen a lot of development since then, and because the performance impact is likely to be very workload-dependent.

[alilleybrinker]

In RFD 4 was opted to pursue gRPC-based plugins in the immediate term. We may add more plugin interface options in the future.