Question regarding major version release cadence

[imsnif]

Hi wasmtime,

It seems to me as if there's a major wasmtime version release every month or so. This is certainly a commendable cadence, but as a dependent of the runtime in a production environment this is a bit more than I can personally keep up with.

I'm happy to stay with older versions - the only thing I'm worried about is security vulnerabilities. I tried to find some sort of document detailing how these are handled (if at all) in older versions but could not find anything.

Could you help me out by pointing me in the right direction?

Thanks!

[alexcrichton]

We've got documentation for the release process in the online book, notably:

• A new major version of Wasmtime will be made available once a month.
• Security bugs and correctness fixes will be backported to the latest two releases of Wasmtime and issued as patch releases.

You can also read about our current policies around security issues online as well.

Outside of the online documentation some background on our current policy is that maintaining older releases for very long periods of time is a significant tax on engineering resources which are already thin for Wasmtime, so that's the primary reason we don't support, for example, year-old releases of Wasmtime. If it's reasonable for your use case we'd love assistance/contributions of engineering resources to the project to help perhaps shift the needle on this over time to maintain more releases and/or set a different cadence.

[bjorn3]

The reason Zellij is using older Wasmtime versions is because it bumps the MSRV much less frequently than Wasmtime.

[alexcrichton]

In case others are interested (I'm sure you've already seen this @bjorn3) our MSRV policy is here and it's definitely one we're open to change as well. The tradeoff though is pretty similar to the release process where we don't have infinite engineering resources but it's also not set in stone such that it can never change.

Again at least personally I'm always willing to reconsider these policies and talk about them further. In the past though most discussions have bottomed out in "well I just want my project's MSRV to be compatible" or "I want to have security updates for as long as possible". What we have now is hoping to strike a balance between maintainer time and usefulness to embedders, and the main sources of feedback we have today are (a) the embeddings that maintainers work on and (b) the frequency/feedback of issues like this.

[imsnif]

Thanks for the answers and the links! I now understand the process much better and know what to expect.

As for contributing engineering resources: I dare say Zellij has even less engineering resources than wasmtime, seeing as it's mostly just me maintaining it with the occasional outside help hat tip to @bjorn3. I do appreciate the work done by the wasmtime maintainers and wish I could give back, but unfortunately I don't see this ever happening in a reasonable way.

As for my point of view in aforementioned discussion: we use wasmtime as a runtime for our plugin system. We run untrusted user written plugins that run on user machines and need to be both backwards and forwards compatible for a very long time (the scope is measured in years and I dare hope decades). As an application/infrastructure developer it is of course my wish to have one un-changing version that will never add new features and only provide security critical updates that are relevant for my use-case.

But I of course realize this is not feasible for a project like wasmtime or a technology as new as wasm/wasi. Development and innovation must happen and one cannot simply stand still and provide support for existing embedders. Especially non-paying ones.

That being said, I feel that asking embedders to go through a significant breaking upgrade every 3 months lest they lose security updates is a lot to ask. Especially considering that in our case we would also potentially need to try and provide a path for old plugins to still work with newer versions and vice versa.

Again though, I'm only providing this feedback because you said it is valuable to you. I'm very happy with wasmtime and am sure I'll be able to smooth over any rough edges even if things stay exactly as they are.

[tschneidereit]

As Alex said, we're definitely open to reconsidering parts of the policy here: both regarding which releases we do security fixes for, and MSRV. Can you say which of these is more important to your use case?

Independently, we care a lot about not breaking content, so at least in theory you shouldn't have to worry about old plug-ins not working anymore.

[alexcrichton]

Yes thank you again for discussing this here, it's definitely appreciated! Everything you say makes sense to me as well.

In addition to Till's question, have you experienced breakage updating Wasmtime in the past? Our goal is that each major version of wasmtime has little different than the previous version API-wise. If there are areas of significant churn though that's a good sign that we should sit down and perhaps redesign or design the area again to maximize future-compatibilty.

[bjorn3]

So far the only Wasmtime update Zellij has done is from 21.0.1 (the initial version I picked when rewriting Zellij's plugin infra to use Wasmer to Wasmtime) to 21.0.2 (which had a bugfix) If I just bump the Wasmtime version to 29.0 everything at least still compiles. I just saw that zellij-org/zellij#3945 will bump the MSRV of Zellij. I might open a Zellij PR to update Wasmtime once it merges.

[alexcrichton]

Oh nice! In theory that's the experience we're aiming for at this time. We have the flexibility to change things as necessary but embedders by-and-large don't have to worry about it unless they're using practically the whole surface area of wasmtime (which I suspect Zellij is not).

This brings up a few points/questions for me:

• If an upgrade of wasmtime is ever difficult in the future, please feel free to file an issue and/or raise something on Zulip.
• Does Zellij have an MSRV policy currently? Our choice of "stable minus two" is pretty arbitrary and changing that to three or four I don't think would be unreasonable if that would match Zellij.

[imsnif]

As Alex said, we're definitely open to reconsidering parts of the policy here: both regarding which releases we do security fixes for, and MSRV. Can you say which of these is more important to your use case?

The former, definitely. In my ideal world we would ever only have to update patch releases for security reasons.

Independently, we care a lot about not breaking content, so at least in theory you shouldn't have to worry about old plug-ins not working anymore.

That's great to hear! This is honestly the most important part for me.

If an upgrade of wasmtime is ever difficult in the future, please feel free to file an issue and/or raise something on Zulip.

That's very kind and appreciated. Even though for now @bjorn3 is doing the bulk of the work and his contributions are very much appreciated - I'd like not to have to rely on him always being there.

Does Zellij have an MSRV policy currently? Our choice of "stable minus two" is pretty arbitrary and changing that to three or four I don't think would be unreasonable if that would match Zellij.

Generally no. I look at the MSRV as I do on our other dependencies - I would prefer never to have to update it/them and usually only do so if they contain critical security fixes that are relevant for us, if the version we're using is at its end of life or if we're asked to by one of our current or future packagers (eg. Debian - which is in the process of packaging wasmtime now btw).

So whichever middle-ground you'd be comfortable with would be very welcome!

[alexcrichton]

I've added an agenda item for the next Wasmtime project meeting about discussing extending either our MSRV window or security release window to raise the discussion here.

Otherwise thanks again for your feedback @imsnif it's quite valuable! I can otherwise reaffirm what @tschneidereit said in that:

• In-wasm content is highly unlikely to break between releases. Basically all users of Wasmtime have the same motivations as you which is to keep the same wasm blob running semantically the same way across releases.
• Wasmtime APIs are subject to breakage across releases but we also don't intentionally break things "just because". As @bjorn3 found the upgrade from 21 to 29 didn't result in any compiler errors at least and that's our goal otherwise. We do document all minor breaking changes in RELEASES.md for each release, however.

Regardless now seems as good a time as any to consider the N constant in these two windows, so that's why I've added an agenda item for them.

[imsnif]

Very cool! Thank you so much for giving this attention.

Just something to consider regarding said N constant: when wasmtime for example gets packaged in debian, their support life cycle is significantly longer than the scale we're talking about here: https://www.debian.org/releases/ (tl;dr five years if my understanding is correct). So if it's interesting for you to also make the life of packagers easier with regards to security updates, it might be something you want to keep in mind.

[cfallin]

One thing to add re: distribution/downstream packaging is that MSRV can be pretty important in that context: at one point I was trying to get Wasmtime working on OpenBSD (#2980) and their version of rustc is frozen at each release, so up to six months old. (rustup doesn't have binaries on that platform and it's pretty niche, so may not weigh too much here, but it's an example of timelines that our consumers may have that stretch longer than ours.)

[bjorn3]

For Debian you did only need an MSRV that matches the rustc in current unstable, right? But at the same time they need security updates for the Wasmtime version they packaged for the full lifecycle of the release. They aren't going to upgrade Wasmtime for the lifetime of the release once the feature freeze happens except for security releases. Even regular API/ABI compatible updates are not done except for a couple of packages like browsers.

[cfallin]

eg. Debian - which is in the process of packaging wasmtime now btw

As an aside, if you have any communications with the folks doing that: they appear to be custom-patching Wasmtime to remove WASI, which is fairly perplexing, and the comment on the patch suggests a misunderstanding that these are "Wasm-only" crates. I emailed the maintainer directly a bit ago when I first noticed this but never got a reply. They'll eventually find out it's essential when they try to package and test the CLI, but...

[cfallin]

But at the same time they need security updates for the Wasmtime version they packaged for the full lifecycle of the release

On the one hand, a five-year LTS release is a huge quantitative leap from our current policies; but on the other hand, if it's one specific release, then this actually doesn't seem too bad: it adds one more branch we need to consider whenever we have a CVE, and backport to if necessary; rustup should allow us to use the period-appropriate compiler to test the fix. (The ability to work in the snapshotted world seems like the biggest issue but crate version pinning and rustup have solved that IIUC.)

[imsnif]

As an aside, if you have any communications with the folks doing that: they appear to be custom-patching Wasmtime to remove WASI,

All I know about is this thread, but maybe @WeepingClown13 can help us out? (I hope the ping is alright)