Properly document v0.2.1 ABI

[mathetake]

Unfortunately, Proxy-Wasm ABI hasn't been appropriately documented since the beginning and has started being used by Envoy and Istio without the proper reference except for the code base of Envoy and Proxy-Wasm SDK/host implementations. That has caused a lot of confusion to anyone interested in this project, not only the end users but also those who is willing to contirbute.

As the former/current leads of this project discussed in the comments #38 (comment) to break this standstill situation which has lasted for the last three years, we agreed on correctly documenting the currently implemented v0.2.1 ABI which is the de-factor in this space (implemented Envoy and other proxies).

After this is resolved, we can incrementally discuss and fix the issues associated with the current ABI, such as #5 #32 #38.

cc @vikaschoudhary16 @jcchavezs @anuraaga @PiotrSikora @martijneken @mpwarres @codefromthecrypt @M4tteoP

[mathetake]

@mpwarres could you provide us with the timeline you might have in your team for this? I do not expect @PiotrSikora to put any update on their documentation work given that my comments and nudges have been ignored multiple times. (@PiotrSikora feel free to correct me if you have any intention to continue your work).

[PiotrSikora]

That has caused a lot of confusion to anyone interested in this project, not only the end users but also those who is willing to contirbute.

While I agree that the v0.2.1 spec is needed and long overdue, the lack of it hasn't stopped 5+ SDKs and 7+ hosts from being implemented, and it's hardly the biggest issue of the project.

The end users shouldn't be looking at the spec at all, the only reason they do is lack of the documentation in the SDKs, but @mpwarres and his team are working on improving that.

@mpwarres could you provide us with the timeline you might have in your team for this? I do not expect @PiotrSikora to put any update on their documentation work given that my comments and nudges have been ignored multiple times. (@PiotrSikora feel free to correct me if you have any intention to continue your work).

I clearly stated that I'm actively working on it and future improvements to the spec a month ago in #38 (comment), and @mpwarres reiterated our plan that's being executed last week in #38 (comment), so I'm not sure why you keep trying to say otherwise... You could have reached out to me on Slack, where we spoke many times, instead of misrepresenting my position in public.

Also, the last time we spoke about Proxy-Wasm, you've said that you cannot contribute to it anymore (which is fine, things change), and I haven't seen you reviewing any PRs in the proxy-wasm org in the last year or two, even though the reviews were assigned to you, so suddenly reappearing and using the maintainer status that was granted to you years ago to go on a crusade closing other people's PRs and issues, removing my content and actively excluding me from that process (without even cc'ing or requesting review from me on that PR) is extremely distasteful and disappointing.

[mathetake]

“Believe” may have been the wrong word, because the situation is awkward. It is well known there is a problem with this specification, and I took action to remove this document due to the issues it caused in forms of unnecessary confusion. Not all of these came into the issues list, but many have. I know deleting a document can feel personal, but this is about the project health and a better alternative to forking it or continued abandonment.

We (really not only me!) would really appreciate it if you could answer the following questions.

1. Are you still interested in actively working on this project again?

   • For those seeing this comment, @PiotrSikora has already resigned from the implementation owner in Envoy: Update WASM extension maintainer from PiotrSikora to mpwarres in OWNERS.md envoyproxy/envoy#25111 and he has been virtually the only blocker of any update to the specification.
   • "Actively" I meant if you can work on issues/pr triaging and comments, and work with contributors and users on a daily basis, not abandoning them. Saying "I am working on it" and disappearing forever just like Consider support wasi file read ABI proxy-wasm-cpp-host#127 (comment) is one of the examples which is dishonest to users and contirbutors.

2. If you still have intention, what's your plan as the project lead? Could you clarify the process for the project's health regarding the comment by @mpwarres Add proxy_log_destination ABI #38 (comment) How do you want to proceed? What's the disagreement between @mpwarres and you?

Hope this will clarify your position here and either way works for me!

[mathetake]

ps I thought I posted ^ in #40, not here but won't be a problem. Anyone tagged feel free to comment also!

[mpwarres]

It's encouraging to see the eagerness to move the ABI and project as a whole forward! A few responses to some of the questions and comments above.

@mathetake asked:

@mpwarres could you provide us with the timeline you might have in your team for this?

My team's immediate focus over the next couple of weeks will be to fill out and update the C++ and Rust SDK documentation. From offline conversation, @PiotrSikora has been working on the ABI, so we agreed that he would make the ABI doc changes while I work on SDK doc. I think the timeframe for the ABI documentation was also roughly 2 weeks (from now).

File-wise, I don't feel strongly about whether the vNEXT documentation is removed now vs. in a later PR that also adds doc for the current ABI (either way it gets removed), but was leaving that to Piotr.

Despite the friction in this comment thread I think it's worth pointing out that from a technical perspective, I think everyone is actually aligned in what the next steps are, namely documenting the current ABI, and creating a new version with incremental fixes.

One last clarification, re: this comment:

How do you want to proceed? What's the disagreement between @mpwarres and you?

I'm not sure what might have given the impression that there's a disagreement between @PiotrSikora and me--there isn't one. If it's the delay, that's due to the practical reality that we are each juggling other responsibilities in addition to this project.

[vikaschoudhary16]

I think the timeframe for the ABI documentation was also roughly 2 weeks (from now).

@mpwarres sorry I could not get this point on timeline. IIUC, here timeline was suggested as may-end. After that we did not get any responses on the status or timeline despite multiple requests/pings

(FWIW, I'm working on the existing & "the minimal bugfix" ABI specs right now. I'll have them out later this month.)

[jcchavezs]

I think proxy-wasm ecosystem and ultimately the community around is bigger than any individual. Something that several projects depend on cannot depend on a single person which isolated work on it, with no clear scopes or timelines, that isn't sustainable in OSS. As someone working in Wasm, writing implementations and willing for features I would like to see this project more community driven rather than having to wait for it to match someone's timeline and given the current situation, with uncertainty on when/how is this moving forward.

[gzurl]

We at Wasm Labs are looking into implementing proxy-wasm support for the Apache Web Server. We are glad to see there are renewed efforts around the specification. Looking forward to working with everyone!

[younes-io]

I was using the ABI for proxy dev but now it seems like it's not available anymore :/
Also, plenty of blog posts point to that ABI spec, but now they point to a "404 Not found" page...

[jcchavezs]

@PiotrSikora @mpwarres I wonder the status of the work, you keep saying you are working on it (one in future improvements on the spec, the other in documenting the current one) I guess all that behind the scenes so you guys are coming up with two PRs that will address both ABI changes and docs? Could we have some transparency around that? Usually starting a draft PR helps to be aware on what is going on, also what are the other pieces in the ecosystem suppose to do in the meanwhile? Shall we wait for you guys to do your secret work and expect a PR that probably have been discussed with any of the stakeholders here?

This overall situation is really crapy. As per proxy-wasm/proxy-wasm-rust-sdk#201 (comment) the suggestion is to wait until you guys come up with this long awaited work but that does not guarantee any of it will be merged since you did not bother to make it collaboratively or visible in any way?

I think many of us around ProxyWasm feel uncertainity and dissapointment around this whole situation because the endless "I am coming" is doing nothing but killing the credibility on this spec. It is fine to not have time to work on this, there is a community willing to do the work, what is not fine is to keep the project as a hostage and depicting the community.

[PiotrSikora]

“Believe” may have been the wrong word, because the situation is awkward. It is well known there is a problem with this specification, and I took action to remove this document due to the issues it caused in forms of unnecessary confusion.

IMHO, removing that document didn't solve any issues, and instead introduced new ones (e.g. #41 (comment)).

I know deleting a document can feel personal, but this is about the project health and a better alternative to forking it or continued abandonment.

I didn't take the document being removed personally. However, I don't appreciate the way it was done, and other personal attacks.

We (really not only me!) would really appreciate it if you could answer the following questions.

Who's "we"?

1. Are you still interested in actively working on this project again?

   • For those seeing this comment, @PiotrSikora has already resigned from the implementation owner in Envoy: Update WASM extension maintainer from PiotrSikora to mpwarres in OWNERS.md envoyproxy/envoy#25111 and he has been virtually the only blocker of any update to the specification.
   • "Actively" I meant if you can work on issues/pr triaging and comments, and work with contributors and users on a daily basis, not abandoning them. Saying "I am working on it" and disappearing forever just like Consider support wasi file read ABI proxy-wasm-cpp-host#127 (comment) is one of the examples which is dishonest to users and contirbutors.

I am actively working on Proxy-Wasm, but not everything is public (yet).

Regarding Envoy - I'm no longer involved with the Envoy implementation of Proxy-Wasm and I handed it off to @mpwarres. This was clearly communicated to everybody who was involved with Proxy-Wasm around that time.

Regarding issues/PRs - I'm pretty much the only one triaging, reviewing and merging PRs in proxy-wasm org. I certainly don't see you doing any of the work there, even though you're listed in CODEOWNERS, so it's weird that you're complaining about this.

Regarding file access - that's a bad example, because I've said that I'm not working on it (see: envoyproxy/envoy#22557 (comment)) and Tetrate was supposed to contribute it (see: envoyproxy/envoy#22557 (comment)). I don't believe you guys ever did that, but now there are 3 people from Tetrate here complaining about deliverables?

2. If you still have intention, what's your plan as the project lead? Could you clarify the process for the project's health regarding the comment by @mpwarres Add proxy_log_destination ABI #38 (comment) How do you want to proceed? What's the disagreement between @mpwarres and you?

The plan is to document the ABI spec and C++ and Rust SDKs, and iterate from there. I have a pretty good sense for the next version of the ABI that would help with a more iterative process.

There was never any disagreement between @mpwarres and myself.

[PiotrSikora]

I think the timeframe for the ABI documentation was also roughly 2 weeks (from now).

@mpwarres sorry I could not get this point on timeline. IIUC, here timeline was suggested as may-end. After that we did not get any responses on the status or timeline despite multiple requests/pings

(FWIW, I'm working on the existing & "the minimal bugfix" ABI specs right now. I'll have them out later this month.)

I was hoping to publish ABI v0.2.1 spec before my time off (hence end of May), but that didn't happen. I'm working on it now, and I hope to have it out in the next few days.

[PiotrSikora]

I think proxy-wasm ecosystem and ultimately the community around is bigger than any individual. Something that several projects depend on cannot depend on a single person which isolated work on it, with no clear scopes or timelines, that isn't sustainable in OSS. As someone working in Wasm, writing implementations and willing for features I would like to see this project more community driven rather than having to wait for it to match someone's timeline and given the current situation, with uncertainty on when/how is this moving forward.

There are at least 2 people from 2 different companies committed to working on Proxy-Wasm OSS (@mpwarres and myself), so it's not blocked on a single person.

@PiotrSikora @mpwarres I wonder the status of the work, you keep saying you are working on it (one in future improvements on the spec, the other in documenting the current one) I guess all that behind the scenes so you guys are coming up with two PRs that will address both ABI changes and docs? Could we have some transparency around that? Usually starting a draft PR helps to be aware on what is going on

What transparency are you expecting to see here, exactly? I don't see publishing incomplete work as a standard practice.

also what are the other pieces in the ecosystem suppose to do in the meanwhile? Shall we wait for you guys to do your secret work and expect a PR that probably have been discussed with any of the stakeholders here?

What other pieces do you mean? I don't believe that documenting current ABI spec or C++ and Rust SDKs is going to affect https://github.com/corazawaf/coraza-proxy-wasm in any way.

This overall situation is really crapy. As per proxy-wasm/proxy-wasm-rust-sdk#201 (comment) the suggestion is to wait until you guys come up with this long awaited work but that does not guarantee any of it will be merged since you did not bother to make it collaboratively or visible in any way?

The suggestion is to avoid duplicating work. How else would you handle it?

I think many of us around ProxyWasm feel uncertainity and dissapointment around this whole situation because the endless "I am coming" is doing nothing but killing the credibility on this spec. It is fine to not have time to work on this, there is a community willing to do the work, what is not fine is to keep the project as a hostage and depicting the community.

Everybody is welcome to contribute to the project. Majority of the PRs are merged, but being open to contributions doesn't mean accepting everything without giving it any thought... That's how you end up with a broken kitchen-sink.

When @mathetake was actively involved, I gave him maintainer access and added to the CODEOWNERS, so we're not trying to exclude anybody from the community, and in fact a lot was done to make it a collaborative project from the beginning (e.g. drafting ABI spec and creating proxy-wasm-cpp-host instead of doing everything in Envoy).

Regarding "community willing to do the work" - there is plenty of maintenance work and known issues to solve, but I don't see anybody from this thread doing that work, so what exactly do you mean here?

[PiotrSikora]

Still working on it. I should have it out tomorrow evening, or over the weekend at the latest.

[PiotrSikora]

I'm 95% done, but it will slip until tomorrow.

[PiotrSikora]

See: #42.