Exit statuses #11
Comments
|
@sunfishcode, I opened what I think is a similar issue here: WebAssembly/WASI#524. Can you take a look at that one and see if it should be merged here? I'm a 👍 for standardizing the error codes; if we have a consistent set of error codes then testing with wasi-testsuite will get easier. |
|
I'll leave WebAssembly/WASI#524 open to track clarifying this for preview1. For preview2, I'm now leaning towards saying "anything from [0,255] is permitted". Looking at programs in the real world, there are lots of programs that use values all throughout this range, and there doesn't seem to be a clean place to draw a boundary. So it seems best to just let the ambiguities through and let users figure it out. |
That works for me 👍 |
This was referenced May 11, 2023
|
I agree that open [0-255] is more pragmatic, though not sure if anyone is taking advantage of the value type allowing larger values. In any case, open avoids a false expectations values will be used consistently. |
|
i guess there are related but separate topics: a. how wasi exit code looks like for a., [0-255] or even full u32 range is fine for me. IMO, the idea to report wasi exit code via a host exit code is a hack at best. when you really need to know the exact exit code (eg. testsuite) |
|
With the migration of .NET to WASI P2, we are now hitting this lack of support for exit codes problem (same context as bytecodealliance/wasmtime#6352). It looks like a significant API gap - exit codes are a pretty old and established way to communicate execution summary back to the system. Indeed, the current API already includes an 'exit code', it's just a boolean-only value. Exit codes are an application-specific concern (i. e. it is the application that defines what the exit codes mean, example: https://github.com/microsoft/binskim/blob/main/docs/UserGuide.md#--rich-return-code-truefalse), so I don't see the need to attach any specific meanings to any specific values, or reserve any of them. By the same logic, the exit code should be returned as-is from the host to the system for embeddings where the WASM application is the root of execution (i. e. |
dicej
added a commit
to dicej/runtimelab
that referenced
this issue
Jun 25, 2024
As of this writing, WASIp2 [does not support process exit statuses](WebAssembly/wasi-cli#11) beyond 0 (success) and 1 (failure). To work around this, I've modified the relevant tests to return 0 on success instead of 100. Signed-off-by: Joel Dice <[email protected]>
This was referenced Jul 8, 2024
|
FYI, I spoke with @sunfishcode and @lukewagner about this the other day, and we agreed that adding a new |
dicej
added a commit
to dicej/wasi-cli
that referenced
this issue
Jul 11, 2024
Whereas the existing `exit` function only accepts a `result` parameter (i.e. binary success or failure), this function allows the instance to report any status code from 0 to 255 to the host, with 0 usually meaning "success" and other values having their own meaning depending on the context. Fixes WebAssembly#11 Signed-off-by: Joel Dice <[email protected]>
dicej
added a commit
to dicej/runtimelab
that referenced
this issue
Jul 17, 2024
This adds `WasiHttpHandler`, a new implementation of `HttpMessageHandler` based on [wasi:http/outgoing-handler](https://github.com/WebAssembly/wasi-http/blob/v0.2.0/wit/handler.wit), plus tweaks to `System.Threading` to allow async `Task`s to work in a single-threaded context, with `ThreadPool` work items dispatched from an application-provided event loop. WASIp2 supports asynchronous I/O and timers via `wasi:io/poll/pollable` resource handles. One or more of those handles may be passed to `wasi:io/poll/poll`, which will block until at least one of them is ready. In order to make this model play nice with C#'s `async`/`await` and `Task` features, we need to reconcile several constraints: - WASI is currently single-threaded, and will continue to be that way for a while. - C#'s `async`/`await` and `Task` features require a working `ThreadPool` implementation capable of deferring work. - A WASI component can export an arbitrary number of functions to the host, and though they will always be called synchronously from the host, they need to be able to perform asynchronous operations before returning. - WASIp3 (currently in the design and prototype phase) will support asynchronous exports, with the top level event loop running in the host instead of the guest, and `wasi:io/poll/pollable` will no longer exist. Therefore, we don't want to add any temporary public APIs to the .NET runtime which will become obsolete when WASIp3 arrives. The solution we arrived at looks something like this: - Tweak the existing `ThreadPool` implementation for WASI so that methods such as `RequestWorkerThread` don't throw `PlatformNotSupportedException`s (i.e. allow work items to be queued even though the "worker thread" is always the same one that is queuing the work) - Add two new methods to `Thread`: - `internal static void Dispatch`: Runs an iteration of event loop, draining the `ThreadPool` queue of ready work items and calling `wasi:io/poll/poll` with any accumulated `pollable` handles - `internal static Task Register(int pollableHandle)`: Registers the specified `pollable` handle to be `poll`ed during the next call to `Dispatch` - Note that these methods are `internal` because they're temporary and should not be part of the public API, but they are intended to be called via `UnsafeAccessor` by application code (or more precisely, code generated by `wit-bindgen` for the application) The upshot is that application code can use `wit-bindgen` (either directly or via the new `componentize-dotnet` package) to generate async export bindings which will provide an event loop backed by `Thread.Dispatch`. Additionally, `wit-bindgen` can transparently convert any `pollable` handles returned by WASI imports into `Task`s via `Thread.Register`, allowing the component to `await` them, pass them to a combinator such as `Task.WhenEach`, etc. Later, when WASIp3 arrives and we update the .NET runtime to target it, we'll be able to remove some of this code (and the corresponding code in `wit-bindgen`) without requiring significant changes to the application developer's experience. This PR contains a few C# source files that were generated by `wit-bindgen` from the official WASI WIT files, plus scripts to regenerate them if desired. Signed-off-by: Joel Dice <[email protected]> switch to `wasm32-wasip2` and update WASI test infra Now that we're using WASI-SDK 22, we can target `wasm32-wasip2`, which produces components by default and includes full `wasi:sockets` support. In order to run those components, I've updated the test infrastructure to use Wasmtime 21 (the latest release as of this writing). Other changes of note: - Tweaked src/coreclr/jit/compiler.cpp to make `Debug` builds work on Linux - Added libWasiHttp.a, which includes the encoded component type (in the form of a pre-generated Wasm object file) and a `cabi_realloc` definition. Both of these are generated by `wit-bindgen` and required by `wasm-component-ld` to generate a valid component. - Added a `FindWasmHostExecutableAndRun.sh` script for running the WASI tests on UNIX-style platforms. Signed-off-by: Joel Dice <[email protected]> various WASI build tweaks Signed-off-by: Joel Dice <[email protected]> quote libWasiHttp.a path in custom linker arg Signed-off-by: Joel Dice <[email protected]> fix wasm-component-ld download command Signed-off-by: Joel Dice <[email protected]> tweak EmccExtraArgs in CustomMain.csproj so wasm-component-ld understands it Signed-off-by: Joel Dice <[email protected]> update CMake minimum version in wasi-sdk-p2.cmake Signed-off-by: Joel Dice <[email protected]> use `HeaderDescriptor` to sort content and response headers Signed-off-by: Joel Dice <[email protected]> allow building native WASI test code in src/tests/build.sh Signed-off-by: Joel Dice <[email protected]> allow WASI runtime tests to be built and run on non-Windows systems Signed-off-by: Joel Dice <[email protected]> update runtime tests to work with WASIp2 As of this writing, WASIp2 [does not support process exit statuses](WebAssembly/wasi-cli#11) beyond 0 (success) and 1 (failure). To work around this, I've modified the relevant tests to return 0 on success instead of 100. Signed-off-by: Joel Dice <[email protected]> fix CI for Windows builds; remove unused file Signed-off-by: Joel Dice <[email protected]> disable sprintf warning in llvmlssa.cpp on macOS Signed-off-by: Joel Dice <[email protected]> remove LibraryWorld_cabi_realloc.o I didn't mean to add this to Git. Signed-off-by: Joel Dice <[email protected]> rename `generate-bindings.sh` files for clarity This makes it more obvious that, though they are similar, they each have a different job. Signed-off-by: Joel Dice <[email protected]> update to `wit-bindgen` 0.27.0 and regenerate bindings Signed-off-by: Joel Dice <[email protected]> reorganize code; add HttpClient smoke test - move System/WASIp2 to System/Threading/WASIp2 - remove generated `cabi_realloc` functions since `wasi-libc` will provide one - add HttpClient test to SmokeTests/SharedLibrary Note that I put the HttpClient test in SmokeTests/SharedLibrary since we were already using NodeJS for that test, and adding a simple loopback webserver to SharedLibraryDriver.mjs was easiest option available to keep the whole test self-contained. Signed-off-by: Joel Dice <[email protected]> implement SystemNative_SysLog for WASI Signed-off-by: Joel Dice <[email protected]> increase NodeJS stack trace limit to 200 Signed-off-by: Joel Dice <[email protected]> give guest no filesystem access in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> switch to Trace.Assert into HttpClient smoke test Signed-off-by: Joel Dice <[email protected]> rename WASIp2 directory to Wasi Signed-off-by: Joel Dice <[email protected]> fix non-GET methods and add HttpClient echo test Signed-off-by: Joel Dice <[email protected]> use azure NPM rename - WasiEventLoop.RegisterWasiPollable - WasiEventLoop.DispatchWasiEventLoop to make it less confusing on the Thread class - unification of gen-buildsys - cleanup pal_process_wasi.c fix build? more buffer /echo request body in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> fix gen-buildsys.sh regression Signed-off-by: Joel Dice <[email protected]> allow only infinite `HttpClient.Timeout`s on WASI This temporary code will be reverted once we support `System.Threading.Timer` on WASI in a forthcoming PR. Signed-off-by: Joel Dice <[email protected]> use `&` operator to simplify install-jco.ps1 Signed-off-by: Joel Dice <[email protected]> remove redundant `CheckWasmSdks` target from SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> split `FindWasmHostExecutable.sh` out of `FindWasmHostExecutableAndRun.sh` Signed-off-by: Joel Dice <[email protected]> replace component type object files with WIT files This updates `wit-bindgen` and `wasm-component-ld`, which now support producing and consuming component type WIT files as an alternative to binary object files. These files are easier to audit from a security perspective. Signed-off-by: Joel Dice <[email protected]> preserve slashes in path in SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]>
dicej
added a commit
to dicej/runtimelab
that referenced
this issue
Aug 1, 2024
This adds `WasiHttpHandler`, a new implementation of `HttpMessageHandler` based on [wasi:http/outgoing-handler](https://github.com/WebAssembly/wasi-http/blob/v0.2.0/wit/handler.wit), plus tweaks to `System.Threading` to allow async `Task`s to work in a single-threaded context, with `ThreadPool` work items dispatched from an application-provided event loop. WASIp2 supports asynchronous I/O and timers via `wasi:io/poll/pollable` resource handles. One or more of those handles may be passed to `wasi:io/poll/poll`, which will block until at least one of them is ready. In order to make this model play nice with C#'s `async`/`await` and `Task` features, we need to reconcile several constraints: - WASI is currently single-threaded, and will continue to be that way for a while. - C#'s `async`/`await` and `Task` features require a working `ThreadPool` implementation capable of deferring work. - A WASI component can export an arbitrary number of functions to the host, and though they will always be called synchronously from the host, they need to be able to perform asynchronous operations before returning. - WASIp3 (currently in the design and prototype phase) will support asynchronous exports, with the top level event loop running in the host instead of the guest, and `wasi:io/poll/pollable` will no longer exist. Therefore, we don't want to add any temporary public APIs to the .NET runtime which will become obsolete when WASIp3 arrives. The solution we arrived at looks something like this: - Tweak the existing `ThreadPool` implementation for WASI so that methods such as `RequestWorkerThread` don't throw `PlatformNotSupportedException`s (i.e. allow work items to be queued even though the "worker thread" is always the same one that is queuing the work) - Add two new methods to `Thread`: - `internal static void Dispatch`: Runs an iteration of event loop, draining the `ThreadPool` queue of ready work items and calling `wasi:io/poll/poll` with any accumulated `pollable` handles - `internal static Task Register(int pollableHandle)`: Registers the specified `pollable` handle to be `poll`ed during the next call to `Dispatch` - Note that these methods are `internal` because they're temporary and should not be part of the public API, but they are intended to be called via `UnsafeAccessor` by application code (or more precisely, code generated by `wit-bindgen` for the application) The upshot is that application code can use `wit-bindgen` (either directly or via the new `componentize-dotnet` package) to generate async export bindings which will provide an event loop backed by `Thread.Dispatch`. Additionally, `wit-bindgen` can transparently convert any `pollable` handles returned by WASI imports into `Task`s via `Thread.Register`, allowing the component to `await` them, pass them to a combinator such as `Task.WhenEach`, etc. Later, when WASIp3 arrives and we update the .NET runtime to target it, we'll be able to remove some of this code (and the corresponding code in `wit-bindgen`) without requiring significant changes to the application developer's experience. This PR contains a few C# source files that were generated by `wit-bindgen` from the official WASI WIT files, plus scripts to regenerate them if desired. Signed-off-by: Joel Dice <[email protected]> switch to `wasm32-wasip2` and update WASI test infra Now that we're using WASI-SDK 22, we can target `wasm32-wasip2`, which produces components by default and includes full `wasi:sockets` support. In order to run those components, I've updated the test infrastructure to use Wasmtime 21 (the latest release as of this writing). Other changes of note: - Tweaked src/coreclr/jit/compiler.cpp to make `Debug` builds work on Linux - Added libWasiHttp.a, which includes the encoded component type (in the form of a pre-generated Wasm object file) and a `cabi_realloc` definition. Both of these are generated by `wit-bindgen` and required by `wasm-component-ld` to generate a valid component. - Added a `FindWasmHostExecutableAndRun.sh` script for running the WASI tests on UNIX-style platforms. Signed-off-by: Joel Dice <[email protected]> various WASI build tweaks Signed-off-by: Joel Dice <[email protected]> quote libWasiHttp.a path in custom linker arg Signed-off-by: Joel Dice <[email protected]> fix wasm-component-ld download command Signed-off-by: Joel Dice <[email protected]> tweak EmccExtraArgs in CustomMain.csproj so wasm-component-ld understands it Signed-off-by: Joel Dice <[email protected]> update CMake minimum version in wasi-sdk-p2.cmake Signed-off-by: Joel Dice <[email protected]> use `HeaderDescriptor` to sort content and response headers Signed-off-by: Joel Dice <[email protected]> allow building native WASI test code in src/tests/build.sh Signed-off-by: Joel Dice <[email protected]> allow WASI runtime tests to be built and run on non-Windows systems Signed-off-by: Joel Dice <[email protected]> update runtime tests to work with WASIp2 As of this writing, WASIp2 [does not support process exit statuses](WebAssembly/wasi-cli#11) beyond 0 (success) and 1 (failure). To work around this, I've modified the relevant tests to return 0 on success instead of 100. Signed-off-by: Joel Dice <[email protected]> fix CI for Windows builds; remove unused file Signed-off-by: Joel Dice <[email protected]> disable sprintf warning in llvmlssa.cpp on macOS Signed-off-by: Joel Dice <[email protected]> remove LibraryWorld_cabi_realloc.o I didn't mean to add this to Git. Signed-off-by: Joel Dice <[email protected]> rename `generate-bindings.sh` files for clarity This makes it more obvious that, though they are similar, they each have a different job. Signed-off-by: Joel Dice <[email protected]> update to `wit-bindgen` 0.27.0 and regenerate bindings Signed-off-by: Joel Dice <[email protected]> reorganize code; add HttpClient smoke test - move System/WASIp2 to System/Threading/WASIp2 - remove generated `cabi_realloc` functions since `wasi-libc` will provide one - add HttpClient test to SmokeTests/SharedLibrary Note that I put the HttpClient test in SmokeTests/SharedLibrary since we were already using NodeJS for that test, and adding a simple loopback webserver to SharedLibraryDriver.mjs was easiest option available to keep the whole test self-contained. Signed-off-by: Joel Dice <[email protected]> implement SystemNative_SysLog for WASI Signed-off-by: Joel Dice <[email protected]> increase NodeJS stack trace limit to 200 Signed-off-by: Joel Dice <[email protected]> give guest no filesystem access in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> switch to Trace.Assert into HttpClient smoke test Signed-off-by: Joel Dice <[email protected]> rename WASIp2 directory to Wasi Signed-off-by: Joel Dice <[email protected]> fix non-GET methods and add HttpClient echo test Signed-off-by: Joel Dice <[email protected]> use azure NPM rename - WasiEventLoop.RegisterWasiPollable - WasiEventLoop.DispatchWasiEventLoop to make it less confusing on the Thread class - unification of gen-buildsys - cleanup pal_process_wasi.c fix build? more buffer /echo request body in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> fix gen-buildsys.sh regression Signed-off-by: Joel Dice <[email protected]> allow only infinite `HttpClient.Timeout`s on WASI This temporary code will be reverted once we support `System.Threading.Timer` on WASI in a forthcoming PR. Signed-off-by: Joel Dice <[email protected]> use `&` operator to simplify install-jco.ps1 Signed-off-by: Joel Dice <[email protected]> remove redundant `CheckWasmSdks` target from SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> split `FindWasmHostExecutable.sh` out of `FindWasmHostExecutableAndRun.sh` Signed-off-by: Joel Dice <[email protected]> replace component type object files with WIT files This updates `wit-bindgen` and `wasm-component-ld`, which now support producing and consuming component type WIT files as an alternative to binary object files. These files are easier to audit from a security perspective. Signed-off-by: Joel Dice <[email protected]> preserve slashes in path in SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> temporarily disable ThreadPoolWorkQueue.Dispatch assertion See dotnet/runtime#104803 Signed-off-by: Joel Dice <[email protected]> update `wit-bindgen` to version 0.28.0 Signed-off-by: Joel Dice <[email protected]>
dicej
added a commit
to dicej/runtimelab
that referenced
this issue
Aug 14, 2024
This adds `WasiHttpHandler`, a new implementation of `HttpMessageHandler` based on [wasi:http/outgoing-handler](https://github.com/WebAssembly/wasi-http/blob/v0.2.0/wit/handler.wit), plus tweaks to `System.Threading` to allow async `Task`s to work in a single-threaded context, with `ThreadPool` work items dispatched from an application-provided event loop. WASIp2 supports asynchronous I/O and timers via `wasi:io/poll/pollable` resource handles. One or more of those handles may be passed to `wasi:io/poll/poll`, which will block until at least one of them is ready. In order to make this model play nice with C#'s `async`/`await` and `Task` features, we need to reconcile several constraints: - WASI is currently single-threaded, and will continue to be that way for a while. - C#'s `async`/`await` and `Task` features require a working `ThreadPool` implementation capable of deferring work. - A WASI component can export an arbitrary number of functions to the host, and though they will always be called synchronously from the host, they need to be able to perform asynchronous operations before returning. - WASIp3 (currently in the design and prototype phase) will support asynchronous exports, with the top level event loop running in the host instead of the guest, and `wasi:io/poll/pollable` will no longer exist. Therefore, we don't want to add any temporary public APIs to the .NET runtime which will become obsolete when WASIp3 arrives. The solution we arrived at looks something like this: - Tweak the existing `ThreadPool` implementation for WASI so that methods such as `RequestWorkerThread` don't throw `PlatformNotSupportedException`s (i.e. allow work items to be queued even though the "worker thread" is always the same one that is queuing the work) - Add two new methods to `Thread`: - `internal static void Dispatch`: Runs an iteration of event loop, draining the `ThreadPool` queue of ready work items and calling `wasi:io/poll/poll` with any accumulated `pollable` handles - `internal static Task Register(int pollableHandle)`: Registers the specified `pollable` handle to be `poll`ed during the next call to `Dispatch` - Note that these methods are `internal` because they're temporary and should not be part of the public API, but they are intended to be called via `UnsafeAccessor` by application code (or more precisely, code generated by `wit-bindgen` for the application) The upshot is that application code can use `wit-bindgen` (either directly or via the new `componentize-dotnet` package) to generate async export bindings which will provide an event loop backed by `Thread.Dispatch`. Additionally, `wit-bindgen` can transparently convert any `pollable` handles returned by WASI imports into `Task`s via `Thread.Register`, allowing the component to `await` them, pass them to a combinator such as `Task.WhenEach`, etc. Later, when WASIp3 arrives and we update the .NET runtime to target it, we'll be able to remove some of this code (and the corresponding code in `wit-bindgen`) without requiring significant changes to the application developer's experience. This PR contains a few C# source files that were generated by `wit-bindgen` from the official WASI WIT files, plus scripts to regenerate them if desired. Signed-off-by: Joel Dice <[email protected]> switch to `wasm32-wasip2` and update WASI test infra Now that we're using WASI-SDK 22, we can target `wasm32-wasip2`, which produces components by default and includes full `wasi:sockets` support. In order to run those components, I've updated the test infrastructure to use Wasmtime 21 (the latest release as of this writing). Other changes of note: - Tweaked src/coreclr/jit/compiler.cpp to make `Debug` builds work on Linux - Added libWasiHttp.a, which includes the encoded component type (in the form of a pre-generated Wasm object file) and a `cabi_realloc` definition. Both of these are generated by `wit-bindgen` and required by `wasm-component-ld` to generate a valid component. - Added a `FindWasmHostExecutableAndRun.sh` script for running the WASI tests on UNIX-style platforms. Signed-off-by: Joel Dice <[email protected]> various WASI build tweaks Signed-off-by: Joel Dice <[email protected]> quote libWasiHttp.a path in custom linker arg Signed-off-by: Joel Dice <[email protected]> fix wasm-component-ld download command Signed-off-by: Joel Dice <[email protected]> tweak EmccExtraArgs in CustomMain.csproj so wasm-component-ld understands it Signed-off-by: Joel Dice <[email protected]> update CMake minimum version in wasi-sdk-p2.cmake Signed-off-by: Joel Dice <[email protected]> use `HeaderDescriptor` to sort content and response headers Signed-off-by: Joel Dice <[email protected]> allow building native WASI test code in src/tests/build.sh Signed-off-by: Joel Dice <[email protected]> allow WASI runtime tests to be built and run on non-Windows systems Signed-off-by: Joel Dice <[email protected]> update runtime tests to work with WASIp2 As of this writing, WASIp2 [does not support process exit statuses](WebAssembly/wasi-cli#11) beyond 0 (success) and 1 (failure). To work around this, I've modified the relevant tests to return 0 on success instead of 100. Signed-off-by: Joel Dice <[email protected]> fix CI for Windows builds; remove unused file Signed-off-by: Joel Dice <[email protected]> disable sprintf warning in llvmlssa.cpp on macOS Signed-off-by: Joel Dice <[email protected]> remove LibraryWorld_cabi_realloc.o I didn't mean to add this to Git. Signed-off-by: Joel Dice <[email protected]> rename `generate-bindings.sh` files for clarity This makes it more obvious that, though they are similar, they each have a different job. Signed-off-by: Joel Dice <[email protected]> update to `wit-bindgen` 0.27.0 and regenerate bindings Signed-off-by: Joel Dice <[email protected]> reorganize code; add HttpClient smoke test - move System/WASIp2 to System/Threading/WASIp2 - remove generated `cabi_realloc` functions since `wasi-libc` will provide one - add HttpClient test to SmokeTests/SharedLibrary Note that I put the HttpClient test in SmokeTests/SharedLibrary since we were already using NodeJS for that test, and adding a simple loopback webserver to SharedLibraryDriver.mjs was easiest option available to keep the whole test self-contained. Signed-off-by: Joel Dice <[email protected]> implement SystemNative_SysLog for WASI Signed-off-by: Joel Dice <[email protected]> increase NodeJS stack trace limit to 200 Signed-off-by: Joel Dice <[email protected]> give guest no filesystem access in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> switch to Trace.Assert into HttpClient smoke test Signed-off-by: Joel Dice <[email protected]> rename WASIp2 directory to Wasi Signed-off-by: Joel Dice <[email protected]> fix non-GET methods and add HttpClient echo test Signed-off-by: Joel Dice <[email protected]> use azure NPM rename - WasiEventLoop.RegisterWasiPollable - WasiEventLoop.DispatchWasiEventLoop to make it less confusing on the Thread class - unification of gen-buildsys - cleanup pal_process_wasi.c fix build? more buffer /echo request body in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> fix gen-buildsys.sh regression Signed-off-by: Joel Dice <[email protected]> allow only infinite `HttpClient.Timeout`s on WASI This temporary code will be reverted once we support `System.Threading.Timer` on WASI in a forthcoming PR. Signed-off-by: Joel Dice <[email protected]> use `&` operator to simplify install-jco.ps1 Signed-off-by: Joel Dice <[email protected]> remove redundant `CheckWasmSdks` target from SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> split `FindWasmHostExecutable.sh` out of `FindWasmHostExecutableAndRun.sh` Signed-off-by: Joel Dice <[email protected]> replace component type object files with WIT files This updates `wit-bindgen` and `wasm-component-ld`, which now support producing and consuming component type WIT files as an alternative to binary object files. These files are easier to audit from a security perspective. Signed-off-by: Joel Dice <[email protected]> preserve slashes in path in SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> temporarily disable ThreadPoolWorkQueue.Dispatch assertion See dotnet/runtime#104803 Signed-off-by: Joel Dice <[email protected]> update `wit-bindgen` to version 0.28.0 Signed-off-by: Joel Dice <[email protected]> upgrade to wasi-sdk 24 and wit-bindgen 0.29.0 Signed-off-by: Joel Dice <[email protected]> check for WASI in `PhysicalFileProvider.CreateFileWatcher` Signed-off-by: Joel Dice <[email protected]>
dicej
added a commit
to dicej/runtimelab
that referenced
this issue
Aug 15, 2024
This adds `WasiHttpHandler`, a new implementation of `HttpMessageHandler` based on [wasi:http/outgoing-handler](https://github.com/WebAssembly/wasi-http/blob/v0.2.0/wit/handler.wit), plus tweaks to `System.Threading` to allow async `Task`s to work in a single-threaded context, with `ThreadPool` work items dispatched from an application-provided event loop. WASIp2 supports asynchronous I/O and timers via `wasi:io/poll/pollable` resource handles. One or more of those handles may be passed to `wasi:io/poll/poll`, which will block until at least one of them is ready. In order to make this model play nice with C#'s `async`/`await` and `Task` features, we need to reconcile several constraints: - WASI is currently single-threaded, and will continue to be that way for a while. - C#'s `async`/`await` and `Task` features require a working `ThreadPool` implementation capable of deferring work. - A WASI component can export an arbitrary number of functions to the host, and though they will always be called synchronously from the host, they need to be able to perform asynchronous operations before returning. - WASIp3 (currently in the design and prototype phase) will support asynchronous exports, with the top level event loop running in the host instead of the guest, and `wasi:io/poll/pollable` will no longer exist. Therefore, we don't want to add any temporary public APIs to the .NET runtime which will become obsolete when WASIp3 arrives. The solution we arrived at looks something like this: - Tweak the existing `ThreadPool` implementation for WASI so that methods such as `RequestWorkerThread` don't throw `PlatformNotSupportedException`s (i.e. allow work items to be queued even though the "worker thread" is always the same one that is queuing the work) - Add two new methods to `Thread`: - `internal static void Dispatch`: Runs an iteration of event loop, draining the `ThreadPool` queue of ready work items and calling `wasi:io/poll/poll` with any accumulated `pollable` handles - `internal static Task Register(int pollableHandle)`: Registers the specified `pollable` handle to be `poll`ed during the next call to `Dispatch` - Note that these methods are `internal` because they're temporary and should not be part of the public API, but they are intended to be called via `UnsafeAccessor` by application code (or more precisely, code generated by `wit-bindgen` for the application) The upshot is that application code can use `wit-bindgen` (either directly or via the new `componentize-dotnet` package) to generate async export bindings which will provide an event loop backed by `Thread.Dispatch`. Additionally, `wit-bindgen` can transparently convert any `pollable` handles returned by WASI imports into `Task`s via `Thread.Register`, allowing the component to `await` them, pass them to a combinator such as `Task.WhenEach`, etc. Later, when WASIp3 arrives and we update the .NET runtime to target it, we'll be able to remove some of this code (and the corresponding code in `wit-bindgen`) without requiring significant changes to the application developer's experience. This PR contains a few C# source files that were generated by `wit-bindgen` from the official WASI WIT files, plus scripts to regenerate them if desired. Signed-off-by: Joel Dice <[email protected]> switch to `wasm32-wasip2` and update WASI test infra Now that we're using WASI-SDK 22, we can target `wasm32-wasip2`, which produces components by default and includes full `wasi:sockets` support. In order to run those components, I've updated the test infrastructure to use Wasmtime 21 (the latest release as of this writing). Other changes of note: - Tweaked src/coreclr/jit/compiler.cpp to make `Debug` builds work on Linux - Added libWasiHttp.a, which includes the encoded component type (in the form of a pre-generated Wasm object file) and a `cabi_realloc` definition. Both of these are generated by `wit-bindgen` and required by `wasm-component-ld` to generate a valid component. - Added a `FindWasmHostExecutableAndRun.sh` script for running the WASI tests on UNIX-style platforms. Signed-off-by: Joel Dice <[email protected]> various WASI build tweaks Signed-off-by: Joel Dice <[email protected]> quote libWasiHttp.a path in custom linker arg Signed-off-by: Joel Dice <[email protected]> fix wasm-component-ld download command Signed-off-by: Joel Dice <[email protected]> tweak EmccExtraArgs in CustomMain.csproj so wasm-component-ld understands it Signed-off-by: Joel Dice <[email protected]> update CMake minimum version in wasi-sdk-p2.cmake Signed-off-by: Joel Dice <[email protected]> use `HeaderDescriptor` to sort content and response headers Signed-off-by: Joel Dice <[email protected]> allow building native WASI test code in src/tests/build.sh Signed-off-by: Joel Dice <[email protected]> allow WASI runtime tests to be built and run on non-Windows systems Signed-off-by: Joel Dice <[email protected]> update runtime tests to work with WASIp2 As of this writing, WASIp2 [does not support process exit statuses](WebAssembly/wasi-cli#11) beyond 0 (success) and 1 (failure). To work around this, I've modified the relevant tests to return 0 on success instead of 100. Signed-off-by: Joel Dice <[email protected]> fix CI for Windows builds; remove unused file Signed-off-by: Joel Dice <[email protected]> disable sprintf warning in llvmlssa.cpp on macOS Signed-off-by: Joel Dice <[email protected]> remove LibraryWorld_cabi_realloc.o I didn't mean to add this to Git. Signed-off-by: Joel Dice <[email protected]> rename `generate-bindings.sh` files for clarity This makes it more obvious that, though they are similar, they each have a different job. Signed-off-by: Joel Dice <[email protected]> update to `wit-bindgen` 0.27.0 and regenerate bindings Signed-off-by: Joel Dice <[email protected]> reorganize code; add HttpClient smoke test - move System/WASIp2 to System/Threading/WASIp2 - remove generated `cabi_realloc` functions since `wasi-libc` will provide one - add HttpClient test to SmokeTests/SharedLibrary Note that I put the HttpClient test in SmokeTests/SharedLibrary since we were already using NodeJS for that test, and adding a simple loopback webserver to SharedLibraryDriver.mjs was easiest option available to keep the whole test self-contained. Signed-off-by: Joel Dice <[email protected]> implement SystemNative_SysLog for WASI Signed-off-by: Joel Dice <[email protected]> increase NodeJS stack trace limit to 200 Signed-off-by: Joel Dice <[email protected]> give guest no filesystem access in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> switch to Trace.Assert into HttpClient smoke test Signed-off-by: Joel Dice <[email protected]> rename WASIp2 directory to Wasi Signed-off-by: Joel Dice <[email protected]> fix non-GET methods and add HttpClient echo test Signed-off-by: Joel Dice <[email protected]> use azure NPM rename - WasiEventLoop.RegisterWasiPollable - WasiEventLoop.DispatchWasiEventLoop to make it less confusing on the Thread class - unification of gen-buildsys - cleanup pal_process_wasi.c fix build? more buffer /echo request body in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> fix gen-buildsys.sh regression Signed-off-by: Joel Dice <[email protected]> allow only infinite `HttpClient.Timeout`s on WASI This temporary code will be reverted once we support `System.Threading.Timer` on WASI in a forthcoming PR. Signed-off-by: Joel Dice <[email protected]> use `&` operator to simplify install-jco.ps1 Signed-off-by: Joel Dice <[email protected]> remove redundant `CheckWasmSdks` target from SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> split `FindWasmHostExecutable.sh` out of `FindWasmHostExecutableAndRun.sh` Signed-off-by: Joel Dice <[email protected]> replace component type object files with WIT files This updates `wit-bindgen` and `wasm-component-ld`, which now support producing and consuming component type WIT files as an alternative to binary object files. These files are easier to audit from a security perspective. Signed-off-by: Joel Dice <[email protected]> preserve slashes in path in SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> temporarily disable ThreadPoolWorkQueue.Dispatch assertion See dotnet/runtime#104803 Signed-off-by: Joel Dice <[email protected]> update `wit-bindgen` to version 0.28.0 Signed-off-by: Joel Dice <[email protected]> upgrade to wasi-sdk 24 and wit-bindgen 0.29.0 Signed-off-by: Joel Dice <[email protected]> check for WASI in `PhysicalFileProvider.CreateFileWatcher` Signed-off-by: Joel Dice <[email protected]>
dicej
added a commit
to dicej/runtimelab
that referenced
this issue
Aug 26, 2024
This adds `WasiHttpHandler`, a new implementation of `HttpMessageHandler` based on [wasi:http/outgoing-handler](https://github.com/WebAssembly/wasi-http/blob/v0.2.0/wit/handler.wit), plus tweaks to `System.Threading` to allow async `Task`s to work in a single-threaded context, with `ThreadPool` work items dispatched from an application-provided event loop. WASIp2 supports asynchronous I/O and timers via `wasi:io/poll/pollable` resource handles. One or more of those handles may be passed to `wasi:io/poll/poll`, which will block until at least one of them is ready. In order to make this model play nice with C#'s `async`/`await` and `Task` features, we need to reconcile several constraints: - WASI is currently single-threaded, and will continue to be that way for a while. - C#'s `async`/`await` and `Task` features require a working `ThreadPool` implementation capable of deferring work. - A WASI component can export an arbitrary number of functions to the host, and though they will always be called synchronously from the host, they need to be able to perform asynchronous operations before returning. - WASIp3 (currently in the design and prototype phase) will support asynchronous exports, with the top level event loop running in the host instead of the guest, and `wasi:io/poll/pollable` will no longer exist. Therefore, we don't want to add any temporary public APIs to the .NET runtime which will become obsolete when WASIp3 arrives. The solution we arrived at looks something like this: - Tweak the existing `ThreadPool` implementation for WASI so that methods such as `RequestWorkerThread` don't throw `PlatformNotSupportedException`s (i.e. allow work items to be queued even though the "worker thread" is always the same one that is queuing the work) - Add two new methods to `Thread`: - `internal static void Dispatch`: Runs an iteration of event loop, draining the `ThreadPool` queue of ready work items and calling `wasi:io/poll/poll` with any accumulated `pollable` handles - `internal static Task Register(int pollableHandle)`: Registers the specified `pollable` handle to be `poll`ed during the next call to `Dispatch` - Note that these methods are `internal` because they're temporary and should not be part of the public API, but they are intended to be called via `UnsafeAccessor` by application code (or more precisely, code generated by `wit-bindgen` for the application) The upshot is that application code can use `wit-bindgen` (either directly or via the new `componentize-dotnet` package) to generate async export bindings which will provide an event loop backed by `Thread.Dispatch`. Additionally, `wit-bindgen` can transparently convert any `pollable` handles returned by WASI imports into `Task`s via `Thread.Register`, allowing the component to `await` them, pass them to a combinator such as `Task.WhenEach`, etc. Later, when WASIp3 arrives and we update the .NET runtime to target it, we'll be able to remove some of this code (and the corresponding code in `wit-bindgen`) without requiring significant changes to the application developer's experience. This PR contains a few C# source files that were generated by `wit-bindgen` from the official WASI WIT files, plus scripts to regenerate them if desired. Signed-off-by: Joel Dice <[email protected]> switch to `wasm32-wasip2` and update WASI test infra Now that we're using WASI-SDK 22, we can target `wasm32-wasip2`, which produces components by default and includes full `wasi:sockets` support. In order to run those components, I've updated the test infrastructure to use Wasmtime 21 (the latest release as of this writing). Other changes of note: - Tweaked src/coreclr/jit/compiler.cpp to make `Debug` builds work on Linux - Added libWasiHttp.a, which includes the encoded component type (in the form of a pre-generated Wasm object file) and a `cabi_realloc` definition. Both of these are generated by `wit-bindgen` and required by `wasm-component-ld` to generate a valid component. - Added a `FindWasmHostExecutableAndRun.sh` script for running the WASI tests on UNIX-style platforms. Signed-off-by: Joel Dice <[email protected]> various WASI build tweaks Signed-off-by: Joel Dice <[email protected]> quote libWasiHttp.a path in custom linker arg Signed-off-by: Joel Dice <[email protected]> fix wasm-component-ld download command Signed-off-by: Joel Dice <[email protected]> tweak EmccExtraArgs in CustomMain.csproj so wasm-component-ld understands it Signed-off-by: Joel Dice <[email protected]> update CMake minimum version in wasi-sdk-p2.cmake Signed-off-by: Joel Dice <[email protected]> use `HeaderDescriptor` to sort content and response headers Signed-off-by: Joel Dice <[email protected]> allow building native WASI test code in src/tests/build.sh Signed-off-by: Joel Dice <[email protected]> allow WASI runtime tests to be built and run on non-Windows systems Signed-off-by: Joel Dice <[email protected]> update runtime tests to work with WASIp2 As of this writing, WASIp2 [does not support process exit statuses](WebAssembly/wasi-cli#11) beyond 0 (success) and 1 (failure). To work around this, I've modified the relevant tests to return 0 on success instead of 100. Signed-off-by: Joel Dice <[email protected]> fix CI for Windows builds; remove unused file Signed-off-by: Joel Dice <[email protected]> disable sprintf warning in llvmlssa.cpp on macOS Signed-off-by: Joel Dice <[email protected]> remove LibraryWorld_cabi_realloc.o I didn't mean to add this to Git. Signed-off-by: Joel Dice <[email protected]> rename `generate-bindings.sh` files for clarity This makes it more obvious that, though they are similar, they each have a different job. Signed-off-by: Joel Dice <[email protected]> update to `wit-bindgen` 0.27.0 and regenerate bindings Signed-off-by: Joel Dice <[email protected]> reorganize code; add HttpClient smoke test - move System/WASIp2 to System/Threading/WASIp2 - remove generated `cabi_realloc` functions since `wasi-libc` will provide one - add HttpClient test to SmokeTests/SharedLibrary Note that I put the HttpClient test in SmokeTests/SharedLibrary since we were already using NodeJS for that test, and adding a simple loopback webserver to SharedLibraryDriver.mjs was easiest option available to keep the whole test self-contained. Signed-off-by: Joel Dice <[email protected]> implement SystemNative_SysLog for WASI Signed-off-by: Joel Dice <[email protected]> increase NodeJS stack trace limit to 200 Signed-off-by: Joel Dice <[email protected]> give guest no filesystem access in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> switch to Trace.Assert into HttpClient smoke test Signed-off-by: Joel Dice <[email protected]> rename WASIp2 directory to Wasi Signed-off-by: Joel Dice <[email protected]> fix non-GET methods and add HttpClient echo test Signed-off-by: Joel Dice <[email protected]> use azure NPM rename - WasiEventLoop.RegisterWasiPollable - WasiEventLoop.DispatchWasiEventLoop to make it less confusing on the Thread class - unification of gen-buildsys - cleanup pal_process_wasi.c fix build? more buffer /echo request body in SharedLibraryDriver.mjs Signed-off-by: Joel Dice <[email protected]> fix gen-buildsys.sh regression Signed-off-by: Joel Dice <[email protected]> allow only infinite `HttpClient.Timeout`s on WASI This temporary code will be reverted once we support `System.Threading.Timer` on WASI in a forthcoming PR. Signed-off-by: Joel Dice <[email protected]> use `&` operator to simplify install-jco.ps1 Signed-off-by: Joel Dice <[email protected]> remove redundant `CheckWasmSdks` target from SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> split `FindWasmHostExecutable.sh` out of `FindWasmHostExecutableAndRun.sh` Signed-off-by: Joel Dice <[email protected]> replace component type object files with WIT files This updates `wit-bindgen` and `wasm-component-ld`, which now support producing and consuming component type WIT files as an alternative to binary object files. These files are easier to audit from a security perspective. Signed-off-by: Joel Dice <[email protected]> preserve slashes in path in SharedLibrary.csproj Signed-off-by: Joel Dice <[email protected]> temporarily disable ThreadPoolWorkQueue.Dispatch assertion See dotnet/runtime#104803 Signed-off-by: Joel Dice <[email protected]> update `wit-bindgen` to version 0.28.0 Signed-off-by: Joel Dice <[email protected]> upgrade to wasi-sdk 24 and wit-bindgen 0.29.0 Signed-off-by: Joel Dice <[email protected]> check for WASI in `PhysicalFileProvider.CreateFileWatcher` Signed-off-by: Joel Dice <[email protected]> switch back to WASI 0.2.0 0.2.1 is not yet widely supported, and causes [trouble](bytecodealliance/jco#486) for Jco, which rely on for the `SharedLibrary` test. Signed-off-by: Joel Dice <[email protected]> remove use of `WeakReference` from `WasiEventLoop` This was causing `HttpClient` timeout tests in the `SharedLibrary` smoke test suite to fail, apparently due to `TimerQueue.SetNextTimer` calling `WasiEventLoop.RegisterWasiPollable`, attaching a continuation to the resulting `Task` and then letting go of the reference, allowing it to be GC'd. Signed-off-by: Joel Dice <[email protected]> skip unsupported signal handling on WASI Signed-off-by: Joel Dice <[email protected]> throw PlatformNotSupportedException in ManualResetEventSlim.Wait on WASI Otherwise, we end up in an infinite loop. Signed-off-by: Joel Dice <[email protected]> Revert "switch back to WASI 0.2.0" This reverts commit a8608b4. enable `NameResolution` and `Sockets` on WASI Signed-off-by: Joel Dice <[email protected]> set `SocketsHttpHandler.IsEnabled` to `false` on WASI ...at least until we get `System.Net.Sockets` working. Signed-off-by: Joel Dice <[email protected]>
|
This is implemented in wasmtime as of bytecodealliance/wasmtime#9381 behind the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently the exit status of a command is a
result<_, _>, meaning it's a boolean success or failure. That has the advantage that we don't need to think about the meanings of exit statuses across platforms, or hidden conventions between programs and their environments. However, it is more restrictive than the intersection of Unix and Windows, so we should consider supporting this intersection.I was earlier under the impression that Windows had a convention of using error codes for exit statuses, and it appears some programs do do that, but it's not as common as I initially thought.
So the exit status space looks something like this:
0 = success
1 = generic failure
2 = custom uses in the wild
3 = abort on Windows
...various custom uses in the wild...
???-124 - Perhaps we should reserve some codes here in case Unix reserves them in the future?
125 - container failed to run (Unix)
126 - command not executable (Unix)
127 - command not found (Unix)
128 and up = program exited with signal (Unix)
256 and up = unrepresentable (Unix)
That suggests we should change the type to
result<0, u8>and have the runtime trap if theu8value is:resultrather than the error codeOpen questions:
3, to avoid ambiguity about whether a 3 is coming from WASI or from a host Windowsabort?The text was updated successfully, but these errors were encountered: