Merge branch main into dev/checkpoint_and_restore #3874
Merged
+7,991
−2,903
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Merge branch main into dev/shared_heap
Merge branch main into dev/shared_heap
And enable merged os_mmap for aot data and text sections except on platform nuttx and esp-idf. Fix issue that aarch64 AOT module fails to load on android: #2274
Merge branch main into dev/merge_aot_data_text
Merge branch main into dev/shared_heap
Make wamrc normalize "arm64" to "aarch64v8". Previously the only way to make the "arm64" target was to not specify a target on 64 bit arm-based mac builds. Now arm64 and aarch64v8 are treated as the same. Make aot_loader accept "aarch64v8" on arm-based apple (as well as accepting legacy "arm64" based aot targets). This also removes __APPLE__ and __MACH__ from the block that defaults size_level to 1 since it doesn't seem to be supported for aarch64: `LLVM ERROR: Only small, tiny and large code models are allowed on AArch64`
Fix the compilation error of this CI: https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/10575515238 ``` /__w/wasm-micro-runtime/wasm-micro-runtime/bloaty/third_party/abseil-cpp/absl/debugging/failure_signal_handler.cc:139:32: error: no matching function for call to 'max(long int, int)' 139 | size_t stack_size = (std::max(SIGSTKSZ, 65536) + page_mask) & ~page_mask; | ~~~~~~~~^~~~~~~~~~~~~~~~~ ```
Merge branch dev/merge_aot_data_text into main to keep the commit history.
…ck is enabled (#3754) In the AOT compiler, allow the user to control stack boundary check when the boundary check is enabled (e.g. `wamrc --bounds-checks=1`). Now the code logic is: 1. When `--stack-bounds-checks` is not set, it will be the same value as `--bounds-checks`. 2. When `--stack-bounds-checks` is set, it will be the option value no matter what the status of `--bounds-checks` is.
…64 or riscv64 (#3755) Mac on aarch64 uses posix_memmap.c os_mmap which doesn't do anything with the flag MMAP_MAP_32BIT for that build so this condition ends up asserting unless the mapping ends up in the first 4 gigs worth of addressable space. Thsi PR changes to call os_mmap with MMAP_MAP_32BIT flag only when the target is x86-64 or riscv64, and the macro __APPLE__ isn't enabled. The behavior is similar to what the posix os_mmap does.
The specific commit has been deleted, I am pointing to the same commit in the main branch though.
- Only retry on EAGAIN, ENOMEM or EINTR. - On EINTR, don't count it against the retry budget, just keep retrying. EINTR can happen in bursts. - Log the errno on failure, and don't conditionalize that logging on BH_ENABLE_TRACE_MMAP. In other parts of the code, error logging is not conditional on that define, while turning on that tracing define makes things overly verbose.
…g and other cases (#3768)
Those parameters can be used to reduce the size of the AOT code. There's going to be more changes related to AOT code size reduction, this is just the initial step. p.s. #3758
Fixes to enable building iwasm_shared and iwasm_static libraries on win32.
Minimum support: - [x] accept (WasmEdge) customized model parameters. metadata. - [x] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [x] basic - [x] chatml - [x] gemma - [x] llama - [x] qwen --- In the future, to support if required: - [ ] Target [wasmedge-ggml examples](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml) - [ ] command-r. (>70G memory requirement) - [ ] embedding. (embedding mode) - [ ] grammar. (use the grammar option to constrain the model to generate the JSON output) - [ ] llama-stream. (new APIS `compute_single`, `get_output_single`, `fini_single`) - [ ] llava. (image representation) - [ ] llava-base64-stream. (image representation) - [ ] multimodel. (image representation) - [ ] Target [llamaedge](https://github.com/LlamaEdge/LlamaEdge)
- Implement TINY / STANDARD frame modes - tiny mode is only able to keep track on the IP and func idx, STANDARD mode provides more capabilities (parameters, stack pointer etc.). - Implement FRAME_PER_FUNCTION / FRAME_PER_CALL modes - frame per function adds code at the beginning and at the end of each function for allocating / deallocating stack frame, whereas in per-call mode the frame is allocated before each call. The exception is call to the imported function, where frame-per-function mode also allocates the stack before the `call` instruction (as it can't instrument the imported function). At the moment TINY + FRAME_PER_FUNCTION is automatically enabled in case GC and perf profiling are disabled and `values` call stack feature is not requested. In all the other cases STANDARD + FRAME_PER_CALL is used. STANDARD + FRAME_PER_FUNCTION and TINY + FRAME_PER_CALL are currently not implemented but possible, and might be enabled in the future. ps. #3758
It seems failing too frequently. cf. #3776
I'm not sure we want to use C99 %tu here. While C99 %zu is more widely used in WAMR, %tu is rare (if any) and I'm not sure if it's ubiquitously implemented in platforms we support.
The definition of os_get_invalid_handle in platform_internal.h did not match the declaration in platform_api_extension.h.
Also add a script that converts instruction pointers to function indexes (or function names). #3758
Aider is AI pair programming in your terminal: https://aider.chat
…ons/actions/upload-artifact-4.4.0 build(deps): bump actions/upload-artifact from 3.1.0 to 4.4.0
…ons/ossf/scorecard-action-2.4.0 build(deps): bump ossf/scorecard-action from 2.3.1 to 2.4.0
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.9 to 3.26.11. - [Release notes](https://github.com/github/codeql-action/releases) - [Commits](github/codeql-action@v3.26.9...v3.26.11) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Currently, the open-source builds of wamrc set WASM_ENABLE_DUMP_CALL_STACK, which causes these two fields to be emitted. They are required by aot_emit_exception.c. Internally at Google, we don't enable call stack dumps, so we've been using the attached patch to make sure the fields are emitted anyway.
The execute_post_instantiate_functions may be triggered by wasm_cluster_spawn_exec_env, in which the exec_env_tls can be NULL and cause the assertion invalid. p.s. #3839.
Set shared_heap_list to NULL with lock, and destroy shared_heap_list_lock. Signed-off-by: wenlingyun1 <[email protected]>
The default iwasm building in Windows MSVC enables libc-uvwasi because libc-wasi isn't supported at the beginning. Since libc-wasi had been refactored and is supported in Windows msys2 building, and libc-wasi supports more functionalities(e.g. sockets) than libc-uvwasi, this PR fixes some issues to enable libc-wasi in windows MSVC buidlings.
They had been moved to wamr-app-framework repo: https://github.com/bytecodealliance/wamr-app-framework/tree/main/samples
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.4.0 to 4.4.3. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v4.4.0...v4.4.3) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.11 to 3.26.12. - [Release notes](https://github.com/github/codeql-action/releases) - [Commits](github/codeql-action@v3.26.11...v3.26.12) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Implement the shared heap feature for interpreter, aot and llvm jit.
Add below runtime APIs:
```C
wasm_shared_heap_t
wasm_runtime_create_shared_heap(SharedHeapInitArgs *init_args);
bool
wasm_runtime_attach_shared_heap(wasm_module_inst_t module_inst,
wasm_shared_heap_t shared_heap);
void
wasm_runtime_detach_shared_heap(wasm_module_inst_t module_inst);
uint64_t
wasm_runtime_shared_heap_malloc(wasm_module_inst_t module_inst, uint64_t size,
void **p_native_addr);
void
wasm_runtime_shared_heap_free(wasm_module_inst_t module_inst, uint64_t ptr);
```
And allow wasm app to call API shared_heap_malloc and shared_heap_free:
```C
void *shared_heap_malloc(uint32_t size);
void shared_heap_free(void *ptr);
```
- Clear some compile warnings - Fix some typos - Fix llvm LICENSE link error - Remove unused aot file and binarydump bin - Add checks when loading AOT exports
- For Windows, llvm libs need to cache more directories, so use a multi-line environment variable for paths - Remove conditionally build directories `win32build`, just use `build` for all platform - Add Windows wamrc and iwasm(disable lib pthread semaphore and fast jit for now) build in release CI
Add runtime API wasm_runtime_set_native_stack_boundary. p.s. #3816
Sort the module instance's export functions with the function name, and use binary search to lookup the wasm/aot function.
… and Exception Handling (#3866) As suggested in #3829, in release CI, we add zip/tar.gz artifacts named iwasm-gc-eh-{version}-{platform} for `iwasm` which supports features garbage collection and exception handling(classic interpreter only). Also, add a command line option to control GC heap size for `iwasm` on the Windows platform.
Previously, the loader would check the name of a new export against all existing exports, leading to a quadratic running time. This change makes the loader parse the entire export section. The exports are then sorted by name, then adjacent exports are checked for uniqueness.
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.12 to 3.26.13. - [Release notes](https://github.com/github/codeql-action/releases) - [Commits](github/codeql-action@v3.26.12...v3.26.13) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* exclude fuzz test for scoreboard scan * ci ignore osv-scanner.toml file name inconsistency
…lds (#3872) Co-authored-by: Deniz Sokmen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.