diff --git a/configs/contributing.sidebar.json b/configs/contributing.sidebar.json index 1855f45d..eded96a4 100644 --- a/configs/contributing.sidebar.json +++ b/configs/contributing.sidebar.json @@ -26,6 +26,10 @@ { "title": "Coding Conventions", "path": "/docs/contributing/coding-conventions" + }, + { + "title": "Adding to the App Catalog", + "path": "/docs/contributing/adding-to-the-app-catalog" } ] } diff --git a/content/docs/concepts/compatibility.mdx b/content/docs/concepts/compatibility.mdx index 2c2770d5..db660940 100644 --- a/content/docs/concepts/compatibility.mdx +++ b/content/docs/concepts/compatibility.mdx @@ -6,11 +6,40 @@ description: | --- -This page contains extracts from [an article published to **USENIX +Parts of this document contains extracts from [an article published to **USENIX ;login**](https://www.usenix.org/publications/loginonline/unikraft-and-coming-age-unikernels) about Unikraft. +One challenge with Unikraft is the potential porting effort required for new applications. +To address this, Unikraft includes a **binary-compatibility layer** which allows the running of Linux binaries (ELFs) on top of Unikraft. + +For such a layer to work, Unikraft has the explicit goal to comply, as much as possible, with Linux's ABI (_Application Binary Interface_), providing a large range of its system call interface. +Concretely, in Unikraft this is handled through its [**system call shim layer**](/docs/internals/syscall-shim) (also called **syscall shim**), which provides Linux-style mappings of system call numbers to actual system call handler functions. + + +Currently, binary-compatibility is available on x86_64, with work ongoing to make this available for AArch64. +Also note that KVM is currently the only supported hypervisor, with QEMU and Firecracker as VMMs (_Virtual Machine Monitors_). + + + +Currently, binaries have to be built as PIE (*Position-Independent Executables*). +This is the default build mode for the majority of Linux distributions, so it shouldn't cause any problems. + + + +Note that, because Linux binaries are included, constructing new Linux binaries requires a Linux or Linux-compatible development environement (such as WSL - _Windows Subsystem for Linux_). +This is only the case for building binaries -- prebuilt binaries can be used and the ELF loader app itself can be built on multiple platforms (Linux, Windows, macOS). + + +Binary-compatible apps are located in the Unikraft [application `catalog` repository](https://github.com/unikraft/catalog). +Follow the guides below to know how to use and develop for the application catalog: + +- [Using the Application Catalog](/guides/using-the-app-catalog) +- [Behind the Scenes with the Application Catalog](/guides/catalog-behind-the-scenes) +- [Using Firecracker in the Application Catalog](/guides/catalog-using-firecracker) +- [Adding Applications to the Catalog](/docs/contributing/adding-to-the-app-catalog) + While Linux has over 350 system calls, previous [studies](https://dl.acm.org/doi/10.1145/2901318.2901341) have shown through static analysis that only a subset (224) are needed to run a Ubuntu diff --git a/content/docs/contributing/adding-to-the-app-catalog.mdx b/content/docs/contributing/adding-to-the-app-catalog.mdx new file mode 100644 index 00000000..156ad4f6 --- /dev/null +++ b/content/docs/contributing/adding-to-the-app-catalog.mdx @@ -0,0 +1,1253 @@ +--- +title: Adding Applications to the Catalog +description: | + This document presents the steps to add application to the catalog. + It is primarily focused on binary-compatible applications. + It is meant to be used by contributors and application developers who want to have a particular application or example featured in the catalog. +--- + +New application should make their way in the [`catalog` repository](https://github.com/unikraft/catalog). +If porting an actual end-user application, that should be part of the `library/` subdirectory, in a directory titled `/` (e.g. `nginx/1.25`, `lua/5.4.4`). +Example applications, generally those demonstrating a given feature of a framework or of a programming language go to the `examples/` directory. + +Adding a new application requires the creation of: + +- [optional] Source code files of the application. + The application may be built from source code files provided in the app directory. + Or the source code files may be scripts (in scripted / interpreted programming languages) that implement the application. +- [optional] Configuration and data files used by the application. +- A `Dockerfile` to generate the filesystem for the application. + The filesystem consists of the application binary executable (`ELF`) or scripts, depending libraries, configuration files, data files. + These files may either be pulled from an existing Docker image, or they may be build / copied from (source code) files provided by the user. +- A `Kraftfile` that details the build and run specification of the application. +- A `README.md` files that documents the steps required to build, run and test the application. + +We demonstrate these steps for three apps: + +- Redis, built as a binary-compatible app +- an asynchronous web server in Rust using Tokio, packaged and run as a binary-compatible app that uses the `base` kernel in the registry +- a Python Flask application, that is packaged on top of a native Python 3.10 environment + +The rough steps for adding a new application to the catalog are: + +1. Build, configure and run the application in a Docker environment. + Potentially create a `Dockerfile`. +1. Determine the minimal set of components required to run the application: executable, libraries, configuration files. +1. Use a `Dockerfile` to build and configure the application, and then extract the minimal set of components in a minimized Docker environment. + Run the application in the minimized Docker environment. +1. Construct the application `Kraftfile` and build, configure and run the application with Unikraft. +1. Contribute the application to the [`catalog` repository](https://github.com/unikraft/catalog). + +## Redis + +Redis is an end-user application, so it goes in the `library/` subdirectory of the [`catalog` repository](https://github.com/unikraft/catalog). +We add the latest version of Redis available as a [DockerHub image](https://hub.docker.com/_/redis) image, namely `7.2.4` at the time of this writing. + +Our first step is to run Redis in a Docker environment. +Afterward we move ro run it with Unikraft. + +Using a Docker environment is a two step process: + +1. Run Redis as it is in the Docker environment. +1. Run Redis in a minimized Docker environment. + +### Run Redis As It Is in Docker + +To Run Redis as it is, use the command: + +```bash +docker run --rm redis:7.2-bookworm +``` + +This will pull the Redis Debian Bookworm image from DockerHub and run it: + +```text +Unable to find image 'redis:7.2-bookworm' locally +7.2-bookworm: Pulling from library/redis +2f44b7a888fa: Already exists +c55535369ffc: Pull complete +3622841bf0aa: Pull complete +91a62ca7377a: Pull complete +fdd219d1f4ab: Pull complete +fdf07fe2fb4c: Pull complete +4f4fb700ef54: Pull complete +fba604e70bfe: Pull complete +Digest: sha256:b5ddcd52d425a8e354696c022f392fe45fca928f68d6289e6bb4a709c3a74668 +Status: Downloaded newer image for redis:7.2-bookworm +1:C 25 Jan 2024 10:47:59.385 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +1:C 25 Jan 2024 10:47:59.385 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +1:C 25 Jan 2024 10:47:59.385 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=1, just started +1:C 25 Jan 2024 10:47:59.385 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf +1:M 25 Jan 2024 10:47:59.385 * monotonic clock: POSIX clock_gettime +1:M 25 Jan 2024 10:47:59.386 * Running mode=standalone, port=6379. +1:M 25 Jan 2024 10:47:59.386 * Server initialized +1:M 25 Jan 2024 10:47:59.386 * Ready to accept connections tcp +``` + +From the message above we derive some information: + +- The `vm.overcommit_memory=1` option should be enabled. + This is Linux kernel configuration for certain use-cases. + Since we only care about a Unikraft run, we ignore it. + +- There should be a configuration file passed as a runtime argument. + Otherwise, it uses a default one. + We'll get to that later. + +- Redis accepts connections on port 6379, so networking support should be enabled. + +For the latter, let's run Redis with networking support from Docker: + +```bash +docker run --rm -p 6379:6379 redis:7.2-bookworm +``` + +The Redis server is now available on port `6379` on `localhost`. + +To test it, use the Redis client, `redis-cli`. +If not available, install it. +On a Debian/Ubuntu system the install command is, as `root` (prefix with `sudo` if required): + +```bash +apt install redis-tools +``` + +Now test the Redis server inside Docker: + +```bash +$ redis-cli -h localhost +``` +``` +localhost:6379> ping +PONG +localhost:6379> set a 1 +OK +localhost:6379> get a +"1" +localhost:6379> +``` + +Everything works OK. + +### Getting Redis Dependencies + +To get Redis dependencies, we have to inspect the Docker environment. +Firstly we inspect the Docker image: + +```bash +docker inspect redis:7.2-bookworm +``` + +We filter out relevant information from the output: + +```json +"Env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "GOSU_VERSION=1.17", + "REDIS_VERSION=7.2.4", + "REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-7.2.4.tar.gz", + "REDIS_DOWNLOAD_SHA=8d104c26a154b29fd67d6568b4f375212212ad41e0c2caa3d66480e78dbd3b59" +], +"Cmd": [ + "redis-server" +], +"ArgsEscaped": true, +"Image": "", +"Volumes": { + "/data": {} +}, +"WorkingDir": "/data", +"Entrypoint": [ + "docker-entrypoint.sh" +], +``` + +Then we run a Docker instance and start a shell: + +```bash +docker run --rm -p 6379:6379 -it redis:7.2-bookworm /bin/bash +``` + +We get a console / shell of running inside Docker, in the `WorkingDir` option above (`/data`): + +```bash +root@8b346198f54d:/data# +``` + +Our goal is to know the path to the executable, the library dependencies, other required files. +We use the commands below to locate the executable and get the library dependencies: + +```bash +root@8b346198f54d:/data# ldd $(which redis-server) +``` +``` +linux-vdso.so.1 (0x00007fffb7d39000) +libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff32f07d000) +libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007ff32efd3000) +libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007ff32eb51000) +libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff32e970000) +/lib64/ld-linux-x86-64.so.2 (0x00007ff32f6f5000) +``` + +We also start Redis to ensure everything works OK: + +```bash +root@8b346198f54d:/data# /usr/local/bin/redis-server +``` +``` +17:C 25 Jan 2024 11:07:55.418 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +17:C 25 Jan 2024 11:07:55.419 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +17:C 25 Jan 2024 11:07:55.419 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=17, just started +17:C 25 Jan 2024 11:07:55.419 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/local/bin/redis-server /path/to/redis.conf +17:M 25 Jan 2024 11:07:55.420 * monotonic clock: POSIX clock_gettime + _._ + _.-``__ ''-._ + _.-`` `. `_. ''-._ Redis 7.2.4 (00000000/0) 64 bit + .-`` .-```. ```\/ _.,_ ''-._ + ( ' , .-` | `, ) Running in standalone mode + |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 + | `-._ `._ / _.-' | PID: 17 + `-._ `-._ `-./ _.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | https://redis.io + `-._ `-._`-.__.-'_.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | + `-._ `-._`-.__.-'_.-' _.-' + `-._ `-.__.-' _.-' + `-._ _.-' + `-.__.-' + +17:M 25 Jan 2024 11:07:55.436 * Server initialized +17:M 25 Jan 2024 11:07:55.436 * Ready to accept connections tcp +``` + +Redis starts OK. + +A crude way to determine other dependencies is to trace the opened files, with `strace`. +First install `strace` in the container: + +```bash +apt update +apt install -y strace +``` + +Now trace the `openat` system call: + +```bash +root@8b346198f54d:/data# strace -e openat /usr/local/bin/redis-server > /dev/null +``` +``` +openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libssl.so.3", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/dev/urandom", O_RDONLY) = 3 +openat(AT_FDCWD, "/usr/lib/ssl/openssl.cnf", O_RDONLY) = -1 ENOENT (No such file or directory) +openat(AT_FDCWD, "/proc/sys/vm/overcommit_memory", O_RDONLY) = 5 +openat(AT_FDCWD, "/sys/kernel/mm/transparent_hugepage/enabled", O_RDONLY) = 5 +openat(AT_FDCWD, "/sys/devices/system/clocksource/clocksource0/current_clocksource", O_RDONLY) = 5 +openat(AT_FDCWD, "/proc/sys/net/core/somaxconn", O_RDONLY) = 6 +openat(AT_FDCWD, "dump.rdb", O_RDONLY) = 8 +openat(AT_FDCWD, "dump.rdb", O_RDONLY) = 8 +openat(AT_FDCWD, "/proc/self/stat", O_RDONLY) = 8 +``` + +Apart from the library files, Redis requires the `/etc/localtime`, `/dev/unrandom` and some `/sys` and `/proc` files. +The `dump.rdb` file is probably a dump of the previous run. +`/sys` and `/proc` files are usually not mandatory. +`/etc/localtime` and `/dev/urandom` may also not be strictly required. + +So we have a list of dependencies. + +### Constructing the Minimized Docker Environment + +With the information above we construct a minimized Docker environment in a `Dockerfile`: + +```dockerfile +FROM redis:7.2-bookworm as build + +FROM scratch + +# Redis binary +COPY --from=build /usr/local/bin/redis-server /usr/bin/redis-server + +# Redis libraries +COPY --from=build /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6 +COPY --from=build /lib/x86_64-linux-gnu/libssl.so.3 /lib/x86_64-linux-gnu/libssl.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libcrypto.so.3 /lib/x86_64-linux-gnu/libcrypto.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 +COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 +COPY --from=build /etc/ld.so.cache /etc/ld.so.cache +``` + +We then build an image from the `Dockerfile`: + +```bash +$ docker build --tag minimal-redis . +``` +``` +[+] Building 1.3s (12/12) FINISHED docker:default + => [internal] load .dockerignore 0.3s + => => transferring context: 2B 0.0s + => [internal] load build definition from Dockerfile 0.5s + => => transferring dockerfile: 689B 0.0s + => [internal] load metadata for docker.io/library/redis:7.2-bookworm 0.0s + => [build 1/1] FROM docker.io/library/redis:7.2-bookworm 0.0s + => CACHED [stage-1 1/7] COPY --from=build /usr/local/bin/redis-server /usr/bin/redis-server 0.0s + => CACHED [stage-1 2/7] COPY --from=build /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6 0.0s + => CACHED [stage-1 3/7] COPY --from=build /lib/x86_64-linux-gnu/libssl.so.3 /lib/x86_64-linux-gnu/libssl.so.3 0.0s + => CACHED [stage-1 4/7] COPY --from=build /lib/x86_64-linux-gnu/libcrypto.so.3 /lib/x86_64-linux-gnu/libcrypto.so.3 0.0s + => CACHED [stage-1 5/7] COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 0.0s => CACHED [stage-1 6/7] COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 0.0s + => CACHED [stage-1 7/7] COPY --from=build /etc/ld.so.cache /etc/ld.so.cache 0.0s + => exporting to image 0.1s + => => exporting layers 0.0s + => => writing image sha256:9e95efccc19fc473a6718741ad5e70398a345361fef2f03187b8fe37a2573bab 0.0s + => => naming to docker.io/library/minimal-redis +``` + +We verify the creation of the image: + +```bash +$ docker image ls minimal-redis +``` +``` +REPOSITORY TAG IMAGE ID CREATED SIZE +minimal-redis latest 4d857719dd2c About a minute ago 24.3MB +``` + +And now we can start Redis inside the minimal image: + +```bash +$ docker run --rm -p 6379:6379 minimal-redis /usr/bin/redis-server +``` +``` +1:C 25 Jan 2024 11:28:55.083 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. +1:C 25 Jan 2024 11:28:55.083 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +1:C 25 Jan 2024 11:28:55.083 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=1, just started +1:C 25 Jan 2024 11:28:55.083 # Warning: no config file specified, using the default config. In order to specify a config file use /usr/bin/redis-server /path/to/redis.conf +1:M 25 Jan 2024 11:28:55.083 * monotonic clock: POSIX clock_gettime +1:M 25 Jan 2024 11:28:55.084 * Running mode=standalone, port=6379. +1:M 25 Jan 2024 11:28:55.084 * Server initialized +1:M 25 Jan 2024 11:28:55.084 * Ready to accept connections tcp +``` + +It started, we also check it works correctly via `redis-cli`: + +```bash +$ redis-cli -h localhost +``` +``` +localhost:6379> ping +PONG +localhost:6379> set a 1 +OK +localhost:6379> get a +"1" +localhost:6379> +``` + +Everything is OK. +We created a minimized Docker image for Redis inside a `Dockerfile`. + +### Setting Redis with Unikraft + +With the `Dockerfile` now available, we require a `Kraftfile` to run Redis with Unikraft. +Since we are adding a new application, we will create an embedded initrd configuration. +For that, we copy-paste [the `Kraftfile` from Node](https://github.com/unikraft/catalog/blob/main/library/node/18/Kraftfile) and update the `name` and `cmd` configuration. +The `Kraftfile` will have the following contents: + +```yaml +spec: v0.6 + +name: redis + +rootfs: ./Dockerfile + +cmd: ["/usr/bin/redis-server"] + +[...] +``` + +Next we build the Unikraft kernel image: + +```bash +kraft build --no-cache --no-update --log-type basic --log-level debug --plat qemu --arch x86_64 +``` + +Next we run the image: + +```bash +kraft run --log-type basic --log-level debug -p 6379:6379 +``` +``` + D kraftkit 0.7.3 + D using platform=qemu + D cannot run because: no arguments supplied runner=linuxu + D cannot run because: no arguments supplied runner=kernel + D using runner=kraftfile-unikraft + D qemu-system-x86_64 -version + D qemu-system-x86_64 -accel help + D qemu-system-x86_64 -append /usr/bin/redis-server -cpu host,+x2apic,-pmu -daemonize -device virtio-net-pci,mac=02:b0:b0:ab:80:01,netdev=hostnet0 -device pvpanic -device sga -display none -enable-kvm -kernel /home/razvand/unikraft/catal +og/library/redis/7.2/.unikraft/build/redis_qemu-x86_64 -machine pc,accel=kvm -m size=64M -monitor unix:/home/razvand/.local/share/kraftkit/runtime/6a798339-4157-4708-8030-8ec9c40ec390/qemu_mon.sock,server,nowait -name 6a798339-4157-4708-80 +30-8ec9c40ec390 -netdev user,id=hostnet0,hostfwd=tcp::6379-:6379 -nographic -no-reboot -S -parallel none -pidfile /home/razvand/.local/share/kraftkit/runtime/6a798339-4157-4708-8030-8ec9c40ec390/machine.pid -qmp unix:/home/razvand/.local/s +hare/kraftkit/runtime/6a798339-4157-4708-8030-8ec9c40ec390/qemu_control.sock,server,nowait -qmp unix:/home/razvand/.local/share/kraftkit/runtime/6a798339-4157-4708-8030-8ec9c40ec390/qemu_events.sock,server,nowait -rtc base=utc -serial file +:/home/razvand/.local/share/kraftkit/runtime/6a798339-4157-4708-8030-8ec9c40ec390/machine.log -smp cpus=1,threads=1,sockets=1 -vga none + E could not start qemu instance: dial unix /home/razvand/.local/share/kraftkit/runtime/6a798339-4157-4708-8030-8ec9c40ec390/qemu_control.sock: connect: no such file or directory +``` + +The error message lets us know there is a problem with running the application, so we check the debug file: + +```bash +$ cat /home/razvand/.local/share/kraftkit/runtime/6a798339-4157-4708-8030-8ec9c40ec390/machine.log +``` +``` +[...] +en1: Added +en1: Interface is up +Powered by Unikraft Telesto (0.16.1~644821db) +[ 0.138996] ERR: [appelfloader] redis-server: Failed to initialize ELF parser +[ 0.140238] ERR: [appelfloader] : Resource exhaustion (10) +``` + +The message `Resource exhaustion` lets us know that maybe we not running with enough memory, so we go for `256M` of memory: + +```bash +kraft run --log-type basic --log-level debug -M 256M -p 6379:6379 +``` + +This indeed is the issue and the output message confirms the starting of the server: + +``` + D kraftkit 0.7.3 + D using platform=qemu + D cannot run because: no arguments supplied runner=linuxu + D cannot run because: no arguments supplied runner=kernel + D using runner=kraftfile-unikraft + D qemu-system-x86_64 -version + D qemu-system-x86_64 -accel help + D qemu-system-x86_64 -append /usr/bin/redis-server -cpu host,+x2apic,-pmu -daemonize -device virtio-net-pci,mac=02:b0:b0:01:cd:01,netdev=hostnet0 -device pvpanic -device sga -display none -enable-kvm -kernel /home/razvand/unikraft/catalog/library/redis/7.2/.unikraft/build/redis_qemu-x86_64 -machine pc,accel=kvm -m size=244M -monitor unix:/home/razvand/.local/share/kraftkit/runtime/a97b85de-91b2-4745-8104-625e870aea65/qemu_mon.sock,server,nowait -name a97b85de-91b2-4745-8104-625e870aea65 -netdev user,id=hostnet0,hostfwd=tcp::6379-:6379 -nographic -no-reboot -S -parallel none -pidfile /home/razvand/.local/share/kraftkit/runtime/a97b85de-91b2-4745-8104-625e870aea65/machine.pid -qmp unix:/home/razvand/.local/share/kraftkit/runtime/a97b85de-91b2-4745-8104-625e870aea65/qemu_control.sock,server,nowait -qmp unix:/home/razvand/.local/share/kraftkit/runtime/a97b85de-91b2-4745-8104-625e870aea65/qemu_events.sock,server,nowait -rtc base=utc -serial file:/home/razvand/.local/share/kraftkit/runtime/a97b85de-91b2-4745-8104-625e870aea65/machine.log -smp cpus=1,threads=1,sockets=1 -vga none +en1: Interface is up +Powered by Unikraft Telesto (0.16.1~644821db) +1:C 25 Jan 2024 12:06:06.081 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo +1:C 25 Jan 2024 12:06:06.082 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=1, just started +1:C 25 Jan 2024 12:06:06.084 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf +[ 0.187817] ERR: [libposix_process] Ignore updating resource 7: cur = 10032, max = 10032 +1:M 25 Jan 2024 12:06:06.089 * Increased maximum number of open files to 10032 (it was originally set to 1024). +1:M 25 Jan 2024 12:06:06.091 * monotonic clock: POSIX clock_gettime + _._ + _.-``__ ''-._ + _.-`` `. `_. ''-._ Redis 7.2.4 (00000000/0) 64 bit + .-`` .-```. ```\/ _.,_ ''-._ + ( ' , .-` | `, ) Running in standalone mode + |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 + | `-._ `._ / _.-' | PID: 1 + `-._ `-._ `-./ _.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | https://redis.io + `-._ `-._`-.__.-'_.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | + `-._ `-._`-.__.-'_.-' _.-' + `-._ `-.__.-' _.-' + `-._ _.-' + `-.__.-' + +1:M 25 Jan 2024 12:06:06.111 # Warning: Could not create server TCP listening socket ::*:6379: unable to bind socket, errno: 97 +1:M 25 Jan 2024 12:06:06.114 * Server initialized +1:M 25 Jan 2024 12:06:06.115 * Ready to accept connections tcp +en1: Set IPv4 address 10.0.2.15 mask 255.255.255.0 gw 10.0.2.2 +``` + +However, the warning of being unable to bind the socket is problematic. +Using `redis-cli` lets us know, there is a problem with Redis: + +```bash +$ redis-cli -h localhost +``` +``` +Could not connect to Redis at localhost:6379: Connection refused +not connected> +``` + +The error is due to a likely absence of full IPv6 support. +We require a configuration file that binds directly to IPv4. + +### Configure Redis for Unikraft + +To fix the above issue we use the [existing Redis 7.0 configuration for Unikraft](https://github.com/unikraft/catalog/blob/main/library/redis/7.0/rootfs/redis.conf). +This is for a native (i.e. non-bincompat) configuration, but it doesn't matter. + +This requires an update to the `Dockerfile`, that needs to include the configuration file. +The new `Dockerfile` is: + +```dockerfile +FROM redis:7.2-bookworm as build + +FROM scratch + +# Redis binary +COPY --from=build /usr/local/bin/redis-server /usr/bin/redis-server + +# Redis libraries +COPY --from=build /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6 +COPY --from=build /lib/x86_64-linux-gnu/libssl.so.3 /lib/x86_64-linux-gnu/libssl.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libcrypto.so.3 /lib/x86_64-linux-gnu/libcrypto.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 +COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 +COPY --from=build /etc/ld.so.cache /etc/ld.so.cache + +# Redis configuration +COPY ./redis.conf /etc/redis.conf +``` + +We also update the `cmd` option in the `Kraftfile`: + +```yaml +cmd: ["/usr/bin/redis-server", "/etc/redis.conf"] +``` + +We rebuild the image: + +```bash +rm -fr .config* .unikraft* +kraft build --no-cache --no-update --log-type basic --log-level debug --plat qemu --arch x86_64 +``` + +And we rerun it: + +```bash +kraft rm --all +kraft run --log-type basic --log-level debug -M 256M -p 6379:6379 +``` + +Everything seems to be OK, according to the output: + +``` + _._ + _.-``__ ''-._ + _.-`` `. `_. ''-._ Redis 7.2.4 (00000000/0) 64 bit + .-`` .-```. ```\/ _.,_ ''-._ + ( ' , .-` | `, ) Running in standalone mode + |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 + | `-._ `._ / _.-' | PID: 1 + `-._ `-._ `-./ _.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | https://redis.io + `-._ `-._`-.__.-'_.-' _.-' + |`-._`-._ `-.__.-' _.-'_.-'| + | `-._`-._ _.-'_.-' | + `-._ `-._`-.__.-'_.-' _.-' + `-._ `-.__.-' _.-' + `-._ _.-' + `-.__.-' + +1:M 25 Jan 2024 12:15:36.099 * Server initialized +1:M 25 Jan 2024 12:15:36.100 * Ready to accept connections tcp +en1: Set IPv4 address 10.0.2.15 mask 255.255.255.0 gw 10.0.2.2 +``` + +We use `redis-cli` to query the server: + +```bash +redis-cli -h localhost +``` + +This currently doesn't work because of an issue with Unikraft. +But everything we did on the application side is OK. + +### Contributing to the Application Catalog + +With the Redis application now set, we can make a contribution to the [`catalog` repository](https://github.com/unikraft/catalog). +For that three additional steps need to be taken: + +1. Create a `README.md` file. +1. Create a GitHub workflow for the application, following the [existing workflow files](https://github.com/unikraft/catalog/tree/main/.github/workflows). +1. Update the badge listing in the [top-level `README.md` file](https://github.com/unikraft/catalog/blob/main/README.md). + +Then create a commit with the `Dockerfile`, `Kraftfile`, `README.md`, the new GitHub workflow file and updates to the [top-level `README.md` file](https://github.com/unikraft/catalog/blob/main/README.md). +And submit a pull request. + +## Rust Tokio Web Server + +A Rust web server is not an end-user application, so we consider it an example, and it goes in the `examples/` subdirectory of the [`catalog` repository](https://github.com/unikraft/catalog). +It will make use of the [`base` image](https://github.com/unikraft/catalog/tree/main/library/base) in the Unikraft registry. + +For this we follow the steps: + +We first create the required source code and build files for a Tokio web server. +That is, the items required for a native build and run. + +The source code file is `src/main.rs` as below: + +```rust +use std::net::SocketAddr; +use tokio::net::TcpListener; +use tokio::io::{AsyncReadExt, AsyncWriteExt}; + +#[tokio::main] +async fn main() -> Result<(), Box> { + let addr = SocketAddr::from(([0, 0, 0, 0], 8080)); + let listener = TcpListener::bind(&addr).await?; + + println!("Listening on: http://{}", addr); + + loop { + let (mut stream, _) = listener.accept().await?; + + tokio::spawn(async move { + loop { + let mut buffer = [0; 1024]; + let _ = stream.read(&mut buffer).await; + + let contents = "Hello, world!\r\n"; + let content_length = contents.len(); + let response = format!("HTTP/1.1 200 OK\r\nContent-Length: {content_length}\r\n\r\n{contents}"); + let _ = stream.write_all(response.as_bytes()).await; + } + }); + } +} +``` + +The build file is `Cargo.toml` as below: + +```toml +[package] +name = "http-tokio" +version = "0.1.0" +edition = "2021" + +[dependencies] +tokio = {version = "1", features = ["rt-multi-thread", "net", "time", "macros", "io-util"] } +``` + +### Build and Run in a Docker Environment + +Both for the eventual Unikraft run, but also to have an environment with everything set, it's easier to build and run the Rust Tokio web server in a Docker environment. +We start from the [Rust Docker image on DockerHub](https://hub.docker.com/_/rust). +We use version `1.73.0-bookworm`. + +For this, we create the following `Dockerfile`: + +```dockerfile +FROM rust:1.73.0-bookworm AS build + +WORKDIR /src + +COPY ./src /src/src +COPY ./Cargo.toml /src/Cargo.toml + +RUN cargo build +``` + +We then build an image from the `Dockerfile`: + +```bash +$ docker build -t http-tokio . +``` +``` +[+] Building 36.9s (10/10) FINISHED docker:default + => [internal] load .dockerignore 0.6s + => => transferring context: 2B 0.0s + => [internal] load build definition from Dockerfile 0.9s + => => transferring dockerfile: 158B 0.2s + => [internal] load metadata for docker.io/library/rust:1.73.0-bookworm 2.8s + => [1/5] FROM docker.io/library/rust:1.73.0-bookworm@sha256:25fa7a9aa4dadf6a466373822009b5361685604dbe151b030182301f1a3c2f58 0.0s + => [internal] load build context 0.3s + => => transferring context: 1.16kB 0.0s + => CACHED [2/5] WORKDIR /src 0.0s + => [3/5] COPY ./src /src/src 1.6s + => [4/5] COPY ./Cargo.toml /src/Cargo.toml 1.3s + => [5/5] RUN cargo build 24.0s + => exporting to image 4.2s + => => exporting layers 4.0s + => => writing image sha256:63d718eb15b0a8c2f07c3daa6686542555ae41738872cdc6873b407101d7f9ad 0.1s + => => naming to docker.io/library/http-tokio +``` + +We verify the creation of the image: + +```bash +$ docker image ls http-tokio +``` +``` +REPOSITORY TAG IMAGE ID CREATED SIZE +http-tokio latest 63d718eb15b0 About a minute ago 1.63GB +``` + +It's a pretty large image. +The Rust environment and the Tokio dependencies occupy quite a bit of space. + +And now we can start the Tokio web server from the Docker image: + +```bash +$ docker run --rm -p 8080:8080 http-tokio /src/target/debug/http-tokio +``` +``` +Listening on: http://0.0.0.0:8080 +``` + +The server starts and waits for connections on TCP port `8080`. + +To test it, we query the server: + +```bash +$ curl localhost:8080 +``` +``` +Hello, world! +``` + +A `Hello, world!` message is printed, so everything works OK. + +### Getting Dependencies + +To get the dependencies, we have to inspect the Docker environment. +We run a Docker instance and start a shell: + +```bash +docker run --rm -p 8080:8080 -it http-tokio /bin/bash +``` + +We get a console / shell of running inside Docker: + +```bash +root@8b346198f54d:/data# +``` + +Our goal is to know the path to the executable, the library dependencies, other required files. +We use the commands below to locate the executable and get the library dependencies: + +```bash +root@66e910817179:/src# ls -F --color=auto target/debug/ +``` +``` +build/ deps/ examples/ http-tokio* http-tokio.d incremental/ +``` + +And then `ldd` to find the dynamically linked shared objects which the application depends: + +```bash +root@66e910817179:/src# ldd target/debug/http-tokio +``` +``` +linux-vdso.so.1 (0x00007fffa8331000) +libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f35fd805000) +libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f35fd726000) +libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f35fd545000) +/lib64/ld-linux-x86-64.so.2 (0x00007f35fd97d000) +``` + +We also start the server to ensure everything works OK: + +```bash +$ docker run --rm -p 8080:8080 -it http-tokio /bin/bash +``` +``` +root@66e910817179:/src# +``` + +It starts OK. + +A crude way to determine other dependencies is to trace the opened files, with `strace`. +First install `strace` in the container: + +```bash +apt update +apt install -y strace +``` + +Now trace the `openat` system call: + +```bash +root@8fbdd8d1010d:/src# strace -e openat ./target/debug/http-tokio +``` +``` +openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/proc/self/cgroup", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/proc/self/mountinfo", O_RDONLY|O_CLOEXEC) = 3 +openat(AT_FDCWD, "/sys/fs/cgroup/cpu.max", O_RDONLY|O_CLOEXEC) = 3 +Listening on: http://0.0.0.0:8080 +``` + +Apart from the library files, the server requires some `/proc` files, that are typically not required. +So we have a list of dependencies comprised of the shared libraries. + +### Constructing the Minimized Docker Environment + +With the information above we construct a minimized Docker environment in the `Dockerfile`: + +```dockerfile +FROM rust:1.73.0-bookworm AS build + +WORKDIR /src + +COPY ./src /src/src +COPY ./Cargo.toml /src/Cargo.toml + +RUN cargo build + +FROM scratch + +# Server binary +COPY --from=build /src/target/debug/http-tokio /server + +# System libraries +COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 +COPY --from=build /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6 +COPY --from=build /lib/x86_64-linux-gnu/libgcc_s.so.1 /lib/x86_64-linux-gnu/libgcc_s.so.1 +COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 +``` + +We then build an image from the `Dockerfile`: + +```bash +$ docker build --tag minimal-http-tokio . +``` +``` +[+] Building 1.3s (12/12) FINISHED docker:default + => [internal] load .dockerignore 0.3s + => => transferring context: 2B 0.0s + => [internal] load build definition from Dockerfile 0.5s + => => transferring dockerfile: 689B 0.0s + => [internal] load metadata for docker.io/library/redis:7.2-bookworm 0.0s + => [build 1/1] FROM docker.io/library/redis:7.2-bookworm 0.0s + => CACHED [stage-1 1/7] COPY --from=build /usr/local/bin/redis-server /usr/bin/redis-server 0.0s + => CACHED [stage-1 2/7] COPY --from=build /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6 0.0s + => CACHED [stage-1 3/7] COPY --from=build /lib/x86_64-linux-gnu/libssl.so.3 /lib/x86_64-linux-gnu/libssl.so.3 0.0s + => CACHED [stage-1 4/7] COPY --from=build /lib/x86_64-linux-gnu/libcrypto.so.3 /lib/x86_64-linux-gnu/libcrypto.so.3 0.0s + => CACHED [stage-1 5/7] COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 0.0s => CACHED [stage-1 6/7] COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 0.0s + => CACHED [stage-1 7/7] COPY --from=build /etc/ld.so.cache /etc/ld.so.cache 0.0s + => exporting to image 0.1s + => => exporting layers 0.0s + => => writing image sha256:9e95efccc19fc473a6718741ad5e70398a345361fef2f03187b8fe37a2573bab 0.0s + => => naming to docker.io/library/minimal-redis +``` + +We verify the creation of the image: + +```bash +$ docker build -t minimal-http-tokio . +``` +``` +[+] Building 19.8s (15/15) FINISHED docker:default + => [internal] load .dockerignore 0.6s + => => transferring context: 2B 0.0s + => [internal] load build definition from Dockerfile 0.3s + => => transferring dockerfile: 594B 0.0s + => [internal] load metadata for docker.io/library/rust:1.73.0-bookworm 1.5s + => [build 1/5] FROM docker.io/library/rust:1.73.0-bookworm@sha256:25fa7a9aa4dadf6a466373822009b5361685604dbe151b030182301f1a3c2f58 0.0s + => [internal] load build context 0.2s + => => transferring context: 1.16kB 0.0s + => CACHED [build 2/5] WORKDIR /src 0.0s + => CACHED [build 3/5] COPY ./src /src/src 0.0s + => CACHED [build 4/5] COPY ./Cargo.toml /src/Cargo.toml 0.0s + => CACHED [build 5/5] RUN cargo build 0.0s + => [stage-1 1/5] COPY --from=build /src/target/debug/http-tokio /server 3.0s + => [stage-1 2/5] COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 2.3s + => [stage-1 3/5] COPY --from=build /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/libm.so.6 2.2s + => [stage-1 4/5] COPY --from=build /lib/x86_64-linux-gnu/libgcc_s.so.1 /lib/x86_64-linux-gnu/libgcc_s.so.1 2.4s + => [stage-1 5/5] COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 2.3s + => exporting to image 1.6s + => => exporting layers 1.5s + => => writing image sha256:33190a2c1ddeee8b0a4cef83f691717e4ae85af4834a8a7518ba0948b27de12e 0.1s + => => naming to docker.io/library/minimal-http-tokio +``` + +And now we can start the server inside the minimal image: + +```bash +$ docker run --rm -p 8080:8080 minimal-http-tokio /server +``` +``` +Listening on: http://0.0.0.0:8080 +``` + +It started, we also check it works correctly by querying it: + +```bash +$ curl localhost:8080 +``` +``` +Hello, world! +``` + +Everything is OK. +We created a minimized Tokio Rust image inside a `Dockerfile`. + +### Using Unikraft + +With the `Dockerfile` now available, we require a `Kraftfile` to run the Rust Tokio server with Unikraft. +Since we are adding an example, we will use the [`base` image](https://github.com/unikraft/catalog/tree/main/library/base) part of the Unikraft registry. +The `Kraftfile` will have the following contents: + +```yaml +spec: v0.6 + +runtime: base:latest + +rootfs: ./Dockerfile + +cmd: ["/server"] +``` + +Next we use `kraft run` to pull the `base` image, pack the Rust Tokio filesystem application and run it with `base`: + +```bash +kraft run --log-type basic --log-level debug -p 8080:8080 +``` + +We get the output: + +``` + D kraftkit 0.7.3 + D using platform=qemu + D cannot run because: no arguments supplied runner=linuxu + D cannot run because: no arguments supplied runner=kernel + D cannot run because: cannot run project build without unikraft runner=kraftfile-unikraft + D using runner=kraftfile-runtime + D querying oci catalog name=base plat=qemu update=false version=latest + D querying manifest catalog name=base plat=qemu update=false version=latest + i pulling unikraft.org/base:latest +[...] + D qemu-system-x86_64 -append vfs.fstab=[ "initrd0:/:extract:::" ] -- /server -cpu host,+x2apic,-pmu -daemonize -device virtio-net-pci,mac=02:b0:b0:a5:d6:01,netdev=hostnet0 -device pvpanic -device sga -display none -enable-kvm -initrd /h +ome/razvand/unikraft/catalog/examples/tmp/http-tokio/.unikraft/build/initramfs-x86_64.cpio -kernel /tmp/kraft-run-1911975420/unikraft/bin/kernel -machine pc,accel=kvm -m size=64M -monitor unix:/home/razvand/.local/share/kraftkit/runtime/ef +6a273d-f066-4674-8d06-b85a10068f13/qemu_mon.sock,server,nowait -name ef6a273d-f066-4674-8d06-b85a10068f13 -netdev user,id=hostnet0,hostfwd=tcp::8080-:8080 -nographic -no-reboot -S -parallel none -pidfile /home/razvand/.local/share/kraftkit +/runtime/ef6a273d-f066-4674-8d06-b85a10068f13/machine.pid -qmp unix:/home/razvand/.local/share/kraftkit/runtime/ef6a273d-f066-4674-8d06-b85a10068f13/qemu_control.sock,server,nowait -qmp unix:/home/razvand/.local/share/kraftkit/runtime/ef6a +273d-f066-4674-8d06-b85a10068f13/qemu_events.sock,server,nowait -rtc base=utc -serial file:/home/razvand/.local/share/kraftkit/runtime/ef6a273d-f066-4674-8d06-b85a10068f13/machine.log -smp cpus=1,threads=1,sockets=1 -vga none + E could not start qemu instance: dial unix /home/razvand/.local/share/kraftkit/runtime/ef6a273d-f066-4674-8d06-b85a10068f13/qemu_control.sock: connect: no such file or directory +``` + +The error message lets us know there is a problem with running the application, so we check the debug file: + +```bash +$ cat /home/razvand/.local/share/kraftkit/runtime/ef6a273d-f066-4674-8d06-b85a10068f13/machine.log +``` +``` +[...] +en1: Added +en1: Interface is up +[ 0.107061] ERR: [libukcpio] /./server: Failed to load content: Input/output error (5) +[ 0.108430] CRIT: [libvfscore] Failed to extract cpio archive to /: -3 +[ 0.109524] ERR: [libukboot] Init function at 0x14a230 returned error -5 +``` + +The failure to extract contents can be an issue related to the amount of memory used, so we go for `256M` of memory: + +```bash +kraft run --log-type basic --log-level debug -M 256M -p 8080:8080 +``` + +This, indeed works, with the output: + +``` + D qemu-system-x86_64 -append vfs.fstab=[ "initrd0:/:extract:::" ] -- /server -cpu host,+x2apic,-pmu -daemonize -device virtio-net-pci,mac=02:b0:b0:79:ab:01,netdev=hostnet0 -device pvpanic -device sga -display none -enable-kvm -initrd /home/razvand/unikraft/catalog/examples/tmp/http-tokio/.unikraft/build/initramfs-x86_64.cpio -kernel /tmp/kraft-run-4233433423/unikraft/bin/kernel -machine pc,accel=kvm -m size=244M -monitor unix:/home/razvand/.local/share/kraftkit/runtime/0fb3fe09-4a1b-4545-9e7d-0c38f0da2335/qemu_mon.sock,server,nowait -name 0fb3fe09-4a1b-4545-9e7d-0c38f0da2335 -netdev user,id=hostnet0,hostfwd=tcp::8080-:8080 -nographic -no-reboot -S -parallel none -pidfile /home/razvand/.local/share/kraftkit/runtime/0fb3fe09-4a1b-4545-9e7d-0c38f0da2335/machine.pid -qmp unix:/home/razvand/.local/share/kraftkit/runtime/0fb3fe09-4a1b-4545-9e7d-0c38f0da2335/qemu_control.sock,server,nowait -qmp unix:/home/razvand/.local/share/kraftkit/runtime/0fb3fe09-4a1b-4545-9e7d-0c38f0da2335/qemu_events.sock,server,nowait -rtc base=utc -serial file:/home/razvand/.local/share/kraftkit/runtime/0fb3fe09-4a1b-4545-9e7d-0c38f0da2335/machine.log -smp cpus=1,threads=1,sockets=1 -vga none +en1: Interface is up +Powered by Unikraft Telesto (0.16.1~b1fa7c5) +Listening on: http://0.0.0.0:8080 +en1: Set IPv4 address 10.0.2.15 mask 255.255.255.0 gw 10.0.2.2 +``` + +We also check it works correctly by querying it: + +```bash +$ curl localhost:8080 +``` +``` +Hello, world! +``` + +Everything is OK. +We create the setup for running a minimized Rust Tokio image with Unikraft. + +### Contributing to the Application Catalog + +With the Rust Tokio example now set, we can make a contribution to the [`catalog` repository](https://github.com/unikraft/catalog). +For that three additional steps need to be taken: + +1. Create a `README.md` file. +1. Update the examples listing in the [top-level `README.md` file](https://github.com/unikraft/catalog/blob/main/README.md). + +Then create a commit with the `Dockerfile`, `Kraftfile`, `README.md`, and updates to the [top-level `README.md` file](https://github.com/unikraft/catalog/blob/main/README.md). +And submit a pull request. + +## Python Flask + +A Python Flask program is not an end-user application, so we consider it an example, and it goes in the `examples/` subdirectory of the [`catalog` repository](https://github.com/unikraft/catalog). +It will make use of the [`python` image](https://github.com/unikraft/catalog/tree/main/library/python/3.10) in the Unikraft registry. + +We first create the required source code and build files for a simple Python Flask web server. +That is, the items required for a native build and run. + +The source code file is `server.py` as below: + +```python +from flask import Flask +app = Flask(__name__) + +@app.route('/') +def hello(): + return "Hello, World!\n" + +if __name__ == '__main__': + app.run(host='0.0.0.0', port=8080) +``` + +We also define a `requirements.txt` file: + +```requirements +flask +``` + +### Build and Run in a Docker Environment + +Both for the eventual Unikraft run, but also to have an environment with everything set, it's easier to build and run the Python Flask server in a Docker environment. +We start from the [Python Docker image on DockerHub](https://hub.docker.com/_/python). +We use version `3.10.11` since it's the one used by the [Python `library/` entry in the `catalog` repository](https://github.com/unikraft/catalog/tree/main/library/python/3.10). + +For this, we create the following `Dockerfile`: + +```dockerfile +FROM python:3.10.11 AS build + +WORKDIR /src + +COPY ./server.py /src/server.py +COPY ./requirements.txt /src/requirements.txt + +RUN pip install -r requirements.txt +``` + +We then build an image from the `Dockerfile`: + +```bash +$ docker build -t http-python-flask . +``` +``` +[+] Building 20.7s (10/10) FINISHED docker:default + => [internal] load .dockerignore 0.4s + => => transferring context: 2B 0.0s + => [internal] load build definition from Dockerfile 0.6s + => => transferring dockerfile: 198B 0.0s + => [internal] load metadata for docker.io/library/python:3.10.11 2.0s + => CACHED [1/5] FROM docker.io/library/python:3.10.11@sha256:f5ef86211c0ef0db2e3059787088221602cad7e11b238246e406aa7bbd7edc41 0.0s + => [internal] load build context 0.4s + => => transferring context: 66B 0.0s + => [2/5] WORKDIR /src 2.5s + => [3/5] COPY ./server.py /src/server.py 1.8s + => [4/5] COPY ./requirements.txt /src/requirements.txt 1.7s + => [5/5] RUN pip install -r requirements.txt 9.0s + => exporting to image 1.8s + => => exporting layers 1.7s + => => writing image sha256:963165fda5d969860361401757a53e2544a597b84ace1ab2142aaf0e7247fb88 0.1s + => => naming to docker.io/library/http-python-flask +``` + +We verify the creation of the image: + +```bash +$ docker image ls http-python-flask +``` +``` +REPOSITORY TAG IMAGE ID CREATED SIZE +http-python-flask latest 963165fda5d9 43 seconds ago 923MB +``` + +It's a pretty large image. +The Python environment and the Flask dependencies occupy quite a bit of space. + +And now we can start the Python Flask web server from the Docker image: + +```bash +$ docker run --rm -p 8080:8080 http-python-flask /usr/local/bin/python3.10 /src/server.py + * Serving Flask app 'server' + * Debug mode: off +WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. + * Running on all addresses (0.0.0.0) + * Running on http://127.0.0.1:8080 + * Running on http://172.17.0.5:8080 +Press CTRL+C to quit +``` + +The server starts and waits for connections on TCP port `8080`. + +To test it, we query the server: + +```bash +$ curl localhost:8080 +``` +``` +Hello, World! +``` + +A `Hello, World!` message is printed, so everything works OK. + +### Constructing the Minimized Docker Environment + +With the information above we construct a minimized Docker environment in the `Dockerfile`: + +```dockerfile +FROM python:3.10.11 AS base + +WORKDIR /app + +COPY requirements.txt /app + +RUN pip3 install -r requirements.txt --no-cache-dir + +FROM scratch + +COPY --from=base /usr/local/lib/python3.10 /usr/local/lib/python3.10 +COPY ./server.py /app/server.py +``` + +We then build an image from the `Dockerfile`: + +```bash +$ docker build -t minimal-http-python-flask . +``` +``` +[+] Building 18.1s (11/11) FINISHED docker:default + => [internal] load .dockerignore 0.5s + => => transferring context: 2B 0.0s + => [internal] load build definition from Dockerfile 0.3s + => => transferring dockerfile: 319B 0.0s + => [internal] load metadata for docker.io/library/python:3.10.11 0.8s + => [build 1/4] FROM docker.io/library/python:3.10.11@sha256:f5ef86211c0ef0db2e3059787088221602cad7e11b238246e406aa7bbd7edc41 0.0s + => [internal] load build context 0.2s + => => transferring context: 66B 0.0s + => CACHED [build 2/4] WORKDIR /src 0.0s + => CACHED [build 3/4] COPY ./requirements.txt /src/requirements.txt 0.0s + => CACHED [stage-1 1/2] COPY ./server.py /server.py 0.0s + => [build 4/4] RUN pip install -r requirements.txt 7.0s + => [stage-1 2/2] COPY --from=build /usr/local/lib/python3.10 /usr/local/lib/python3.10 3.4s + => exporting to image 1.4s + => => exporting layers 1.2s + => => writing image sha256:76f8451f95098275585836b03e06a16dd905734097d6a3ff90762e39a480bd8b 0.0s + => => naming to docker.io/library/minimal-http-python-flask 0.1s +``` + +We verify the creation of the image: + +```bash +$ docker image ls minimal-http-python-flask +``` +``` +REPOSITORY TAG IMAGE ID CREATED SIZE +minimal-http-python-flask latest 76f8451f9509 10 seconds ago 51MB +``` + +This image doesn't possess a Python interpreter. +We rely on the Unikraft registry image to provide tha. + +### Using Unikraft + +With the `Dockerfile` now available, we require a `Kraftfile` to run the Python Flask server with Unikraft. +Since we are adding an example, we will use the [`python:3.10` image](https://github.com/unikraft/catalog/tree/main/library/python/3.10) part of the Unikraft registry. +The `Kraftfile` will have the following contents: + +```yaml +spec: v0.6 + +runtime: unikraft.org/python:3.10 + +rootfs: ./Dockerfile + +cmd: ["/server.py"] +``` + +Next we use `kraft run` to pull the `python` image, pack the Python Flask filesystem application and run it with `python`: + +```bash +kraft run --log-type basic --log-level debug -p 8080:8080 +``` + +We get the output: + +``` + D kraftkit 0.7.3 + D using platform=qemu + D cannot run because: no arguments supplied runner=linuxu + D cannot run because: no arguments supplied runner=kernel + D cannot run because: cannot run project build without unikraft runner=kraftfile-unikraft + D using runner=kraftfile-runtime + D querying oci catalog name=unikraft.org/python plat=qemu update=false version=3.10 + D querying manifest catalog name=unikraft.org/python plat=qemu update=false version=3.10 + D querying oci catalog name=unikraft.org/python plat=qemu update=true version=3.10 + D querying manifest catalog name=unikraft.org/python plat=qemu update=true version=3.10 + i pulling unikraft.org/python:3.10 +[...] + D qemu-system-x86_64 -append vfs.fstab=[ "initrd0:/:extract:::" ] -- /server.py -cpu host,+x2apic,-pmu -daemonize -device virtio-net-pci,mac=02:b0:b0:ba:2c:01,netdev=hostnet0 -device pvpanic -device sga -display none -enable-kvm -initrd + /home/razvand/unikraft/catalog/examples/tmp/http-python3.12-flask/.unikraft/build/initramfs-x86_64.cpio -kernel /tmp/kraft-run-3997990667/unikraft/bin/kernel -machine pc,accel=kvm -m size=64M -monitor unix:/home/razvand/.local/share/kraft +kit/runtime/4667ae02-d991-4135-af68-ba22698ecd72/qemu_mon.sock,server,nowait -name 4667ae02-d991-4135-af68-ba22698ecd72 -netdev user,id=hostnet0,hostfwd=tcp::8080-:8080 -nographic -no-reboot -S -parallel none -pidfile /home/razvand/.local/ +share/kraftkit/runtime/4667ae02-d991-4135-af68-ba22698ecd72/machine.pid -qmp unix:/home/razvand/.local/share/kraftkit/runtime/4667ae02-d991-4135-af68-ba22698ecd72/qemu_control.sock,server,nowait -qmp unix:/home/razvand/.local/share/kraftki +t/runtime/4667ae02-d991-4135-af68-ba22698ecd72/qemu_events.sock,server,nowait -rtc base=utc -serial file:/home/razvand/.local/share/kraftkit/runtime/4667ae02-d991-4135-af68-ba22698ecd72/machine.log -smp cpus=1,threads=1,sockets=1 -vga none + E could not start qemu instance: dial unix /home/razvand/.local/share/kraftkit/runtime/4667ae02-d991-4135-af68-ba22698ecd72/qemu_control.sock: connect: no such file or directory +``` + +The error message lets us know there is a problem with running the application, so we check the debug file: + +```bash +$ cat /home/razvand/.local/share/kraftkit/runtime/ef6a273d-f066-4674-8d06-b85a10068f13/machine.log +``` +``` +[...] +Booting from ROM... +[ 0.000000] CRIT: [libkvmplat] Assertion failure: mr_prio == 0 || ml_prio == 0 +``` + +The failure to extract contents can be an issue related to the amount of memory used, so we go for `512M` of memory: + +```bash +kraft run --log-type basic --log-level debug -M 512M -p 8080:8080 +``` + +This, indeed works, with the output: + +``` + D qemu-system-x86_64 -append vfs.fstab=[ "initrd0:/:extract:::" ] -- /server.py -cpu host,+x2apic,-pmu -daemonize -device virtio-net-pci,mac=02:b0:b0:7e:03:01,netdev=hostnet0 -device pvpanic -device sga -display none -enable-kvm -initrd /home/razvand/unikraft/catalog/examples/tmp/http-python3.12-flask/.unikraft/build/initramfs-x86_64.cpio -kernel /tmp/kraft-run-3035028343/unikraft/bin/kernel -machine pc,accel=kvm -m size=488M -monitor unix:/home/razvand/.local/share/kraftkit/runtime/355437d0-52d6-443f-9906-f12be299a9cb/qemu_mon.sock,server,nowait -name 355437d0-52d6-443f-9906-f12be299a9cb -netdev user,id=hostnet0,hostfwd=tcp::8080-:8080 -nographic -no-reboot -S -parallel none -pidfile /home/razvand/.local/share/kraftkit/runtime/355437d0-52d6-443f-9906-f12be299a9cb/machine.pid -qmp unix:/home/razvand/.local/share/kraftkit/runtime/355437d0-52d6-443f-9906-f12be299a9cb/qemu_control.sock,server,nowait -qmp unix:/home/razvand/.local/share/kraftkit/runtime/355437d0-52d6-443f-9906-f12be299a9cb/qemu_events.sock,server,nowait -rtc base=utc -serial file:/home/razvand/.local/share/kraftkit/runtime/355437d0-52d6-443f-9906-f12be299a9cb/machine.log -smp cpus=1,threads=1,sockets=1 -vga none +Powered by +o. .o _ _ __ _ +Oo Oo ___ (_) | __ __ __ _ ' _) :_ +oO oO ' _ `| | |/ / _)' _` | |_| _) +oOo oOO| | | | | (| | | (_) | _) :_ + OoOoO ._, ._:_:_,\_._, .__,_:_, \___) + Telesto 0.16.1~b1fa7c5 + * Serving Flask app 'server' + * Debug mode: off +WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. + * Running on all addresses (0.0.0.0) + * Running on http://127.0.0.1:8080 + * Running on http://0.0.0.0:8080 +Press CTRL+C to quit +``` + +We also check it works correctly by querying it: + +```bash +$ curl localhost:8080 +``` +``` +Hello, World! +``` + +Everything is OK. +We create the setup for running a minimized Python Flask image with Unikraft. + +### Contributing to the Application Catalog + +With the Python Flask example now set, we can make a contribution to the [`catalog` repository](https://github.com/unikraft/catalog). +For that three additional steps need to be taken: + +1. Create a `README.md` file. +1. Update the examples listing in the [top-level `README.md` file](https://github.com/unikraft/catalog/blob/main/README.md). + +Then create a commit with the `Dockerfile`, `Kraftfile`, `README.md`, and updates to the [top-level `README.md` file](https://github.com/unikraft/catalog/blob/main/README.md). +And submit a pull request. diff --git a/content/guides/bincompat.mdx b/content/guides/bincompat.mdx deleted file mode 100644 index 5f7b8307..00000000 --- a/content/guides/bincompat.mdx +++ /dev/null @@ -1,651 +0,0 @@ ---- -title: Binary Compatibility -description: | - This guide presents the Unikraft binary compatibility layer. - The binary compatibility layer (bincompat) is used to run unmodified Linux binaries (ELFs) on top of Unikraft. ---- - -## Intro - -One of the obstacles when aiming to use Unikraft is the porting effort of new applications. -This process can be made painless through the use of Unikraft's **binary compatibility layer**. -Binary compatibility allows you to run pre-built Linux binaries (ELFs) on top of Unikraft. -This is done without any porting effort while maintaining the benefits of Unikraft: reduced kernel memory footprint, high degree of configurability of library components. - -For this, Unikraft must provide a similar ABI (_Application Binary Interface_) with the Linux kernel. -This means that Unikraft has to provide a similar system call interface that Linux kernel provides, a [POSIX](https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/)-compatible interface. -For this, the [**system call shim layer**](/docs/internals/syscall-shim) (also called **syscall shim**) was created. -The system call shim layer provides Linux-style mappings of system call numbers to actual system call handler functions. - - -Currently, binary compatibility is available on x86_64. -Work is being carried out to make it work on AArch64 as well. -Also, KVM is currently the only supported hypervisor. - - - -Currently, binaries have to be built as PIE (*Position-Independent Executables*). -This is the default build mode of the majority of Linux distributions, so it shouldn't cause any problems. - - - -Note that, because Linux binaries are included, constructing new Linux binaries requires a Linux or Linux-compatible development environement (such as WSL - _Windows Subsystem for Linux_). -This is only the case for building binaries. -Prebuilt binaries and the ELF loader app itself can be built on multiple platforms (Linux, Windows, macOS). - - -## Setup - -To set up, build and run Linux ELFs with [`app-elfloader`](https://github.com/unikraft/app-elfloader), we recommend you use [the `run-app-elfloader` repository](https://github.com/unikraft/run-app-elfloader). -Along with the [`run-app-elfloader`](https://github.com/unikraft/run-app-elfloader) repository, we collected pre-built applications that you can use in binary compatibility mode. -Those are located in the [`static-pie-apps`](https://github.com/unikraft/static-pie-apps/) and [`dynamic-apps`](https://github.com/unikraft/dynamic-apps/) repositories. -These are pre-built applications, so no time must be spent on compiling them. -They need to be cloned and then used. - -The following repositories need to be cloned: - -```console -git clone https://github.com/unikraft/run-app-elfloader -git clone https://github.com/unikraft/static-pie-apps -git clone https://github.com/unikraft/dynamic-apps -``` - -## Quick Runs - -### Hello World - -In order to quickly run a `helloworld` application in binary compatibility mode, you can use the `run.sh` script in the `run-app-elfloader` repository: - -```console -cd run-app-elfloader/ -./run.sh -d -r ../dynamic-apps/lang/c/helloworld/ helloworld -``` - -You will see the following output: - -```text -SeaBIOS (version rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org) -Booting from ROM..TEST nofollow -Powered by -o. .o _ _ __ _ -Oo Oo ___ (_) | __ __ __ _ ' _) :_ -oO oO ' _ `| | |/ / _)' _` | |_| _) -oOo oOO| | | | | (| | | (_) | _) :_ - OoOoO ._, ._:_:_,\_._, .__,_:_, \___) - Atlas 0.13.1~d20aa7cb -[...] -Hello, World! -``` - -This will run a dynamically linked `helloworld` application. -Currently, the unikernel doesn't shut down. -To close the running instance use `Ctrl+c`; -if that doesn't work use `Ctrl+a x`, that is press `Ctrl+a` and then, separately, press `x`. - -The `-r` option passed to the `run.sh` script (together with the `../dynamic-apps/lang/c/helloworld/`) is the root filesystem of the application. -The root filesystem contains the binary ELF, the required dynamic libraries (shared objects) and any support files (configuration files, data files etc.) - -The `-d` option disables KVM support. -We use it for portability, in case you run this on a virtual machine, or on a system that doesn't provide KVM support. - -### HTTP Server - -Networking support requires the `-n` option to be passed to the `run.sh` script. -And it also requires admin privileges (to create the required network interface), so we use `sudo`. -So, in order to run an HTTP server (let's go for the one written in Go), we use, while inside the `run-app-elfloader/` directory: - -```console -sudo ./run.sh -d -n -r ../dynamic-apps/lang/go/http_server /http_server -``` - -You will see the following output: - -```text -Booting from ROM..1: Set IPv4 address 172.44.0.2 mask 255.255.255.0 gw 172.44.0.1 -en1: Added -en1: Interface is up -Powered by -o. .o _ _ __ _ -Oo Oo ___ (_) | __ __ __ _ ' _) :_ -oO oO ' _ `| | |/ / _)' _` | |_| _) -oOo oOO| | | | | (| | | (_) | _) :_ - OoOoO ._, ._:_:_,\_._, .__,_:_, \___) - Prometheus 0.14.0~4cce8306-custom -``` - -Note that the server listens for connections on the `172.44.0.2` IP address. -And, by checkig the source code, we know it's using the `8080` port. -So we query that address: - -```console -curl 172.44.0.2:8080 -``` - -This results in a simple `hello` message, signaling it works correctly: - -```text -hello -``` - -### Nginx - -The same steps as those for the HTTP server are used for Nginx. - -To run Nginx in bincompat mode, we use the command below, while inside the `run-app-elfloader` directory: - -```console -sudo ./run.sh -d -n -r ../dynamic-apps/nginx /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf -``` - -You will see the following output: - -```text -Booting from ROM..1: Set IPv4 address 172.44.0.2 mask 255.255.255.0 gw 172.44.0.1 -en1: Added -en1: Interface is up -Powered by -o. .o _ _ __ _ -Oo Oo ___ (_) | __ __ __ _ ' _) :_ -oO oO ' _ `| | |/ / _)' _` | |_| _) -oOo oOO| | | | | (| | | (_) | _) :_ - OoOoO ._, ._:_:_,\_._, .__,_:_, \___) - Prometheus 0.14.0~4cce8306-custom -``` - -Note that the server listens for connections on the `172.44.0.2` IP address, on the HTTP port (`80`). -So we query that address: - -```console -curl 172.44.0.2 -``` - -This results in the standard Nginx HTML output: - -```text - - - -Welcome to nginx! - - - -

Welcome to nginx!

-

If you see this page, the nginx web server is successfully installed and -working. Further configuration is required.

- -

For online documentation and support please refer to -nginx.org.
-Commercial support is available at -nginx.com.

- -

Thank you for using nginx.

- - -``` - -### run_app.sh - -[The `run-app-elfloader` repository](https://github.com/unikraft/run-app-elfloader) provides the `run_app.sh` directory for quickly running apps. -It calls `run.sh` behind the scenes. - -To get a list of possible applications, run the script without arguments, while inside the `run-app-elfloader/` directory: - -```console -./run_app.sh -``` - -It will generate the following output: - -```text -Usage: ./run_app.sh [-l] -Possible apps: -bc bc_static bzip2 client client_go client_go_static client_static echo ffmpeg -gnupg gzip gzip_static haproxy helloworld helloworld_cpp helloworld_cpp_static -helloworld_go helloworld_go_static helloworld_lua helloworld_perl -helloworld_python helloworld_rust helloworld_rust_static_gnu -helloworld_rust_static_musl helloworld_static http_server http_server_cpp -http_server_go http_server_python http_server_rust ls nginx nginx_static -openssl python redis redis7 redis_static server server_go server_go_static -server_static sqlite3 sqlite3_static - - -l - use dynamic loader explicitly -``` - -The list of apps are arguments to be passed to the script. - -Use the commands below to run, respectively, the helloworld, HTTP server, and Nginx apps: - -```console -./run_app.sh helloworld -./run_app.sh http_server -./run_app.sh nginx -``` - -The behavior is identical to the above sections, given it runs the `run.sh` script behind the scenes. - -Take a look at the `run_app.sh` script; -there is a function for each application run, that invokes `run.sh`. -The three functions used for the helloworld, HTTP server and Nginx apps are: - -```bash -run_helloworld() -{ - ./run.sh -d -r ../dynamic-apps/lang/c/helloworld "$extra_args" /helloworld -} - -run_http_server() -{ - ./run.sh -d -n -r ../dynamic-apps/lang/c/http_server "$extra_args" /http_server -} - -run_nginx() -{ - ./run.sh -d -n -r ../dynamic-apps/nginx/ "$extra_args" /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf -} -``` - -You can see they use the same `run.sh` commands we used above. - -### Practice: Run Binary Applications - -Use the `run_app.sh` script to run all applications available. -After each run, close the running instance with `Ctrl+c` or `Ctrl+a x`. -Recall that applications that require networking support (i.e. those where the `-n` option is passed to the `run.sh` script) need be run with admin rights; -use `sudo` in fron the the `run_app.sh` commands. - -Use the `run.sh` script on as many applications as possible. -Check the contents of the `run_app.sh` script and run the corresponding commands. - -## Entire Filesystem Runs - -As you've seen, running an application in binary compatibility mode requires a filesytem (storing the Linux binary, dynamic libraries and support files) and the command line used to start the application. -To quickly test a new application, we can use the entire Linux filesystem, (i.e. passing `/` as the filesystem path). - -For example, to run the `/bin/ls` Linux executable with Unikraft, we would use the `run.sh` script such as below, in the `run-app-elfloader/` directory: - -```console -./run.sh -r / /bin/ls -``` - -Similarly, to run `grep`, use the command below: - -```console -./run.sh -r / /bin/grep "bash" /etc/passwd -``` - -The commands mount the entire host filesystem to Unikraft and, in doing so, make all executables available to be tested. - -### Practice: Run Filesystem Executables - -Run as many executables as possible from the host filesystem on top of Unikraft, using the binary compatibility layer. -As potential items, use `/bin/head`, `/usr/bin/sort`, `/bin/zip`. -A good option would be Python. -You need the path to the actual Linux executable, not a symbolic link. - - -Note that certain executables will not work due to features not being supported by Unikraft: - -- Applications using multiple processes or forking are not supported. - For example, `gcc` spawns multiple processes, so it will not work. -- Applications that make use of terminal features. - For example, terminal viewers (`less`) or editors (`nano`, `vi`) will not work. -- Applications that use a GUI will not work. - For example Firefox or Gedit will not work. - - -## Debugging Binary Compatibility - -It can happen that there are issues with Unikraft when running binary compatible apps. -There may be missing system calls, unimplemented arguments, ABI incompatibilities. -So we need debugging features. - -### System Call Tracing - -The most direct way to debug binary compatibility is via system call tracing (i.e. listing system calls and their arguments). -To assist with that, the `run-app-elfloader` repository contains an `app-elfloader` image with tracing support: `app-elfloader_qemu-x86_64_strace`. -To use that image, pass the `-k` option to the `run.sh` script. -For example, to run the helloworld application with tracing we use: - -```console -./run.sh -k app-elfloader_qemu-x86_64_strace -r ../dynamic-apps/lang/c/helloworld/ /helloworld -``` - -This results in the output below, consisting of system calls being made, along with the printing of the `Hello, World!` message: - -```text -brk(NULL) = va:0x47f800000 -uname(utsname:{sysname="Unikraft", nodename="unikraft", ...}) = OK -access("/etc/ld.so.nohwcap", F_OK) = No such file or directory (-2) -access("/etc/ld.so.preload", R_OK) = No such file or directory (-2) -[...] -mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, fd:-1, 0) = va:0x10003f3000 -arch_prctl(0x1002, 0x10003f3f00, ...) = 0x0 -mprotect(va:0x10003e9000, 16384, PROT_READ) = OK -mprotect(va:0x400601000, 4096, PROT_READ) = OK -mprotect(va:0x47f22a000, 4096, PROT_READ) = OK -fstat(fd:1, stat:{st_size=0, st_mode=020000, ...}) = OK -ioctl(0x1, 0x5401, ...) = 0x0 -brk(NULL) = va:0x47f800000 -brk(va:0x47f821000) = va:0x47f821000 -Hello, World! -write(fd:1, "Hello, World!\x0A", 14) = 14 -``` - -### Full Debug Messages - -We can also use extensive debugging provided by Unikraft. -Note that this will give **a lot** of output and will slow things down considerably. - -To assist with that, the `run-app-elfloader` repository contains an `app-elfloader` image with full debug message support: `app-elfloader_qemu-x86_64_full-debug`. -To use that image, pass the `-k` option to the `run.sh` script. -For example, to run the helloworld application with full debug support, use: - -```console -./run.sh -k app-elfloader_qemu-x86_64_full-debug -r ../dynamic-apps/lang/c/helloworld/ /helloworld -``` - -This results in the output below, consisting of extensive debug messages, system calls being made, along with the printing of the `Hello, World!` message: - -```text -[...] -fstat(fd:1, stat:{st_size=0, st_mode=020000, ...}) = OK -[ 5.045493] dbg: [libsyscall_shim] Binary system call request "ioctl" (16) at ip:0x10001178e8 (arg0=0x1, arg1=0x5401, ...) -[ 5.048418] dbg: [libvfscore] (int) uk_syscall_r_ioctl((int) 0x1, (unsigned long int) 0x5401, (void*) 0x40009fb80) -ioctl(0x1, 0x5401, ...) = 0x0 -[ 5.052490] dbg: [libsyscall_shim] Binary system call request "brk" (12) at ip:0x10001180f9 (arg0=0x0, arg1=0x10003edc40, ...) -[ 5.055469] dbg: [appelfloader] (void *) uk_syscall_r_brk((void *) 0x0) -[ 5.057158] dbg: [appelfloader] Outside of brk range, return current brk 0x47f800000 -brk(NULL) = va:0x47f800000 -[ 5.060265] dbg: [libsyscall_shim] Binary system call request "brk" (12) at ip:0x10001180f9 (arg0=0x47f821000, arg1=0x10003edc40, ...) -[ 5.063398] dbg: [appelfloader] (void *) uk_syscall_r_brk((void *) 0x47f821000) -[ 5.065240] dbg: [appelfloader] zeroing 0x47f800000-0x47f821000... -[ 5.066905] dbg: [appelfloader] brk @ 0x47f821000 (brk heap region: 0x47f800000-0x47fa00000) -brk(va:0x47f821000) = va:0x47f821000 -[ 5.070504] dbg: [libsyscall_shim] Binary system call request "write" (1) at ip:0x1000112104 (arg0=0x1, arg1=0x47f800260, ...) -[ 5.073497] dbg: [libvfscore] (ssize_t) uk_syscall_r_write((int) 0x1, (const void *) 0x47f800260, (size_t) 0xe) -[ 5.076049] dbg: [libvfscore] (ssize_t) uk_syscall_r_writev((int) 0x1, (const struct iovec *) 0x40009f730, (int) 0x1) -Hello, World! -write(fd:1, "Hello, World!\x0A", 14) = 14 -[ 5.080710] dbg: [libsyscall_shim] Binary system call request "exit_group" (231) at ip:0x10000e6ab6 (arg0=0x0, arg1=0x3c, ...) -[ 5.083937] dbg: [libposix_process] (int) uk_syscall_r_exit_group((int) 0x0) -[ 5.085801] dbg: [libposix_process] Terminating PID 1: Self-killing TID 1... -[...] -``` - -When encountering problems with binary compatibility mode, use either system call tracing or full debug messages to assist in understanding what's wrong. - -### Using GDB - -Tracing and debug messages may not be enough to identify the cause of certain issues. -For that you want to follow the control flow of the application, be able to follow the instructions and print variable values. -In short, you require the use of a debugger, such as GDB. - -See instructions in [the `README.md` file of the `app-elfloader` repository](https://github.com/unikraft/app-elfloader) about the use of GDB for debugging. - -### Practice: Run Applications with Debugging Enabled - -Run as many applications as you can with debugging support in binary compatibility: both system call tracing and full debug messages. -Run applications from [the `dynamic-apps` repository](https://github.com/unikraft/dynamic-apps) and applications from the entire Linux filesystem. - -## Creating an Application-Specific Root Filesystem - -Applications in [the `run-app-elfloader` repository](https://github.com/unikraft/run-app-elfloader) use a directory as their root filesystem. -This contains: - -- The application binary -- Required dynamic libraries (shared objects) -- Support files: configuration files, data files, language-specific libraries - -Having such as a directory is important when packing an application. -Only the required files are added to it, similar to a container making thre result image, as small as possible. - -Application binaries can be obtained in two ways: - -- Pre-built binaries extracted from a package, container or filesystem -- Built from source code - -Supported binaries must be PIE (**Position-Independent Executables**), either static or dynamic. - -### Pre-built Binaries - -Once a dynamic binary application is obtained, we need to extract the required dynamic libraries. -This step is only required for dynamic binaries; -static binaries aren't using dynamic libraries. -For this we use [the `extract.sh` script](https://github.com/unikraft/dynamic-apps/tree/master/extract.sh) in the `dynamic-apps` repository. - -To get the syntax of the script, run it without arguments: - -```console -./extract.sh -``` - -It prints the output: - -```text -Binary to extract not provided. - -Usage: ./extract.sh [] - - Default extract path is current directory -``` - -The `extract.sh` script will take an `ELF` file as the argument and an optional directory that stores the root filesystem. -If no directory is provided, the current directory is used as the root filesystem. -The script will then populate the root directory with the binary and dynamic libraries. - -The command below uses the script to create the root filesystem directory for `grep`: - -```console -./extract.sh /usr/bin/grep grep -``` - -The command output presents the copying of the binary and the required dynamic libraries: - -```text -Copying /usr/bin/grep ... -Copying /lib/x86_64-linux-gnu/libpcre.so.3 ... -Copying /lib/x86_64-linux-gnu/libc.so.6 ... -Copying /lib64/ld-linux-x86-64.so.2 ... -``` - -We'll also copy the `/etc/passwd` file as test file: - -```console -cp --parents /etc/passwd grep/ -``` - -The resulting directory consists the properly organized filesystem for the application: - -```text -grep/ -|-- etc/ -| `-- passwd -|-- lib/ -| `-- x86_64-linux-gnu/ -| |-- libc.so.6* -| `-- libpcre.so.3 -|-- lib64/ -| `-- ld-linux-x86-64.so.2* -`-- usr/ - `-- bin/ - `-- grep* -``` - -After all this is done, we can go back to the `run-app-elfloader` repository and use the `run.sh` script to run the application we just prepared: - -```console -./run.sh -r ../dynamic-apps/grep/ /usr/bin/grep bash /etc/passwd -``` - -The command will search for the `bash` string in the `/etc/passwd` file. -Note that paths are absolute in the application root filesystem. - -The command output will be similar to: - -```text -SeaBIOS (version 1.15.0-1) -Booting from ROM..Powered by -o. .o _ _ __ _ -Oo Oo ___ (_) | __ __ __ _ ' _) :_ -oO oO ' _ `| | |/ / _)' _` | |_| _) -oOo oOO| | | | | (| | | (_) | _) :_ - OoOoO ._, ._:_:_,\_._, .__,_:_, \___) - Prometheus 0.14.0~4cce8306-custom -root:x:0:0:root:/root:/bin/bash -unikraft:x:1000:1000:Unikraft User,,,:/home/unikraft:/bin/bash -``` - -### Custom Applications - -The steps above assumed the existence of a pre-built binary. -Let's consider custom applications that we have written. -For example, we create a simple helloworld application in C++. - -We create the application as `helloworld.cpp`: - -```cpp -#include - -int main() -{ - std::cout << "Hello World!" << std::endl; - return 0; -} -``` - -We then build the application: - -```console -g++ -fPIC -pie -Wall -o helloworld helloworld.cpp -``` - -The `-fPIC` or `-pie` flags are typically default build flags. -We added them just to be sure. - -We are now in possession of the binary executable `helloworld`, so we apply the steps laid out in section [Pre-built Binaries](/guides/bincompat/#pre-built-binaries). -Namely, using the `extract.sh` script to extract the binary and the dynamic libraries in the application root filesystem, and running the resulting filesystem using `run.sh`. - -### Practice: Application Filesystems - -Create application root filesystems for application that are already part of your Linux host filesystem. -Follow the steps in the section [Pre-built Binaries](/guides/bincompat/#pre-built-binaries). - -Recall to target binaries that don't use the GUI, nor the terminal screen, nor are multi-process. - -Aim to create pull requests with the new application filesystems in [the `dynamic-apps` repository](https://github.com/unikraft/dynamic-apps). - -### Practice: Custom Applications in Interpreted Languages - -Create your own applications in your preferred interpreted language. -Choose among the languages that are already part of [the `dynamic-apps` repository](https://github.com/unikraft/dynamic-apps) (the `lang/` directory): Python, Lua, Perl, Ruby. - -Add your scripts in the application filesystem for the respective programming language. -Then run it with the `run.sh` script. - -Aim to create pull requests with the new application filesystems in [the `dynamic-apps` repository](https://github.com/unikraft/dynamic-apps), in the corresponding subdirectory of the `lang/` directory. - -### Practice: Custom Applications in Compiled Languages - -Create your own applications in your preferred compiled language (C, C++, Rust, Go, Objective-C). -Build the source code into a dynamic PIE ELF. - -Then create application root filesystems for application that are already part of your Linux host filesytem. -Aim to create pull requests with the new application filesystems in [the `dynamic-apps` repository](https://github.com/unikraft/dynamic-apps), in the corresponding subdirectory of the `lang/` directory. - -## Build `app-elfloader` - -Using `./run.sh`, we used the pre-built `app-elfloader` images from [the `run-app-elfloader` repository](https://github.com/unikraft/run-app-elfloader): - -- `app-elfloader_qemu-x86_64`: the standard image -- `app-elfloader_qemu-x86_64_strace`: the image with system call tracing -- `app-elfloader_qemu-x86_64_full-debug`: the image with full debug messages. - -However, if new changes are added to Unikraft, or we want to test potential changes ourselves (pull requests, branches), we need to re-build the `app-elfloader` from [its repository](https://github.com/unikraft/app-elfloader). - -In order to build our own `app-elfloader` image, follow the instructions in the [`app-elfloader` README file](https://github.com/unikraft/app-elfloader#readme), the ["Set Up"](https://github.com/unikraft/app-elfloader#set-up) and the ["Scripted Building and Running"](https://github.com/unikraft/app-elfloader#scripted-building-and-running) sections. -In short, the instructions present you with different ways to build, using the scripts in the `scripts/build/` directory: - -- 9pfs or initrd filesystem -- KraftKit-based build or Make-based build -- QEMU or Firecracker VMM -- Building the standard, system call tracing or full debug message image - -Running the image is easiest to be done via the scripts in the `scripts/run/` directory. -These scripts invoke KraftKit or Firecracker or QEMU behind the scenes. - -Note that the `run.sh` script in [the `run-app-elfloader` repository](https://github.com/unikraft/run-app-elfloader) can only be used for QEMU and 9pfs filesystem. - -### Building and Running Nginx - -As an example, let's build `app-elfloader` and run Nginx in binary compatibility mode. -Let's go for a 9pfs build, both with KraftKit and with Make. - -The steps are: - -1. Set up `app-elfloader` by following [the instructions in its documentation](https://github.com/unikraft/app-elfloader#set-up). - -1. Enter the repository clone (i.e. the `elfloader/` directory) and run the `./generate.py` script the generates the scripts in `scripts/build/` and `scripts/run/` directories: - - ```console - ./scripts/generate.py - ls -R ./scripts - ``` - -1. Build the ELF loader with KraftKit: - - ```console - ./scripts/build/kraft-qemu-x86_64-9pfs.sh - ``` - -1. Build the ELF Loader with Make: - - ```console - ./scripts/build/make-qemu-x86_64-9pfs.sh - ``` - -1. Run the resulting image with KraftKit: - - ```console - ./scripts/run/kraft-qemu-x86_64-9pfs-nginx.sh - ``` - -1. Rn the resulting image with QEMU: - - ```console - ./scripts/run/qemu-x86_64-9pfs-nginx.sh - ``` - -1. Test - -1. Run the resulting images from KraftKit and QEMU with `run.sh`: - - ```console - sudo pkill -f firecracker - sudo pkill -f qemu - sudo ip link set dev virbr0 down - sudo ip link del dev virbr0 - sudo ./run.sh -n -k ../elfloader/.unikraft/build/elfloader-qemu-x86_64-9pfs_qemu-x86_64 -r ../dynamic-apps/nginx /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf - - sudo pkill -f firecracker - sudo pkill -f qemu - sudo ip link set dev virbr0 down - sudo ip link del dev virbr0 - sudo ./run.sh -n -k ../elfloader/workdir/build/elfloader_qemu-x86_64 -r ../dynamic-apps/nginx /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf - ``` - -1. Test all runs with `curl` on a different console: - - ```console - curl http://172.44.0.2 - ``` - -### Practice: Build `app-elfloader` and Run Applications - -Build `app-elfloader` in different configurations (filesystem, VMMs, KraftKit / Make). -Run different applications with it in different ways: KraftKit, QEMU, Firecracker, `run.sh`. diff --git a/content/guides/catalog-behind-the-scenes.mdx b/content/guides/catalog-behind-the-scenes.mdx new file mode 100644 index 00000000..37b85ee0 --- /dev/null +++ b/content/guides/catalog-behind-the-scenes.mdx @@ -0,0 +1,345 @@ +--- +title: "Behind the Scenes with the Application Catalog" +description: | + This guide presents internal technical information about the application catalog. + It shows what is happening behind the scenes and how you can get more control on the build and run phases. +--- + +The [guide on using the application catalog](/guides/using-the-app-catalog) provides the user-friendly information on using the Unikraft application registry and the [`catalog` repository](https://github.com/unikraft/catalog). +It presents some hints into what's happening behind the scenes, but it aims to keep the user away for this. +This guide takes a deep dive into the internals of configuring, building and running Unikernel applications from the catalog. +It is aimed for those more technically inclined who would be interested in understanding what's happening behind the scenes and maybe [contribute to the application catalog](/docs/contributing/adding-to-the-app-catalog). + +Similar to the [guide on using the application catalog](/guides/using-the-app-catalog), we will use two applications: + +- [the `nginx/1.25` application](https://github.com/unikraft/catalog/tree/main/library/nginx/1.25) +- [the `http-go1.21` example](https://github.com/unikraft/catalog/tree/main/examples/http-go1.21) + +## NGINX + +For the [`nginx/1.25` bincompat application](https://github.com/unikraft/catalog/tree/main/library/nginx/1.25), there is a build phase and a run phase. +The build phase creates the output kernel, and the run phase launches a Unikraft virtual machine instance from the kernel. + +The kernel is a join of the actual Unikraft kernel and the application filesystem, packed as an initial ramdisk. +We call the packed initial ramdisk the **embedded initial ramdisk** or **embedded initrd**. + +### Configuration + +The build and run configuration is part of the [`Kraftfile`](https://github.com/unikraft/catalog/blob/main/library/nginx/1.25/Kraftfile). + +The `Kraftfile` defines the: + +- resulting image name: `nginx` +- the command line to start the application: `/usr/bin/nginx` +- path to the template `app-elfloader` +- paths and versions of repositories (`unikraft`, `lwip`, `libelf`) +- configuration options: i.e. the `CONFIG_...` option enables the emdedded initrd build +- build and run targets: currently only x86_64-based builds are available, and only KVM-based builds, using QEMU or Firecracker +- root filesystem used to build the (embedded) initrd + +The root filesystem is generated from a `Dockerfile` specification, as configured in the `Kraftfile`. +The `Dockerfile` specification collects the required files (binary executable, depending libraries, configuration files, data files): + +```dockerfile +FROM --platform=linux/x86_64 nginx:1.25.3-bookworm AS build + +# These are normally syminks to /dev/stdout and /dev/stderr, which don't +# (currently) work with Unikraft. We remove them, such that NGINX will create +# them by hand. +RUN rm /var/log/nginx/error.log +RUN rm /var/log/nginx/access.log + +FROM scratch + +# NGINX binaries, modules, configuration, log and runtime files +COPY --from=build /usr/sbin/nginx /usr/bin/nginx +COPY --from=build /usr/lib/nginx /usr/lib/nginx +COPY --from=build /etc/nginx /etc/nginx +COPY --from=build /etc/passwd /etc/passwd +COPY --from=build /etc/group /etc/group +COPY --from=build /var/log/nginx /var/log/nginx +COPY --from=build /var/cache/nginx /var/cache/nginx +COPY --from=build /var/run /var/run + +# Libraries +COPY --from=build /lib/x86_64-linux-gnu/libcrypt.so.1 /lib/x86_64-linux-gnu/libcrypt.so.1 +COPY --from=build /lib/x86_64-linux-gnu/libpcre2-8.so.0 /lib/x86_64-linux-gnu/libpcre2-8.so.0 +COPY --from=build /lib/x86_64-linux-gnu/libssl.so.3 /lib/x86_64-linux-gnu/libssl.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libcrypto.so.3 /lib/x86_64-linux-gnu/libcrypto.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libz.so.1 /lib/x86_64-linux-gnu/libz.so.1 +COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 +COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 +COPY --from=build /etc/ld.so.cache /etc/ld.so.cache + +# Custom configuration files, including using a single process for Nginx +COPY ./conf/nginx.conf /etc/nginx/nginx.conf +COPY ./conf/unikraft.local.crt /etc/nginx/unikraft.local.crt +COPY ./conf/unikraft.local.key /etc/nginx/unikraft.local.key + +# Web root +COPY ./wwwroot /wwwroot +``` + +The `Dockerfile` is being interpreted via [`BuildKit`](https://docs.docker.com/build/buildkit/), hence the need to set up the `BuildKit` container. + +### Build Phase + +The build command requires the `BuildKit` container to be configured beforehand: + +```bash +docker run -d --name buildkitd --privileged moby/buildkit:latest +export KRAFTKIT_BUILDKIT_HOST=docker-container://buildkitd +``` + +The build command is: + +```bash +kraft build --plat qemu --arch x86_64 +``` + +`kraft build` goes through the following steps: + +1. It generates the root filesystem, via BuildKit from the `Dockerfile` specification. +1. It packs the root filsystem in an initial ramdisk (initrd). +1. It builds the kernel, using the configuration in the `Kraftfile`. +1. It embeds the initrd in the output kernel file. + +The resulting embedded kernel image is `.unikraft/build/nginx_qemu-x86_64`: + +```bash +$ ls -lh .unikraft/build/nginx_qemu-x86_64 +``` +``` +-rwxr-xr-x 2 razvand docker 15M Jan 2 21:23 .unikraft/build/nginx_qemu-x86_64 +``` + +### Run Phase + +This image is run with a command such as: + +```bash +kraft run -W -p 8080:80 . +``` + +It can also be run manually with `qemu-system-x86_64`: + +```bash +qemu-system-x86_64 \ + -kernel .unikraft/build/nginx_qemu-x86_64 \ + -nographic \ + -m 128M \ + -device virtio-net-pci,mac=02:b0:b0:d3:d2:01,netdev=hostnet0 \ + -netdev user,id=hostnet0,hostfwd=tcp::8080-:80 \ + -append "/usr/bin/nginx" \ + -cpu max +``` + +This starts a QEMU virtual machine instance. +Query it using: + +```bash +curl http://localhost:8080 +``` + +If you want use a bridge interface, first create the bridge interface as `root` (prefix with `sudo` if required): + +```bash +kraft net create -n 172.44.0.1/24 virbr0 +``` + +An the run manually with `qemu-system-x86_64` as `root` (prefix with `sudo` if required): + +```bash +qemu-system-x86_64 \ + -kernel .unikraft/build/nginx_qemu-x86_64 \ + -nographic \ + -m 128M \ + -netdev bridge,id=en0,br=virbr0 -device virtio-net-pci,netdev=en0 \ + -append "netdev.ip=172.44.0.2/24:172.44.0.1 -- /usr/bin/nginx" \ + -cpu max +``` + +This starts a QEMU virtual machine instance. +Query it using: + +```bash +curl http://172.44.0.2 +``` + +To close the running QEMU instance, use `Ctrl+a x` in the QEMU console. + +## HTTP Go Server + +For the [`http-go1.21` bincompat example](https://github.com/unikraft/catalog/tree/main/examples/http-go1.21), there is no build phase, only a run phase. +The example it's using a prebuilt kernel image. +The prebuilt [`base` kernel image](https://github.com/unikraft/catalog/tree/main/library/base) is pulled from the registry, from `unikraft.org/base`. +This happens during the run phase. + +### Configuration + +The run configuration is part of the [`Kraftfile`](https://github.com/unikraft/catalog/blob/main/examples/http-go1.21/Kraftfile): + +```yaml +spec: v0.6 + +runtime: base:latest + +rootfs: ./Dockerfile + +cmd: ["/server"] +``` + +The `Kraftfile` defines: + +- the runtime image to use, containing the kernel: `unikraft.org/base:latest' (it can be summarized as just `base:latest`) +- the root filesystem used, defined in a `Dockerfile` +- the command line to start the application: `/server` +- the available run targets: currently only x86_64-based builds are available, and only KVM-based builds, using QEMU or Firecracker + +The root filesystem is generated from a `Dockerfile` specification, as configured in the `Kraftfile`. +The `Dockerfile` specification collects the required files (binary executable, depending libraries, configuration files, data files): + +```dockerfile +FROM golang:1.21.3-bookworm AS build + +WORKDIR /src + +COPY ./server.go /src/server.go + +RUN set -xe; \ + CGO_ENABLED=1 \ + go build \ + -buildmode=pie \ + -ldflags "-linkmode external -extldflags '-static-pie'" \ + -tags netgo \ + -o /server server.go \ + ; + +FROM scratch + +COPY --from=build /server /server +COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/ +COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ +``` + +The Dockerfile is being interpreted via [BuildKit](https://docs.docker.com/build/buildkit/), hence the need to set up the BuildKit container. + +### Run Phase + +The run command requires the `BuildKit` container to be configured beforehand: + +```bash +docker run -d --name buildkitd --privileged moby/buildkit:latest +export KRAFTKIT_BUILDKIT_HOST=docker-container://buildkitd +``` + +The run command is: + +```bash +kraft run -W -p 8080:8080 . +``` + +`kraft run` goes through the following steps: + +1. It pulls the kernel package from the registry, from `unikraft.org/base:latest`. +1. It generates the root filesystem, via BuildKit from the `Dockerfile` specification. + The generation of the root filesystem implies the building the Go source code files into a binary executable (`ELF`). + The executable, together with the depending libraries is then extracted into the root filesystem. +1. It packs the root filesystem in an initial ramdisk (initrd). +1. It runs the kernel attaching the initrd and using the command line in the specification: `/http_server`. + +The resulting initrd image is `.unikraft/build/initramfs.cpio`. + +```bash +$ ls -lh .unikraft/build/initramfs.cpio +``` +``` +-rw-r--r-- 1 root root 8.9M Jan 4 18:16 .unikraft/build/initramfs-x86_64.cpio +``` + +To view the contents of the root filesystem you can use `cpio`: + +```bash +$ cpio -itv < .unikraft/build/initramfs.cpio +``` +``` +d--------- 0 root root 0 Jan 1 1970 /lib +d--------- 0 root root 0 Jan 1 1970 /lib/x86_64-linux-gnu +-rwxr-xr-x 1 root root 1922136 Sep 30 11:31 /lib/x86_64-linux-gnu/libc.so.6 +d--------- 0 root root 0 Jan 1 1970 /lib64 +-rwxr-xr-x 1 root root 210968 Sep 30 11:31 /lib64/ld-linux-x86-64.so.2 +-rwxr-xr-x 1 root root 7151306 Jan 4 18:16 /server +18136 blocks +``` + +The kernel image is pulled into a temporary directory. + +To run the application manually, first pull the kernel image from `unikraft.org/base:latest`: + +```bash +kraft pkg pull -w base unikraft.org/base:latest +``` + +The kernel image is `base/unikraft/bin/kernel`: + +```bash +$ tree +base/ +`-- unikraft/ + `-- bin/ + `-- kernel + +3 directories, 1 file + +$ ls -lh base/unikraft/bin/kernel +-rw-rw-r-- 1 razvand razvand 1.6M Jan 25 14:48 base/unikraft/bin/kernel +``` + +You can run the application manually with `qemu-system-x86_64` and the passing of the `-kernel`, `-initrd` and `-append` arguments: + +```bash +qemu-system-x86_64 \ + -kernel base/unikraft/bin/kernel \ + -nographic \ + -m 128M \ + -device virtio-net-pci,mac=02:b0:b0:d3:d2:01,netdev=hostnet0 \ + -netdev user,id=hostnet0,hostfwd=tcp::8080-:8080 \ + -append "vfs.fstab=[ \"initrd0:/:extract:::\" ] -- /server" \ + -initrd .unikraft/build/initramfs-x86_64.cpio \ + -cpu max +``` + +This starts a QEMU virtual machine instance. +Query it using: + +```bash +curl http://localhost:8080 +``` + +If you want use a bridge interface, first create the bridge interface as `root` (prefix with `sudo` if required): + +```bash +kraft net create -n 172.44.0.1/24 virbr0 +``` + +An the run manually with `qemu-system-x86_64` as `root` (prefix with `sudo` if required): + +```bash +qemu-system-x86_64 \ + -kernel base/unikraft/bin/kernel \ + -nographic \ + -m 128M \ + -netdev bridge,id=en0,br=virbr0 -device virtio-net-pci,netdev=en0 \ + -append "netdev.ip=172.44.0.2/24:172.44.0.1 vfs.fstab=[ \"initrd0:/:extract:::\" ] -- /server" \ + -initrd .unikraft/build/initramfs-x86_64.cpio \ + -cpu max +``` + +This starts a QEMU virtual machine instance. +Query it using: + +```bash +curl http://172.44.0.2:8080 +``` + +To close the running QEMU instance, use `Ctrl+a x` in the QEMU console. diff --git a/content/guides/catalog-using-firecracker.mdx b/content/guides/catalog-using-firecracker.mdx new file mode 100644 index 00000000..1a1c708c --- /dev/null +++ b/content/guides/catalog-using-firecracker.mdx @@ -0,0 +1,244 @@ +--- +title: "Using Firecracker in the Application Catalog" +description: | + This guide presents internal technical information about using Firecracker with the application catalog. +--- + +This guide is a companion to the ["Behind the Scenes with the Application Catalog"](/guides/catalog-behind-the-scenes). +It details how to use [Firecracker](https://firecracker-microvm.github.io/) to run Unikraft applications. +Firecracker is not yet fully support by KraftKit: +building can be done with KraftKit, but running is to be done manually. + + +Note that Firecracker requires [hardware virtualization support (KVM)](/docs/concepts/virtualization). +Because of that, it can't be used in a virtual machine to run Unikraft applications. +You have to run in a native Linux install. + + +## Installing Firecracker + +To use Firecracker, you need to download a [Firecracker release](https://github.com/firecracker-microvm/firecracker/releases). +For this guide we use the `1.4.0` release: + +```bash +cd /tmp +wget https://github.com/firecracker-microvm/firecracker/releases/download/v1.4.0/firecracker-v1.4.0-x86_64.tgz +tar xzf firecracker-v1.4.0-x86_64.tgz +``` + +You can use the command below to make the `firecracker-x86_64` executable available globally in the command line: + +```bash +sudo cp release-v1.4.0-x86_64/firecracker-v1.4.0-x86_64 /usr/local/bin/firecracker-x86_64 +``` + +Similar to the ["Application Catalog: Behind the Scenes" guide](/guides/catalog-internals), we will use two applications: + +- [the `nginx/1.25` application](https://github.com/unikraft/catalog/tree/main/library/nginx/1.25) +- [the `http-go1.21` example](https://github.com/unikraft/catalog/tree/main/examples/http-go1.21) + +## NGINX + +Use the steps below to build and run the NGINX binary-compatible application: + +1. If not already configured, configure the BuildKit container: + + ```bash + docker run -d --name buildkitd --privileged moby/buildkit:latest + export KRAFTKIT_BUILDKIT_HOST=docker-container://buildkitd + ``` + +1. Enter the NGINX binary-compatible directory: + + ```bash + cd catalog/library/nginx/1.25 + ``` + +1. Build the application for the Firecracker (`fc`) platform: + + ```bash + kraft build --plat fc --arch x86_64 + ``` + + The resulting kernel file is `.unikraft/build/nginx_fc-x86_64`. + +1. As `root` (prefix with `sudo` if required), create a network tap interface: + + ```bash + ip tuntap add dev tap0 mode tap + ip address add 172.45.0.1/24 dev tap0 + ip link set dev tap0 up + ``` + +1. Create the Firecracker JSON configuration file `fc-x86_64.json`: + + ```json + { + "boot-source": { + "kernel_image_path": ".unikraft/build/nginx_fc-x86_64", + "boot_args": ".unikraft/build/nginx_fc-x86_64 netdev.ip=172.45.0.2/24:172.45.0.1 -- /usr/bin/nginx" + }, + "drives": [], + "machine-config": { + "vcpu_count": 1, + "mem_size_mib": 128, + "smt": false, + "track_dirty_pages": false + }, + "cpu-config": null, + "balloon": null, + "network-interfaces": [ + { + "iface_id": "net1", + "guest_mac": "06:00:ac:10:00:02", + "host_dev_name": "tap0" + } + ], + "vsock": null, + "logger": { + "log_path": "/tmp/firecracker.log", + "level": "Debug", + "show_level": true, + "show_log_origin": true + }, + "metrics": null, + "mmds-config": null, + "entropy": null + } + ``` + +1. Run as `root` (prefix with `sudo` if required): + + ```bash + rm -f /tmp/firecracker.log + touch /tmp/firecracker.log + rm -f /tmp/firecracker.socket + firecracker-x86_64 --api-sock /tmp/firecracker.socket --config-file fc-x86_64.json + ``` + +1. Query the unikernel instance: + + ```bash + curl http://172.45.0.2 + ``` + +To close the running Firecracker instance, kill the corresponding process. +In another console, run as `root` (prefix with `sudo` if required): + +```bash +pkill -f firecracker +``` + +## HTTP Go Server + +Use the steps below to build and run the HTTP Go server as a binary-compatible application. + +1. If not already configured, configure the BuildKit container: + + ```bash + docker run -d --name buildkitd --privileged moby/buildkit:latest + export KRAFTKIT_BUILDKIT_HOST=docker-container://buildkitd + ``` + +1. Enter the HTTP Go server example directory: + + ```bash + cd catalog/examples/http-go1.21/ + ``` + +1. Pull the unikernel `base` image for the Firecracker (`fc`) platform: + + ```bash + kraft pkg pull -w base unikraft.org/base:latest --plat fc --arch x86_64 + ``` + +1. Use `kraft run` to trigger the build the root filesystem as an initrd: + + ```bash + sudo KRAFTKIT_BUILDKIT_HOST=docker-container://buildkitd kraft run --plat fc --arch x86_64 . + ``` + + Note that this will cause an error similar to the one below, as KraftKit does not yet support running Firecracker: + + ```text + E [PUT /actions][400] createSyncActionBadRequest &{FaultMessage:The requested operation is not supported after starting the microVM.} + ``` + + However, the filesystem initird is created in `.unikraft/build/initramfs-x86_64.cpio`: + + ```bash + $ ls -lh .unikraft/build/initramfs-x86_64.cpio + ``` + ``` + -rw-r--r-- 1 razvand razvand 9.7M Jan 26 18:50 .unikraft/build/initramfs-x86_64.cpio + ``` + + We will use the initrd for a manual run of Firecracker. + +1. As `root` (prefix with `sudo` if required), create a network tap interface: + + ```bash + ip tuntap add dev tap0 mode tap + ip address add 172.45.0.1/24 dev tap0 + ip link set dev tap0 up + ``` + +1. Create the Firecracker JSON configuration file `fc-x86_64.json`: + + ```json + { + "boot-source": { + "kernel_image_path": "base/unikraft/bin/kernel", + "boot_args": "kernel netdev.ip=172.45.0.2/24:172.45.0.1 vfs.fstab=[ \"initrd0:/:extract:::\" ] -- /server", + "initrd_path": ".unikraft/build/initramfs-x86_64.cpio" + }, + "drives": [], + "machine-config": { + "vcpu_count": 1, + "mem_size_mib": 512, + "smt": false, + "track_dirty_pages": false + }, + "cpu-config": null, + "balloon": null, + "network-interfaces": [ + { + "iface_id": "net1", + "guest_mac": "06:00:ac:10:00:02", + "host_dev_name": "tap0" + } + ], + "vsock": null, + "logger": { + "log_path": "/tmp/firecracker.log", + "level": "Debug", + "show_level": true, + "show_log_origin": true + }, + "metrics": null, + "mmds-config": null, + "entropy": null + } + ``` + +1. Run as `root` (prefix with `sudo` if required): + + ```bash + rm -f /tmp/firecracker.log + touch /tmp/firecracker.log + rm -f /tmp/firecracker.socket + firecracker-x86_64 --api-sock /tmp/firecracker.socket --config-file fc-x86_64.json + ``` + +1. Query the unikernel instance: + + ```bash + curl http://172.45.0.2:8080 + ``` + +To close the running Firecracker instance, kill the corresponding process. +In another console, run as `root` (prefix with `sudo` if required): + +```bash +pkill -f firecracker +``` diff --git a/content/guides/using-the-app-catalog.mdx b/content/guides/using-the-app-catalog.mdx new file mode 100644 index 00000000..48a28018 --- /dev/null +++ b/content/guides/using-the-app-catalog.mdx @@ -0,0 +1,321 @@ +--- +title: Using the Application Catalog +description: | + This guide presents the Unikraft application catalog and how to use it: build and run applications. + It is meant for anyone interested in using Unikraft and benefiting from technical advantages of unikernels. +--- + +The Unikraft application catalog is a collection of applications and examples that are built and packaged to run with Unikraft. +The application packages are stored in the Unikraft Application Registry, typically identified by a name similar to those used by [DockerHub](https://hub.docker.com/): `unikraft.org/node:18`, `unikraft.org/python:3.10`. + +The catalog stores both native builds and [binary-compatible](/docs/concepts/compatibility) builds of Unikraft applications. +As a user it shouldn't matter to you. +[KraftKit](https://github.com/unikraft/kraftkit) is used to provide the same build and run interface for both native and binary-compatible apps. + +## Running Applications from the Registry + +You can list the applications in the registry by using: + +```bash +$ kraft pkg ls --apps --all --update +``` +``` +TYPE NAME VERSION FORMAT MANIFEST INDEX PLAT +app unikraft.org/base latest oci 18cd70e af5c5ed qemu/x86_64 +app unikraft.org/base latest oci ac5efa1 af5c5ed fc/x86_64 +app unikraft.org/helloworld latest oci 8f6c8ff 01f213a fc/x86_64 +app unikraft.org/helloworld latest oci 6686724 01f213a xen/x86_64 +app unikraft.org/helloworld latest oci 225f853 01f213a linuxu/arm64 +app unikraft.org/helloworld latest oci 45782c1 01f213a qemu/arm64 +app unikraft.org/helloworld latest oci 603e401 01f213a qemu/x86_64 +app unikraft.org/helloworld latest oci c156b11 01f213a linuxu/x86_64 +app unikraft.org/lua 5.4 oci 5aa7cae 22b92ef qemu/x86_64 +app unikraft.org/lua 5.4 oci 4910f7f 22b92ef fc/x86_64 +app unikraft.org/lua 5.4 oci bb9f1bb 22b92ef fc/arm64 +app unikraft.org/lua 5.4 oci a0e36f4 22b92ef qemu/arm64 +app unikraft.org/nginx 1.15 oci 11012dd 05c88f3 qemu/arm64 +app unikraft.org/nginx 1.15 oci 789628a 05c88f3 qemu/x86_64 +app unikraft.org/nginx 1.15 oci c6934f4 05c88f3 fc/x86_64 +app unikraft.org/nginx 1.15 oci 2c70788 05c88f3 fc/arm64 +app unikraft.org/python 3.10 oci ecb5863 1f0939e fc/x86_64 +app unikraft.org/python 3.10 oci 2d74e1b 1f0939e qemu/x86_64 +app unikraft.org/redis 7.0 oci ee39581 1661b08 qemu/arm64 +app unikraft.org/redis 7.0 oci a87f72f 1661b08 qemu/x86_64 +app unikraft.org/redis 7.0 oci 9c726bc 1661b08 fc/arm64 +app unikraft.org/redis 7.0 oci 77d88f3 1661b08 fc/x86_64 +``` + +The `base` entry is a meta-application that is being used in [binary-compatibility mode](/docs/concepts/compatibility) to run other applications and examples. +The other entries may be binary-compatible or native builds. + +To pull and run `helloworld`, for example, use: + +```bash +kraft run -W unikraft.org/helloworld +``` + +This will default to the `x86_64` architecture and to the `qemu` platform. +It will pull and run run the application from the registry: + +``` + i using arch=x86_64 plat=qemu +[+] pulling unikraft.org/helloworld +o. .o _ _ __ _ +Oo Oo ___ (_) | __ __ __ _ ' _) :_ +oO oO ' _ `| | |/ / _)' _` | |_| _) +oOo oOO| | | | | (| | | (_) | _) :_ + OoOoO ._, ._:_:_,\_._, .__,_:_, \___) + Telesto 0.16.1~b1fa7c5 +Hello from Unikraft! + +This message shows that your installation appears to be working correctly. + +For more examples and ideas, visit: https://unikraft.org/docs/ +``` + + +The `-W` option for `kraft run` is used to disable hardware virtualization (KVM). +This is useful if you are running Unikraft applications inside a virtual machine. +If you are running natively (for maximum performance), and if you likely have hardware virtualization support, you can remove the `-W` option. + + +Similarly, we can pull and run Nginx: + +```bash +kraft run -W unikraft.org/nginx +``` +``` + i using arch=x86_64 plat=qemu +[+] pulling unikraft.org/nginx +o. .o _ _ __ _ +Oo Oo ___ (_) | __ __ __ _ ' _) :_ +oO oO ' _ `| | |/ / _)' _` | |_| _) +oOo oOO| | | | | (| | | (_) | _) :_ + OoOoO ._, ._:_:_,\_._, .__,_:_, \___) + Telesto 0.16.1~b1fa7c5 +``` + +Note that the Nginx server starts but waits for a connection. +In order to connect to it, we need to pass a port mapping, similar to [`docker` commands](https://docs.docker.com/network/#published-ports): + +```bash +kraft run -W -p 8080:80 unikraft.org/nginx +``` +``` + i using arch=x86_64 plat=qemu +[+] pulling unikraft.org/nginx +Powered by +o. .o _ _ __ _ +Oo Oo ___ (_) | __ __ __ _ ' _) :_ +oO oO ' _ `| | |/ / _)' _` | |_| _) +oOo oOO| | | | | (| | | (_) | _) :_ + OoOoO ._, ._:_:_,\_._, .__,_:_, \___) + Telesto 0.16.1~b1fa7c5 +``` + +With the command above we mapped the local port `8080` to the internal Nginx Unikraft port `80`: +Query the server to get the index page: + +```bash +curl localhost:8080 +``` +``` + + + + Hello, world! + + +

Hello from Unikraft!

+ +

This message shows that your installation appears to be working correctly.

+ +

For more examples and ideas, visit Unikraft's Documentation.

+ + +``` + +This a local port mapping, running on `localhost`. +For an IP-based network connection, you can use a bridged interface. +First create a bridge interface, as `root` (prefix with `sudo` if required): + +```bash +sudo kraft run --network bridge:virbr0 unikraft.org/nginx +``` +``` + i using arch=x86_64 plat=qemu +[+] pulling unikraft.org/nginx +en1: Interface is up +Powered by +o. .o _ _ __ _ +Oo Oo ___ (_) | __ __ __ _ ' _) :_ +oO oO ' _ `| | |/ / _)' _` | |_| _) +oOo oOO| | | | | (| | | (_) | _) :_ + OoOoO ._, ._:_:_,\_._, .__,_:_, \___) + Telesto 0.16.1~b1fa7c5 +``` + +The IP address used is typically the first available address (`172.44.0.2`, if that doesn't work try `172.44.0.3`) so we query the server using that address: + +```bash +curl 172.44.0.2 +``` +``` + + + + Hello, world! + + +

Hello from Unikraft!

+ +

This message shows that your installation appears to be working correctly.

+ +

For more examples and ideas, visit Unikraft's Documentation.

+ + +``` + +## The Application Catalog Repository + +Application in the registry are built and packaged using the recipes and source code files in the [`catalog` repository](https://github.com/unikraft/catalog). +The `catalog` repository contains two types of entries: + +- end-user applications in the [`library/` directory](https://github.com/unikraft/catalog/tree/main/library) +- examples in the [`examples/` directory](https://github.com/unikraft/catalog/tree/main/examples) + +The end-user appplications are built, packaged and published periodically in the registry. + +We present the steps to building application and running them locally for: + +- [the `nginx/1.25` application](https://github.com/unikraft/catalog/tree/main/library/nginx/1.25) +- [the `http-go1.21` example](https://github.com/unikraft/catalog/tree/main/examples/http-go1.21) + +Both are running in binary-compatibility mode. + +For building and running binary-compatible applications, [KraftKit](https://github.com/unikraft/kraftkit) and [Docker](https://docs.docker.com/engine/install/) are required. + +### NGINX + +Follow the steps below to build and run the NGINX binary-compatible application: + +1. Set up the BuildKit container, if not already running: + + ```bash + docker run -d --name buildkitd --privileged moby/buildkit:latest + export KRAFTKIT_BUILDKIT_HOST=docker-container://buildkitd + ``` + +1. Clone the `catalog` repository: + + ```bash + git clone https://github.com/unikraft/catalog + ``` + +1. Enter the application directory: + + ```bash + cd catalog/library/nginx/1.25 + ``` + +1. Build the application: + + ```bash + kraft build --plat qemu --arch x86_64 + ``` + +1. Run the application with `8080:80` port mapping.: + + ```bash + kraft run -W -p 8080:80 . + ``` + +1. Query the unikernel instance: + + ```bash + curl http://localhost:8080 + ``` + +If, instead of port mapping, you want to use bridge networking, follow the steps: + +1. As `root` (prefix with `sudo` if required), create a network interface bridge: + + ```bash + kraft net create -n 172.44.0.1/24 virbr0 + ``` + +1. Run as `root` (prefix with `sudo` if required): + + ```bash + kraft run --network bridge:virbr0 . + ``` + +1. Query the unikernel instance: + + ```bash + curl http://172.44.0.2 + ``` + +To close the running `kraft` instance, remove the corresponding `kraft` process. +Run, as `root` (prefix with `sudo` if required): + +```bash +kraft rm --all +``` + +### HTTP Go Server + +The NGINX build / run uses a feature called "embedded initrd", that embeds and initial ramdisk with the kernel. +The initial ramdisk contains with the NGINX application binary and depending libraries. +This is generally the case for end-user applications, located in the [`library/` directory](https://github.com/unikraft/catalog/tree/main/library). + +Another approach is to use a `base` image that is not embedding an actual application. +This is the case for examples, located in the [`examples/` directory](https://github.com/unikraft/catalog/tree/main/examples). +The application / example is then passed via an initial ramdisk. +One such example is the [`http-go1.21` example](https://github.com/unikraft/catalog/tree/main/examples/http-go1.21). +Follow the steps below to build and run the example: + +1. Enter the example directory: + + ```bash + cd catalog/examples/http-go1.21 + ``` + +1. Run: + + ```bash + kraft run -W -p 8080:8080 . + ``` + +1. Query the unikernel instance: + + ```bash + curl http://localhost:8080 + ``` + +If, instead of port mapping, you want to use bridge networking, follow the steps: + +1. As `root` (prefix with `sudo` if required), create a network interface bridge: + + ```bash + kraft net create -n 172.44.0.1/24 virbr0 + ``` + +1. Run as `root` (prefix with `sudo` if required): + + ```bash + sudo KRAFTKIT_BUILDKIT_HOST=docker-container://buildkitd kraft run --network bridge:virbr0 . + ``` + +1. Query the unikernel instance: + + ```bash + curl http://172.44.0.2:8080 + ``` + +To close the running `kraft` instance, remove the corresponding `kraft` process. +Run, as `root` (prefix with `sudo` if required): + +```bash +kraft rm --all +``` diff --git a/src/components/mdx-components/codeblock/highlight.tsx b/src/components/mdx-components/codeblock/highlight.tsx index 81987576..e2106195 100644 --- a/src/components/mdx-components/codeblock/highlight.tsx +++ b/src/components/mdx-components/codeblock/highlight.tsx @@ -19,6 +19,10 @@ import 'prismjs/components/prism-clike' import 'prismjs/components/prism-docker' import 'prismjs/components/prism-yaml' import 'prismjs/components/prism-diff' +import 'prismjs/components/prism-rust' +import 'prismjs/components/prism-python' +import 'prismjs/components/prism-toml' +import 'prismjs/components/prism-json' const RE = /{([\d,-]+)}/