1.1 Installation guide

Introduction

This is the installation guide to set up the development environment.

Hardware requirements

The entire setup requires a lot of computing power to run the tools and OMA-ML at the same time in a developer setting. It is recommended to have at least a Quad-core (Physical cores) CPU and 16GB of RAM. Most components of OMA-ML can run on Windows 10/11, Mac OS, or Linux, but the frontend component is dependent on Visual Studio IDE which is currently only available for Windows or Mac OS.

Required Tools

IDE

Frontend development:

• Visual Studio 2022 Community. Download link: https://visualstudio.microsoft.com/de/downloads/

Windows

When installing Visual Studio, the following "workload" packages are required:

• ASP.NET and web development
• Desktop development with C++ (installs C++ compiler for some Adapters)

MS SQL connection issues

The ASP.NET and web development package will install the localDB (MS SQL Database), which is required by the frontend component. In case the MS SQL Database does not install correctly using Windows 11 in combination with specific SSD types, manually install Microsoft SQL Server 2019 Express on another Hard drive. Download link: https://www.microsoft.com/de-de/download/details.aspx?id=101064

If you need to use the manually installed database, the connection string located here: frontend/src/Server/BlazorBoilerplate.Server/appsettings.json must be changed from:

• (localdb)\MSSQLLocalDB -> localhost\SQLEXPRESS In such cases be aware not to submit the appsettings.json when making commits.

Mac

Visual Studio does not offer an integrated MS SQL database. We provide a docker-compose development set up that install MS SQL, Mongo DB and redis.

The development docker-compose can be found at the route of the MetaAutoML project, to start the development services open a terminal inside the MetaAutoML folder and execute: docker-compose -f docker-compose-development.yml up

Whenever you want to debug, this docker-compose must run!

Before the frontend and controller can connect to the development docker-compose services, you must set a few environment variables:

Change the default controller launch config:

To:

Please change the settings back to default when making a pull request!!

Change the launch setting of the frontend:

To the correct path on your system to the controllers app-data/datasets path

Backend development

• Visual Studio Code. Download link: https://code.visualstudio.com/Download
• Extensions: Python, Pylance, Docker, autoDocstring, vscode-proto3, Stardog RDF Grammars

Using Development Containers for Ubuntu only

Visual Studio Code has an extension to work with Docker containers called Dev Containers. We can use this to create a development environment that contains all necessary tools to run the application. Microsoft already has a predefined container for Python that we can use. Follow the tutorial here: code.visualstudio.com with the following alternative steps:

• pick Python 3 as the programming language and choose version 3.9
• skip the node.js installation
• select features for the Dev Container: git, Docker (Moby) Support, reuse host Docker Engine (Docker-from.Docker)
  • we want to use the host Docker engine so that the MetaAutoML containers are visible from the host

When the container setup is done, you should be able to run the hello world example from docker in the VSCode terminal : docker run hello-world. Use a terminal outside of VSCode to check if you can see the container: docker container ls -a. If you find a container using the image hello-world everything works. Launch the MetaAutoML application with docker-compose -f docker-compose.yml -f docker-compose-frontend.yml up. Check whether you can see all containers on your host machine: docker container ls -a. The output should look similar to this:

CONTAINER ID   IMAGE                                            ...
1d72a8c08ed8   hello-world                                      ...
b06ede0fc46f   omaml_blazorboilerplate                          ...
edbb1088bf1f   omaml_controller                                 ...
c307cd71a7c3   redis:alpine                                     ...
32ca7efd34f9   mcr.microsoft.com/mssql/server:2019-latest       ...
9619abf72be9   omaml_sklearn                                    ...
fd720fe65673   omaml_flaml                                      ...
5da1569de508   omaml_mljar                                      ...
3dad8d4b180f   mongo                                            ...
3a8f4830e29f   omaml_mcfly                                      ...
094bbab3fb36   omaml_gluon                                      ...
4e520b0f0e0f   omaml_autokeras                                  ...
8f217ec8dccd   vsc-omaml-e5c48446cdb7952601ad42e9576b95e9-uid   ...
6c86c22f0a6f   moby/buildkit:buildx-stable-1                    ...

We need to find the IP address of the front-end container:

• inspect the docker network with docker network inspect omaml_default
• find the container omaml_blazorboilerplate_1 and copy the IPv4Address into your browser (eg. 172.19.0.12, no port needed)
• if the login page appears, everything works

To complete the setup you can install other VSCode extensions inside your Dev Container without affecting other workspaces.

NOTE: tested on Linux (Ubuntu 22.04), does not work on Windows (WSL2) and probably MacOS out of the box. Some Network configuration required, see this or this.

Misc

The desktop versions are recommended, but the CLI-only version may also be used for docker and git:

• Docker Desktop. Download link: https://www.docker.com/products/docker-desktop/
• Github Desktop. Download link: https://desktop.github.com/
• Python, recommended version 3.9.X: Download link: https://www.python.org/downloads/. Disable maximum path limit when prompted! Important!!
• Redis server. Download link: https://github.com/microsoftarchive/redis/releases
• MongoDB Community Server edition. Installation link: https://www.mongodb.com/try/download/community?tck=docs_server. During the installation wizard select Complete and Install MongoD as a Service when prompted. The rest can be left to default. The installer will also install MongoDBCompass, the client software to inspect MongoDB (Not required but nice to have).

Clone the repository

Open GitHub Desktop and select Clone a repository from the Internet.... Within the new dialog select the URL tab, and insert the repository URL: https://github.com/hochschule-darmstadt/MetaAutoML.git. It is recommended to select a short local path to avoid any issues with long paths within Python (clone the repository on your desktop for example).

Adapter-specific installation issues

Currently unsupported adapters, only necessary if one develop the specific adapter.

AutoCVE Info

When setting up AutoCVE, you must manually install NumPy before installing the requirements.txt. Do this by executing the command pip install numpy.

AutoPytorch

You need to download swig (https://swig.org/download.html) and set the PATH variable accordingly before installing the dependencies. (Info: currently auto Pytorch isn't supported by windows)

AlphaD3M

Important, a Python 3.8.x env is required.

"Windows debug setup not successful yet"

On Windows, you must manually install SciPy: download scipy‑1.4.1‑cp35‑cp35m‑win_amd64.whl. Depending on the venv, you might need to rename the tags. Check compatible tags using: pip debug --verbose. And rename the file appropriately. Then execute it in the venv of AlphaD3M pip install scipy-1.4.1-py35-none-win_amd64.whl.

Using ubuntu, install sudo apt-get install python3.x-dev and python 3.8 and setup a venv in vscode and follow the dockfile installation.