Skip to content

2.3 Adapter architecture

Jump to bottom
Drewlin56 edited this page Mar 3, 2023 · 4 revisions

Introduction

The adapters folder contains the implementations of the individual AutoML solutions as well as shared functionality code between the adapters. Each adapter is composed of the following files (example from AutoKeras adapter):

Adapter file structure

The file name describes which functionality/class can be found within them and the files can be found at one of two different locations inside the adapters folder:

  • If the code is AutoML adapter specific, the file can be found inside the individual adapter. Example: adapters/autokeras/....
  • If the code is shared between all adapters it can be found inside the adapters folder. Example: adapters/Utils/...

The Visual Studio code settings.json and launch.json are configured to find and use the shared files during development/debug/run. When building the adapter dockers the shared files are copied within the AutoML adapter container.

Overview

The adapter exposes a specific port unique to the specific AutoML. Which port each adapter uses is specified within its Dockerfile/launch.json and should not be changed (as the controller has to know the port and cannot search for it).

When the adapter is running it is starting its GRPC server and is waiting for incoming connections. The adapter GRPC interface provides the following functionality to the controller:

  1. StartAutoMl: starts a new AutoML process to search for an ML pipeline
  2. GetAutoMlStatus: poll the latest status from a running AutoML process
  3. ExplainModel: Use a found model to compute it´s probabilities for the explainable AI module
  4. PredictModel: use a found model to predict a new live dataset

For a detailed interface, definition see the GRPC interface file.

Dependency-injection

The AutoML solution search process is a process that takes a long period of time. To avoid any blocking of the controller component the StartAutoMl request goal is to simply create a new training session and then return with a unique session id to the controller. Afterward, the controller can use the session id to periodically poll the adapter with the GetAutoMlStatus to get the latest information about the process.

Since incoming GRPC connections are limited to the connection state, after closing the connection the state will be lost. To permit the independent connection to poll the information after the search state, the adapter uses dependency injection to safely inject the Singleton object of the AdapterScheduler into each incoming request. The AdapterScheduler is managing the entire background threads and can quickly relay the newest messages to incoming requests without blocking as its functions are marked as async allowing them to be coroutines.

Additionally, the adapter-specific AdapterManager is injected directly inside the start_auto_ml function within the AdapterScheduler.

Functionality

Any adapter functionality is implemented as a GRPC procedure and relies on the injected AdapterScheduler instance. All the functions are coroutines and can be called by multiple threads.

start_auto_ml

When the start_auto_ml function is called within the AdapterScheduler, a new adapter-specific AdapterManager instance is injected as a function parameter. The AdapterScheduler generates a new UUID as the session id used to identifier the new AdapterManager when using other GRPC calls. Finally, the AdapterScheduler starts the AdapterManager thread and saves it inside its adapter_managers dictionary.

The AdapterManager thread performs the following steps:

  • Setup the new run environment: generate all paths and folders for this new training
  • Start the AutoML process: calls a new process that initiates the actual AutoML adapter, that performs the search using an AutoML solution. It has to run in a subprocess or else the adapter has no way to read the console and receive the information of the AutoML solution.
  • Capture the AutoML solutions process output: generates new AutoMlStatusReponse messages when the AutoML subprocess generates console output
  • Zip the script: generate the executable python script and then zip everything (the required python files and the found model)
  • Evaluate the model: use a test set to make predictions on the found model and compute the model metric
  • Generate last status message: mark this message as the last status message = AutoML process completed

get_auto_ml_status

While the AdapterManager thread is running, it generates multiple status messages that can be polled using the get_auto_ml_status function. The controller will poll until the last status message is received.

explain_model

After having received the last AutoMlStatus message the controller will start the model explanation process. The controller will call this method several times, to make predictions on sent data. The response to this request contains the probability metric, which is why it is a separate command from the predict_model function

predict_model

When the user uploaded a new live dataset, the controller will request that predictions are made with the new live dataset. The adapter will load the selected model, make predictions, and persist the result in the specified folder.

app-data

This folder represents the point of exchange between the adapter and the controller:

  • /datasets subfolder: the datasets used by the Adapter run are saved here
  • /templates subfolder: containing the jinja2 templates to generate an executable test file for a trained AutoML solution
  • /training/ subfolder: the different training sessions of the adapter are saved here

When a new training is started, the adapter will create a new subfolder inside /training with the user id of the OMA-ML user and the corresponding dataset id of the used dataset, and finally a folder with the training id. The final folder structure: app-data/training/USER_ID/DATASET_ID/TRAINING_ID.

All training-relevant folders are then created within that structure:

  • export: contains the final zip archive of the model
  • job: contains the job json passed to the subprocess
  • model: the save folder when the AutoML solution persists information on the disk
  • result: the final result folder containing the model file, and python scripts to execute the model

In docker or Kubernetes this folder is mounted as a volume for shared use between the controller and the adapters.

Jinja2

After finishing training each AutoML adapter saves the trained model. But since every AutoML solution produces different models requiring different libraries and commands to execute these models can not be used in a general execution script.

Therefore each autoML adapter generates an executable python script that loads and does a prediction with the trained model and the trained dataset. This is done with jinja2 templates which are stored within the app-data/templates folder and are assembled by the TemplateGenerator.py script in the /Utils folder in the AutoML adapter.

The TemplateGenerator.py script assembles a .py file that loads the exported model and the dataset, preprocesses the data where necessary, predicts the test data of the dataset, and saves said predictions to a prediction.csv file. Additionally, a requirements.txt file containing the necessary libraries for the execution of the prediction script is generated.

Component description

The AutoKeras adapter will be used as an example:

Located inside individual AutoML adapters

  • AutokerasServer: The adapter's main entry point initiates the adapter GRPC server.
  • Container: Collection of dependency injection containers, all dependency injected objects are initialized here.
  • AutoKerasAdapterManager: Implementation of adapter specific functionality of the AdapterManager class.
  • AutoML: main routine called as a subprocess to start the AutoML Solution search process
  • AutoKerasAdapter: Implementation of the functionality of the AutoML solution using the AbstractAdapter for shared functionality between all adapters.
  • AutoKerasParameterConfig: Config file for translating the parameters from the ontology to adapter individual.

Shared between AutoML adapters

  • AdapterService: The server implementation of the compiled adapter GRPC interface functionality, the entry point for all incoming GRPC requests is located here.
  • AdapterBGRPC: The compiled GRPC server and client interface definition for the adapters, the server is implemented in the adapters.
  • AdapterScheduler: Main manager object responsible to create new training and persistence sessions on controller requests.
  • AdapterManager: Implementation of all shared functionalities used by the incoming GRPC requests, adapter specific code is found in the inherited AutoKerasAdapterManager object.
  • AbstractAdapter: Abstract class providing the interface for all Adapter interfacing with AutoML solutions.

Shared helper functions

  • prediction_time_sources: provide shared functionality that is required by the individually generated scripts to correctly execute the new model outside of the cluster.
  • AdapterUtils: a collection of helper functions to interact with files (read dataframes) and data (convert features)
  • TemplateGenerator: script to generate the ML model execution files using the template jinja2 files
  • JsonUtils

Parameter Implementation

General

  • Translate_parameter function in AdapterUtils:
    • Input: Task, selected parameters from frontend, ParameterConfig
    • Does: looping through the possible parameters in the ParameterConfig and translating the fitting selected parameters to the adapter individual output dictionary
    • Output: dictionary with adapter individual parameter name as key and parameter individual values as value

In each adapter:

ParameterConfig File: translating the parameter names received from the frontend from to ontology name to the adapter individual name Parts of the ParameterConfig:

  • Task_config: main object of the file. Maps the tasks to the fitting config grafik
  • <Task>_config: List of parameters that the individual task can have. Each List item has the following attributes:
    • Broader_type: the broader ID for the parameter (e.g. “metric”)
    • Specific_type: the autoML specific ID for that parameter (e.g. “metric_autokeras_tabular_classification”
    • Default: the default value in case nothing is selected
    • Expected parameter count: the value is a single entry or a list of values
    • Converting type: the type of the value -> integer or dictionary (of strings/objects)
    • Lookup dictionary: the lookup dictionary to translate the given parameters (from the frontend) to the autoML solution specific parameter
    • Used name by autoML: the name of the parameter that the autoML expects grafik
  • <AutoML>_<parameter>: Lookup dictionary for parameter values
    • Why?: Individual AutoMLs want specific strings/objects given to their functions grafik

Ontology:

Clone this wiki locally