Architecture
API access must be possible for multiple users at the same time, so a request system is implemented. To carry out the prediction and training operations, a request is generated on the system, which will respond with an ID generated to refer to the request, in order to respond to the client as soon as the request has been correctly received. It is necessary to use a request-system also to make sure that operations are not blocking the user pending the result of the prediction or training.
For each request, a directory is created, divided into additional sub-directories specific to the individual QA-Integrator-Model required, which will be used for all the necessary files, in a sandbox-like environment. Each prediction or training request is processed, the directories described above are created and a thread is created for each model indicated by the request. These threads are assigned to a ThreadPoolExecutor specifically configured based on the resources of the system on which it is run, in order to optimize resources and therefore performance.
This is the block diagram of the QA-engine core:
Exposes the system functions externally, checks the correctness of the incoming data, executes the necessary requests through the QA-integrator, and returns the results of these requests to the end user.
The core of the system, integrates the logic to coordinate the various additional modules (QA-Integrator-Model), recalling them when necessary and transmitting the necessary data to them. Subsequently, it also takes care of taking the responses obtained from the modules and returning them via the HTTP API.
Modules added to the system to add support to a specific QA algorithm/model. They are python modules that must guarantee the exchange of information with the QA-integrator, they are written ad-hoc to convert the data from the system format to the specific format of the algorithm (and vice versa for the answers) and then call the necessary external scripts of the algorithm.
Question&Answering algorithm which the system should support. Its operation must be independent by the Engine, and must be callable via some script. The scripts necessary to start the prediction and training operations are those that must be called up via the QA-Integrator-Model.
Each request for prediction and training by the user will have an own lifecycle. Each request can be deleted manually through the API or it will be deleted automatically after a certain period of time from the completion of the operation by this cronjob.