OpenVINO™ Model Server 2023.2

The 2023.2 is a major release with several new features and improvements.

Changes

• Updated OpenVINO backend to version 2023.2.
• MediaPipe framework has been updated to the current latest version 0.10.3.
• Model API used in the OpenVINO Inference MediaPipe Calculator has been updated and included with all its features.

New Features

• Introduced extension of KServe gRPC API with a stream on input and output. That extension is enabled for the servables with MediaPipe graphs. MediaPipe graph is persistent in the scope of the user session. That improves processing performance and supports stateful graphs – for example tracking algorithms. It also enables the use of source calculators. Check more details. 
• Added a demo showcasing gRPC streaming with MediaPipe graph. Check more details.
• Added parameters for gRPC quota configuration and changed default gRPC channel arguments to add rate limits. It will minimize the risks of impact of the service from uncontrolled flow of requests. Check more details.
• Updated python clients requirements to match wide range of python versions from 3.7 to 3.11

Breaking changes

No breaking changes.

Bug fixes

• Handling situation when MediaPipe graph is being added with the same name as previously loaded DAG.
• Fixed returned HTTP status code when MediaPipe graph/DAG is not loaded yet. (previously 404, now 503)
• Corrected error message returned via HTTP when using method other than GET for metadata endpoint - "Unsupported method".

You can use an OpenVINO Model Server public Docker image's based on Ubuntu via the following command:
docker pull openvino/model_server:2023.2  - CPU device support with the image based on Ubuntu20.04
docker pull openvino/model_server:2023.2-gpu - GPU and CPU device support with the image based on Ubuntu22.04
or use provided binary packages.
The prebuilt image is available also on RedHat Ecosystem Catalog