Skip to content

Latest commit

 

History

History
310 lines (246 loc) · 13.5 KB

running_video_analytics_serving.md

File metadata and controls

310 lines (246 loc) · 13.5 KB
Raw

Running Intel(R) DL Streamer Pipeline Server

| Intel(R) DL Streamer Pipeline Server Microservice | Interacting with the Microservice | Real Time Streaming Protocol (RTSP) Re-streaming | Selecting Pipelines and Models at Runtime | Developer Mode | Enabling Hardware Accelerators |

Intel(R) Deep Learning Streamer (Intel(R) DL Streamer) Pipeline Server docker images can be started using standard docker run and docker compose commands. For convenience a simplified run script is provided to pass common options to docker run such as proxies, device mounts, and to expose the default microservice port (8080).

The following sections give an overview of the way the image is run as well as common customization and interaction patterns.

Note: Descriptions and instructions below assume a working knowledge of docker commands and features. For more information see docker documentation.

Intel(R) DL Streamer Pipeline Server Microservice

The default image of the Pipeline Server includes a microservice that exposes a RESTful interface on port 8080. The microservice has endpoints to list, start, stop, and get the status of media analytics pipelines.

Microservice Endpoints

Path Description
GET /models Return supported models.
GET /pipelines Return supported pipelines.
GET /pipelines/{name}/{version} Return pipeline description.
POST /pipelines/{name}/{version} Start new pipeline instance.
GET /pipelines/{name}/{version}/{instance_id} Return pipeline instance summary.
GET /pipelines/{name}/{version}/{instance_id}/status Return pipeline instance status.
DELETE /pipelines/{name}/{version}/{instance_id} Stops a running pipeline or cancels a queued pipeline.

Default Run Commands and Image Names

Command Media Analytics Base Image Image Name Description
./docker/run.sh Intel(R) DL Streamer docker file dlstreamer-pipeline-server-gstreamer Intel(R) DL Streamer based microservice with default pipeline definitions and deep learning models. Exposes port 8080. Mounts the host system's graphics devices.
./docker/run.sh --framework ffmpeg FFmpeg Video Analytics docker file dlstreamer-pipeline-server-ffmpeg FFmpeg Video Analytics based microservice with default pipeline definitions and deep learning models. Mounts the graphics devices.

Interacting with the Microservice

The following examples demonstrate how to start and issue requests to a Pipeline Server Microservice either using the Intel(R) DL Streamer based image or the FFmpeg based image.

Note: The following examples assume that the Video Analytics Serving image has already been built. For more information and instructions on building please see the Getting Started Guide or Building Intel(R) DL Streamer Pipeline Server Docker Images

Note: Both the Intel(R) DL Streamer based microservice and the FFmpeg based microservice use the same default port: 8080 and only one can be started at a time. To change the port please see the command line options for the Pipeline Server microservice.

Starting the Microservice

To start the microservice use standard docker run commands via the provided utility script.

Intel(R) DL Streamer based Microservice

Example:

docker/run.sh -v /tmp:/tmp

FFmpeg Video Analytics based Microservice

Example:

docker/run.sh --framework ffmpeg -v /tmp:/tmp

Issuing Requests

From a new shell use curl to issue requests to the running microservice.

Getting Loaded Pipelines

Example:

Note: In this example we assume you are running FFmpeg Video Analytics based Microservice

curl localhost:8080/pipelines
[
  {
    "description": "Object Detection Pipeline",
    "name": "object_detection",
    <snip>
    "type": "FFmpeg",
    "version": "1"
  },
  {
    "description": "Emotion Recognition Pipeline",
    "name": "emotion_recognition",
    <snip>
    "type": "FFmpeg",
    "version": "1"
  }
]

Detecting Objects in a Sample Video File

Example:

Note: In this example we assume you are running Intel(R) DL Streamer based Microservice

curl localhost:8080/pipelines/object_detection/person_vehicle_bike -X POST -H \
'Content-Type: application/json' -d \
'{
  "source": {
    "uri": "https://github.com/intel-iot-devkit/sample-videos/blob/master/person-bicycle-car-detection.mp4?raw=true",
    "type": "uri"
  },
  "destination": {
    "metadata": {
      "type": "file",
      "path": "/tmp/results.txt",
      "format": "json-lines"
    }
  }
}'
tail -f /tmp/results.txt

{"objects":[{"detection":{"bounding_box":{"x_max":0.0503933560103178,"x_min":0.0,"y_max":0.34233352541923523,"y_min":0.14351698756217957},"confidence":0.6430817246437073,"label":"vehicle","label_id":2},"h":86,"roi_type":"vehicle","w":39,"x":0,"y":62}],"resolution":{"height":432,"width":768},"source":"https://github.com/intel-iot-devkit/sample-videos/blob/master/person-bicycle-car-detection.mp4?raw=true","timestamp":49250000000}

Detection results are published to /tmp/results.txt.

Stopping the Microservice

To stop the microservice use standard docker stop or docker kill commands with the name of the running container.

Intel(R) DL Streamer based Microservice

Example:

docker stop dlstreamer-pipeline-server-gstreamer

FFmpeg Video Analytics based Microservice

Example:

docker stop dlstreamer-pipeline-server-ffmpeg

Real Time Streaming Protocol (RTSP) Re-streaming

Note: RTSP Re-streaming supported only in Intel(R) DL Streamer based Microservice.

The Pipeline Server contains an RTSP server that can be optionally started at launch time. This allows an RTSP client to connect and visualize input video with superimposed bounding boxes.

Enable RTSP in service

docker/run.sh --enable-rtsp

Note: RTSP server starts at service start-up for all pipelines. It uses port 8554 and has been tested with VLC.

Connect and visualize

Note: Leverage REST client when available.

  • Start a pipeline with curl request with frame destination set as rtsp and custom path set. For demonstration, path set as person-detection in example request below.
curl localhost:8080/pipelines/object_detection/person_vehicle_bike -X POST -H \
'Content-Type: application/json' -d \
'{
  "source": {
    "uri": "https://github.com/intel-iot-devkit/sample-videos/blob/master/person-bicycle-car-detection.mp4?raw=true",
    "type": "uri"
  },
  "destination": {
    "metadata": {
      "type": "file",
      "path": "/tmp/results.txt",
      "format": "json-lines"
    },
    "frame": {
      "type": "rtsp",
      "path": "person-detection"
    }
  }
}'
  • Check that pipeline is running using status request before trying to connect to the RTSP server.
  • Re-stream pipeline using VLC network stream with url rtsp://localhost:8554/person-detection.

RTSP destination params.

Note: If the RTSP stream playback is choppy this may be due to network bandwidth. Decreasing the encoding-quality or increasing the cache-length can help.

"frame": {
  "type": "rtsp",
  "path" : <custom rtsp path>(required. When path already exists, throws error),
  "cache-length": (default 30) number of frames to buffer in rtsp pipeline.
  "encoding-quality": (default 85): jpeg encoding quality (0 - 100). Lower values increase compression but sacrifice quality.
  "sync-with-source": (default True) process media at the encoded frame rate (e.g. 30 fps)
  "sync-with-destination": (default True) block processing pipeline if rtsp pipeline is blocked.
}

Selecting Pipelines and Models at Runtime

The models and pipelines loaded by the Pipeline Server microservice can be selected at runtime by volume mounting the appropriate directories when starting the container.

Note: Mounted pipeline definitions must match the media framework supported in the media analytics base image.

Note: Pipeline and Model directories are only scanned once at service start-up. To make modifications the service must be restarted.

Mounting Pipelines and Models into a Intel(R) DL Streamer based Image

Example:

./docker/run.sh --framework gstreamer --pipelines /path/to/my-pipelines --models /path/to/my-models

Enabling Hardware Accelerators

The run script automatically gives docker access (i.e. device, volume mount and device cgroup rule) to the following accelerators

  • iGPU
  • Intel® Neural Compute Stick 2 (Intel® NCS2)
  • HDDL-R cards

You also need to specify the inference device in the parameters section of the Pipeline Server request. Example for GPU below

"parameters": {
   "device": "GPU"
}

See Customizing Pipeline Requests for more information.

The following the table shows docker configuration and inference device name for all accelerators.

Note: Open Visual Cloud base images only support the GPU accelerator. OpenVINO™ base images support all accelerators.

Accelerator Device Volume Mount(s) CGroup Rule Inference Device
GPU /dev/dri GPU
Intel® NCS2 /dev/bus/usb c 189:* rmw MYRIAD
HDDL-R /var/tmp, /dev/shm HDDL

Note: Intel® NCS2 and HDDL-R accelerators are incompatible and cannot be used on the same system.

GPU

The first time inference is run on a GPU there will be a 30s delay while OpenCL kernels are built for the specific device. To prevent the same delay from occurring on subsequent runs a model instance id can be specified in the request.

On Ubuntu20 and later hosts extra configuration, not shown in the above table, is necessary to allow access to the GPU. The docker/run.sh script takes care of this for you, but other deployments will have to be updated accordingly.

Intel® NCS2

Configure your host by following the steps outlined in the OpenVINO™ documentation

Note: These steps require the file 97-myriad-usbboot.rules which can be extracted from the Pipeline Server docker container using the following command:

./docker/run.sh -v ${PWD}:/tmp --entrypoint cp --entrypoint-args "/opt/intel/openvino/inference_engine/external/97-myriad-usbboot.rules /tmp"

Once extracted the file will be in the current directory. Follow the instructions given in the OpenVINO™ documentation to copy it to the correct location.

HDDL-R

Configure your host by downloading the HDDL driver package then installing dependencies and run the hddldaemon on the host as per the HDDL install guide.

The HDDL plug-in in the container communicates with the daemon on the host, so the daemon must be started before running the container.

Developer Mode

The run script includes a --dev flag which starts the container in "developer" mode. "Developer mode," sets docker run options to make development and modification of media analytics pipelines easier.

Note: Pipeline and Model directories are only scanned once at service start-up. When making modifications to the models, pipelines, or source code the service must be restarted for them to take effect.

Developer mode:

  • Starts the container with an interactive bash shell.
  • Volume mounts the local source code, models and pipelines directories. Any changes made to the local files are immediately reflected in the running container.
  • Uses the docker option --network=host. All ports and network interfaces for the host are shared with the container.
  • Uses the docker option --privileged. Operates the container with elevated privileges.

Intel(R) DL Streamer based Image in Developer Mode

Example:

docker/run.sh --dev
pipeline-server@my-host:~$ python3 -m pipeline-server

* Other names and brands may be claimed as the property of others.