Motivation

Some of our collaborators work with human pose estimation, using MMPose: OpenMMLab Pose Estimation Toolbox and Benchmark.

The would like to import the predicted poses into movement, to leverage our filtering + analysis (and in future, visualisation) features.

Format

The pose predictions are done by an MMPoseInferencer class, the usage and outputs of which are described here. The results are then serialised into json for storage, see code here.

This is an example of how the outputs look like:

{
    "frame_id": 0,
    "instances": [
      {
        "keypoints": [
          [
            961.4871847546324,
            39.44995934640923
          ],
          [
            787.3275313021991,
            255.6230511152969
          ],
        ],
        "keypoint_scores": [
          0.009758014231920242,
          0.005633828230202198,
        ],
        "bbox": [
          [
            496.4014892578125,
            0.298583984375,
            1016.2412109375,
            720.661376953125
          ]
        ],
        "bbox_score": 0.863023042678833
      }
    ]
  },
  {
    "frame_id": 1,
    "instances": [
      {
        "keypoints": [

What we can do

Since movement's data structure already supports keypoints and associated confidence scores, its should be easy to write functions that read/write the "keypoints" and "keypoint_scores" information for all instances and frames from/to such json files.

The public-facing functions can be named load_poses.from_mmpose_file() and save_poses.to_mmpose_file() to conform with our existing naming conventions.

The tricky bit is what to do about the bounding boxes. movement currently doesn't support those (but future support is planned, see #167). Maybe in the first instance we can only care about the keypoints, and think about the bboxes, at a later stage.