how to inference based multisports pretrained model ??

[Eurususu]

I used given link https://download.openmmlab.com/mmaction/v1.0/detection/slowfast/slowfast_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/slowfast_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-af666368.pth to download Multisports model .

Then I wannna inference with the model, command like:

python3 demo/demo_spatiotemporal_det.py --config configs/detection/slowfast/slowfast_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb.py --checkpoint weights/slowfast_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb_20230320-af666368.pth --det-config demo/demo_configs/faster-rcnn_r50_fpn_2x_coco_infer.py --det-checkpoint weights/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.45 --action-score-thr 0.5 --label-map tools/data/multisports/label_map.txt --predict-stepsize 8 --output-stepsize 4 --output-fps 8 data/multisports/test/basketball/v_4Aoy9-NoY2I_c600.mp4 work_dirs/slowfast_kinetics400-pretrained-r50_8xb16-4x16x1-8e_multisports-rgb/test.mp4

The result I get is a mess!

But I have seen in multisports official webside, the result in video is

so how to inference video using model to get the simple result? I have tried change detection and action classfication threshold.But it didn't helpful.

[cir7]

Hi, @Eurususu
The SlowFast-Det model takes a two-stage action detection, which would recognize the action of each person in the video. That's why your inference result has so many detected actions, using a higher threshold for classification would make it simpler.
The below image is from the official website and looks like a visualization of ground truth.
Basketball-related actions are pretty hard in the dataset, you can refer to the table in the table from paper:

aerobic videos have fewer people, related actions are easier, and have a better performance, you can try on it.

[Eurususu]

Thanks for answering!!

I have tried higher threshold, the boxes is less, but I still can't get the result as showed on multisports official.

This is threshold 0.9 for action detection.

what can I do to get the good result?

[Eurususu]

@cir7

[cir7]

The video from the official website looks like a visualization of annotation, the model can't achieve the performance.