BU_detection

Object detection, 3D detection, and pose estimation using center point detection:

Objects as Points,
Xingyi Zhou, Dequan Wang, Philipp Krähenbühl,
arXiv technical report (arXiv 1904.07850)
Github

Main results

Object Detection on COCO validation

Backbone	AP / FPS	Flip AP / FPS	Multi-scale AP / FPS
Hourglass-104	40.3 / 14	42.2 / 7.8	45.1 / 1.4
DLA-34	37.4 / 52	39.2 / 28	41.7 / 4
ResNet-101	34.6 / 45	36.2 / 25	39.3 / 4
ResNet-18	28.1 / 142	30.0 / 71	33.2 / 12

Keypoint detection on COCO validation

Backbone	AP	FPS
Hourglass-104	64.0	6.6
DLA-34	58.9	23

3D bounding box detection on KITTI validation

Backbone	FPS	AP-E	AP-M	AP-H	AOS-E	AOS-M	AOS-H	BEV-E	BEV-M	BEV-H
DLA-34	32	96.9	87.8	79.2	93.9	84.3	75.7	34.0	30.5	26.8

All models and details are available in our Model zoo.

Installation

Please refer to INSTALL.md for installation instructions.

Use CenterNet

We support demo for image/ image folder, video, and webcam.

First, download the models (By default, ctdet_coco_dla_2x for detection and multi_pose_dla_3x for human pose estimation) from the Model zoo and put them in CenterNet_ROOT/models/.

You can add --debug 2 to visualize the heatmap outputs. You can add --flip_test for flip test.

To use this CenterNet in your own project, you can

import sys
CENTERNET_PATH = /path/to/CenterNet/src/lib/
sys.path.insert(0, CENTERNET_PATH)

from detectors.detector_factory import detector_factory
from opts import opts

MODEL_PATH = /path/to/model
TASK = 'ctdet' # or 'multi_pose' for human pose estimation
opt = opts().init('{} --load_model {}'.format(TASK, MODEL_PATH).split(' '))
detector = detector_factory[opt.task](opt)

img = image/or/path/to/your/image/
ret = detector.run(img)['results']

ret will be a python dict: {category_id : [[x1, y1, x2, y2, score], ...], }

Preparation

• python src/ave_var.py #get the mean and variance of the input images and fill them in BU.py(line 16&18)

• python src/merge_json.py #get the json files , which will generated in /data/BU/annotations/

• add our own dataset BU in (already set) opts.py & dataset_factory.py

• change the num_class in BU.py

Train Network

python main.py --exp_id coco_res18 --batch_size 5 --lr 1.25e-4 --gpus 0 --load_model ../exp/ctdet/coco_res18/model_best.pth

The network applied now is the res_18, which could be changed in opts.py to gain higher percision.

Detection

python demo.py --demo ../data/JPEGImages/ --load_model ../exp/ctdet/coco_res18/model_best.pth

The output images will be saved in imaged_rj or imaged

Test

 python test.py --load_model ../exp/ctdet/coco_res18/model_best.pth --non_prefetch_test

ONNX & TensorRT

For the project's application in C++ version(maybe...), the format of ret could be changed. There are two formats of ret, which could be chosen in msra_resnet.py

Benchmark Evaluation and Training

After installation, follow the instructions in DATA.md to setup the datasets. Then check GETTING_STARTED.md to reproduce the results in the paper. We provide scripts for all the experiments in the experiments folder.

Develop

If you are interested in training CenterNet in a new dataset, use CenterNet in a new task, or use a new network architecture for CenterNet, please refer to DEVELOP.md. Also feel free to send us emails for discussions or suggestions.

Third-party resources

• Keras Implementation: keras-centernet from see--.
• CenterNet + DeepSORT tracking implementation: centerNet-deep-sort from kimyoon-young.
• Blogs on training CenterNet on custom datasets (in Chinese): ships from Rhett Chen and faces from linbior.