README.md

2s-KAGCN

Two-Stream Adaptive Kolmogorov-Arnold Network Graph Convolutional Networks for Skeleton-Based Action Recognition

Note

This is based on 2s-AGCN repository. The main innovation has been the introduction of a new backbone, the so-called KAN network. We used the efficient-Kan version, but in the future more efficient models could be introduced. Wait for those in the next months!

Data Preparation

• Download the raw data from NTU-RGB+D and Skeleton-Kinetics. Then put them under the data directory:

   -data\  
     -kinetics_raw\  
       -kinetics_train\
         ...
       -kinetics_val\
         ...
       -kinetics_train_label.json
       -keintics_val_label.json
     -nturgbd_raw\  
       -nturgb+d_skeletons\
         ...
       -samples_with_missing_skeletons.txt
  

• Preprocess the data with

  python data_gen/ntu_gendata.py

  python data_gen/kinetics-gendata.py.

• Generate the bone data with:

  python data_gen/gen_bone_data.py

Training & Testing

Change the config file depending on what you want.

`python main.py --config ./config/nturgbd-cross-view/train_joint.yaml`

`python main.py --config ./config/nturgbd-cross-view/train_bone.yaml`

To ensemble the results of joints and bones, run test firstly to generate the scores of the softmax layer.

`python main.py --config ./config/nturgbd-cross-view/test_joint.yaml`

`python main.py --config ./config/nturgbd-cross-view/test_bone.yaml`

Then combine the generated scores with:

`python ensemble.py` --datasets ntu/xview

To do:

1. Complete training for the whole script and compare results with official paper.
2. Use additional models to compare results and select the best model.

Citation

Please cite the following paper if you use this repository in your reseach.

@inproceedings{2sagcn2019cvpr,  
      title     = {Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition},  
      author    = {Lei Shi and Yifan Zhang and Jian Cheng and Hanqing Lu},  
      booktitle = {CVPR},  
      year      = {2019},  
}

@article{shi_skeleton-based_2019,
    title = {Skeleton-{Based} {Action} {Recognition} with {Multi}-{Stream} {Adaptive} {Graph} {Convolutional} {Networks}},
    journal = {arXiv:1912.06971 [cs]},
    author = {Shi, Lei and Zhang, Yifan and Cheng, Jian and LU, Hanqing},
    month = dec,
    year = {2019},
}

Contact

For any questions, feel free to contact: [email protected]