reproducing

[meryusha]

Dear team,
Thank you again for your work on the code!
I tried training SVT from scratch on Kinetics and one other dataset but didn't manage to get good results. I used the script provided in scripts/train.sh. Then I noticed that you pushed some modifications and changed the backbone to Dino.
I have a question: can I reproduce the results from the paper using the current code and scripts/train.sh? Or are there more updates to code to be done? Do you think the issues with my training were because of the wrong backbone?
Thank you very much!

[kahnchana]

Thank you for your interest. Also, we are extremely sorry for any lack of clarity in the repo.

The backbone we use is the ViT-B/16 architecture. We simply use self-supervised pre-trained weights (from DINO) for initialization of our model (as highlighted in our paper). This has a considerable impact on training and convergence.

Previously, I was loading this through the config files used during training (to allow different initializations to be tested) and now hard coded it into the model code to load the DINO weights by default.

In terms of the code, you should be able to reproduce our results with the current version of our code.

[eeehco]

Hello, have you successfully reproduced the results of the paper with a dataset other than kinesics?

[kahnchana]

Sorry, we have performed self-supervised training only with Kinetics. We were hoping to explore with SSv2 as a future direction, but did not get around to it yet.