Question about generating result jitter

[cyk990422]

Hi, thank you so much for sharing such an interesting job. May I ask if the results generated by your VAE in the early stage of training, such as 1000-2000 epochs, will have motion jitter, and do you need to add some smoothing filters such as euros?

[ChenFengYe]

Hi, thanks for your attention. I have not added such euro filters, but it could help. You can refer to
https://github.com/mkocabas/VIBE/blob/851f779407445b75cd1926402f61c931568c6947/demo.py#L229

For our implementation, we just trained the VAE part much longer for better motion quality. You can set it to 3000 or 4000 epochs.

[cyk990422]

Thank you for your timely reply. Have you tried to use only rot6D as input for training? Is there any motion jitter?

[ChenFengYe]

I have tried rot6d in the early research, which is much smooth compared to the rotation vector.
Rot6d even is the greatest representation for the motion-only generation, I believe, to avoid motion jitter.
However, it could cause global artifacts, like foot-sliding on the ground. If you do not care about that, rot6d is great.

[cyk990422]

Thank you very much for your patience, I will try it