Optimizing with multithreading

[AmmarRashed]

Hello,
Loved the repo so I wanted to leave this script here in case someone needs it to run your script on a specific video. Normally, your code would take about 8fps. With the multithreading implementation below it goes up to 50-60 fps, and finally outputs a pandas dataframe.

[abdullah-2022-skipq]

@AmmarRashed Hi, Appreciate the contribution, it helped me learn threading concepts. May I ask what specs did you test your code on? I have to run this on videos ranging from 50-70 minutes, currently testing it on 16 GB RAM, and it crashes the RAM within a couple of minutes.

Not sure if it can be improved, but I'm looking for ways to optimize this for longer videos.

[AmmarRashed]

@abdullah-2022-skipq Hi, I think you can reduce the batch-size, and if it still runs out of memory, you can increase the batch size and remove the parallelization part. I remember testing with different configurations. Longer videos are trickier. The fastest I got eventually was using this method> https://github.com/AmmarRashed/EmoNetTCN/blob/9bf8d5c559323980242fe98663e4f4c5888b788c/face_detection.py#L44

The trick is to skip every other frame effectively reducing the FPS of the video, and avoiding the serialization overhead of multithreading. Worked quite well.

[abdullah-2022-skipq]

@AmmarRashed Hi. Thank you very much for the response. I tried reducing the batch size to 2, take only one frame per second instead of all or every other along with numerous other approaches but the RAM still crashed for longer videos (60+ minutes). I think this is due to the fact that the entire processing is being done in memory and memory is cleaned only after the execution from all threads is completed.

What I have done now is that I save the frames as jpgs and then I run the gaze detection code (not from this repo, it's based on face_recognition python package - I'll try on this one as well if the accuracy I'm currently getting is not as good as this one) on these jpgs in multiple threads.

The original code without multi-threading or preprocessing took around 65-75 minutes on an hour long video, now it takes 20 minutes without multi-threading and 7 minutes on multithreading with 4 threads.