Video Parameters Opening Black Frames

[MBRINLE1]

I have successfully used Simba with multiple projects, however, with a new dataset I am working with I am having a problem with calculating the distance.

• I am not experiencing any errors when importing the video or DLC data. However, when I go to video parameters and select calculate distance SimBA displays a completely black frame and throws no errors on the SimBA prompt window or the anaconda prompt. This has happend with > 5 videos from different animals and different days. I have previously been able to use these videos successfully when I was testing the pipeline.
• If I select random points to calculate the distance from and continue to correct for outliers, I can define ROIs (it pulls up a useable frame to define the ROI's).
• It doesn't appear like there are any black frames in the videos I am importing.

I am using simBA version 1.87.5 on a Windows 11 computer.
Anaconda Version 3.6.13

I am attaching a screen shot of what I see for the same video when I click on "calculate distance" under the "video parameters" tab and "define ROIs" under the ROIs tab.

[sronilsson]

@MBRINLE1 - thanks for reporting @MBRINLE1 I appreciate it.

First, odd timing - I was just working on this pixel-per-millimeter graphical interface 10min ago before seeing this message. It was not very well written so I updated it. Before I dig and try and replicate - can you confirm that it is also not working in the latest version of SimBA?

If you do pip install simba-uw-tf-dev --upgrade and make sure you have version 1.97.7, how does it look?

if that doesn't work, could you send me a video that thows the issue and I can see if I can replicate it? You could share it through a gdrive link.

[MBRINLE1]

After the update, I am unable to open simba at all and getting the following errors. I am working with simba in a virtual environment for python 3.6.13 to avoid overwriting dependencies required for other pipelines run on the computer.

I have attempted this both under my user and opening anaconda as admin. I have also ensured all users have permissions to the pertinent folders via (folder properties --> securities). None of which have changed the error message.

Log:
(Simba36) C:\Users\mbrinle1>simba
Access is denied.

(Simba36) C:\Users\mbrinle1>python -m simba
C:\ProgramData\miniconda3\envs\Simba36\python.exe: No module named simba.main; 'simba' is a package and cannot be directly executed

[sronilsson]

Hi @MBRINLE1 - when you updated SimBA, did you update it in the same anaconda environment SimBA was previously running in, or did you create a new environment and installed updated SimBA there?

When you created the Simba36 environment, did you use the anaconda navigator graphical interface, or did you create the the environment using the command line?

[MBRINLE1]

Yes, I updated in the same environment as previously installed. The Simba36 environment was created through the command prompt. On Jul 23, 2024, at 11:23 AM, Simon Nilsson ***@***.***> wrote:  You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Hi @MBRINLE1<https://github.com/MBRINLE1> - when you updated SimBA, did you update it in the same anaconda environment SimBA was previously running in, or did you create a new environment and installed updated SimBA there? When you created the Simba36 environment, did you use the anaconda navigator graphical interface, or did you create the the environment using the command line? — Reply to this email directly, view it on GitHub<#374 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVFMYY5E5K4UXI47BKCLWUDZNZYOVAVCNFSM6AAAAABLIMLPKKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVGU2TAOJUGU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[sronilsson]

I can't replicate it immediately on my machines - if you do try in a brand new conda environment, do you get the same error?

[MBRINLE1]

Okay I got around the access denied errors, may have just been programs running on other users? I have the updated Simba running (1.97.7) and continue to see a black frame when trying to configure video parameters, but no problems with the frame that is pulled up for ROI. I just uploaded a new video and am having the error with the new upload as well. [cid:eb6c5a39-468f-4744-87c0-0a471c4c31aa]

…

______________________________________ Mia Brinley Fox, PT, DPT (she/her) Graduate Student ​The University of Tennessee Health Science Center Integrated Biomedical Science Program Neuroscience Track

________________________________ From: Simon Nilsson ***@***.***> Sent: Tuesday, July 23, 2024 1:34 PM To: sgoldenlab/simba ***@***.***> Cc: Fox, Mia B ***@***.***>; Mention ***@***.***> Subject: [Ext] Re: [sgoldenlab/simba] Video Parameters Opening Black Frames (Issue #374) You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> I can't replicate it immediately on my machines - if you do try in a brand new conda environment, do you get the same error? — Reply to this email directly, view it on GitHub<#374 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVFMYY2A27NJ6YIV3NEHCUTZN2O43AVCNFSM6AAAAABLIMLPKKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBVHE3DMOBVHE>. You are receiving this because you were mentioned.

[sronilsson]

Hi @MBRINLE1 - yes, could have been some permission error, if the files are being used elsewhere: do you have the file the video file that is playing up? I can take a look at it and why you see why you see a black screen in the pixel per millimeter conversion interface.

[sronilsson]

... glancing at the the code, it looks like the pixel per millimeter interface pulls up the very first frame of the video (frame number 0), while the ROI interface reads the second frame and shows it (frame number 1).

I know you mentioned earlier that there are no black frames in the video, but just to confirm - is it at all possible that the very first video frame of the offending videos are black?

EDIT:

What we could do to fix it, is insert a conditional - "if image is all black (or uniform single-color) pixels, read in the next frame until SimBA finds a non-black non-uniform color frame", although I would only scan the first few seconds of the videos or so, to protect against situations where users have videos that are all black throughout and SimBA would scan the whole video for an hour..

[MBRINLE1]

Wow, I think that is it. Will report back if that fixes the problem. Thanks for the thorough communication/ assistance! Mia On Jul 24, 2024, at 11:38 AM, Simon Nilsson ***@***.***> wrote:  You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> ... glancing at the the code, it looks like the pixel per millimeter interface pulls up the very first frame of the video (frame number 0), while the ROI interface reads the second frame and shows it (frame number 1). I know you mentioned earlier that there are no black frames in the video, but just to confirm - is it at all possible that the very first video frame of the offending videos are black? — Reply to this email directly, view it on GitHub<#374 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVFMYYZ54R5YRWJ6MWF7CGDZN7DATAVCNFSM6AAAAABLIMLPKKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBYGMZDOOJSGI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[sronilsson]

Alright thanks @MBRINLE1

PS. No need to dig and try to remove that black framem if that turns out the case, to get it running in the pixel per millimeter interface - I will insert a fix to to view the first non-black, non-uniform color frame. That said it would probably be good idea to locate where the black frame is coming from at the source to avoid it from happening in feature as it will be an odd frame to analyze in pose-estimation.

[MBRINLE1]

It turns out that our original videos we are importing do not have a black frame. But when I pull up the video from the project_folder/videos/ path it does have a black frame. Hence the confusion on me not seeing any black frames in the videos. Is it possible that somewhere in the script it could be adding that black frame? From our end that is what it looks like is happening. On Jul 24, 2024, at 12:05 PM, Simon Nilsson ***@***.***> wrote:  Alright thanks @MBRINLE1<https://github.com/MBRINLE1> PS. No need to dig and try to remove that black framem if that turns out the case, to get it running in the pixel per millimeter interface - I will insert a fix to to view the first non-black, non-uniform color frame. That said it would probably be good idea to locate where the black frame is coming from at the source to avoid it from happening in feature as it will be an odd frame to analyze in pose-estimation. — Reply to this email directly, view it on GitHub<#374 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVFMYY6CVRCTKNN6YXSUENTZN7GD3AVCNFSM6AAAAABLIMLPKKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBYGM4TCMJSGQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[sronilsson]

When you import videos into SimBA, SimBA just copies the files into the project_folder/videos directory, so I can't see how the black frame was introduced then.

If you have the original videos, you could just replace the video in the project_folder/videos directory with the original video.

Did you perform any operation/formatting on the video after importing it into the SimBA project?

[MBRINLE1]

Thanks for your attention to this, I think the frame was added with the video editing software somehow. All is working now! ***@***.***

…

_________________________________ Mia Brinley Fox, PT, DPT (she/her) Graduate Student The University of Tennessee Health Science Center Integrated Biomedical Science Program Neuroscience Track On Jul 24, 2024, at 11:24 AM, Simon Nilsson ***@***.******@***.***>> wrote: When you import videos into SimBA, SimBA just copies the files into the project_folder/videos directory, so I can't see how the black frame was introduced then. If you have the original videos, you could just replace the video in the project_folder/videos directory with the original video. Did you perform any operation/formatting on the video after importing it into the SimBA project? — Reply to this email directly, view it on GitHub<#374 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVFMYY5TQB5YHUAYLJZESVTZN7ILLAVCNFSM6AAAAABLIMLPKKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBYGQZTIOBWG4>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[sronilsson]

Great thanks for letting me know @MBRINLE1!