If our video does not initially contain the object we want to detect, how should we modify the code?

[Zhujx-zjjx]

As stated in the question, when the object I want to detect does not appear at the beginning of my video, the code will report an error when running. What method should I use to eliminate this hidden danger?

[rentainhe]

As stated in the question, when the object I want to detect does not appear at the beginning of my video, the code will report an error when running. What method should I use to eliminate this hidden danger?

You can set the start frame to 10 or later which may be helpful to avoid this issue. Or you can check each frame to determine a suitable start frame for your demo.

Or you can implement a check function to determine which frame the object first appears in, and then decide whether to start tracking from this frame.

[Zhujx-zjjx]

I agree with your comments. The problem is that designing a check function might be not easy, could you give me some suggestions about this check function?

[rentainhe]

I agree with your comments. The problem is that designing a check function might be not easy, could you give me some suggestions about this check function?

I think you can try this solution:

• Firstly, run grounding model on a set of frames, like frame 1, frame 3, frame 5, and collect all the grounding results of your specific classname
• Then, you can try to use NMS op to filter objects with large overlap areas
• Use the grounding boxes as your initial prompt for tracking

You can try this solution for your scenarios.

And I think this is a good topic for us to discuss how to use SAM 2 better

[Paul0302]

I faced this issue recently either, I'm using grounded_sam2_tracking_demo_custom_gd1.5.py, and i modify the Grounded-Dino part. First is to check if the input_boxes is empty or not, if yes than detect next frame until the object appear. Hope this can help you