Error Releasing VideoCapture in OpenCV Using GStreamer Pipeline

[gasparramoa]

Dear all,

Thank you for developing such a helpful library.

Problem:

I am trying to get frames using opencv.
I know that libcamera does not entirely support cv2 (I already created another issue for that kbingham/libcamera#96).
I am actually able to get frames via cv2.VideoCapture through GStreamer using the following:

import cv2
camera = "/base/soc/i2c0mux/i2c@1/imx219@10"
pipeline = f"libcamerasrc camera-name={camera} ! video/x-raw,width=1640,height=1232,format=BGR,framerate=40/1 ! videoconvert ! appsink"
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)

However, when I try to release the capture (for ending the program or maybe adjusting raw image size): cap.release()
I get the following error:

[1:39:52.854669803] [1885] ERROR DeviceEnumerator device_enumerator.cpp:172 Removing media device /dev/media4 while still in use
[1:39:52.854709759] [1885] ERROR DeviceEnumerator device_enumerator.cpp:172 Removing media device /dev/media3 while still in use

System:

• OS Bullseye LITE
• Arducam IMX219
• Rbpi 4
• Libcamera: v0.0.5+83-bde9b04f
• Python 3.9.2

What I already tried:

I updated my libcamera from v0.0.5 to v0.3.0+65-6ddd79b5 (clone this repo and use the build instructions)
I updated the libcamera-apps build to rpicam-apps build: 511941fdb1fe-intree
I build the custom cv2 version of advait-0, used cv2.VideoCapture(0, cv2.CAP_LIBCAMERA), but unfortunately I got the same problem. I cannot release the camera without an error in DeviceEnumerator.

More Details:

This has definitely to do with gstlibcamerasrc.cpp. Apparently this error happened before and this was the solution: fa63d42
I am confident it has to do with that... I also activated the logs of Gstreamer and Libcamera and present them here as soon as I start the release():

(...)
[2:28:30.630557394] [2011] DEBUG RPiDelayedControls delayed_controls.cpp:240 frame 147 started
[2:28:30.630634539] [2011] DEBUG RPiDelayedControls delayed_controls.cpp:283 Queue is empty, auto queue no-op.
[2:28:30.655460147] [2011] DEBUG RPI pipeline_base.cpp:1356 Frame start 148
[2:28:30.655547513] [2011] DEBUG RPiDelayedControls delayed_controls.cpp:240 frame 148 started
[2:28:30.655623807] [2011] DEBUG RPiDelayedControls delayed_controls.cpp:283 Queue is empty, auto queue no-op.

Trying to Releasing camera...True
0:00:03.891307777  2007  0x16aa820 DEBUG           libcamerasrc gstlibcamerasrc.cpp:691:gst_libcamera_src_task_leave:<libcamerasrc0> Streaming thread is about to stop
[2:28:30.669317521] [2013] DEBUG Camera camera.cpp:1385 Stopping capture
0:00:03.909948584  2007  0x169b900 DEBUG           libcamerasrc gstlibcamerasrc.cpp:713:gst_libcamera_src_close:<libcamerasrc0> Releasing resources
[2:28:30.688185411] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1539 /dev/video0[13:cap]: Releasing buffers
[2:28:30.688691723] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1254 /dev/video0[13:cap]: 0 buffers requested.
[2:28:30.688885030] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1539 /dev/video20[14:out]: Releasing buffers
[2:28:30.689265033] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1254 /dev/video20[14:out]: 0 buffers requested.
[2:28:30.689374269] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1539 /dev/video21[15:cap]: Releasing buffers
[2:28:30.689745217] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1254 /dev/video21[15:cap]: 0 buffers requested.
[2:28:30.689864860] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1539 /dev/video22[16:cap]: Releasing buffers
[2:28:30.690043667] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1254 /dev/video22[16:cap]: 0 buffers requested.
[2:28:30.690706177] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1539 /dev/video23[17:cap]: Releasing buffers
[2:28:30.690889540] [2013] DEBUG V4L2 v4l2_videodevice.cpp:1254 /dev/video23[17:cap]: 0 buffers requested.
[2:28:30.691083699] [2011] ERROR DeviceEnumerator device_enumerator.cpp:172 Removing media device /dev/media4 while still in use
[2:28:30.691222489] [2011] ERROR DeviceEnumerator device_enumerator.cpp:172 Removing media device /dev/media3 while still in use
Segmentation fault

[ShevaDavid]

faced the same problem.
@gasparramoa did you find any workaround for it ?

[nvvnst]

I have the same error.

• RPi 4 with OS Bullseye 64-bit
• imx477 and imx296 sensors

I tried both libcamera v0.0.5 and v0.3.0+65-6ddd79b5 versions.

Test script for rerpoducing error:

#include <opencv2/opencv.hpp>
#include <string>
#include <iostream>

int main() {
    std::string gst_pipeline = "libcamerasrc ! video/x-raw,framerate=30/1,width=1440,height=1080 ! appsink";
    cv::VideoCapture cam_(gst_pipeline, cv::CAP_GSTREAMER);
    int counter = 20;
    cv::Mat frame;
    while (counter > 0) {
        if (!cam_.read(frame)) {
            std::cout << "Could not read frame from the video stream." << std::endl;
            break;
        }
        counter--;
        std::cout << counter << std::endl;
    }
    cam_.release();
}

And gdb backtrace:

[Thread 0x7fe77fd640 (LWP 2536) exited]
[0:09:19.619052205] [2532] ERROR DeviceEnumerator device_enumerator.cpp:166 Removing media device /dev/media0 while still in use
[0:09:19.619125072] [2532] ERROR DeviceEnumerator device_enumerator.cpp:166 Removing media device /dev/media2 while still in use
[Thread 0x7fede1f640 (LWP 2532) exited]

Thread 1 "test" received signal SIGBUS, Bus error.
0x0000000000000006 in ?? ()
(gdb) backtrace
#0  0x0000000000000006 in  ()
#1  0x0000007fedee0b38 in libcamera::thread::postMessage(std::unique_ptr<libcamera::Message, std::default_delete<libcamera::Message> >, libcamera::Object*) ()
    at /lib/aarch64-linux-gnu/libcamera-base.so.0.0
#2  0x0000007fedede330 in libcamera::Object::postMessage(std::unique_ptr<libcamera::Message, std::default_delete<libcamera::Message> >) () at /lib/aarch64-linux-gnu/libcamera-base.so.0.0
#3  0x0000007fedede3e0 in libcamera::Object::deleteLater() ()
    at /lib/aarch64-linux-gnu/libcamera-base.so.0.0
#4  0x0000007fedf8b7e8 in  () at /lib/aarch64-linux-gnu/libcamera.so.0.0
#5  0x0000007fedfaad88 in libcamera::FrameBufferAllocator::~FrameBufferAllocator() ()
    at /lib/aarch64-linux-gnu/libcamera.so.0.0
#6  0x0000007fee097264 in  () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstlibcamera.so
#7  0x0000007ff700a960 in g_object_unref () at /lib/aarch64-linux-gnu/libgobject-2.0.so.0
#8  0x0000007fee098508 in  () at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstlibcamera.so
#9  0x0000007ff700a960 in g_object_unref () at /lib/aarch64-linux-gnu/libgobject-2.0.so.0
#10 0x0000007ff70ab7b4 in  () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#11 0x0000007ff70e609c in gst_mini_object_unref () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#12 0x0000007ff7112ab0 in  () at /lib/aarch64-linux-gnu/libgstreamer-1.0.so.0
#13 0x0000007ff6e93928 in  () at /lib/aarch64-linux-gnu/libgstapp-1.0.so.0
#14 0x0000007ff700a8d0 in g_object_unref () at /lib/aarch64-linux-gnu/libgobject-2.0.so.0
#15 0x0000007ff7f699a4 in cv::GStreamerCapture::~GStreamerCapture() ()
    at /usr/local/lib/aarch64-linux-gnu/libopencv_videoio.so.407
#16 0x0000007ff7f39b44 in cv::VideoCapture::release() ()
    at /usr/local/lib/aarch64-linux-gnu/libopencv_videoio.so.407
#17 0x00000055555512a4 in main ()
(gdb)

[gasparramoa]

So far, I am still looking for a workaround for this. But I am confident it has to do with what Laurent Pinchart(@pinchartl) worked on in this issue: fa63d42.

[nvvnst]

I tested this on RPi OS Bookworm 64 bit and faced the same error.

[naushir]

I presume this is still an issue? To be honest, I don't know anything about gstreamer so I cannot really advise on this issue. Perhaps the libcamera mailing list might be a better place for this query?

[gasparramoa]

Hi @naushir! Thanks for reaching out. This is really important for the library. I was hoping that @pinchartl would come up with a solution since he/she has worked on this specific problem before. I will create a query for the libcamera mailing list, but I am not sure if that will reach more people or not.