Troubleshooting Memory Leaks in Video Detection Models

[micedevai]

Troubleshooting Memory Leaks in Video Detection Models

While performing detection inference on a video, a memory leak is observed even after resource cleanup. During the inference process, memory usage increases over time, eventually causing the process to slow down, while the memory usage should ideally remain stable.

Installations

To set up the necessary environment, run the following installation commands:

!python -m pip install pyyaml
!pip install 'git+https://github.com/facebookresearch/detectron2.git'
!pip install opencv-python
!pip install torch

Then, import the required libraries:

import cv2, os, numpy as np, tqdm, time, math, psutil, torch
from collections import defaultdict
from google.colab.patches import cv2_imshow
from tqdm import tqdm
from detectron2.utils.logger import setup_logger
setup_logger()
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.video_visualizer import VideoVisualizer
from detectron2.utils.visualizer import ColorMode, Visualizer
from detectron2.data import MetadataCatalog

Set Up Configuration and Initialize the Predictor

Configure the Detectron2 model for object detection:

cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml")
cfg.MODEL.DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
predictor = DefaultPredictor(cfg)

Open the Video and Initialize Video Writer

Set up the video capture and writer for processing:

cap = cv2.VideoCapture("/content/Input_Video.mp4")
fps = int(cap.get(cv2.CAP_PROP_FPS))
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
video_writer = cv2.VideoWriter("/content/Output_Video.mp4", cv2.VideoWriter_fourcc(*'mp4v'), fps, (width, height))

Process the Video

Process each frame of the video, run object detection, and draw bounding boxes:

start_time = time.time()
frame_idx = 0

while cap.isOpened():
    success, frame = cap.read()
    if not success: break

    outputs = predictor(frame)
    boxes = outputs["instances"].to("cpu").pred_boxes if outputs["instances"].has("pred_boxes") else None
    if boxes:
        for box in boxes:
            x1, y1, x2, y2 = map(int, box.tolist())
            cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 2)
    video_writer.write(frame)
    frame_idx += 1

Resource Cleanup

Ensure proper cleanup after processing:

cap.release()
video_writer.release()
cv2.destroyAllWindows()

Observed Behavior

Memory usage is monitored before and after inference:

memory_before = psutil.Process(os.getpid()).memory_info().rss / (1024 * 1024)

# Video Inference

memory_after = psutil.Process(os.getpid()).memory_info().rss / (1024 * 1024)
print(f"Memory difference: {memory_after - memory_before:.2f} MB")

Output:

Memory difference: 621.10 MB

Expected Behavior:

For a video detection task, memory usage should remain stable across frames of similar resolution. However, in this case, the memory usage steadily increases, which is indicative of a potential memory leak.

Environment:

• Detectron2 version: 0.6
• Python version: 3.10.12
• OS: Ubuntu 22.04
• GPU: NVIDIA-SMI 535.104.05, CUDA 12.2

[github-actions]

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template.
The following information is missing: "Instructions To Reproduce the Issue and Full Logs";