frame_number exceeding video length

[vbelagali]

Hi authors,

I am currently trying to set up the ego4d dataset for HOI-ref. But I find that some entries in the dataset have frame number higher than the length of the video. I have attached the code I am using to extract frames.

`
def save_ref_frame(frame_path=None, ego4d_root="/data/"):
video_id = frame_path.split("/")[-2]
frame_str = frame_path.split("/")[-1].rstrip(".jpg")
frame_number = int(frame_str.split("_")[-1])
video_path = f'{ego4d_root}/Ego4D/v2/clips/{video_id}.mp4'
target_frame_path = f'{ego4d_root}/{frame_path}'
frame_count = frame_number
video = cv2.VideoCapture(video_path)
if not video.isOpened():
print("Error: Could not open video.", video_path)
exit()
video.set(cv2.CAP_PROP_POS_FRAMES, frame_count)
success, frame = video.read()
print(f"video id: {video_id}, frame: {frame_str}")
print(f"video len : {int(video.get(cv2.CAP_PROP_FRAME_COUNT))}")
assert success == True
parent_path = Path(target_frame_path).parent
Path(parent_path).mkdir(parents=True, exist_ok=True)
cv2.imwrite(target_frame_path, frame)

def process(meta_file="hoiqa_dataset/ego4d-train.json"):
failed_metas = []
with open(meta_file, 'r') as file:
metas = json.load(file)
success = 0
for i, meta in tqdm(enumerate(metas[700000:]), total=len(metas)):
save_ref_frame(meta["image_path"])
success = success + 1

if name == "main":
process()
`

I get the following error and the video len is less than the frame number

I used the ego4d cli tool as follows to download the videos.
ego4d --output_directory="/data/Ego4D/" \ --datasets clips \ --version v2 \ --video_uid_file ego4d-clip-ids.txt \ --aws_profile_name ego4d

Can you please let me know if I am doing something wrong or can you please provide scripts to extract frames from ego4d videos ?

Thank you,
Varun

[Sid2697]

Hello @vbelagali,

You'll have to use the videos from clip folder in Ego4D/v2 and account for the offset of the video by using this video['parent_start_frame']. Let me know if there is any confusion.

Regards,
Siddhant Bansal

[vbelagali]

Hi Siddhant,

Thanks for the guidance. Can you please confirm if folder name is clip or clips ?

I can locate the parent_frame_start in manifest.csv file

Do I need to subtract the parent_frame_start from the frame_number to get the correct offset ?

[Sid2697]

Hello @vbelagali , yes, you do need to subtract to get the correct offset.