Question about how to detect events in video

[xiaokj37]

In the paper, author mentioned using DPC-KNN to segment the video to get multiple events, and then process each event separately.
I would like to ask where this part of the code is implemented, I don't seem to find it.

[jpthu17]

I am sorry to reply to you so late because I have been busy with other projects recently.

The relevant code is below, we are currently trying to improve this part of the code and will update our code base in the near future.

Chat-UniVi/ChatUniVi/model/arch.py

Lines 135 to 153 in f63c247

	cls_features = torch.mean(image_features, dim=1, keepdim=False).unsqueeze(0).clone()
	token_dict = {'x': cls_features,
	'token_num': cls_features.size(1),
	'idx_token': torch.arange(cls_features.size(1))[None, :].repeat(
	cls_features.size(0), 1),
	'agg_weight': cls_features.new_ones(cls_features.size(0), cls_features.size(1),
	1),
	'mask': None}
	down_dict, token_dict = self.get_model().ctm3(token_dict)
	events = OrderedDict()
	max_len = 0
	for id, i in enumerate(down_dict["idx_token"][0].tolist()):
	if i not in events:
	events[i] = [id]
	else:
	events[i].append(id)
	max_len = len(events[i]) if max_len < len(events[i]) else max_len

[xiaokj37]

Thanks for your reply.
I am now trying to reproduce the results of the video event slicing given in the paper. You have given the visualization code for the image, is it possible to give the processing code for the video and the corresponding reference video.
I'm waiting for your early reply.