what kind of matching is this?

[royw99]

Is this dense matching or key point matching?

[royw99]

OmniGlue chooses to focus on sparse methods

The paper seems to suggest that the only the descriptor generated by Superpoint is improved but the coordinates (keypoints) are unchanged and it is still sparse matching

However, my colmap 3d reconstruction pipeline ends with poor performance based normally associated with dense matching (because the keypoints of every image are changed when that image is paired up with other distinct images)

[arjunkarpur]

Our method is considered a sparse keypoint matching method. We use extracted SuperPoint features as input to OmniGlue, without any modifications to the pretrained SP model. Any issues with keypoint repeatability are likely due to the behavior of SuperPoint detection. That being said, SuperPoint should always yield the same keypoints/descriptors for a fixed input image.

Could you try your SfM pipeline with SuperPoint + SuperGlue/LightGLue/MNN? This would help us determine if the issue is with SuperPoint detection or with OmniGlue matches

---

...(because the keypoints of every image are changed when that image is paired up with other distinct images)

One thing to note is that the OmniGlue.FindMatches() fn only returns matching keypoints, not the full set of extracted keypoints. You would need to modify the code slightly here to return the raw output of self.sp_extract() to get all extracted keypoints.

[royw99]

Thanks for the reply. I have separated the key point detection and descriptor section from the omniglue matching process. The 3d reconstruction work now, but still doesn't work as good as other models, such as superpoint + Lightglue. Maximizing the number of keypoints detected helps to squeeze a little more accuracy, but I can't push it any further. any advice?

[arjunkarpur]

In combination with more keypoints, I would recommend lowering the match_threshold filter value to a very low value (e.g. 0.0 < threshold < 1e-3) to allow the maximum amount of candidate matches.

You can also try setting pad_random_features=True for the calls to self.sp_extract() here. When fewer than N(=1024 by default) keypoints are detected, the method will sample keypoints randomly in the image in order to provide additional context on the image to the matching model. This is especially helpful when # keypoints is higher (2k, 4k, 8k).

[royw99]

In combination with more keypoints, I would recommend lowering the match_threshold filter value to a very low value (e.g. 0.0 < threshold < 1e-3) to allow the maximum amount of candidate matches.

You can also try setting pad_random_features=True for the calls to self.sp_extract() here. When fewer than N(=1024 by default) keypoints are detected, the method will sample keypoints randomly in the image in order to provide additional context on the image to the matching model. This is especially helpful when # keypoints is higher (2k, 4k, 8k).

Hi, thanks for the tips. I have tried your tricks but it still didn't work that will (may be it has to do with the distribution of the dataset or maybe my implementation in other part is mistaken).