XFeat + LightGlue

[guipotje]

Hello everyone,

I'm training some LightGlue variations (finding a neat trade-off between model size vs accuracy) and soon I will update the repo with the model and weights in the next weeks!

You can follow this issue if you are interested.

Best,

Guilherme

[cxb1998]

That's great! Looking forward to your work

[guipotje]

Hey guys, just released a version of LightGlue matcher, please check out in README

[muchluv525]

大家好，刚刚发布了一个版本的 LightGlue 匹配器，请在 README 中查看

Hello, author, didn't you say that XFEAT performance is better than Superpoint, why look at the data this time Superpoint is stronger

[guipotje]

Hi @muchluv525,

Please notice that we trained a smaller and faster version of LightGlue.

Other than that, there are still a few reasons that SuperPoint + LightGlue might be still better than XFeat + LG (full size)
1 - Descriptor embedding: XFeat extracts a much more compact (64-D vs 256-D) descriptors;
2 - SuperPoint has a much larger backbone.

[zhouzq-thu]

And the number of layers is just 6. BTW, would you upload the training code of LightGlue for XFeat?

[muchluv525]

嗨，

请注意，我们训练了一个_更小、更快_的 LightGlue 版本。

除此之外，SuperPoint + LightGlue 可能仍然优于 XFeat + LG（全尺寸）有几个原因 1 - 描述符嵌入：XFeat 提取更紧凑（64-D 与 256-D）的描述符; 2 - SuperPoint 具有更大的主干网。

我明白了，谢谢你的回答，我是初学者，期待对xfeat进行更详细的解释

[noahzn]

@guipotje Have you tried training Xfeat and LightGlue end-to-end?

[guipotje]

Hello @noahzn, I haven't tried to train it end-to-end. It might deliver some improvements, as mentioned in SuperGlue's paper (section 5.4), when backpropagating through the descriptors. However, it also might lead to less generalization to different scenes.

[zhangchang0127]

Hello, I encountered an error while converting the ONNX model using torch.export. Here is the code we used for the model conversion.

This is the error.

[EndlessPeak]

Hello,

Thank you very much for your great work.

I am curious about the modifications made to the lightglue network structure to achieve the balance between inference accuracy and speed mentioned in the README. Will this part of the code be made publicly available?

I checked the match_lighterglue.py file, but it does not provide more information on this aspect.

[noahzn]

Hello @noahzn, I haven't tried to train it end-to-end. It might deliver some improvements, as mentioned in SuperGlue's paper (section 5.4), when backpropagating through the descriptors. However, it also might lead to less generalization to different scenes.

Hi @guipotje I might try end-to-end training, do you have any idea about the implementation? Since XFeat and LightGlue use different homography code for training, I'm wondering if it's possible to keep their own random homography code, but only optimize two networks together. This may need XFeat to return a loss and then we add it to LightGlue's loss and backprop together. Do you think this is the minimal effort for an end-to-end training of the two networks?