diff --git a/README.md b/README.md index 2a70128..49fe5d9 100644 --- a/README.md +++ b/README.md @@ -1,247 +1,3 @@ # GV-Bench: Benchmarking Local Feature Matching for Geometric Verification of Long-term Loop Closure Detection -### [Arxiv](https://arxiv.org/abs/2407.11736) - - - - - -## Abstract -Visual loop closure detection is an important module in visual simultaneous localization and mapping (SLAM), which associates current camera observation with previously visited places. Loop closures correct drifts in trajectory estimation to build a globally consistent map. However, a false loop closure can be fatal, so verification is required as an additional step to ensure robustness by rejecting the false positive loops. Geometric verification has been a well-acknowledged solution that leverages spatial clues provided by local feature matching to find true positives. Existing feature matching methods focus on homography and pose estimation in long-term visual localization, lacking references for geometric verification. To fill the gap, this paper proposes a unified benchmark targeting geometric verification of loop closure detection under long-term conditional variations. Furthermore, we evaluate six representative local feature matching methods (handcrafted and learning-based) under the benchmark, with in-depth analysis for limitations and future directions. - - - -## News -- :star: Paper is release on [arxiv](https://arxiv.org/abs/2407.11736). -- :tada: The paper is accepted by IROS 2024! -- :rocket: Releasing the visualization of [image matching](./assets/appendix.pdf) results. ([google drive](https://drive.google.com/file/d/1145hQb812E0HaPGekdpD04bEbjuej4Lx/view?usp=drive_link)) -- :rocket: Releasing the benchmark (easy)! Checkout the image pairs from `dataset/gt` and images from [google drive](https://drive.google.com/drive/folders/1E8m353fi3hv-gaytJuRPLhFeNLPWTak6?usp=sharing) - - - - - -## Installation -We use part of the HLoc code for feature extraction and matching. -```bash -git clone && cd GV-Bench -git submodule init -git submodule update -cd third_party/Hierarchival-Localization -git checkout gvbench # this is a customized fork version -python -m pip install -e . -``` -## Replicate Results in Exps -We provide the [output results]() with the format shown below. You can use these results directly. -```bash -$seq_$feature_$match.log -$seq_$feature_$match.npy # with following format -``` -```python -np.save(str(export_dir), { - 'prob': num_matches_norm, - 'qImages': qImages, - 'rImages': rImages, - 'gt': labels, - 'inliers': inliers_list, - 'all_matches': pointMaps, - 'precision': precision, - 'recall': recall, - 'TH': TH, - 'average_precision': average_precision, - 'Max Recall': r_recall - }) -``` -### Replicate from scratch -To get standard feature detection and matching results, we proposed to use [hloc](https://github.com/cvg/Hierarchical-Localization). - -- Download the dataset sequences from [google drive](https://drive.google.com/file/d/1145hQb812E0HaPGekdpD04bEbjuej4Lx/view?usp=drive_link) and put it under the `dataset/` folder. - -- Extract and match feature using hloc. - - Extract features: SIFT, SuperPoint, and DISK - ```bash - python third_party/Hierarchical-Localization/gvbench_utils.py config/${seq}.yaml --extraction - ``` - - Match features: SIFT-NN, SIFT-LightGlue (Not yet implemented), SuperPoint-NN, DISK-NN, SuperPoint-SuperGlue, SuperPoint-LightGlue, DISK-LightGlue, LoFTR - ```bash - # all methods except LoFTR - python third_party/Hierarchical-Localization/gvbench_utils.py config/${seq}.yaml --matching - - # LoFTR is different from above methods thus - python third_party/Hierarchical-Localization/gvbench_utils.py config/${seq}.yaml --matching_loftr - ``` - - - Image pairs files - - We prepare pairs (GT) file for matching under `dataset/gt` foler. - - Make sure to use the fork hloc for feature extraction and matching `https://github.com/jarvisyjw/Hierarchical-Localization.git -b gvbench` - -- Evaluation - - We provide out-of-box scripts - - ```bash - cd GV-Bench/scripts - bash ./evaluation # run script with - #sequence name: day, night, night-hard, season, season-hard, weather - ``` - -## Benchmark Usage -- Using customized local features for geometric verification (GV). - - TODO - - - - -- Visualization - - Demos are presented in `plot_data.ipynb` - -## Acknowledgement -- This work builds upon [hloc](https://github.com/cvg/Hierarchical-Localization), thanks for their amazing work. -- Contact: `jingwen.yu@connect.ust.hk` - +### [[Arxiv]](https://arxiv.org/abs/2407.11736) [[Project Page]](https://jarvisyjw.github.io/GV-Bench/) \ No newline at end of file