Skip to content

Commit

Permalink
update doc
Browse files Browse the repository at this point in the history
  • Loading branch information
@wangxiaomeng030
wangxiaomeng030 committed May 22, 2024
1 parent 4c82d3d commit cd5eb9e
Show file tree
Hide file tree
Showing 3 changed files with 40 additions and 5 deletions.
11 changes: 6 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ We provide a set of pre-implemented deep-learning based SLAM algorithms.

**Replica/office0**

| [nice-slam](https://github.com/cvg/nice-slam) | [co-slam](https://github.com/HengyiWang/Co-SLAM) | [Vox-Fusion](https://github.com/zju3dv/Vox-Fusion) | [Point_SLAM](https://github.com/eriksandstroem/Point-SLAM) | [splaTAM](https://github.com/spla-tam/SplaTAM) | [DPVO](https://github.com/princeton-vl/DPVO) |
| :----------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| <img src="./docs/imgs/nice_slam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/co_slam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/vox_fusion.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/point_slam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/splatam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/dpvo.gif" alt="dpvo" style="zoom: 50%;" /> |
| [nice-slam](https://github.com/cvg/nice-slam) | [co-slam](https://github.com/HengyiWang/Co-SLAM) | [Vox-Fusion](https://github.com/zju3dv/Vox-Fusion) | [Point_SLAM](https://github.com/eriksandstroem/Point-SLAM) | [splaTAM](https://github.com/spla-tam/SplaTAM) | [DPVO](https://github.com/princeton-vl/DPVO) | [NeuralRecon](https://github.com/zju3dv/NeuralRecon) |
| :----------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| <img src="./docs/imgs/nice_slam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/co_slam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/vox_fusion.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/point_slam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/splatam.gif" alt="nice-slam" style="zoom: 50%;" /> | <img src="./docs/imgs/dpvo.gif" alt="dpvo" style="zoom: 50%;" /> | <img src="./docs/imgs/neural_recon.gif" alt="neural_recon" style="zoom: 50%;" /> |

## Quickstart

Expand Down Expand Up @@ -144,7 +144,7 @@ usage: ds-viewer [-h] [--config.vis-dir PATH] [--config.save-rendering {True,Fal

- --config.algorithm-name

NOTE: When use splaTAM or DPVO, algorithm name should be set. (default: None)
NOTE: When use splaTAM or DPVO or NeuralRecon, algorithm name should be set. (default: None)

#### ds-eval

Expand Down Expand Up @@ -180,7 +180,7 @@ Here are the comparison results on **Replica** datasets. The results of the orig

The algorithms with _X suffix are the corresponding algorithms in the XRDSLAM framework. For details, see [benchmark](docs/benchmark.md).

(DPVO is a visual odometry (VO) algorithm that does not generate meshes, and DPVO paper does not provide results for the Replica datasets. Therefore, here we only present the trajectory metrics from XRDSLAM framework. The comparison results of DPVO for the Euroc dataset can be found in the [benchmark](docs/benchmark.md).)
(DPVO is a visual odometry (VO) algorithm that does not generate meshes, and DPVO paper does not provide results for the Replica datasets. Therefore, here we only present the trajectory metrics from XRDSLAM framework. The comparison results of DPVO for the Euroc dataset can be found in the [benchmark](docs/benchmark.md). NeuralRecon is a mapping algorithm, and NeuralRecon paper does not provide results for the Replica datasets. Therefore, here we only present the mesh metrics from XRDSLAM framework. The comparison results of NeuralRecon for the 7Scenes dataset can be found in the [benchmark](docs/benchmark.md).)

Note: The default configuration in the algorithm is suitable for Replica. If you use other datasets, you need to modify the corresponding configuration items in slam/configs/input_config.py.

Expand All @@ -197,6 +197,7 @@ Note: The default configuration in the algorithm is suitable for Replica. If you
| SplaTAM | 0.36 | 34.11 | 0.97 | 0.10 | - | - | - | - | - | - | - |
| SplaTAM_X | 0.40 | 34.44 | 0.96 | 0.09 | - | - | - | - | - | - | - |
| DPVO_X | 0.31 | - | - | - | - | - | - | - | - | - | - |
| NeuralRecon_X | - | - | - | - | 13.29 | 7.43 | 9.51 | - | 5.87 | 19.36 | 38.13 |

## License

Expand Down
34 changes: 34 additions & 0 deletions docs/benchmark.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ A single Replica dataset has a total of 2000 frames, rendering is performed ever

## 3D metrics

Note: When evaluating 3D metrics, the default value of distance_thresh is 0.01. However, for NeuralRecon, the default value of distance_thresh is 0.05. Therefore, 3D metrics are provided under these two threshold values respectively.

| Algorithm | Metric | Room0 | Room1 | Room2 | Office0 | Office1 | Office2 | Office3 | Office4 | Average |
| ------------ | --------------------- | ----- | ----- | ----- | ------- | ------- | ------- | ------- | ------- | ------- |
| NICE-SLAM | Depth L1[cm] - | 1.81 | 1.44 | 2.04 | 1.39 | 1.76 | 8.33 | 4.99 | 2.01 | 2.97 |
Expand Down Expand Up @@ -114,6 +116,18 @@ A single Replica dataset has a total of 2000 frames, rendering is performed ever
| | Acc. [cm]- | 1.45 | 1.14 | 1.20 | 1.04 | 0.85 | 1.32 | 1.56 | 1.48 | 1.25 |
| | Comp. [cm]- | 3.62 | 3.07 | 3.01 | 1.69 | 2.37 | 3.66 | 3.53 | 4.02 | 3.12 |
| | Comp. Ratio[<5cm %] + | 87.60 | 89.13 | 89.43 | 93.00 | 89.42 | 85.85 | 85.56 | 85.23 | 88.15 |
| NeuralRecon_X | Precision [%] + | 13.06 | 10.91 | 13.53 | 13.64 | 16.73 | 17.85 | 10.98 | 9.67 | 13.29 |
| | Recall [%] + | 6.48 | 6.63 | 8.44 | 8.86 | 9.09 | 9.71 | 5.29 | 5.01 | 7.43 |
| | F1[%] + | 8.66 | 8.25 | 10.40 | 10.74 | 11.78 | 12.57 | 7.13 | 6.60 | 9.51 |
| | Acc. [cm]- | 5.47 | 6.24 | 6.31 | 5.25 | 3.60 | 6.06 | 7.05 | 7.00 | 5.87 |
| | Comp. [cm]- | 24.02 | 13.70 | 20.03 | 10.69 | 11.48 | 25.13 | 28.52 | 21.38 | 19.36 |
| | Comp. Ratio[<5cm %] + | 34.26 | 39.52 | 40.36 | 51.55 | 45.00 | 35.44 | 30.63 | 28.29 | 38.13 |
| NeuralRecon_X | Precision [%] + | 66.12 | 57.03 | 59.09 | 69.01 | 73.34 | 63.89 | 60.87 | 50.10 | 62.43 |
| distance_thresh=0.05 | Recall [%] + | 35.06 | 40.10 | 40.98 | 51.96 | 45.14 | 35.88 | 31.72 | 29.15 | 38.74 |
| | F1[%] + | 45.82 | 47.09 | 48.40 | 59.29 | 55.88 | 45.95 | 41.71 | 36.86 | 47.62 |
| | Acc. [cm]- | 5.47 | 6.23 | 6.30 | 5.19 | 3.60 | 6.07 | 7.08 | 7.03 | 5.87 |
| | Comp. [cm]- | 24.11 | 13.76 | 19.88 | 10.71 | 11.56 | 25.21 | 28.43 | 21.45 | 19.38 |
| | Comp. Ratio[<5cm %] + | 34.28 | 39.42 | 40.52 | 51.48 | 44.86 | 35.39 | 30.76 | 28.29 | 38.12 |

# Euroc

Expand All @@ -125,3 +139,23 @@ Note: DPVO run with monocular data and requires scale correction during trajecto
| ----------------------- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ------- |
| DPVO | 8.7 | 5.5 | 15.8 | 13.7 | 11.4 | 5.0 | 14.0 | 8.6 | 5.7 | 4.9 | 21.1 | 10.5 |
| DPVO_X | 10.0 | 7.4 | 11.8 | 15.2 | 8.7 | 9.4 | 15.9 | 10.2 | 6.6 | 6.4 | 12.3 | 10.4 |

# 7-Scenes

## 3D metrics

Note: The distance_thresh is 0.05 for both NeuralRecon_X and NeuralRecon.

The original paper didn't offer individual dataset results. Hence, only XRDSLAM results per dataset are provided here. Also, due to the absence of default parameters for 7-Scenes in the original code, there are minor discrepancies between the results of NeuralRecon_X and the original NeuralRecon.

| Algorithm | Precision [%] + | Recall [%] + | F1[%] + | Acc. [cm]- | Comp. [cm]- |
| ------------- | --------------- | ------------ | ------- | ---------- | ----------- |
| NeuralRecon | 38.9 | 22.7 | 28.2 | 10.0 | 22.8 |
| NeuralRecon_X | 37.51 | 21.24 | 26.55 | 14.23 | 32.21 |

| Method | Metric | chess-seq-03 | chess-seq-05 | fire-seq-03 | fire-seq-04 | heads-seq-01 | office-seq-02 | office-seq-06 | office-seq-07 | office-seq-09 | pumpkin-seq-01 | pumpkin-seq-07 | redkitchen-seq-03 | redkitchen-seq-04 | redkitchen-seq-06 | stairs-seq-01 | stairs-seq-04 | Average |
| ------------- | --------------- | ------------ | ------------ | ----------- | ----------- | ------------ | ------------- | ------------- | ------------- | ------------- | -------------- | -------------- | ----------------- | ----------------- | ----------------- | ------------- | ------------- | ------- |
| NeuralRecon_X | Precision [%] + | 48.37 | 36.21 | 40.08 | 47.90 | 51.00 | 26.64 | 24.83 | 31.73 | 42.93 | 39.97 | 26.39 | 24.93 | 44.30 | 35.07 | 40.95 | 38.87 | 37.51 |
| | Recall [%] + | 30.98 | 27.37 | 31.88 | 30.45 | 24.57 | 18.24 | 25.93 | 22.37 | 26.35 | 17.91 | 11.78 | 9.34 | 19.80 | 16.49 | 15.32 | 11.15 | 21.24 |
| | F1[%] + | 37.77 | 31.18 | 35.51 | 37.24 | 33.16 | 21.65 | 25.37 | 26.24 | 32.66 | 24.74 | 16.29 | 13.59 | 27.37 | 22.43 | 22.30 | 17.31 | 26.55 |
| | Acc. [cm]- | 7.68 | 8.97 | 8.88 | 7.56 | 9.74 | 25.26 | 23.20 | 12.90 | 8.82 | 13.71 | 26.97 | 25.95 | 11.24 | 10.97 | 16.01 | 9.96 | 14.23 |
Binary file added docs/imgs/neural_recon.gif
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit cd5eb9e

Please sign in to comment.