Skip to content

Latest commit

 

History

History
95 lines (81 loc) · 3.37 KB

README.md

File metadata and controls

95 lines (81 loc) · 3.37 KB
Raw

Rendered Database

[Goal] Make Rendered Database using Rendered Image Obtained by Trained 3D Gaussian Splatting

Get Rendered Image using 3D Gaussian Splatting Rasterization

  • Benchmark Dataset → Tanks and Temples
Render Img1 (Novel View Synthesis) Render Img2 (Novel View Synthesis) Render Img3 (Novel View Synthesis)

Prerequisites

  1. 3D Gaussian Splatting
  2. NetVLAD Pytorch
  3. Faiss
  4. Anaconda
  5. OpenCV with Contrib
  6. RoMa
  7. LightGlue
  8. SuperPoint
  9. Feature Matching

Download pre-trained model

Download pre-trained model

Preparing Dataset (in example folder)

(1) cameras.txt → [IMAGE_ID, CAMERA TYPE, IMAGE Width, IMAGE HEIGHT, ~]
(2) images.txt → [IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME]
(3) image folder

Combined Preparing Dataset (Make KeyFrame DB)

  • Using NetVLAD Method to make global descriptor

    • KeyFrame = {Index, Image, Keypoints, Descriptor(local or global), Camera Pose, Camera Type, Camera Params}

  • Final Database Extension file is HDF5

Method of Keypoint and Descriptor (in config.yaml)

  • extractor_method (extractor_utils.py):

    • 0 → apply ORB
    • 1 → apply SIFT
    • 2 → apply AKAZE
    • 3 → apply SuperPoint Model
    • roma_based_extractor function → apply RoMa Model

  • descriptor_method (extractor_utils.py):

    • 0 → apply ORB
    • 1 → apply SURF
    • 2 → apply DAISY
    • 3 → apply AKAZE
    • 4 → apply SuperPoint Model

  • matcher_mode (matching_utils.py):

    • calculate_score function → apply BF Matcher (Brute-Force) + KNN Matches
    • roma_based_extractor function → apply RoMa Model
    • lightglue_matcher function → apply LightGlue Model

  • REJECTION_MODE (using cv2):

    • apply cv2.fundamentalMatrix

Example Results

  • (1) Find the Nearest Distance using Faiss obtained by VLAD vector (Find Candidates)

  • (2) Find Feature Matching using Local Descriptor (Finda the Best Candidates)

    Matching Results (ORB + DAISY + distance=0.99)
    Matching Results (RoMa indoor + reproj=0.01)