Improving Feature-based Visual Localization by Geometry-Aided Matching (2211.08712v2)

Abstract: Feature matching is crucial in visual localization, where 2D-3D correspondence plays a major role in determining the accuracy of camera pose. A sufficient number of well-distributed 2D-3D correspondences is essential for accurate pose estimation due to noise. However, existing 2D-3D feature matching methods rely on finding nearest neighbors in the feature space and removing outliers using hand-crafted heuristics, which may lead to potential matches being missed or the correct matches being filtered out. In this work, we propose a novel method called Geometry-Aided Matching (GAM), which incorporates both appearance information and geometric context to address this issue and to improve 2D-3D feature matching. GAM can greatly boost the recall of 2D-3D matches while maintaining high precision. We apply GAM to a new hierarchical visual localization pipeline and show that GAM can effectively improve the robustness and accuracy of localization. Extensive experiments show that GAM can find more real matches than hand-crafted heuristics and learning baselines. Our proposed localization method achieves state-of-the-art results on multiple visual localization datasets. Experiments on Cambridge Landmarks dataset show that our method outperforms the existing state-of-the-art methods and is six times faster than the top-performed method. The source code is available at https://github.com/openxrlab/xrlocalization.

• The paper introduces GAM to enhance 2D-3D feature matching by incorporating geometric constraints, increasing match recall and precision.
• It employs a Bipartite Matching Neural Network with a Hungarian pooling layer to convert many-to-many matches into reliable one-to-one correspondences.
• Experimental results on datasets like Cambridge Landmarks and Aachen Day-Night demonstrate superior accuracy and speed over previous methods.

Overview of Geometry-Aided Feature-Based Visual Localization

The paper "Improving Feature-based Visual Localization by Geometry-Aided Matching" presents a novel method called Geometry-Aided Matching (GAM) to enhance 2D-3D feature matching in visual localization tasks. The authors recognize a critical challenge: the need for a copious number of well-distributed matches for accurate pose estimation in the presence of noise. The conventional approach, which relies on finding nearest neighbors and hand-crafted heuristics to eliminate outliers, often misses potential matches or discards correct ones due to these heuristics' limitations.

GAM addresses these challenges by combining appearance information with geometric context, enhancing the recall of 2D-3D matches while maintaining precision. GAM is integrated into a hierarchical visual localization pipeline, showcasing significant robustness and accuracy improvements. This paper presents experiments and results that underscore the superiority of GAM over both hand-crafted heuristics and existing learning baselines.

Key Contributions

1. Geometry-Aided Matching (GAM): The proposed method increases the recall of 2D-3D matches by incorporating geometric constraints into the matching process. It leverages both appearance and geometric context to refine the matching process, overcoming the limitations of simple nearest-neighbor approaches.
2. Bipartite Matching Neural Network (BMNet): At the core of GAM, BMNet effectively handles many-to-many candidate matches, predicting geometric priors for each match and ensuring a one-to-one correspondence through a Hungarian pooling layer.
3. Hierarchical Localization with Scene Retrieval: The method embeds GAM into a hierarchical model that enhances retrieval precision through co-visibility information, leading to improved pose estimation accuracy across challenging datasets.
4. State-of-the-Art Results: The proposed method outperformed existing state-of-the-art systems on various visual localization datasets, particularly showcasing efficiency with faster operation while maintaining precision.

Experimental Validation

The authors conducted extensive experiments, demonstrating GAM's superior performance in retrieving real 2D-3D correspondences compared to hand-crafted methods such as NN-ratio test and learned models adapted from 2D-2D matching strategies. On the Cambridge Landmarks and Aachen Day-Night datasets, GAM showed improvements in both matching recall and precision, resulting in lower positional and angular errors in pose estimation.

One standout result was on the Cambridge Landmarks dataset, where the proposed solution outperformed the top prior method in both translational and rotational accuracy while being six times faster. Such efficiency gains, without losing precision, are particularly valuable for real-time applications such as augmented reality and autonomous systems.

Implications and Future Directions

The implications of this paper extend to visual SLAM and AR environments, where efficient and precise localization is pivotal. The novel integration of geometric reasoning in feature matching presents opportunities to further explore end-to-end learning systems where different modules could be optimized individually rather than jointly. Furthermore, the method's adaptability to incorporate more expansive scenes without substantial overhead suggests its potential scalability to larger environments.

For future work, one could investigate more sophisticated descriptor representations or explore different geometric reasoning architectures to further boost precision. The approach can be generalized to other correspondence tasks beyond visual localization, potentially benefiting multi-modal learning systems where appearance and contextual cues need to be jointly optimized.

The method’s source code availability allows for broader research community engagement and validation across new environments and datasets, ensuring that the results and methods proposed stand the test of time and adaptability across diverse visual localization challenges.