Learning Combinatorial Embedding Networks for Deep Graph Matching (1904.00597v3)

Abstract: Graph matching refers to finding node correspondence between graphs, such that the corresponding node and edge's affinity can be maximized. In addition with its NP-completeness nature, another important challenge is effective modeling of the node-wise and structure-wise affinity across graphs and the resulting objective, to guide the matching procedure effectively finding the true matching against noises. To this end, this paper devises an end-to-end differentiable deep network pipeline to learn the affinity for graph matching. It involves a supervised permutation loss regarding with node correspondence to capture the combinatorial nature for graph matching. Meanwhile deep graph embedding models are adopted to parameterize both intra-graph and cross-graph affinity functions, instead of the traditional shallow and simple parametric forms e.g. a Gaussian kernel. The embedding can also effectively capture the higher-order structure beyond second-order edges. The permutation loss model is agnostic to the number of nodes, and the embedding model is shared among nodes such that the network allows for varying numbers of nodes in graphs for training and inference. Moreover, our network is class-agnostic with some generalization capability across different categories. All these features are welcomed for real-world applications. Experiments show its superiority against state-of-the-art graph matching learning methods.

• The paper presents an end-to-end differentiable framework that leverages combinatorial embeddings and permutation loss to tackle NP-complete graph matching.
• It integrates GNNs to capture higher-order affinities, outperforming state-of-the-art methods in accuracy and flexibility.
• The approach achieves class-agnostic generalization, offering promising applications in computer vision tasks like visual tracking and action recognition.

Learning Combinatorial Embedding Networks for Deep Graph Matching

The paper "Learning Combinatorial Embedding Networks for Deep Graph Matching" by Runzhong Wang, Junchi Yan, and Xiaokang Yang addresses the complex problem of graph matching using deep learning techniques. Graph matching (GM) involves finding correspondences between nodes of two or more graphs—to maximize the affinity of matching nodes and edges while dealing with inherent computational challenges due to its NP-completeness. The paper presents a novel approach centered around a differentiable deep network that learns an affinity model for graph matching.

Key Contributions

1. End-to-End Differentiable Pipeline: The authors propose a deep learning framework integrating a supervised permutation loss to tackle graph matching. This approach captures the combinatorial nature of the problem, which traditional shallow models struggle with, particularly in the presence of noise and outliers. The network learns to parameterize intra-graph and cross-graph affinities using deep embeddings, instead of conventional parametric forms like Gaussian kernels.
2. Permutation Loss for Graph Matching: The use of a permutation loss, based on the Sinkhorn algorithm, is an innovative feature of this work. This loss effectively guides the learning process by leveraging a more inherent representation of the graph matching task, making it adaptable to variable-sized graphs during training and inference.
3. Graph Neural Networks (GNNs) for Embedding: The authors are among the first to employ GNNs for node embedding in graph matching tasks within the computer vision domain. This allows the model to encode not only second-order but also higher-order affinities, fundamentally enriching the features considered in the matching process.
4. Class-Agnostic Model with Generalization Effects: The proposed architecture is class-agnostic, hinting at an inherent versatility that extends to different application domains without specific retraining for new categories.

Strong Numerical Results and Implications

The paper presents experimental results that demonstrate the superiority of the proposed method over several state-of-the-art approaches, particularly against deep learning methods like the one proposed by Zanfir et al. The method outperforms traditional graph matching algorithms, such as the structured SVM-based approaches, in terms of accuracy and flexibility.

Theoretical and Practical Implications

From a theoretical perspective, this research provides evidence supporting the potential of integrating permutation-based learning in neural networks for combinatorial problems like graph matching. Practically, the approach holds promise in computer vision applications such as visual tracking and action recognition, where noise and structural variations are common.

Furthermore, the paper opens avenues for future research into extending permutation-based learning models to handle other complex combinatorial optimization problems within and beyond graph theory.

Future Directions in AI

Looking ahead, the integration of deep learning techniques with combinatorial optimization models could be further developed to tackle other NP-complete problems using end-to-end trainable networks. The insights from the permutation learning applied here can inspire new architectures that effectively balance computational efficiency and the modeling power needed for real-world applications in AI. This could potentially pave the way for developing more generalized frameworks for solving a wide array of graph-structured data challenges in AI.