LocalGCL: Local-aware Contrastive Learning for Graphs (2402.17345v1)
Abstract: Graph representation learning (GRL) makes considerable progress recently, which encodes graphs with topological structures into low-dimensional embeddings. Meanwhile, the time-consuming and costly process of annotating graph labels manually prompts the growth of self-supervised learning (SSL) techniques. As a dominant approach of SSL, Contrastive learning (CL) learns discriminative representations by differentiating between positive and negative samples. However, when applied to graph data, it overemphasizes global patterns while neglecting local structures. To tackle the above issue, we propose \underline{Local}-aware \underline{G}raph \underline{C}ontrastive \underline{L}earning (\textbf{\methnametrim}), a self-supervised learning framework that supplementarily captures local graph information with masking-based modeling compared with vanilla contrastive learning. Extensive experiments validate the superiority of \methname against state-of-the-art methods, demonstrating its promise as a comprehensive graph representation learner.
- “Arnetminer: extraction and mining of academic social networks,” in SIGKDD, 2008.
- “Self-supervised graph transformer on large-scale molecular data,” 2020.
- “Utilizing graph machine learning within drug discovery and development,” Briefings in bioinformatics, 2021.
- “Graph neural networks for social recommendation,” in WWW, 2019.
- Ishan Misra and Laurens van der Maaten, “Self-supervised learning of pretext-invariant representations,” in CVPR, 2020.
- “What do self-supervised vision transformers learn?,” arXiv:2305.00729, 2023.
- “Prioritizing network communities,” Nature communications, 2018.
- “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv:1810.04805, 2018.
- “A simple framework for contrastive learning of visual representations,” in ICML, 2020.
- “Graph contrastive learning with augmentations,” NeurIPS, 2020.
- “Graph contrastive learning with adaptive augmentation,” in WWW, 2021.
- “Cuco: Graph representation with curriculum contrastive learning.,” in IJCAI, 2021.
- “Masked autoencoders are scalable vision learners,” in CVPR, 2022.
- “Variational graph auto-encoders,” arXiv:1611.07308, 2016.
- “Adversarially regularized graph autoencoder for graph embedding,” arXiv:1802.04407, 2018.
- “How powerful are graph neural networks?,” 2018.
- “Semi-supervised classification with graph convolutional networks,” arXiv:1609.02907, 2016.
- “Graph attention networks,” in ICLR, 2018.
- Kihyuk Sohn, “Improved deep metric learning with multi-class n-pair loss objective,” NeurIPS, 2016.
- “Unsupervised feature learning via non-parametric instance discrimination,” in CVPR, 2018.
- “Libsvm: a library for support vector machines,” ACM transactions on intelligent systems and technology (TIST), 2011.
- “Weisfeiler-lehman graph kernels.,” JMLR, 2011.
- “Deep graph kernels,” in SIGKDD, 2015.
- “Graphmae: Self-supervised masked graph autoencoders,” in SIGKDD, 2022, pp. 594–604.
- “Strategies for pre-training graph neural networks,” arXiv:1905.12265, 2019.
- “Zinc 15–ligand discovery for everyone,” Journal of chemical information and modeling, 2015.
- “Moleculenet: a benchmark for molecular machine learning,” Chemical science, 2018.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.