Abstract
In this paper, we present LiGNN, a deployed large-scale Graph Neural Networks (GNNs) Framework. We share our insight on developing and deployment of GNNs at large scale at LinkedIn. We present a set of algorithmic improvements to the quality of GNN representation learning including temporal graph architectures with long term losses, effective cold start solutions via graph densification, ID embeddings and multi-hop neighbor sampling. We explain how we built and sped up by 7x our large-scale training on LinkedIn graphs with adaptive sampling of neighbors, grouping and slicing of training data batches, specialized shared-memory queue and local gradient optimization. We summarize our deployment lessons and learnings gathered from A/B test experiments. The techniques presented in this work have contributed to an approximate relative improvements of 1% of Job application hearing back rate, 2% Ads CTR lift, 0.5% of Feed engaged daily active users, 0.2% session lift and 0.1% weekly active user lift from people recommendation. We believe that this work can provide practical solutions and insights for engineers who are interested in applying Graph neural networks at large scale.
Overview
-
LiGNN, a framework for integrating Graph Neural Networks (GNNs) at LinkedIn, aims to optimize the network's ecosystem by addressing the unique challenges of applying GNNs in a large-scale, dynamic environment.
-
The framework overcomes obstacles such as GNN training scalability, integrating diverse entities, addressing the cold start problem, and adapting to platform dynamics through innovative algorithmic and deployment strategies.
-
Technological advancements within LiGNN, including Kubernetes-based training infrastructure, adaptive sampling, graph densification techniques, and temporal graph modeling, have significantly enhanced efficiency and performance in recommendation systems.
-
LiGNN's deployment across LinkedIn's various domains has resulted in marked improvements in job and people recommendations, ad click-through rates, and user engagement, demonstrating the wide applicability and impact of GNNs in large-scale recommendation systems.
Deploying Large-Scale Graph Neural Networks at LinkedIn: Insights and Innovations
Introduction
LinkedIn, the world's largest professional network, has successfully integrated Graph Neural Networks (GNNs) at an unprecedented scale with its LiGNN framework. This integration aims to optimize LinkedIn's ecosystem, a complex web of professional connections and interactions among members, companies, and various entities. LiGNN encompasses sophisticated algorithmic advancements and strategic deployment mechanisms that address the unique challenges of applying GNNs in a dynamic, large-scale environment.
Challenges and Technological Advancements
LiGNN's development and deployment at LinkedIn presented numerous challenges, notably GNN training scalability, the integration of diverse entities into a unified graph embedding space, addressing the "cold start" problem, and adapting to the platform's dynamic nature. Through inventive approaches, including adaptive sampling and temporal graph architectures, LiGNN has achieved remarkable efficiency and marked improvements in various recommendation systems on LinkedIn.
GNN Training at Scale
LiGNN's training infrastructure utilizes a Kubernetes-based cluster, integrating Microsoft's DeepGNN graph engine for efficient real-time graph sampling. Implementations such as adaptive neighbor sampling and specialized data processing mechanisms have significantly improved GNN training stability and reduced training times by a factor of 7.
Graph Densification and Multi-Hop Sampling
To mitigate issues of graph sparsity and enhance representation learning, LiGNN employs graph densification techniques and multi-hop sampling strategies. These methods have proven effective in enriching graph representations and accelerating the sampling process, demonstrating notable improvements in various production applications.
Temporal Modeling
The framework introduces a simplified and scalable approach to temporal graph modeling, distinct from conventional temporal GNNs. By modifying neighbor sampling to encapsulate temporal dynamics, and incorporating transformer models for sequence encoding, LiGNN captures time-sensitive interactions, contributing to improved model performance across multiple use cases.
Deployment Lessons and Near-Line Inference Architecture
The deployment of LiGNN has provided critical insights, particularly in addressing feature freshness through a GNN model inference pipeline for near real-time embedding generation. Challenges related to integrating LiGNN with LinkedIn's systems prompted the development of specific Apache Beam components, enhancing the pipeline's efficiency and applicability.
Impact Across LinkedIn
LiGNN's deployment across various domains within LinkedIn, including job and people recommendations, ads, and feed post recommendations, has shown substantial improvements. For instance, LiGNN contributed to increases in job application hearing back rates, ad click-through rates, and overall user engagement metrics. The framework demonstrates the practical value and wide applicability of GNNs in enhancing recommendation systems at scale.
Conclusion
The development and deployment of LiGNN at LinkedIn highlight the potential of graph neural networks in transforming large-scale recommendation systems. By addressing key challenges through innovative approaches to training scalability, cold start solutions, and temporal modeling, LiGNN has achieved significant improvements in various application domains. This work serves as a valuable reference for engineers and researchers interested in applying GNNs at scale, promising further advancements and broader applications in the field of AI.
Create an account to read this summary for free: