Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network (2402.11518v2)

Published 18 Feb 2024 in cs.LG and cs.CL

Abstract: Heterogeneous information networks (HIN) have gained increasing popularity in recent years for capturing complex relations between diverse types of nodes. Meta-structures are proposed as a useful tool to identify the important patterns in HINs, but hand-crafted meta-structures pose significant challenges for scaling up, drawing wide research attention towards developing automatic search algorithms. Previous efforts primarily focused on searching for meta-structures with good empirical performance, overlooking the importance of human comprehensibility and generalizability. To address this challenge, we draw inspiration from the emergent reasoning abilities of LLMs. We propose ReStruct, a meta-structure search framework that integrates LLM reasoning into the evolutionary procedure. ReStruct uses a grammar translator to encode the meta-structures into natural language sentences, and leverages the reasoning power of LLMs to evaluate their semantic feasibility. Besides, ReStruct also employs performance-oriented evolutionary operations. These two competing forces allow ReStruct to jointly optimize the semantic explainability and empirical performance of meta-structures. Furthermore, ReStruct contains a differential LLM explainer to generate and refine natural language explanations for the discovered meta-structures by reasoning through the search history. Experiments on eight representative HIN datasets demonstrate that ReStruct achieves state-of-the-art performance in both recommendation and node classification tasks. Moreover, a survey study involving 73 graduate students shows that the discovered meta-structures and generated explanations by ReStruct are substantially more comprehensible. Our code and questionnaire are available at https://github.com/LinChen-65/ReStruct.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Exploring the potential of large language models (llms) in learning on graphs. arXiv preprint arXiv:2307.03393 (2023).
  2. Diffmg: Differentiable meta graph search for heterogeneous graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 279–288.
  3. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 135–144.
  4. Talk like a graph: Encoding graphs for large language models. arXiv preprint arXiv:2310.04560 (2023).
  5. Luciano Floridi and Massimo Chiriatti. 2020. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines 30 (2020), 681–694.
  6. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of The Web Conference 2020. 2331–2341.
  7. GPT4Graph: Can Large Language Models Understand Graph Structured Data? An Empirical Evaluation and Benchmarking. arXiv preprint arXiv:2305.15066 (2023).
  8. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532 (2023).
  9. Genetic meta-structure search for recommendation on heterogeneous information network. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 455–464.
  10. Meta structure: Computing relevance in large heterogeneous information networks. In Proceedings of the 22nd ACM SIGKDD International conference on knowledge discovery and data mining. 1595–1604.
  11. Semi-supervised Learning over Heterogeneous Information Networks by Ensemble of Meta-graph Guided Random Walks.. In IJCAI. 1944–1950.
  12. Differentiable meta multigraph search with partial message propagation on heterogeneous information networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8518–8526.
  13. Leveraging meta-path contexts for classification in heterogeneous information networks. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 912–923.
  14. GraphMSE: efficient meta-path selection in semantically aligned feature space for graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4206–4214.
  15. Towards graph foundation models: A survey and beyond. arXiv preprint arXiv:2310.11829 (2023).
  16. Large Language Model Agent for Hyper-Parameter Optimization. arXiv:2402.01881 [cs.LG]
  17. Discovering meta-paths in large heterogeneous information networks. In Proceedings of the 24th international conference on world wide web. 754–764.
  18. Automatic meta-path discovery for effective graph-based recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 1563–1572.
  19. Reinforced neighborhood selection guided multi-relational graph neural networks. ACM Transactions on Information Systems (TOIS) 40, 4 (2021), 1–46.
  20. Mathematical discoveries from program search with large language models. Nature (2023), 1–3.
  21. A survey of heterogeneous information network analysis. IEEE Transactions on Knowledge and Data Engineering 29, 1 (2016), 17–37.
  22. Reflexion: Language agents with verbal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems.
  23. Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. arXiv preprint arXiv:2307.07697 (2023).
  24. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment 4, 11 (2011), 992–1003.
  25. Graphgpt: Graph instruction tuning for large language models. arXiv preprint arXiv:2310.13023 (2023).
  26. Reinforcement learning based meta-path discovery in large-scale heterogeneous information networks. In Proceedings of the aaai conference on artificial intelligence, Vol. 34. 6094–6101.
  27. Can Language Models Solve Graph Problems in Natural Language? arXiv preprint arXiv:2305.10037 (2023).
  28. Heterogeneous graph attention network. In The world wide web conference. 2022–2032.
  29. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022).
  30. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837.
  31. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32, 1 (2020), 4–24.
  32. Relation-aware graph convolutional networks for agent-initiated social e-commerce recommendation. In Proceedings of the 28th ACM international conference on information and knowledge management. 529–538.
  33. Similarity modeling on heterogeneous networks via automatic path discovery. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part II 18. Springer, 37–54.
  34. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601 (2023).
  35. Natural language is all a graph needs. arXiv preprint arXiv:2308.07134 (2023).
  36. Graph transformer networks. Advances in neural information processing systems 32 (2019).
  37. Graph Meets LLMs: Towards Large Graph Models. In NeurIPS 2023 Workshop: New Frontiers in Graph Learning.
  38. Graphtext: Graph reasoning in text space. arXiv preprint arXiv:2310.01089 (2023).
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets