GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability (2403.04483v2)
Abstract: Evaluating and enhancing the general capabilities of LLMs has been an important research topic. Graph is a common data structure in the real world, and understanding graph data is a crucial part for advancing general intelligence. To evaluate and enhance the graph understanding abilities of LLMs, in this paper, we propose a benchmark named GraphInstruct, which comprehensively includes 21 classical graph reasoning tasks, providing diverse graph generation pipelines and detailed reasoning steps. Based on GraphInstruct, we further construct GraphLM through efficient instruction-tuning, which shows prominent graph understanding capability. In order to enhance the LLM with graph reasoning capability as well, we propose a step mask training strategy, and construct a model named GraphLM+. As one of the pioneering efforts to enhance the graph understanding and reasoning abilities of LLMs, extensive experiments have demonstrated the superiority of GraphLM and GraphLM+ over other LLMs. We look forward to more researchers exploring the potential of LLMs in the graph data mining domain through GraphInstruct. Our code for generating GraphInstruct is released publicly at: https://github.com/CGCL-codes/GraphInstruct.
- Flamingo: a visual language model for few-shot learning. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
- Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, RecSys ’23, page 1007–1014, New York, NY, USA. Association for Computing Machinery.
- Albert-László Barabási and Réka Albert. 1999. Emergence of scaling in random networks. Science, 286(5439):509–512.
- Graphllm: Boosting graph reasoning ability of large language model. CoRR, abs/2310.05845.
- Exploring the potential of large language models (llms) in learning on graphs. CoRR, abs/2307.03393.
- Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023).
- Chatlaw: Open-source legal large language model with integrated external knowledge bases.
- Talk like a graph: Encoding graphs for large language models. CoRR, abs/2310.04560.
- Gpt4graph: Can large language models understand graph structured data ? an empirical evaluation and benchmarking. CoRR, abs/2305.15066.
- Explanations as features: Llm-based features for text-attributed graphs. CoRR, abs/2305.19523.
- Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
- Audiogpt: Understanding and generating speech, music, sound, and talking head. CoRR, abs/2304.12995.
- Glen Jeh and Jennifer Widom. 2003. Scaling personalized web search. In Proceedings of the Twelfth International World Wide Web Conference, WWW 2003, Budapest, Hungary, May 20-24, 2003, pages 271–279. ACM.
- Mistral 7b. CoRR, abs/2310.06825.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
- Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
- Ecomgpt: Instruction-tuning large language models with chain-of-task tasks for e-commerce. CoRR, abs/2308.06966.
- Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6).
- Wizardcoder: Empowering code large language models with evol-instruct. CoRR, abs/2306.08568.
- OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
- Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
- Graphgpt: Graph instruction tuning for large language models.
- Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.09288.
- Huatuo: Tuning llama model with chinese medical knowledge.
- Can language models solve graph problems in natural language? In Thirty-seventh Conference on Neural Information Processing Systems.
- Duncan J Watts and Steven H Strogatz. 1998. Collective dynamics of ‘small-world’networks. nature, 393(6684):440–442.
- Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
- Bloomberggpt: A large language model for finance. CoRR, abs/2303.17564.
- Evaluating spatial understanding of large language models. CoRR, abs/2310.14540.
- GPT can solve mathematical problems without a calculator. CoRR, abs/2309.03241.
- Dyval: Graph-informed dynamic evaluation of large language models. In The Twelfth International Conference on Learning Representations.