LLMs achieve adult human performance on higher-order theory of mind tasks (2405.18870v2)

Abstract: This paper examines the extent to which LLMs have developed higher-order theory of mind (ToM); the human ability to reason about multiple mental and emotional states in a recursive manner (e.g. I think that you believe that she knows). This paper builds on prior work by introducing a handwritten test suite -- Multi-Order Theory of Mind Q&A -- and using it to compare the performance of five LLMs to a newly gathered adult human benchmark. We find that GPT-4 and Flan-PaLM reach adult-level and near adult-level performance on ToM tasks overall, and that GPT-4 exceeds adult performance on 6th order inferences. Our results suggest that there is an interplay between model size and finetuning for the realisation of ToM abilities, and that the best-performing LLMs have developed a generalised capacity for ToM. Given the role that higher-order ToM plays in a wide range of cooperative and competitive human behaviours, these findings have significant implications for user-facing LLM applications.

• The paper demonstrates that GPT-4 and Flan-PaLM achieve human-level performance on complex higher-order ToM tasks using the MoToMQA benchmark.
• The study employs rigorous testing with controlled human and LLM experiments, highlighting the impact of prompt design and anchoring effects.
• Results indicate that model size and fine-tuning are crucial for attaining advanced ToM capabilities, with implications for AI ethics and real-world applications.

LLMs and Theory of Mind Tasks

The paper "LLMs achieve adult human performance on higher-order theory of mind tasks" (2405.18870) presents an extensive paper on the capabilities of LLMs in executing Theory of Mind (ToM) tasks, which involves reasoning about recursive mental and emotional states. It introduces a new benchmark, Multi-Order Theory of Mind Question {content} Answer (MoToMQA), to test LLMs against human performance.

Introduction to Theory of Mind in LLMs

Theory of Mind (ToM) is a critical aspect of human social intelligence, enabling individuals to infer and interpret the mental states of others. Previous studies focused primarily on second-order ToM tasks, but this paper extends the analysis to orders 2 through 6, using a novel benchmark based on the Imposing Memory Task (IMT). The findings indicate that GPT-4 and Flan-PaLM achieve adult-level ToM capability, particularly excelling at higher-order tasks.

Experimental Setup and Procedures

Human Testing

Human participants were screened for English proficiency and assigned to read and respond to stories containing ToM and factual statements. The survey setup included story conditions that controlled for recall ability, and question ordering to test potential anchoring effects.

LLM Testing

Five models, including GPT-3.5, GPT-4, LaMDA, PaLM, and Flan-PaLM, were subjected to the MoToMQA. The LLMs processed stories and provided responses based on log probabilities of candidate answers. The paper considered the influence of prompt design and anchoring effects on LLM performance.

Results

The paper revealed significant performance disparities among the tested models on ToM tasks, with GPT-4 and Flan-PaLM showing the highest accuracy, comparable to human performance.

Figure 1: Performance of humans and various LLMs on ToM tasks up to order 6.

• GPT-4 Performance: Exceeds human performance on 6th-order ToM tasks, indicating a strong ability to manage complex recursive reasoning.
• Flan-PaLM Performance: Achieves near-human performance but struggles with higher-order tasks relative to GPT-4.
• Comparison Among Models: Smaller and non-finetuned models like LaMDA exhibit lower ToM capacities compared to their larger, instruction-finetuned counterparts, illustrating the importance of model size and finetuning.

Discussion

The paper underscores the relationship between LLM size and ToM capabilities, suggesting that beyond a certain scale, LLMs develop advanced cognitive abilities. This might be attributed to scaling laws governing neural architecture efficiency. Additionally, finetuned models like Flan-PaLM show improved ToM performance due to their enhanced capacity for following human-like instructions.

High-performance models have practical implications in real-world applications involving complex social interactions and moral reasoning. However, there are inherent risks associated with LLMs possessing advanced ToM abilities, such as the potential for manipulation, demanding robust ethics and alignment strategies.

Limitations and Future Research

The paper's constraints include limited linguistic diversity and evaluation scope, emphasizing English-only benchmarks. Further research is needed to develop culturally comprehensive datasets and extend the tasks to higher ToM orders. Exploring multimodal training paradigms could also enhance the generalizability and robustness of ToM capabilities in LLMs.

Conclusion

The paper demonstrates that LLMs, specifically GPT-4 and Flan-PaLM, can perform ToM tasks at a level commensurate with adult humans, marking significant progress in AI's cognitive modeling abilities. This highlights the necessity for continuous research into the mechanisms underpinning these capabilities and their implications for AI ethics and alignment.

These findings offer a new perspective on assessing cognitive tasks executed by LLMs, challenging preconceived notions about their understanding and proposing a requirement to recognize LLM-induced cognitive equivalence.