PersLLM: A Personified Training Approach for Large Language Models (2407.12393v5)

Abstract: LLMs exhibit human-like intelligence, enabling them to simulate human behavior and support various applications that require both humanized communication and extensive knowledge reserves. Efforts are made to personify LLMs with special training data or hand-crafted prompts, while correspondingly faced with challenges such as insufficient data usage or rigid behavior patterns. Consequently, personified LLMs fail to capture personified knowledge or express persistent opinion. To fully unlock the potential of LLM personification, we propose PersLLM, a framework for better data construction and model tuning. For insufficient data usage, we incorporate strategies such as Chain-of-Thought prompting and anti-induction, improving the quality of data construction and capturing the personality experiences, knowledge, and thoughts more comprehensively. For rigid behavior patterns, we design the tuning process and introduce automated DPO to enhance the specificity and dynamism of the models' personalities, which leads to a more natural opinion communication. Both automated metrics and expert human evaluations demonstrate the effectiveness of our approach. Case studies in human-machine interactions and multi-agent systems further suggest potential application scenarios and future directions for LLM personification.

• The paper presents PersLLM, which integrates personality traits into LLM training to create more authentic and dynamic human-like interactions.
• It employs psychology-grounded data construction and Direct Preference Optimization to embed consistent and evolving personality characteristics.
• Experiments in single-agent, multi-agent, and human-agent settings demonstrate improved interaction quality and realistic simulation of distinct personalities.

PersLLM: A Personified Training Approach for LLMs

The paper "PersLLM: A Personified Training Approach for LLMs" explores the integration of personality traits into LLMs to enhance their utility in various applications, such as social simulations, human-machine interactions, and collaborative systems. The authors recognize the necessity for LLMs to exhibit distinct personality characteristics to improve their practical applicability, addressing the uniform response patterns often seen in current models.

Introduction

The authors identify a deficiency in current LLMs: a lack of personified traits that would allow them to simulate human interactions authentically. They propose a personified training methodology called PersLLM, which incorporates psychology-grounded principles such as social practice, consistency, and dynamic development directly into LLM parameters, thereby enhancing personality traits.

Figure 1: Schematic diagram for PersLLM's personified training approach, displaying data collection, annotation, and tuning strategies.

Methodology

Personified Data Construction

The construction of personified data involves collecting raw data on target personalities. This includes biographies, personal writings, and third-party descriptions that offer insights into an individual's experiences, knowledge, and speech style. The raw data is then annotated through conversational tuning strategies, including the use of temporal labels for dynamic development, anti-induced data to maintain consistency, and Chain-of-Thought (CoT) prompting to ensure depth in reasoning.

Model Training

The annotated conversational data is integrated into model training, where personality features are embedded into the LLM parameters rather than relying solely on external prompt engineering or retrieval-augmented generation (RAG). The approach uses automatic Direct Preference Optimization (DPO) to differentiate responses across different temporal stages and personalities.

Experiments and Results

Single-agent Evaluation

The paper includes a single-agent evaluation using the Harry Potter dataset, demonstrating that PersLLM generates responses more aligned with distinct personalities compared to baseline methods. The authors utilize BLEU and ROUGE metrics to assess the model's performance in text generation tasks.

Figure 2: Comparison of attitudes, knowledge reserves, and temporal changes in personalities of personified models versus role-play instruction tuning models.

Multi-agent Communication

PersLLM shows efficacy in simulating human-like interactions in multi-agent settings, particularly in maintaining opinion consistency and fostering collaborative creativity among agents. Conflict and cooperation instances are tested among different personality models, such as Harry and Voldemort, and John Nash and Huiyin Lin.

Figure 3: Conflict between Harry Potter and Lord Voldemort showcasing PersLLM's ability to maintain distinct personalities and prevent opinion convergence.

Figure 4: Cooperation between John Nash and Huiyin Lin, illustrating PersLLM's capability to enable specialized interdisciplinary dialogues.

Human-Agent Interaction

Human-agent interaction tests suggest that personified models provide a superior interactive experience compared to generic models. The evaluations, conducted with real participants, highlight improvements in trust, companionship, character simulation, and user engagement.

Figure 5: Evaluation of human interaction with the PersLLM Huiyin Lin agent, showing enhanced satisfaction in multiple metrics.

Conclusion

The PersLLM framework successfully models dynamic personality traits in LLMs, showing potential benefits for applications in human-like conversational agents and simulated social research. The personified approach emphasizes consistency, adaptability, and the realistic portrayal of complex human interactions across various contexts. Future work will aim to improve real-time personality modeling and address ethical concerns related to privacy and the misuse of personification technologies.