Large Language Models Understand and Can be Enhanced by Emotional Stimuli
(2307.11760)Abstract
Emotional intelligence significantly impacts our daily behaviors and interactions. Although LLMs are increasingly viewed as a stride toward artificial general intelligence, exhibiting impressive performance in numerous tasks, it is still uncertain if LLMs can genuinely grasp psychological emotional stimuli. Understanding and responding to emotional cues gives humans a distinct advantage in problem-solving. In this paper, we take the first step towards exploring the ability of LLMs to understand emotional stimuli. To this end, we first conduct automatic experiments on 45 tasks using various LLMs, including Flan-T5-Large, Vicuna, Llama 2, BLOOM, ChatGPT, and GPT-4. Our tasks span deterministic and generative applications that represent comprehensive evaluation scenarios. Our automatic experiments show that LLMs have a grasp of emotional intelligence, and their performance can be improved with emotional prompts (which we call "EmotionPrompt" that combines the original prompt with emotional stimuli), e.g., 8.00% relative performance improvement in Instruction Induction and 115% in BIG-Bench. In addition to those deterministic tasks that can be automatically evaluated using existing metrics, we conducted a human study with 106 participants to assess the quality of generative tasks using both vanilla and emotional prompts. Our human study results demonstrate that EmotionPrompt significantly boosts the performance of generative tasks (10.9% average improvement in terms of performance, truthfulness, and responsibility metrics). We provide an in-depth discussion regarding why EmotionPrompt works for LLMs and the factors that may influence its performance. We posit that EmotionPrompt heralds a novel avenue for exploring interdisciplinary knowledge for human-LLMs interaction.
Overview
-
LLMs exhibit potential emotional intelligence that can be tapped to improve their performance.
-
Various LLMs were evaluated across tasks with EmotionPrompt, showing up to 10.9% improvement in generative tasks.
-
A human study confirmed that emotional prompts enhanced LLMs' truthfulness and responsibility.
-
Analysis revealed emotional stimuli affect LLMs' attention mechanisms, emphasizing positive words.
-
The study suggests that emotion augmentation like EmotionPrompt could advance AI interactions and prompt further research.
Introduction
Emotional intelligence plays a significant role in human behavior and interactions, prompting a question in the realm of AI: can LLMs, which push the boundaries in various tasks, also understand and be enhanced by emotional stimuli? A recent study scrutinizes LLMs from this perspective, aiming to uncover their potential emotional intelligence.
Exploration of Emotional Intelligence in LLMs
The study examines various LLMs, such as Flan-T5-Large, Vicuna, Llama 2, BLOOM, ChatGPT, and GPT-4, across 45 tasks that represent both deterministic and generative applications. The LLMs' grasp of emotional intelligence is judged by injecting emotional contexts into prompts—coined EmotionPrompt—in automatic as well as human-assisted evaluations. Results indicate that LLMs performance can be significantly enhanced with 8.00% relative improvement in deterministic tasks, and about 10.9% average improvement in the performance of generative tasks.
Human Study and Generative Tasks
To assess more nuanced aspects of generative tasks, such as quality and creativity, which elude automatic evaluation, a human study with 106 participants was conducted. The study endorsed that emotion-augmented prompts aided the LLMs, leading to improved outcomes in terms of performance, truthfulness, and responsibility. LLMs' generative abilities were assessed through various metrics, deriving significant improvements almost across the board with emotional prompts.
Analysis and Insights
Understanding why EmotionPrompt works is key. The study explore the LLM’s attention mechanism, observing that emotional stimuli not only enrich the representation of prompts but also tend to assign greater weight to positive words. An exploration into diverse emotional stimuli reveals that specific prompts outperform others, suggesting multiple factors at play, including task complexity and metrics employed. An examination of features like LLM model size, temperature, and pre-training strategies sheds light on varying influences on performance enhancement through EmotionPrompt.
Conclusion and Future Work
The research concludes that emotional intelligence, when implemented in the form of EmotionPrompt, can improve the performance of LLMs. EmotionPrompt, an approach that infuses emotional context into prompts, showcases promise in evolving human-LLM interaction, bearing implications for advancements in both AI and social sciences. Moving forward, the research opens up new questions and potential explorations at the intersection of psychology and LLM implementation, calling for deeper analysis into the emotional intelligence exhibited by AI systems.
Create an account to read this summary for free: