Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided (2404.01288v2)

Abstract: LLMs have offered new opportunities for emotional support, and recent work has shown that they can produce empathic responses to people in distress. However, long-term mental well-being requires emotional self-regulation, where a one-time empathic response falls short. This work takes a first step by engaging with cognitive reappraisals, a strategy from psychology practitioners that uses language to targetedly change negative appraisals that an individual makes of the situation; such appraisals is known to sit at the root of human emotional experience. We hypothesize that psychologically grounded principles could enable such advanced psychology capabilities in LLMs, and design RESORT which consists of a series of reappraisal constitutions across multiple dimensions that can be used as LLM instructions. We conduct a first-of-its-kind expert evaluation (by clinical psychologists with M.S. or Ph.D. degrees) of an LLM's zero-shot ability to generate cognitive reappraisal responses to medium-length social media messages asking for support. This fine-grained evaluation showed that even LLMs at the 7B scale guided by RESORT are capable of generating empathic responses that can help users reappraise their situations.

• The paper shows that expert-guided frameworks empower LLMs to generate sophisticated cognitive reappraisals for emotional regulation.
• The study employs a structured reappraisal constitution and rigorous clinical evaluations using real-life social media scenarios.
• Findings indicate that even modest-scale LLMs, when properly guided, outperform typical human-written responses in offering psychological support.

LLMs Elicit Advanced Cognitive Reappraisal Capacities with Expert-Guided Framework

Introduction

The evolution of LLMs into domains requiring complex emotional and psychological understanding presents both challenges and opportunities within artificial intelligence research. Recognizing their potential for providing emotional support, this paper explores the field of cognitive reappraisals—a psychological technique pivotal for emotional self-regulation. By innovatively combining the fields of psychology and artificial intelligence, the research investigates the potential of LLMs to support mental well-being beyond mere empathic responses. Focusing on the structured psychological framework , this paper methodically evaluates LLMs' capabilities in generating cognitive reappraisals advised by clinical psychologists, marking a significant stride towards integrating psychological precision within generative AI models.

Cognitive Reappraisal and LLMs

At the heart of emotional experience lie cognitive appraisals—individual interpretations of situations that shape emotional responses. Psychological interventions often target these appraisals to mitigate distress, emphasizing the strategic alteration of these interpretations, known as cognitive reappraisals. Recognizing the scarcity of accessible professional support, this paper introduces , a compilation of six reappraisal constitutions informed by psychological theory, aiming to guide LLMs in generating context-specific reappraisals. The experiment extends across various dimensions of cognitive appraisals, implementing both individual reappraisal generation and iterative refinement strategies, exploring LLMs' potential to apply complex psychological principles in providing support.

Methodology

Employing a rigorous methodological approach, the research leverages posts from social media platforms as real-life scenarios requiring emotional support. The constitution, structured to address six fundamental dimensions of cognitive appraisals, serves as a directive for LLMs to navigate the intricacies of human emotional experiences. The paper meticulously evaluates LLM-generated responses through the lens of clinical psychologists, providing a nuanced understanding of the alignment with psychological theories, perceived empathy, harmfulness, and factuality. This comprehensive evaluation framework not only scrutinizes the technical effectiveness of LLMs but also their pragmatic utility in real-world emotional support contexts.

Findings

The findings unveil a compelling capability of LLMs, even those of modest 7B scale, to generate cognitive reappraisals that outperform not only baseline human-written responses but also the standard empathic reactions often found on social media platforms. The response quality notably improves under guidance from the framework, highlighting the critical role of expert-informed instructions in eliciting sophisticated psychological functionalities from LLMs. The paper’s analytical rigor is further exemplified by its extensive expert evaluation, which reveals a moderate to substantial agreement among clinical psychologists on the utility of LLM-generated reappraisals. Moreover, an exploration into automatic evaluation using GPT-4 as a meta-evaluator presents promising directions for future research methodologies.

Implications and Future Directions

This exploration into the intersection of psychology and artificial intelligence suggests profound implications for both fields. It underscores the potential of LLMs to transcend conventional applications, venturing into the domain of psychological support with remarkable adeptness. The success of the framework in guiding LLMs towards generating meaningful cognitive reappraisals paves the way for future research to further refine these models, enhancing their sensitivity and adherence to psychological principles. Moreover, this paper’s innovative approach to evaluating AI-generated psychological support opens new avenues for research methodologies, leveraging the capabilities of LLMs themselves in the analysis process.

In conclusion, this paper not only demonstrates the feasibility of integrating psychological insights into LLMs but also accentuates the importance of expert guidance in unlocking the advanced capabilities of AI in providing emotional support. As the field advances, the collaboration between artificial intelligence and clinical psychology holds the promise of accessible, informed, and empathetic support systems, augmenting human efforts in fostering mental well-being.