Recourse for reclamation: Chatting with generative language models
(2403.14467)Abstract
Researchers and developers increasingly rely on toxicity scoring to moderate generative language model outputs, in settings such as customer service, information retrieval, and content generation. However, toxicity scoring may render pertinent information inaccessible, rigidify or "value-lock" cultural norms, and prevent language reclamation processes, particularly for marginalized people. In this work, we extend the concept of algorithmic recourse to generative language models: we provide users a novel mechanism to achieve their desired prediction by dynamically setting thresholds for toxicity filtering. Users thereby exercise increased agency relative to interactions with the baseline system. A pilot study ($n = 30$) supports the potential of our proposed recourse mechanism, indicating improvements in usability compared to fixed-threshold toxicity-filtering of model outputs. Future work should explore the intersection of toxicity scoring, model controllability, user agency, and language reclamation processes -- particularly with regard to the bias that many communities encounter when interacting with generative language models.
Overview
-
The paper introduces a method to integrate algorithmic recourse in Generative Language Models (GLMs) to allow users to adjust toxicity thresholds for a more personalized interaction.
-
This study conducts a pilot with 30 participants to compare the effectiveness of fixed-thresholding versus dynamic-thresholding systems in conversations about identity.
-
Findings suggest that participants preferred the dynamic thresholding system, noting improvements in usability and expressing a desire for more control over content filtering.
-
The research advocates for further development of this approach to better meet diverse user needs and enhance the inclusivity and responsiveness of AI interfaces.
Extending Algorithmic Recourse to Conversational AI Interactions
Introduction to the Study
Recent advancements in Generative Language Models (GLMs) have significantly influenced various sectors, including customer service, information retrieval, and content generation. To mitigate potential harm, developers commonly deploy toxicity scores, filtering out content deemed offensive or harmful. This paper introduces a novel approach, incorporating algorithmic recourse within GLM interactions to enhance user agency and refine toxicity threshold mechanisms. Through a pilot study with 30 participants, the researchers explore the utility of allowing users to dynamically adjust toxicity thresholds, thereby customizing their interaction based on personal or context-specific tolerances.
Problem Context and Proposed Solution
Toxicity scoring, while essential for moderating content, can inadvertently restrict access to relevant information and impede the process of language reclamation for marginalized communities. Recognizing these challenges, the authors propose a dynamic threshold system for toxicity filtering, granting users more control over what content is filtered. This system distinguishes between absolute toxicity thresholds set by platforms and user-defined tolerances. Through a two-step feedback mechanism, users can decide to view content flagged by the system and subsequently determine whether similar content should be filtered in future interactions.
Methodological Approach
The study utilized a within-participants design, comparing a fixed-thresholding system (control condition) with the proposed dynamic-thresholding system (recourse condition). Participants engaged in conversations on the theme of "identity" with a GLM, encountering both system defaults and the recourse mechanism depending on the condition. Post-interaction, participants rated the system's usability and provided qualitative feedback. Analysis centered on the feasibility of the recourse mechanism, user satisfaction, and the types of themes emerging from user experiences.
Findings and Observations
Participants widely exercised the recourse option when available, indicating a strong preference for a more customizable experience. Quantitative analysis revealed improvements in usability scores in the recourse condition, suggesting that dynamic thresholding could enhance user satisfaction. Qualitatively, participants highlighted the impact of safety responses on their interaction strategies and expressed frustration with the default filtering system's limitations. Notably, the mechanism's perceived complexity suggested further optimization is needed to fulfill user expectations of control.
Implications and Future Directions
The research presents algorithmic recourse as a viable strategy for improving dialogue interfaces, emphasizing the importance of user empowerment in interactions with AI. This study highlights the need for future investigations into optimal threshold settings and the feasibility of expanding user control over AI interactions. Given the pilot nature of this study, there is a significant opportunity for further research, particularly in understanding diverse user populations' needs and expectations.
Conclusion
The incorporation of algorithmic recourse into GLM interactions represents a promising avenue toward aligning AI outputs with user expectations and societal values. By enabling users to tailor toxicity thresholds, this approach fosters a more inclusive and responsive interactive experience. Continued exploration and refinement of this mechanism will be crucial in realizing its full potential for supporting user agency and mitigating biases in AI-generated content.
Create an account to read this summary for free: