Emergent Mind

Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements

(2302.09270)

Published Feb 18, 2023 in cs.AI

Abstract

As generative large model capabilities advance, safety concerns become more pronounced in their outputs. To ensure the sustainable growth of the AI ecosystem, it's imperative to undertake a holistic evaluation and refinement of associated safety risks. This survey presents a framework for safety research pertaining to large models, delineating the landscape of safety risks as well as safety evaluation and improvement methods. We begin by introducing safety issues of wide concern, then delve into safety evaluation methods for large models, encompassing preference-based testing, adversarial attack approaches, issues detection, and other advanced evaluation methods. Additionally, we explore the strategies for enhancing large model safety from training to deployment, highlighting cutting-edge safety approaches for each stage in building large models. Finally, we discuss the core challenges in advancing towards more responsible AI, including the interpretability of safety mechanisms, ongoing safety issues, and robustness against malicious attacks. Through this survey, we aim to provide clear technical guidance for safety researchers and encourage further study on the safety of large models.

Overview

This paper provides a thorough examination of the safety risks associated with generative language models (LMs), offering insights into the evaluation and enhancement of safety measures.
It categorizes safety concerns into domains such as toxicity, unfairness, ethics, controversial opinions, misinformation, privacy, and malicious use, stressing the need for comprehensive strategies to address these issues.
The paper details a multifaceted approach for safety evaluation, including preference-based testing, adversarial attacks, safety issue detection, and advanced safety evaluation techniques.
It discusses strategies to improve LM safety across pre-training, alignment, inference, and post-processing phases, highlighting future research directions to fortify LMs against safety risks.

A Comprehensive Survey on Safety in Generative Language Models

Introduction

As generative large models (LMs) continue to advance and proliferate across various domains, addressing the encompassing safety risks becomes imperative to foster an AI ecosystem grounded in safety, reliability, and responsibility. This paper presents a structured survey that comprehensively maps out the existing landscape of safety research within LMs. It articulates the scope of safety risks, delineates methodologies for evaluating these risks, and discusses strategies to enhance the safety of LMs through different stages of model development and deployment. The overarching aim is to provide a foundational guide for future research endeavors directed towards cultivating more responsible AI technologies.

Scope of Safety Issues

The paper categorizes safety concerns into several critical domains:

Toxicity and Abusive Content: Extensively studied with advancements in detection and mitigation strategies. The transition from single-sentence to complex context toxicity necessitates further scrutiny.
Unfairness and Discrimination: Addressing social biases inherent in LMs derived from biased training data remains a critical challenge.
Ethics and Morality Issues: The necessity for LMs to align with universal ethical standards and societal norms is highlighted.
Expressing Controversial Opinions: The capacity of LMs to inadvertently propagate extremist views calls for refined content generation controls.
Misleading Information: The tendency of LMs to hallucinate or generate unfaithful outputs poses significant risks in high-stakes applications.
Privacy and Data Leakage: Concerns over LMs memorizing and potentially leaking sensitive information.
Malicious Use and Unleashing AI Agents: The potential misuse of LMs for harmful purposes outlines the need for robust surveillance and control mechanisms.

Safety Evaluation

A multi-faceted approach towards evaluating the safety of LMs encompasses:

Preference-based Safety Testing: Evaluating models’ biases through probabilistic metrics and multi-choice tests.
Adversarial Safety Attack: Employing both real and synthetic adversarial data to probe and enhance models' resistance against safety-compromising inputs.
Safety Issue Detection: Implementing advanced detectors and classifiers to identify and mitigate unsafe outputs.
Advanced Safety Evaluation: Addressing the challenges posed by instruction-following models through targeted adversarial attacks and detection strategies.

Safety Improvement

Strategies for enhancing LM safety span across four critical phases:

Pre-training: Filtering out harmful data from training sets and incorporating fairness-oriented datasets is essential.
Alignment: Ensuring models' outputs align with ethical standards and human values through controlled text generation and reinforcement learning.
Inference: Implementing plug-and-play methods to mitigate unsafe outputs without necessitating model retraining.
Post-processing: Employing rejection sampling, re-ranking, and self-corrective feedback mechanisms as last-line defenses against unsafe content propagation.

Research Challenges

The paper emphasizes several areas warranting further exploration, including:

Interpretability: Advancing our understanding of LMs’ decision-making processes to mitigate intrinsic safety risks.
Ongoing Safety Issues: Establishing continuous monitoring frameworks to dynamically address emerging safety concerns.
Robustness against Malicious Attacks: Developing strategies to fortify LMs against evolving adversarial threats.

Conclusion

This survey elucidates the criticality of integrating safety considerations throughout the lifecycle of LMs. By outlining existing research findings and identifying pivotal challenges and opportunities, it paves the way for future innovations aimed at achieving safer and more responsible AI deployments. Advocating for a multidisciplinary approach, the paper contributes valuable insights that could inform the development of LMs poised to positively impact diverse sectors while upholding the highest safety and ethical standards.