ERASER: A Benchmark to Evaluate Rationalized NLP Models (1911.03429v2)

Abstract: State-of-the-art models in NLP are now predominantly based on deep neural networks that are opaque in terms of how they come to make predictions. This limitation has increased interest in designing more interpretable deep models for NLP that reveal the `reasoning' behind model outputs. But work in this direction has been conducted on different datasets and tasks with correspondingly unique aims and metrics; this makes it difficult to track progress. We propose the Evaluating Rationales And Simple English Reasoning (ERASER) benchmark to advance research on interpretable models in NLP. This benchmark comprises multiple datasets and tasks for which human annotations of "rationales" (supporting evidence) have been collected. We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i.e., the degree to which provided rationales influenced the corresponding predictions). Our hope is that releasing this benchmark facilitates progress on designing more interpretable NLP systems. The benchmark, code, and documentation are available at https://www.eraserbenchmark.com/

• The paper introduces ERASER as a benchmark to evaluate the interpretability of NLP models by comparing machine-generated and human rationales using precise metrics.
• It employs diverse datasets and baseline models, including hard selection and soft scoring methods, to address challenges in rationale alignment.
• The study finds that while rationale-level supervision improves alignment with human explanations, it does not always boost overall predictive performance.

Evaluating Rationales and Simple English Reasoning (ERASER): A Benchmark for Interpretable NLP Models

The paper presents the ERASER benchmark, a significant contribution aimed at enhancing the interpretability of NLP models. The focus is on rationalized models—systems that elucidate the reasoning behind their predictions through annotated rationales. Current state-of-the-art NLP models, predominantly based on deep neural networks, often achieve high performance but lack transparency in their decision-making processes. ERASER addresses this gap by providing a standardized framework for evaluating interpretability across multiple tasks and datasets, thereby facilitating consistent and comparable progress.

Benchmark Composition

ERASER comprises multiple datasets with associated human-annotated rationales serving as supporting evidence for labels. These datasets span a variety of tasks, including sentiment analysis, natural language inference, and question answering. A core feature of ERASER is its emphasis on the alignment between model-provided rationales and human rationales, evaluated through a suite of proposed metrics.

Metrics for Evaluation

The benchmark introduces several metrics designed to assess the quality and faithfulness of rationales:

• Agreement with Human Rationales: This metric evaluates how well model-generated rationales correspond with those provided by human annotators. It includes measures like Intersection-Over-Union (IOU) and token-level precision and recall.
• Faithfulness Metrics: These metrics, namely comprehensiveness and sufficiency, aim to determine whether the extracted rationales genuinely informed the model's predictions. Comprehensiveness checks if all necessary input features were considered, while sufficiency measures whether the rationales contained enough information for the given prediction.

Baseline Models

The paper outlines several baseline models tested on the ERASER datasets, distinguishing between those that use 'hard' rationale selection (discrete) and 'soft' scoring models (continuous importance). These models include:

• Hard Selection Models: Algorithms like those proposed by Lei et al., which directly select input snippets as rationales, and pipeline models where separate modules identify and classify rationales.
• Soft Scoring Models: Methods using continuous scoring mechanisms over tokens, such as attention-based BERT representations, gradient-based, and LIME methods.

Empirical Findings

The baseline evaluations reveal key insights:

• Model Diversity: Different datasets necessitate models that can handle varying input lengths and rationale granularities, highlighting a need for adaptable interpretative architectures.
• Effectiveness of Training Supervision: Models leveraging rationale-level supervision exhibit improved alignment with human rationales but do not necessarily translate to better predictive performance.
• Attention Limitations: Attention mechanisms often provide rationales considered plausible but not necessarily faithful, aligning with previous observations about their limitations in conveying actual model reasoning.

Implications and Future Directions

ERASER is poised to advance the field of interpretable NLP by standardizing how rationalization is evaluated across diverse tasks. The proposed metrics offer a starting point for measuring both the alignment with human explanations and the faithfulness of model outputs. Given the nascent state of interpretability research, this benchmark lays foundational work that encourages further investigation into designing models capable of nuanced reasoning and explanation, as well as refining evaluative metrics.

Future research might explore more complex models that adapt extraction strategies based on task requirements or extend ERASER to include multilingual datasets. Additionally, investigations into alternative metrics for rationale quality can provide deeper insights into model interpretability.

Conclusion

ERASER represents a comprehensive step forward in establishing standardized practices for evaluating rationale-based interpretability in NLP models. By providing diverse datasets along with robust evaluative criteria, it offers the research community a valuable tool for developing more transparent and justifiable AI systems in natural language processing.