Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations (1702.07826v2)

Abstract: We introduce AI rationalization, an approach for generating explanations of autonomous system behavior as if a human had performed the behavior. We describe a rationalization technique that uses neural machine translation to translate internal state-action representations of an autonomous agent into natural language. We evaluate our technique in the Frogger game environment, training an autonomous game playing agent to rationalize its action choices using natural language. A natural language training corpus is collected from human players thinking out loud as they play the game. We motivate the use of rationalization as an approach to explanation generation and show the results of two experiments evaluating the effectiveness of rationalization. Results of these evaluations show that neural machine translation is able to accurately generate rationalizations that describe agent behavior, and that rationalizations are more satisfying to humans than other alternative methods of explanation.

• The paper presents a neural machine translation framework that generates human-like explanations from AI state-action pairs.
• It employs an encoder-decoder architecture with attention, trained on human commentary from Frogger to ensure relatable rationalizations.
• Empirical results show that AI rationalization enhances user satisfaction over numeric or action-based explanations in varied game scenarios.

Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations

The paper "Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations" introduces a novel technique for explainable AI termed "AI rationalization". This method leverages neural machine translation to convert state-action pairs into intuitive, human-like natural language explanations. The primary focus is on providing explanations that simulate what a human might articulate, without necessitating a verbatim interpretation of the underlying decision-making process.

Core Contributions and Methodology

The authors collect a natural language corpus from human players who verbalize their thought processes while playing the arcade game Frogger. This corpus serves as a training dataset for an encoder-decoder neural network augmented with attention mechanisms, birthing a model capable of translating the internal representations of an AI system's decisions into coherent rationalizations.

One of the noteworthy contributions is the conceptual separation between explanation and interpretability. While interpretability refers to the structural transparency of algorithms, this work defines explanation as more focused on sequential problems and grounded in natural language for the end-user's convenience. AI rationalization, thus, sidesteps precise accuracy for a more real-time and accessible interaction, proposing an alternative path when systems need to communicate decisions effectively to non-expert users.

Experimental Framework and Results

The empirical evaluation involves two experiments conducted in the Frogger game environment. The authors developed three obstacle-rich scenarios (25%, 50%, and 75% filled maps) to test the effectiveness of the rationalization. The encoder-decoder network was trained to associate rich state-action triples with appropriate natural language responses, outperforming both random and majority-vote baselines by statistically significant margins. These encouraging results suggest that neural machine translation is effective in generating situationally appropriate rationalizations.

To examine human satisfaction with different forms of AI explanations, another evaluation was conducted using three robotic agents issuing different types of explanations. Results from human subjects showed a strong preference for the rationalizing robot over the action-declaring robot and numeric expression, with significant p-values indicating higher satisfaction rankings for the natural language rationalizations. The rationale spanned explanatory power, relatability, ludic nature, and adequate detail, highlighting the richer user-agent rapport when language is used over mere numerics or action declarations.

Implications and Future Research

The implications of AI rationalization potentially span various domains, especially those needing seamless human-agent interactions like healthcare, military, and personal service robotics. By using human-like rationalizations, AI systems can appear more relatable, fostering trust and confidence in their decision-making processes.

Future research could delve into how inaccuracies in rationalizations affect human-agent trust or explore diversified applications in more complex environments. Extending this work could involve enriching state-action representations or experimenting with more advanced neural architectures tailored for specific domains.

Conclusion

The authors have paved a new approach to making AI intelligible and approachable through AI rationalization. By harnessing the power of neural machine translation to synthesize pseudo-human rationalizations, they offer a viable solution to the challenge of making AI decisions accessible to the everyday user. While the presented results show promise, the true potential of AI rationalization in practical, real-world applications beckons further exploration.