Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective
(2401.04374)Abstract
Given the complexity and lack of transparency in deep neural networks (DNNs), extensive efforts have been made to make these systems more interpretable or explain their behaviors in accessible terms. Unlike most reviews, which focus on algorithmic and model-centric perspectives, this work takes a "data-centric" view, examining how data collection, processing, and analysis contribute to explainable AI (XAI). We categorize existing work into three categories subject to their purposes: interpretations of deep models, referring to feature attributions and reasoning processes that correlate data points with model outputs; influences of training data, examining the impact of training data nuances, such as data valuation and sample anomalies, on decision-making processes; and insights of domain knowledge, discovering latent patterns and fostering new knowledge from data and models to advance social values and scientific discovery. Specifically, we distill XAI methodologies into data mining operations on training and testing data across modalities, such as images, text, and tabular data, as well as on training logs, checkpoints, models and other DNN behavior descriptors. In this way, our study offers a comprehensive, data-centric examination of XAI from a lens of data mining methods and applications.
Overview
-
The paper discusses the importance of Explainable AI (XAI) for understanding and trusting AI systems, particularly Deep Neural Networks.
-
It highlights the need for XAI in high-stakes situations and focuses on the 'data-centric' aspects that contribute to AI explainability.
-
The paper categorizes XAI research into three main areas: interpreting deep models, assessing the influence of training data, and extracting domain-specific insights.
-
Various techniques used in XAI, such as LIME, Integrated Gradients, and Shapley values, are examined for their role in enhancing interpretability.
-
The work endeavors to bridge the gap between AI system functionality and human-centric analysis, stressing the connection to societal norms and scientific advancement.
Introduction
The field of AI is shaped by Deep Neural Networks (DNNs), which, due to their opaque nature, pose interpretability challenges. As AI gains prevalence in high-stakes scenarios, being able to trust and understand these systems becomes imperative. This is where Explainable AI (XAI) steps in, offering tools and techniques to make AI decisions more transparent. But beyond the algorithms themselves, this exploration emphasizes the 'data-centric' aspects of XAI – considering the role of data in explainability, from its collection to the way insights are gleaned from it – and categorizes existing work into three key purposes.
Interpretations of Deep Models
Understanding the rationale behind a model's predictions is vital for trust and transparency. Interpretations of deep models dive into feature attributions, highlighting specific data points responsible for predictions. Techniques here may include perturbation-based methods like LIME, which adjust input data and observe the effect on predictions, or differentiation-based methods like Integrated Gradients, which compute gradients as a measure of input feature importance. Activation and attention-based methods further enrich interpretability for models, making sense of how input data is weighted in decision-making.
Influences of Training Data
The quality of machine learning outcomes is a function of training data quality. XAI acknowledges this by pinpointing influential data points, aiding in the refinement of training processes for enhanced simplicity and performance. Methods for evaluating sample contributions include gradient-based approaches for assessing data point influence, and resampling techniques like Leave-One-Out, which measure the impact of removing certain data points from the training set. Game theoretical methods like Shapley values, traditionally applied in cooperative games, are also adapted to quantify the valuation of individual training samples.
Insights of Domain Knowledge
XAI's third purpose is to extract and elucidate domain-specific patterns from data, translating these into actionable insights. This facet of XAI not only serves to interpret AI processes but also aims to advance human understanding within various domains, such as healthcare and finance. XAI enables the discovery of latent patterns which can be crucial for scientific exploration and social value promotion, ensuring AI's decisions align with societal norms and contribute to the broader knowledge base.
Bridging the Interpretability Gap
In summary, the role of XAI extends beyond explaining model decisions to understanding the influence of data and drawing out domain insights that can inform social values and expediate scientific discovery. Through a structured, data-centric approach, this work categorizes and analyzes XAI methodologies, ultimately aiming to better integrate human-centric analyses with AI, for improved human-AI synergy. This comprehensive view of XAI within the context of data mining aids in the ongoing development of AI systems that are not only powerful but also accountable, trustworthy, and transparent.
Create an account to read this summary for free: