- The paper demonstrates that post-hoc GNN explainers lose effectiveness when even minor label noise is present.
- It evaluates explanation quality using fidelity+ and fidelity- metrics across synthetic and real-world datasets.
- Findings include a surprising recovery in performance beyond 50% noise, indicating the need for more robust explainer methods.
On the Robustness of Post-hoc GNN Explainers to Label Noise
The paper "On the Robustness of Post-hoc GNN Explainers to Label Noise" explores an important aspect of the interpretability of Graph Neural Networks (GNNs): the robustness of post-hoc GNN explainers when faced with label noise. In recent years, GNNs have been increasingly utilized for graph-structured data, drawing attention for their effectiveness in various fields but also for their opacity. Post-hoc explainers, such as GNNExplainer and PGExplainer, have been developed to address this limitation and facilitate insight into the decision-making process of GNNs. However, the robustness of these explainers under conditions of label noise, which is prevalent in real-world datasets, remains underexplored.
The paper introduces two central research questions: (i) Are post-hoc GNN explainers robust to malicious label noise? (ii) Does the robustness of the GNN model guarantee effective explanations by the post-hoc explainers? To address these questions, the authors conduct a systematic empirical investigation into the effects of label noise on the explanations generated by post-hoc GNN explainers. These evaluations are conducted across four datasets, two of which are synthetic (BA-2motifs and BA-Multi-Shapes), and two are real-world datasets (MUTAG and Graph-Twitter).
The results demonstrate that post-hoc GNN explainers are indeed susceptible to label noise, which significantly decreases the quality of explanations even when the noise level is relatively minor. This sensitivity is noteworthy considering the robustness observed in GNN models themselves, suggesting that the stability of GNN models does not directly translate to the robustness of their explanations. A particularly surprising finding is that beyond a noise threshold of 50%, explanation effectiveness begins to recover despite increasing noise, hinting at a complex interaction between noise levels and feature recognition by GNN explainers.
The paper utilizes two key metrics for evaluating the quality of explanations: fidelity+ and fidelity-. Notably, the paper criticizes the use of fidelity- as a robust metric in this context, suggesting that low values might not necessarily indicate poor explanation quality in scenarios involving high label noise.
These findings have broad implications. Practically, they suggest that caution should be exercised when deploying GNN explainers in domains where label noise is common. There is also a methodological implication regarding the metrics used to evaluate explanation robustness. Theoretically, the results highlight the need for more noise-resilient explainer methodologies. Future research directions proposed include designing more robust post-hoc explainers, refining evaluation metrics, and developing larger-scale benchmark datasets to better paper these phenomena.
Overall, this investigation provides critical insights into a previously underexplored area of GNN interpretability, emphasizing the necessity for further research to develop robust solutions capable of maintaining fidelity under noise. As AI models continue to be deployed in high-stakes environments, understanding the nuances of their interpretability and robustness will remain crucial.
