Bridging Adversarial Robustness and Gradient Interpretability

Abstract: Adversarial training is a training scheme designed to counter adversarial attacks by augmenting the training dataset with adversarial examples. Surprisingly, several studies have observed that loss gradients from adversarially trained DNNs are visually more interpretable than those from standard DNNs. Although this phenomenon is interesting, there are only few works that have offered an explanation. In this paper, we attempted to bridge this gap between adversarial robustness and gradient interpretability. To this end, we identified that loss gradients from adversarially trained DNNs align better with human perception because adversarial training restricts gradients closer to the image manifold. We then demonstrated that adversarial training causes loss gradients to be quantitatively meaningful. Finally, we showed that under the adversarial training framework, there exists an empirical trade-off between test accuracy and loss gradient interpretability and proposed two potential approaches to resolving this trade-off.

• The paper demonstrates that adversarial training realigns loss gradients with the image manifold, enhancing visual interpretability.
• It introduces a quantitative evaluation framework using ROAR and KAR metrics to link gradient interpretability with varying levels of adversarial robustness.
• The study reveals a trade-off between natural image accuracy and gradient interpretability, urging further advances in training methodologies.

Bridging Adversarial Robustness and Gradient Interpretability

Problem Statement and Motivation

The paper "Bridging Adversarial Robustness and Gradient Interpretability" (1903.11626) investigates the observed phenomenon that adversarially trained deep neural networks (DNNs) yield more visually interpretable loss gradients, aligning closely with salient features in images and human perceptual expectations. Despite wide recognition of this property, prior works lack rigorous explanations regarding the underlying mechanisms and quantitative assessment of gradient interpretability related to adversarial robustness. The authors aim to formally analyze why adversarial training enhances loss gradient interpretability, to establish quantitative frameworks for evaluation, and to identify trade-offs between robustness, interpretability, and test accuracy.

Visual Interpretability of Loss Gradients

The empirical analysis demonstrates that loss gradients from adversarially trained models are confined closer to the image manifold. Using VAE-GAN-based projections, the authors quantify the distance of adversarial examples from the image manifold ($d_\pi$) for both standard and adversarially robust classifiers across MNIST, FMNIST, and CIFAR-10. Adversarially trained models produce adversarial examples that are significantly closer to the image manifold compared to standard DNNs. Furthermore, the strength of the adversarial training (i.e., the magnitude of attack perturbation, $\epsilon$) has a direct effect: higher robustness yields loss gradients increasingly aligned with the data distribution.

The theoretical conjecture, grounded in the boundary tilting perspective, posits that adversarial training removes decision boundary tilting along low variance directions in the data, causing adversarial perturbations (and thus gradients) to move data points within the manifold of natural images. This is experimentally validated on a two-dimensional toy dataset: robust training eliminates boundary tilting and ensures perturbations traverse the manifold, reinforcing both the interpretability and perceptual quality of loss gradients.

Quantitative Interpretability Framework

The authors formalize an attribution-based evaluation framework, considering families of networks ($\mathcal{F}$), attribution methods ($\mathcal{G}$), and metrics ($\mathcal{M}$). They treat loss gradients (and the Gradient $*$ Input variant) as attribution methods and employ Remove and Retrain (ROAR) and Keep and Retrain (KAR) protocols to evaluate interpretability. These metrics quantify how occluding important or unimportant pixels, as indicated by the attribution, degrades or preserves test accuracy after retraining the classifier.

Empirically, there is a strong positive correlation between adversarial robustness and quantitative gradient interpretability (ROAR and KAR scores). As attack strength in adversarial training increases, gradients more accurately reflect the internal representations utilized by the DNN decision process. Moreover, Gradient $*$ Input achieves superior scores compared to plain loss gradients, owing to its global attribution properties.

Trade-off Between Accuracy and Interpretability

Experimental results reveal a near-monotonic negative correlation between test accuracy on natural images and gradient interpretability as measured by both ROAR and KAR. As adversarial robustness increases, models sacrifice standard accuracy but exhibit substantially improved gradient interpretability. The analysis uncovers nuanced effects based on the attack norm used during training: $\ell_2$-trained models yield smoother but less sparse gradients, effective for identifying important features (ROAR), while $\ell_\infty$-trained models are more resistant to interpretability degradation for identifying less important features (KAR). These observations suggest potential research directions in optimizing adversarial objectives and combining training strategies.

The authors propose two practical approaches: combining adversarial training with advanced global attribution methods (e.g., Integrated Gradients, DeepLIFT) and exploring tailored applications or modifications of $\ell_\infty$ adversarial training to mitigate the accuracy-interpretability trade-off.

Implications and Future Directions

This work elucidates a formal connection between adversarial robustness and gradient interpretability. The findings imply that adversarial training acts as an implicit regularizer, promoting gradient alignment with meaningful features and the image manifold, and hence facilitating both human and algorithmic interpretability of DNNs. However, achieving high interpretability via adversarial training comes at the cost of decreased clean test accuracy. These insights motivate future research in devising training schemes or attribution methods that resolve this trade-off, leveraging manifold constraints, hybrid objective functions, or post-hoc interpretability enhancements. Additionally, the results suggest that decision boundary geometry induced by adversarial objectives has substantial impact on the semantic alignment and informativeness of attribution maps.

Conclusion

The paper provides rigorous theoretical and empirical analysis bridging adversarial robustness and gradient interpretability. Adversarially trained DNNs produce loss gradients confined to the image manifold and quantitatively meaningful for attribution, as demonstrated across multiple datasets and evaluation metrics. However, the empirical trade-off between accuracy and interpretability signals the need for further advances in training paradigms and interpretability methodologies. The research lays foundational groundwork for designing robust and interpretable DNNs with practical and theoretical significance for future developments in trusted AI systems.