Improving Explainability of Softmax Classifiers Using a Prototype-Based Joint Embedding Method (2407.02271v2)
Abstract: We propose a prototype-based approach for improving explainability of softmax classifiers that provides an understandable prediction confidence, generated through stochastic sampling of prototypes, and demonstrates potential for out of distribution detection (OOD). By modifying the model architecture and training to make predictions using similarities to any set of class examples from the training dataset, we acquire the ability to sample for prototypical examples that contributed to the prediction, which provide an instance-based explanation for the model's decision. Furthermore, by learning relationships between images from the training dataset through relative distances within the model's latent space, we obtain a metric for uncertainty that is better able to detect out of distribution data than softmax confidence.
- This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019.
- A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597–1607. PMLR, 2020.
- Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pages 1050–1059. PMLR, 2016.
- Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Pprocessing Systems, 30, 2017.
- Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
- Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in Neural Information Processing Systems, 33:7498–7512, 2020.
- Learning with mixture of prototypes for out-of-distribution detection. arXiv preprint arXiv:2402.02653, 2024.
- Making the black box more transparent: Understanding the physical implications of machine learning. Bulletin of the American Meteorological Society, 100(11):2175–2199, 2019.
- Pip-net: Patch-based intuitive prototypes for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2744–2753, 2023.
- Understanding softmax confidence and uncertainty. arXiv preprint arXiv:2106.04972, 2021.
- Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206–215, 2019.
- Classifier-head informed feature masking and prototype-based logit smoothing for out-of-distribution detection. IEEE Transactions on Circuits and Systems for Video Technology, 2024.
- Uncertainty estimation using a single deep deterministic neural network. In International Conference on Machine Learning, pages 9690–9700. PMLR, 2020.
- Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60, 2023.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.