The Adversarial Consistency of Surrogate Risks for Binary Classification (2305.09956v3)
Abstract: We study the consistency of surrogate risks for robust binary classification. It is common to learn robust classifiers by adversarial training, which seeks to minimize the expected $0$-$1$ loss when each example can be maliciously corrupted within a small ball. We give a simple and complete characterization of the set of surrogate loss functions that are \emph{consistent}, i.e., that can replace the $0$-$1$ loss without affecting the minimizing sequences of the original adversarial risk, for any data distribution. We also prove a quantitative version of adversarial consistency for the $\rho$-margin loss. Our results reveal that the class of adversarially consistent surrogates is substantially smaller than in the standard setting, where many common surrogates are known to be consistent.
- Calibration and consistency of adversarial surrogate losses. NeurIps, 2021a.
- On the existence of the adversarial bayes classifier (extended version). arxiv, 2021b.
- A finer calibration analysis for adversarial robustness. arxiv, 2021c.
- H-consistency bounds for surrogate loss minimizers. In Proceedings of the 39th International Conference on Machine Learning. PMLR, 2022.
- Calibrated surrogate losses for adversarially robust classification. arxiv, 2021.
- Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473), 2006.
- On the difficulty of approximately maximizing agreements. Journal of Computer System Sciences, 2003.
- R. Bhattacharjee and K. Chaudhuri. When are non-parametric methods robust? PMLR, 2020.
- R. Bhattacharjee and K. Chaudhuri. Consistent non-parametric methods for maximizing robustness. NeurIps, 2021.
- Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pages 387–402. Springer, 2013.
- The geometry of adversarial training in binary classification. arxiv, 2021.
- G. B. Folland. Real analysis: modern techniques and their applications, volume 40. John Wiley & Sons, 1999.
- N. S. Frank and J. Niles-Weed. Existence and minimax theorems for adversarial surrogate risks in binary classification. arXiv, 2023.
- Adaptive square attack: Fooling autonomous cars with adversarial traffic signs. IEEE Internet of Things Journal, 8(8), 2021.
- Y. Lin. A note on margin-based loss functions in classification. Statistics & Probability Letters, 68(1):73–82, 2004.
- Cross-entropy loss functions: Theoretical analysis and applications, 2023.
- Towards consistency in adversarial classification. arXiv, 2022.
- S. A. Mingyuan Zhang. Consistency vs. h-consistency: The interplay between surrogate loss functions and the scoring function class. NeurIps, 2020.
- Generalizability vs. robustness: Adversarial examples for medical imaging. Springer, 2018.
- R. A. S. Philip M. Long. Consistency versus realizable h-consistency for multiclass classification. ICML, 2013.
- M. S. Pydi and V. Jog. The many faces of adversarial risk. Neural Information Processing Systems, 2021.
- Surrogate regret bounds for proper losses. In Proceedings of the 26th Annual International Conference on Machine Learning, New York, NY, USA, 2009. Association for Computing Machinery.
- I. Steinwart. How to compare different loss functions and their risks. Constructive Approximation, 2007.
- Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
- N. G. Trillos and R. Murray. Adversarial classification: Necessary conditions and geometric flows. arxiv, 2020.
- The multimarginal optimal transport formulation of adversarial multiclass classification. arXiv, 2022.
- On the existence of solutions to adversarial training in multiclass classification, 2023.
- T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. The Annals of Statistics, 2004.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
