Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 37 tok/s
Gemini 2.5 Pro 41 tok/s Pro
GPT-5 Medium 10 tok/s Pro
GPT-5 High 15 tok/s Pro
GPT-4o 84 tok/s Pro
Kimi K2 198 tok/s Pro
GPT OSS 120B 448 tok/s Pro
Claude Sonnet 4 31 tok/s Pro
2000 character limit reached

Alleviating the Effect of Data Imbalance on Adversarial Training (2307.10205v2)

Published 14 Jul 2023 in cs.LG, cs.CR, and cs.CV

Abstract: In this paper, we study adversarial training on datasets that obey the long-tailed distribution, which is practical but rarely explored in previous works. Compared with conventional adversarial training on balanced datasets, this process falls into the dilemma of generating uneven adversarial examples (AEs) and an unbalanced feature embedding space, causing the resulting model to exhibit low robustness and accuracy on tail data. To combat that, we theoretically analyze the lower bound of the robust risk to train a model on a long-tailed dataset to obtain the key challenges in addressing the aforementioned dilemmas. Based on it, we propose a new adversarial training framework -- Re-balancing Adversarial Training (REAT). This framework consists of two components: (1) a new training strategy inspired by the effective number to guide the model to generate more balanced and informative AEs; (2) a carefully constructed penalty function to force a satisfactory feature space. Evaluation results on different datasets and model structures prove that REAT can effectively enhance the model's robustness and preserve the model's clean accuracy. The code can be found in https://github.com/GuanlinLee/REAT.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. In Proc. of the ICML, pages 274–283, 2018.
  2. A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 106:249–259, 2018.
  3. Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss. In Proc. of the NeurIPS, pages 1565–1576, 2019.
  4. Towards Evaluating the Robustness of Neural Networks. In Proc. of the SP, pages 39–57, 2017.
  5. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proc. of the ICML, pages 2206–2216, 2020.
  6. Class-Balanced Loss Based on Effective Number of Samples. In Proc. of the CVPR, pages 9268–9277, 2019.
  7. Explaining and Harnessing Adversarial Examples. In Proc. of the ICLR, 2015.
  8. Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning. In Proc. of the ICIC, pages 878–887, 2005.
  9. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 21(9):1263–1284, 2009.
  10. Deep Residual Learning for Image Recognition. In Proc. of the CVPR, pages 770–778, 2016.
  11. Disentangling Label Distribution for Long-Tailed Visual Recognition. In Proc. of the CVPR, pages 6626–6636, 2021.
  12. Learning Deep Representation for Imbalanced Classification. In Proc. of the CVPR, pages 5375–5384, 2016.
  13. Self-Adaptive Training: beyond Empirical Risk Minimization. In Proc. of the NeurIPS, 2020.
  14. The class imbalance problem: A systematic study. Intelligent data analysis, 6(5):429–449, 2002.
  15. Decoupling Representation and Classifier for Long-Tailed Recognition. In Proc. of the ICLR, 2020.
  16. Exploring Balanced Feature Spaces for Representation Learning. In Proc. of the ICLR, 2021.
  17. Striking the Right Balance With Uncertainty. In Proc. of the CVPR, pages 103–112, 2019.
  18. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
  19. Focal Loss for Dense Object Detection. In Proc. of the ICCV, pages 2999–3007, 2017.
  20. Exploratory Undersampling for Class-Imbalance Learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 39(2):539–550, 2009.
  21. Towards Deep Learning Models Resistant to Adversarial Attacks. In Proc. of the ICLR, 2018.
  22. Exploring the Limits of Weakly Supervised Pretraining. In Proc. of the ECCV, pages 185–201, 2018.
  23. Distributed Representations of Words and Phrases and their Compositionality. In Proc. of the NeurIPS, pages 3111–3119, 2013.
  24. Balanced Meta-Softmax for Long-Tailed Visual Recognition. In Proc. of the NeurIPS, 2020.
  25. Overfitting in adversarially robust deep learning. In Proc. of the ICML, pages 8093–8104, 2020.
  26. Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect. In Proc. of the NeurIPS, 2020.
  27. The Devil Is in Classification: A Simple Framework for Long-Tail Instance Segmentation. In Proc. of the ECCV, pages 728–744, 2020a.
  28. Improving Adversarial Robustness Requires Revisiting Misclassified Examples. In Proc. of the ICLR, 2020b.
  29. Learning to Model the Tail. In Proc. of the NeurIPS, pages 7029–7039, 2017.
  30. Adversarial Weight Perturbation Helps Robust Generalization. In Proc. of the NeurIPS, 2020a.
  31. Adversarial Robustness Under Long-Tailed Distribution. In Proc. of the CVPR, pages 8659–8668, 2021.
  32. Solving Long-Tailed Recognition with Deep Realistic Taxonomic Classifier. In Proc. of the ECCV, pages 171–189, 2020b.
  33. Feature Transfer Learning for Face Recognition With Under-Represented Data. In Proc. of the CVPR, pages 5704–5713, 2019.
  34. Wide Residual Networks. In Proc. of the BMVC, 2016.
  35. Theoretically Principled Trade-off between Robustness and Accuracy. In Proc. of the ICML, pages 7472–7482, 2019.
  36. Efficient adversarial training with transferable adversarial examples. In Proc. of the CVPR, pages 1178–1187, 2020.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
Lightbulb On Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - GuanlinLee/REAT: Adversarial Training Over Long-Tailed Distribution (4 stars)

Don't miss out on important new AI/ML research

See which papers are being discussed right now on X, Reddit, and more:

Explore Trending Papers

“Emergent Mind helps me see which AI papers have caught fire online.”

Philip

Philip

Creator, AI Explained on YouTube