FFT-based Selection and Optimization of Statistics for Robust Recognition of Severely Corrupted Images (2403.14335v1)
Abstract: Improving model robustness in case of corrupted images is among the key challenges to enable robust vision systems on smart devices, such as robotic agents. Particularly, robust test-time performance is imperative for most of the applications. This paper presents a novel approach to improve robustness of any classification model, especially on severely corrupted images. Our method (FROST) employs high-frequency features to detect input image corruption type, and select layer-wise feature normalization statistics. FROST provides the state-of-the-art results for different models and datasets, outperforming competitors on ImageNet-C by up to 37.1% relative gain, improving baseline of 40.9% mCE on severe corruptions.
- “Benchmarking neural network robustness to common corruptions and perturbations,” ICLR, 2019.
- “Hybridaugment++: Unified frequency spectra perturbations for model robustness,” in ICCV, 2023.
- “TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation,” in ICLR, 2023.
- “Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain,” in ICCV, 2021.
- “Frequency domain model augmentation for adversarial attack,” in ECCV, 2022.
- “Imagenet large scale visual recognition challenge,” IJCV, 2015.
- “Learning multiple layers of features from tiny images,” 2009.
- “Improving robustness of feature representations to image deformations using powered convolution in cnns,” in CVPR, 2017.
- “Augmax: Adversarial composition of random augmentations for robust training,” in NeurIPS, 2021.
- “Autoaugment: Learning augmentation strategies from data,” in CVPR, 2019.
- “Pixmix: Dreamlike pictures comprehensively improve safety measures,” CVPR, 2021.
- “AugMix: A simple data processing method to improve robustness and uncertainty,” ICLR, 2020.
- “Are transformers more robust than cnns?,” in NeurIPS, 2021.
- “Improving robustness against common corruptions by covariate shift adaptation,” NeurIPS, 2020.
- “Sita: Single image test-time adaptation,” ArXiv, 2021.
- “Improving robustness against common corruptions with frequency biased models,” in ICCV, 2021.
- “A spectral view of randomized smoothing under common corruptions: Benchmarking and improving certified robustness,” in ECCV, 2022.
- “Improving robustness without sacrificing accuracy with patch gaussian augmentation,” ArXiv, 2019.
- Richard Zhang, “Making convolutional networks shift-invariant again,” in ICML, 2019.
- “A simple way to make neural networks robust against diverse image corruptions,” in ECCV, 2020.
- “The many faces of robustness: A critical analysis of out-of-distribution generalization,” in ICCV, 2021.
- “An algorithm for the machine calculation of complex fourier series,” MCOM, 1965.
- “Comparing partitions,” Journal of Classification, 1985.
- “Multi-scale fast fourier transform based attention network for remote-sensing image super-resolution,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023.
- “On performance of multiscale sparse fast fourier transform algorithm,” Circuits, Systems and Signal Processing, 2020.
- “Deep residual learning for image recognition,” in CVPR, 2016.
- “An image is worth 16x16 words: Transformers for image recognition at scale,” ICLR, 2021.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.