Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook (2205.05173v5)

Published 10 May 2022 in cs.LG

Abstract: Anomaly detection (AD) plays a crucial role in various domains, including cybersecurity, finance, and healthcare, by identifying patterns or events that deviate from normal behaviour. In recent years, significant progress has been made in this field due to the remarkable growth of deep learning models. Notably, the advent of self-supervised learning has sparked the development of novel AD algorithms that outperform the existing state-of-the-art approaches by a considerable margin. This paper aims to provide a comprehensive review of the current methodologies in self-supervised anomaly detection. We present technical details of the standard methods and discuss their strengths and drawbacks. We also compare the performance of these models against each other and other state-of-the-art anomaly detection models. Finally, the paper concludes with a discussion of future directions for self-supervised anomaly detection, including the development of more effective and efficient algorithms and the integration of these techniques with other related fields, such as multi-modal learning.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (137)
  1. Latent space autoregression for novelty detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 481–490.
  2. A comprehensive survey of numeric and symbolic outlier mining techniques. Intell. Data Anal. 10, 521–538. doi:10.3233/IDA-2006-10604.
  3. Ganomaly: Semi-supervised anomaly detection via adversarial training, in: Asian conference on computer vision, Springer. pp. 622–637.
  4. Big self-supervised models advance medical image classification, in: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3458–3468. doi:10.1109/ICCV48922.2021.00346.
  5. Anomaly detection using deep reconstruction and forecasting for autonomous systems. arXiv preprint arXiv:2006.14556 .
  6. Ssdpt: Self-supervised dual-path transformer for anomalous sound detection. Digital Signal Processing 135, 103939. URL: https://www.sciencedirect.com/science/article/pii/S1051200423000349, doi:https://doi.org/10.1016/j.dsp.2023.103939.
  7. Classification-based anomaly detection for general data, in: International Conference on Learning Representations (ICLR).
  8. Mvtec ad — a comprehensive real-world dataset for unsupervised anomaly detection, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9584–9592. doi:10.1109/CVPR.2019.00982.
  9. Salad: Self-supervised aggregation learning for anomaly detection on x-rays, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 468–478.
  10. Detecting anomalies in retinal diseases using generative, discriminative, and self-supervised deep learning. JAMA ophthalmology 140, 185–189.
  11. Neural contextual anomaly detection for time series, in: Raedt, L.D. (Ed.), Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, International Joint Conferences on Artificial Intelligence Organization. pp. 2843–2851. URL: https://doi.org/10.24963/ijcai.2022/394, doi:10.24963/ijcai.2022/394. main Track.
  12. Deep learning for anomaly detection: A survey. URL: https://arxiv.org/abs/1901.03407, doi:10.48550/ARXIV.1901.03407.
  13. Outlier detection: A survey. ACM Computing Surveys 14, 15.
  14. Gccad: Graph contrastive learning for anomaly detection. IEEE Transactions on Knowledge and Data Engineering .
  15. Novelty detection via contrastive learning with negative data augmentation, in: Zhou, Z.H. (Ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, International Joint Conferences on Artificial Intelligence Organization. pp. 606–614. URL: https://doi.org/10.24963/ijcai.2021/84, doi:10.24963/ijcai.2021/84. main Track.
  16. A simple framework for contrastive learning of visual representations, in: International conference on machine learning, PMLR. pp. 1597–1607.
  17. Unsupervised anomaly detection with multi-scale interpolated gaussian descriptors. arXiv preprint arXiv:2101.10043 2.
  18. Masked contrastive learning for anomaly detection, in: Zhou, Z.H. (Ed.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, International Joint Conferences on Artificial Intelligence Organization. pp. 1434–1441. URL: https://doi.org/10.24963/ijcai.2021/198, doi:10.24963/ijcai.2021/198. main Track.
  19. Self-supervised 3d out-of-distribution detection via pseudoanomaly generation, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 95–103.
  20. Learning a similarity metric discriminatively, with application to face verification, in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), pp. 539–546 vol. 1. doi:10.1109/CVPR.2005.202.
  21. Debiased contrastive learning. Advances in Neural Information Processing Systems 33.
  22. Sub-image anomaly detection with deep pyramid correspondences. arXiv preprint arXiv:2005.02357 .
  23. Padim: a patch distribution modeling framework for anomaly detection and localization, in: International Conference on Pattern Recognition, Springer. pp. 475–489.
  24. A survey on gans for anomaly detection. URL: https://arxiv.org/abs/1906.11632, doi:10.48550/ARXIV.1906.11632.
  25. Unsupervised visual representation learning by context prediction, in: Proceedings of the IEEE international conference on computer vision, pp. 1422–1430.
  26. Graph anomaly detection via multi-scale contrastive learning networks with augmented view. arXiv:2212.00535.
  27. An introduction to roc analysis. Pattern Recogn. Lett. 27, 861–874. URL: https://doi.org/10.1016/j.patrec.2005.10.010, doi:10.1016/j.patrec.2005.10.010.
  28. Attribute restoration framework for anomaly detection. IEEE Transactions on Multimedia .
  29. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410 .
  30. Minimizing the cost of environmental management decisions by optimizing statistical thresholds. Ecology Letters 7, 669–675.
  31. Mad: Self-supervised masked anomaly detection task for multivariate time series, in: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. doi:10.1109/IJCNN55064.2022.9892218.
  32. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728 .
  33. Self-supervised classification for detecting anomalous sounds, in: Detection and Classification of Acoustic Scenes and Events Workshop 2020. URL: https://www.amazon.science/publications/self-supervised-classification-for-detecting-anomalous-sounds.
  34. Deep anomaly detection using geometric transformations. Advances in neural information processing systems 31.
  35. Bootstrap your own latent: A new approach to self-supervised learning. arXiv:2006.07733.
  36. Anomalous sound detection using audio representation with machine id based contrastive learning pretraining, in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. doi:10.1109/ICASSP49357.2023.10096054.
  37. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 98–107.
  38. Anomaly detection-inspired few-shot medical image segmentation through self-supervision with supervoxels. Medical Image Analysis 78, 102385.
  39. Momentum contrast for unsupervised visual representation learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729–9738.
  40. Deep anomaly detection with outlier exposure, in: International Conference on Learning Representations. URL: https://openreview.net/forum?id=HyxCxhRcY7.
  41. Using self-supervised learning can improve model robustness and uncertainty. Advances in Neural Information Processing Systems 32.
  42. Learning deep representations by mutual information estimation and maximization, in: International Conference on Learning Representations. URL: https://openreview.net/forum?id=Bklr3j0cKX.
  43. Self-supervised learning for anomalous channel detection in eeg graphs: Application to seizure analysis, in: Proceedings of the AAAI Conference on Artificial Intelligence.
  44. Graph-based time-series anomaly detection: A survey. arXiv preprint arXiv:2302.00058 .
  45. A survey of outlier detection methodologies. Artificial Intelligence Review 22, 85–126. doi:10.1023/B:AIRE.0000045502.10941.a9.
  46. Dasvdd: Deep autoencoding support vector data descriptor for anomaly detection, in: arXiv. arXiv:2106.05410.
  47. Self-supervised acoustic anomaly detection via contrastive learning, in: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
  48. Multivariate time-series anomaly detection with temporal self-supervision and graphs: Application to vehicle failure prediction, in: The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD).
  49. A self-supervised cnn for particle inspection on optical element. IEEE Transactions on Instrumentation and Measurement 70, 1–12.
  50. Efficient time series anomaly detection by multiresolution self-supervised discriminative network. Neurocomputing 491, 261–272. URL: https://www.sciencedirect.com/science/article/pii/S0925231222003435, doi:https://doi.org/10.1016/j.neucom.2022.03.048.
  51. Hop-count based self-supervised anomaly detection on attributed networks, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer. pp. 225–241.
  52. Fighting fake news: Image splice detection via learned self-consistency, in: Proceedings of the European conference on computer vision (ECCV), pp. 101–117.
  53. Anomaly detection on the rail lines using semantic segmentation and self-supervised learning, in: 2021 IEEE Symposium Series on Computational Intelligence (SSCI), IEEE. pp. 1–7.
  54. Anomalybert: Self-supervised transformer for time series anomaly detection using data degradation scheme. arXiv:2305.04468.
  55. Towards parkinson’s disease prognosis using self-supervised learning and anomaly detection, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 3960–3964.
  56. Timeautoad: Autonomous anomaly detection with self-supervised contrastive loss for multivariate time series. IEEE Transactions on Network Science and Engineering 9, 1604–1619. doi:10.1109/TNSE.2022.3148276.
  57. Self-supervised visual feature learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 4037–4058. doi:10.1109/TPAMI.2020.2992393.
  58. Multi-class supervised novelty detection. IEEE transactions on pattern analysis and machine intelligence 36, 2510–2523.
  59. Supervised contrastive learning, in: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 18661–18673. URL: https://proceedings.neurips.cc/paper/2020/file/d89a66c7c80a29b1bdbab0f2a1a94af8-Paper.pdf.
  60. Spatial contrastive learning for anomaly detection and localization. IEEE Access 10, 17366–17376.
  61. Self-supervised complex network for machine sound anomaly detection, in: 2021 29th European Signal Processing Conference (EUSIPCO), pp. 586–590. doi:10.23919/EUSIPCO54536.2021.9615923.
  62. Deep learning with support vector data description. Neurocomputing 165, 111–117.
  63. An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. Journal of Imaging 4, 36.
  64. Cifar-10 (canadian institute for advanced research). URL: http://www.cs.toronto.edu/~kriz/cifar.html.
  65. Learning representations for automatic colorization, in: European Conference on Computer Vision (ECCV).
  66. Phonocardiographic sensing using deep learning for abnormal heartbeat detection. IEEE Sensors Journal 18, 9393–9400.
  67. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems 31.
  68. Cutpaste: Self-supervised learning for anomaly detection and localization, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9664–9674.
  69. Deepfib: Self-imputation for time series anomaly detection. arXiv preprint arXiv:2112.06247 .
  70. Open category detection with PAC guarantees, in: Dy, J., Krause, A. (Eds.), Proceedings of the 35th International Conference on Machine Learning, PMLR. pp. 3169–3178. URL: https://proceedings.mlr.press/v80/liu18e.html.
  71. Graph self-supervised learning: A survey. IEEE Transactions on Knowledge and Data Engineering 35, 5879–5900.
  72. Anomaly detection on attributed networks via contrastive self-supervised learning. IEEE transactions on neural networks and learning systems 33, 2378–2392.
  73. Anomaly detection in dynamic graphs via transformer. IEEE Transactions on Knowledge and Data Engineering .
  74. Explainable deep one-class classification. arXiv preprint arXiv:2007.01760 .
  75. Deep graph level anomaly detection with contrastive learning. Scientific Reports 12, 19867.
  76. On the generalised distance in statistics, in: Proceedings of the National Institute of Sciences of India, pp. 49–55.
  77. An empirical evaluation of deep learning for network anomaly detection, in: 2018 International Conference on Computing, Networking and Communications (ICNC), IEEE. pp. 893–898.
  78. DATE: Detecting anomalies in text via self-supervision of transformers, in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Online. pp. 267–277. URL: https://aclanthology.org/2021.naacl-main.25, doi:10.18653/v1/2021.naacl-main.25.
  79. Date: Detecting anomalies in text via self-supervision of transformers. arXiv preprint arXiv:2104.05591 .
  80. Su-ids: A semi-supervised and unsupervised framework for network intrusion detection, in: International Conference on Cloud Computing and Security, Springer. pp. 322–334.
  81. Self-supervised learning for generalizable out-of-distribution detection. Proceedings of the AAAI Conference on Artificial Intelligence 34, 5216–5223. URL: https://ojs.aaai.org/index.php/AAAI/article/view/5966, doi:10.1609/aaai.v34i04.5966.
  82. Deep learning for anomaly detection: A review. ACM Comput. Surv. 54. URL: https://doi.org/10.1145/3439950, doi:10.1145/3439950.
  83. Self-supervised medical out-of-distribution using u-net vision transformers, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 104–110.
  84. Inpainting transformer for anomaly detection. arXiv preprint arXiv:2104.13897 .
  85. Neural transformation learning for deep anomaly detection beyond images, in: Meila, M., Zhang, T. (Eds.), Proceedings of the 38th International Conference on Machine Learning, PMLR. pp. 8703–8714. URL: https://proceedings.mlr.press/v139/qiu21a.html.
  86. Self-supervised anomaly detection by self-distillation and negative sampling. arXiv preprint arXiv:2201.06378 .
  87. Multi-task self-supervised learning for robust speech recognition, in: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6989–6993. doi:10.1109/ICASSP40776.2020.9053569.
  88. Panda: Adapting pretrained features for anomaly detection and segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2806–2814.
  89. Mean-shifted contrastive loss for anomaly detection. arXiv preprint arXiv:2106.03844 .
  90. Modeling the distribution of normal data in pre-trained deep features for anomaly detection, in: 2020 25th International Conference on Pattern Recognition (ICPR), IEEE. pp. 6726–6733.
  91. Same same but differnet: Semi-supervised defect detection with normalizing flows, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1907–1916.
  92. Fully convolutional cross-scale-flows for image-based defect detection, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1088–1097.
  93. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE .
  94. Deep one-class classification, in: International conference on machine learning, PMLR. pp. 4393–4402.
  95. Deep semi-supervised anomaly detection. arXiv preprint arXiv:1906.02694 .
  96. Self-supervised representation learning via neighborhood-relational encoding, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).
  97. Puzzle-ae: Novelty detection in images through solving puzzles. arXiv:2008.12959.
  98. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery, in: International conference on information processing in medical imaging, Springer. pp. 146–157.
  99. Self-supervised out-of-distribution detection and localization with natural synthetic anomalies (nsa). arXiv preprint arXiv:2109.15222 .
  100. Multi-view contrastive self-supervised learning of accounting data representations for downstream audit tasks. arXiv preprint arXiv:2109.11201 .
  101. Facenet: A unified embedding for face recognition and clustering, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823. doi:10.1109/CVPR.2015.7298682.
  102. {SSD}: A unified framework for self-supervised outlier detection, in: International Conference on Learning Representations. URL: https://openreview.net/forum?id=v5gjXpmR8J.
  103. Anomaly detection for tabular data with internal contrastive learning, in: International Conference on Learning Representations.
  104. Unsupervised anomaly segmentation via deep feature reconstruction. Neurocomputing 424, 9–22.
  105. Learning and evaluating representations for deep one-class classification. arXiv preprint arXiv:2011.02578 .
  106. Anoseg: Anomaly segmentation network using self-supervised learning. arXiv preprint arXiv:2110.03396 .
  107. Self-taught semi-supervised anomaly detection on upper limb x-rays, in: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), IEEE. pp. 1632–1636.
  108. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems 33, 11839–11852.
  109. Support vector data description. Machine learning 54, 45–66.
  110. Multi-scale patch-based representation learning for image anomaly detection and segmentation, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3992–4000.
  111. Mocca: Multi-layer one-class classification for anomaly detection. arXiv e-prints , arXiv–2012.
  112. Self-supervised out-of-distribution detection in brain ct scans. arXiv preprint arXiv:2011.05428 .
  113. Attention guided anomaly localization in images, in: European Conference on Computer Vision, Springer. pp. 485–503.
  114. Semi-supervised anomaly detection algorithms: A comparative summary and future research directions. Knowledge-Based Systems 218, 106878. URL: https://www.sciencedirect.com/science/article/pii/S0950705121001416, doi:https://doi.org/10.1016/j.knosys.2021.106878.
  115. Deep fraud detection on non-attributed graph, in: 2021 IEEE International Conference on Big Data (Big Data), IEEE. pp. 5470–5473.
  116. Student-teacher feature pyramid matching for unsupervised anomaly detection. arXiv preprint arXiv:2103.04257 .
  117. Progress in outlier detection techniques: A survey. Ieee Access 7, 107964–108000.
  118. Deep contrastive one-class time series anomaly detection. arXiv:2207.01472.
  119. Sla22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPTp: Self-supervised anomaly detection with adversarial perturbation. URL: https://arxiv.org/abs/2111.12896, doi:10.48550/ARXIV.2111.12896.
  120. Tutorial: Self-supervised learning, in: Canziani, A., Grant, E. (Eds.), Advances in Neural Information Processing Systems. URL: https://nips.cc/virtual/2021/tutorial/21895.
  121. Contrastive training for improved out-of-distribution detection. URL: https://arxiv.org/abs/2007.05566, doi:10.48550/ARXIV.2007.05566.
  122. Unsupervised feature learning via non-parametric instance discrimination, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  123. Gan-based anomaly detection: A review. Neurocomputing URL: https://www.sciencedirect.com/science/article/pii/S0925231221019482, doi:https://doi.org/10.1016/j.neucom.2021.12.093.
  124. Machine learning and deep learning methods for cybersecurity. Ieee access 6, 35365–35381.
  125. Anomaly detection on electroencephalography with self-supervised learning, in: 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE. pp. 363–368.
  126. Contrastive attributed network anomaly detection with data augmentation, in: Advances in Knowledge Discovery and Data Mining: 26th Pacific-Asia Conference, PAKDD 2022, Chengdu, China, May 16–19, 2022, Proceedings, Part II, Springer. pp. 444–457.
  127. Patch svdd: Patch-level svdd for anomaly detection and segmentation, in: Proceedings of the Asian Conference on Computer Vision.
  128. Draem-a discriminatively trained reconstruction embedding for surface anomaly detection, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8330–8339.
  129. Barlow twins: Self-supervised learning via redundancy reduction, in: Meila, M., Zhang, T. (Eds.), Proceedings of the 38th International Conference on Machine Learning, PMLR. pp. 12310–12320. URL: https://proceedings.mlr.press/v139/zbontar21a.html.
  130. Joint generative-contrastive representation learning for anomalous sound detection, in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. doi:10.1109/ICASSP49357.2023.10095568.
  131. Self-supervised anomaly detection via neural autoregressive flows with active learning, in: NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications. URL: https://openreview.net/forum?id=LdWEo5mri6.
  132. Deep anomaly detection with self-supervised learning and adversarial training. Pattern Recognition 121, 108234. URL: https://www.sciencedirect.com/science/article/pii/S0031320321004155, doi:https://doi.org/10.1016/j.patcog.2021.108234.
  133. Self-supervised tumor segmentation through layer decomposition. arXiv preprint arXiv:2109.03230 .
  134. Time series anomaly detection for smart grids via multiple self-supervised tasks learning, in: 2022 IEEE International Conference on Knowledge Graph (ICKG), IEEE Computer Society, Los Alamitos, CA, USA. pp. 392–397. URL: https://doi.ieeecomputersociety.org/10.1109/ICKG55886.2022.00057, doi:10.1109/ICKG55886.2022.00057.
  135. Anomaly detection for medical images using self-supervised and translation-consistent features. IEEE Transactions on Medical Imaging 40, 3641–3651.
  136. Generative and contrastive self-supervised learning for graph anomaly detection. IEEE Transactions on Knowledge and Data Engineering .
  137. From unsupervised to few-shot graph anomaly detection: A multi-scale contrastive learning approach. arXiv preprint arXiv:2202.05525 .
Citations (22)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now