Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GreenSaliency: A Lightweight and Efficient Image Saliency Detection Method (2404.00253v1)

Published 30 Mar 2024 in eess.IV

Abstract: Image saliency detection is crucial in understanding human gaze patterns from visual stimuli. The escalating demand for research in image saliency detection is driven by the growing necessity to incorporate such techniques into various computer vision tasks and to understand human visual systems. Many existing image saliency detection methods rely on deep neural networks (DNNs) to achieve good performance. However, the high computational complexity associated with these approaches impedes their integration with other modules or deployment on resource-constrained platforms, such as mobile devices. To address this need, we propose a novel image saliency detection method named GreenSaliency, which has a small model size, minimal carbon footprint, and low computational complexity. GreenSaliency can be a competitive alternative to the existing deep-learning-based (DL-based) image saliency detection methods with limited computation resources. GreenSaliency comprises two primary steps: 1) multi-layer hybrid feature extraction and 2) multi-path saliency prediction. Experimental results demonstrate that GreenSaliency achieves comparable performance to the state-of-the-art DL-based methods while possessing a considerably smaller model size and significantly reduced computational complexity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on pattern analysis and machine intelligence, 20(11):1254–1259, 1998.
  2. How is gaze influenced by image transformations? dataset and model. IEEE Transactions on Image Processing, 29:2287–2300, 2019.
  3. Synthesizing supervision for learning deep saliency network without human annotation. IEEE transactions on pattern analysis and machine intelligence, 42(7):1755–1769, 2019.
  4. Saliency driven perceptual image compression. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 227–236, 2021.
  5. Sgdnet: An end-to-end saliency-guided deep neural network for no-reference image quality assessment. In Proceedings of the 27th ACM international conference on multimedia, pages 1383–1391, 2019.
  6. Saliencymix: A saliency guided data augmentation strategy for better regularization. arXiv preprint arXiv:2006.01791, 2020.
  7. Improving deep learning interpretability by saliency guided training. Advances in Neural Information Processing Systems, 34:26726–26739, 2021.
  8. Graph-based visual saliency. Advances in neural information processing systems, 19, 2006.
  9. Shallow and deep convolutional networks for saliency prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 598–606, 2016.
  10. Predicting human eye fixations via an lstm-based saliency attentive model. IEEE Transactions on Image Processing, 27(10):5142–5154, 2018.
  11. Learning to predict where humans look. In 2009 IEEE 12th international conference on computer vision, pages 2106–2113. IEEE, 2009.
  12. Cat2000: A large scale fixation dataset for boosting saliency research. arXiv preprint arXiv:1505.03581, 2015.
  13. Salicon: Saliency in context. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1072–1080, 2015.
  14. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  15. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  16. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  17. Sun: A bayesian framework for saliency using natural statistics. Journal of vision, 8(7):32–32, 2008.
  18. A nonparametric approach to bottom-up visual saliency. Advances in neural information processing systems, 19, 2006.
  19. Large-scale optimization of hierarchical features for saliency prediction in natural images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2798–2805, 2014.
  20. Deep gaze i: Boosting saliency prediction with feature maps trained on imagenet. arXiv preprint arXiv:1411.1045, 2014.
  21. Understanding low-and high-level contributions to fixation prediction. In Proceedings of the IEEE international conference on computer vision, pages 4789–4798, 2017.
  22. Deepgaze iie: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12919–12928, 2021.
  23. Eml-net: An expandable multi-layer network for saliency prediction. Image and vision computing, 95:103887, 2020.
  24. Unified image and video saliency modeling. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, pages 419–435. Springer, 2020.
  25. Salgan: Visual saliency prediction with adversarial networks. In CVPR scene understanding workshop (SUNw), 2017.
  26. Fbnet: Feedback-recursive cnn for saliency detection. In 2021 17th International Conference on Machine Vision and Applications (MVA), pages 1–5. IEEE, 2021.
  27. Salfbnet: Learning pseudo-saliency distribution via feedback convolutional networks. Image and Vision Computing, 120:104395, 2022.
  28. Green learning: Introduction, examples and outlook. Journal of Visual Communication and Image Representation, 90:103685, 2023.
  29. Interpretable convolutional neural networks via feedforward design. Journal of Visual Communication and Image Representation, 60:346–359, 2019.
  30. Pixelhop++: A small successive-subspace-learning-based (SSL-based) model for image classification. In 2020 IEEE International Conference on Image Processing (ICIP), pages 3294–3298. IEEE, 2020.
  31. On supervised feature selection from high dimensional feature spaces. APSIPA Transactions on Signal and Information Processing, 11(1), 2022.
  32. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794, 2016.
  33. Defakehop: A light-weight high-performance deepfake detector. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2021.
  34. Greenbiqa: A lightweight blind image quality assessment method. In 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), pages 1–6. IEEE, 2022.
  35. Tghop: an explainable, efficient, and lightweight method for texture generation. APSIPA Transactions on Signal and Information Processing, 10:e17, 2021.
  36. John Canny. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence, (6):679–698, 1986.
  37. A benchmark of computational models of saliency to predict human fixations. 2012.
  38. Saliency benchmarking made easy: Separating models, maps and metrics. In Proceedings of the European Conference on Computer Vision (ECCV), pages 770–787, 2018.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now