I-SplitEE: Image classification in Split Computing DNNs with Early Exits (2401.10541v1)
Abstract: The recent advances in Deep Neural Networks (DNNs) stem from their exceptional performance across various domains. However, their inherent large size hinders deploying these networks on resource-constrained devices like edge, mobile, and IoT platforms. Strategies have emerged, from partial cloud computation offloading (split computing) to integrating early exits within DNN layers. Our work presents an innovative unified approach merging early exits and split computing. We determine the 'splitting layer', the optimal depth in the DNN for edge device computations, and whether to infer on edge device or be offloaded to the cloud for inference considering accuracy, computational efficiency, and communication costs. Also, Image classification faces diverse environmental distortions, influenced by factors like time of day, lighting, and weather. To adapt to these distortions, we introduce I-SplitEE, an online unsupervised algorithm ideal for scenarios lacking ground truths and with sequential data. Experimental validation using Caltech-256 and Cifar-10 datasets subjected to varied distortions showcases I-SplitEE's ability to reduce costs by a minimum of 55% with marginal performance degradation of at most 5%.
- K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu et al., “A survey on vision transformer,” IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 1, pp. 87–110, 2022.
- K. Bochie, M. S. Gilbert, L. Gantert, M. S. Barbosa, D. S. Medeiros, and M. E. M. Campista, “A survey on deep learning for challenged networks: Applications and trends,” Journal of Network and Computer Applications, vol. 194, p. 103213, 2021.
- Y. Matsubara, M. Levorato, and F. Restuccia, “Split computing and early exiting for deep learning applications: Survey and research challenges,” ACM Computing Surveys, vol. 55, no. 5, pp. 1–30, 2022.
- S. Teerapittayanon, B. McDanel, and H.-T. Kung, “Branchynet: Fast inference via early exiting from deep neural networks,” in 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, 2016, pp. 2464–2469.
- S. Laskaridis, S. I. Venieris, M. Almeida, I. Leontiadis, and N. D. Lane, “Spinn: synergistic progressive inference of neural networks over device and cloud,” in Proceedings of the 26th annual international conference on mobile computing and networking, 2020, pp. 1–15.
- G. Xu, J. Hao, L. Shen, H. Hu, Y. Luo, H. Lin, and J. Shen, “Lgvit: Dynamic early exiting for accelerating vision transformer,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 9103–9114.
- R. G. Pacheco, F. D. Oliveira, and R. S. Couto, “Early-exit deep neural networks for distorted images: Providing an efficient edge offloading,” in 2021 IEEE Global Communications Conference (GLOBECOM). IEEE, 2021, pp. 1–6.
- J. Shao and J. Zhang, “Bottlenet++: An end-to-end approach for feature compression in device-edge co-inference systems,” in 2020 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 2020, pp. 1–6.
- S. Dodge and L. Karam, “Understanding how image quality affects deep neural networks,” in 2016 eighth international conference on quality of multimedia experience (QoMEX). IEEE, 2016, pp. 1–6.
- R. G. Pacheco, F. D. V. R. Oliveira, and R. S. Couto, “Early-exit deep neural networks for distorted images: providing an efficient edge offloading,” in IEEE Global Communications Conf. (GLOBECOM), 2021, pp. 1–6.
- P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine learning, vol. 47, no. 2, pp. 235–256, 2002.
- M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510–4520.
- G. Griffin, A. Holub, and P. Perona, “Caltech-256 object category dataset,” 2007.
- A. Krizhevsky, V. Nair, and G. Hinton, “The cifar-10 dataset,” online: http://www. cs. toronto. edu/kriz/cifar. html, vol. 55, no. 5, 2014.
- Y. Kang, J. Hauswald, C. Gao, A. Rovinski et al., “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” in ACM Computer Architecture News, vol. 45, 2017, pp. 615–629.
- A. E. Eshratifar, A. Esmaili, and M. Pedram, “Bottlenet: A deep learning architecture for intelligent mobile cloud computing services,” in 2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED). IEEE, 2019, pp. 1–6.
- J. Xin, R. Tang, J. Lee, Y. Yu, and J. Lin, “Deebert: Dynamic early exiting for accelerating bert inference,” arXiv preprint arXiv:2004.12993, 2020.
- X. Liu, T. Sun, J. He, L. Wu, X. Zhang, H. Jiang, Z. Cao, X. Huang, and X. Qiu, “Towards efficient NLP: A standard evaluation and A strong baseline,” 2021. [Online]. Available: https://arxiv.org/abs/2110.07038
- W. Ju, W. Bao, D. Yuan, L. Ge, and B. B. Zhou, “Learning early exit for deep neural network inference on mobile devices through multi-armed bandits,” in 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid). IEEE, 2021, pp. 11–20.
- W. Ju, W. Bao, L. Ge, and D. Yuan, “Dynamic early exit scheduling for deep neural network inference through contextual bandits,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 823–832.
- H. N. N U, M. K. Hanawal, and A. Bhardwaj, “Unsupervised early exit in dnns with multiple exits,” in Proceedings of the Second International Conference on AI-ML Systems, ser. AIMLSystems ’22. New York, NY, USA: Association for Computing Machinery, 2023.
- P. Auer et al., “Finite-time analysis of the multiarmed bandit problem,” Machine Learning, vol. 47, pp. 235–256, 2002.
- M. Wang, J. Mo, J. Lin, Z. Wang, and L. Du, “Dynexit: A dynamic early-exit strategy for deep residual networks,” in 2019 IEEE International Workshop on Signal Processing Systems (SiPS). IEEE, 2019, pp. 178–183.
- Z. Kuang, L. Li, J. Gao, L. Zhao, and A. Liu, “Partial offloading scheduling and power allocation for mobile edge computing systems,” IEEE Internet of Things Journal, vol. 6, no. 4, pp. 6774–6785, 2019.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.