Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Smart HPA: A Resource-Efficient Horizontal Pod Auto-scaler for Microservice Architectures (2403.07909v1)

Published 27 Feb 2024 in cs.DC, cs.SY, and eess.SY

Abstract: Microservice architectures have gained prominence in both academia and industry, offering enhanced agility, reusability, and scalability. To simplify scaling operations in microservice architectures, container orchestration platforms such as Kubernetes feature Horizontal Pod Auto-scalers (HPAs) designed to adjust the resources of microservices to accommodate fluctuating workloads. However, existing HPAs are not suitable for resource-constrained environments, as they make scaling decisions based on the individual resource capacities of microservices, leading to service unavailability and performance degradation. Furthermore, HPA architectures exhibit several issues, including inefficient data processing and a lack of coordinated scaling operations. To address these concerns, we propose Smart HPA, a flexible resource-efficient horizontal pod auto-scaler. It features a hierarchical architecture that integrates both centralized and decentralized architectural styles to leverage their respective strengths while addressing their limitations. We introduce resource-efficient heuristics that empower Smart HPA to exchange resources among microservices, facilitating effective auto-scaling of microservices in resource-constrained environments. Our experimental results show that Smart HPA outperforms the Kubernetes baseline HPA by reducing resource overutilization, overprovisioning, and underprovisioning while increasing resource allocation to microservice applications.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. E. Pimentel, W. Pereira, P. H. M. Maia, M. I. Cortés, et al., “Self-adaptive microservice-based systems-landscape and research opportunities,” in International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), pp. 167–178, IEEE, 2021.
  2. F. Rossi, V. Cardellini, F. L. Presti, and M. Nardelli, “Dynamic multi-metric thresholds for scaling applications using reinforcement learning,” IEEE Transactions on Cloud Computing, 2022.
  3. H. Ahmad, I. Dharmadasa, F. Ullah, and M. A. Babar, “A review on c3i systems’ security: Vulnerabilities, attacks, and countermeasures,” ACM Computing Surveys, vol. 55, no. 9, pp. 1–38, 2023.
  4. A. Gördén, “Predicting resource usage on a kubernetes platform using machine learning methods,” 2023.
  5. J. Dobies and J. Wood, Kubernetes operators: Automating the container orchestration platform. O’Reilly Media, 2020.
  6. F. Soppelsa and C. Kaewkasi, Native docker clustering with swarm. Packt Publishing Ltd, 2016.
  7. G. Dumpleton, Deploying to OpenShift: a guide for busy developers. " O’Reilly Media, Inc.", 2018.
  8. N. Zhou, H. Zhou, and D. Hoppe, “Containerization for high performance computing systems: Survey and prospects,” IEEE Transactions on Software Engineering, vol. 49, no. 4, pp. 2722–2740, 2022.
  9. F. Rossi, V. Cardellini, and F. L. Presti, “Hierarchical scaling of microservices in kubernetes,” in IEEE International Conference on Autonomic Computing and Self-organizing Systems (ACSOS), pp. 28–37, IEEE, 2020.
  10. J. Liu, S. Zhang, Q. Wang, and J. Wei, “Coordinating fast concurrency adapting with autoscaling for slo-oriented web applications,” IEEE Transactions on Parallel and Distributed Systems, vol. 33, no. 12, pp. 3349–3362, 2022.
  11. M. ZargarAzad and M. Ashtiani, “An auto-scaling approach for microservices in cloud computing environments,” 2023.
  12. P. Bodik, A. Fox, M. J. Franklin, M. I. Jordan, and D. A. Patterson, “Characterizing, modeling, and generating workload spikes for stateful services,” in 1st ACM Symposium on Cloud Computing (SoCC), pp. 241–252, 2010.
  13. T.-T. Nguyen, Y.-J. Yeom, T. Kim, D.-H. Park, and S. Kim, “Horizontal pod autoscaling in kubernetes for elastic container orchestration,” Sensors, vol. 20, no. 16, p. 4621, 2020.
  14. A. Abdel Khaleq and I. Ra, “Intelligent microservices autoscaling module using reinforcement learning,” Cluster Computing, pp. 1–12, 2023.
  15. A. F. Baarzi and G. Kesidis, “Showar: Right-sizing and efficient scheduling of microservices,” in ACM Symposium on Cloud Computing (SoCC), pp. 427–441, 2021.
  16. G. Yu, P. Chen, and Z. Zheng, “Microscaler: Cost-effective scaling for microservice applications in the cloud with an online learning approach,” IEEE Transactions on Cloud Computing, vol. 10, no. 2, pp. 1100–1116, 2020.
  17. A. U. Gias, G. Casale, and M. Woodside, “Atom: Model-driven autoscaling for microservices,” in IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pp. 1994–2004, IEEE, 2019.
  18. E. D. Nitto, L. Florio, and D. A. Tamburri, “Autonomic decentralized microservices: The gru approach and its evaluation,” Microservices: Science and Engineering, pp. 209–248, 2020.
  19. F. Rossi, V. Cardellini, F. L. Presti, and M. Nardelli, “Geo-distributed efficient deployment of containers with kubernetes,” Computer Communications, vol. 159, pp. 161–174, 2020.
  20. M. Imdoukh, I. Ahmad, and M. G. Alfailakawi, “Machine learning-based auto-scaling for containerized applications,” Neural Computing and Applications, vol. 32, pp. 9745–9760, 2020.
  21. F. Rossi, V. Cardellini, and F. L. Presti, “Self-adaptive threshold-based policy for microservices elasticity,” in 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 1–8, IEEE, 2020.
  22. D. Weyns, B. Schmerl, V. Grassi, S. Malek, R. Mirandola, C. Prehofer, J. Wuttke, J. Andersson, H. Giese, and K. M. Göschka, “On patterns for decentralized control in self-adaptive systems,” in Software Engineering for Self-Adaptive Systems II: International Seminar, Dagstuhl Castle, Germany, October 24-29, 2010 Revised Selected and Invited Papers, pp. 76–107, Springer, 2013.
  23. Replication Package for SmartHPA. https://github.com/HussainAhmad05/Smart_HPA.git.
  24. M. R. Hossen and M. A. Islam, “A lightweight workload-aware microservices autoscaling with qos assurance,” arXiv e-prints, pp. arXiv–2202, 2022.
  25. D. Balla, C. Simon, and M. Maliosz, “Adaptive scaling of kubernetes pods,” in IEEE/IFIP Network Operations and Management Symposium (NOMS), pp. 1–5, IEEE, 2020.
  26. Y. Al-Dhuraibi, F. Paraiso, N. Djarallah, and P. Merle, “Autonomic vertical elasticity of docker containers with elasticdocker,” in IEEE 10th international conference on cloud computing (CLOUD), pp. 472–479, IEEE, 2017.
  27. J. Santos, T. Wauters, B. Volckaert, and F. De Turck, “gym-hpa: Efficient auto-scaling via reinforcement learning for complex microservice-based applications in kubernetes,” in NOMS IEEE/IFIP Network Operations and Management Symposium, pp. 1–9, IEEE, 2023.
  28. B. Liu, R. Buyya, and A. Nadjaran Toosi, “A fuzzy-based auto-scaler for web applications in cloud computing environments,” in 16th International Conference on Service-Oriented Computing (ICSOC), pp. 797–811, Springer, 2018.
  29. V. Persico, D. Grimaldi, A. Pescape, A. Salvi, and S. Santini, “A fuzzy approach based on heterogeneous metrics for scaling out public clouds,” IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 8, pp. 2117–2130, 2017.
  30. H. Arabnejad, C. Pahl, P. Jamshidi, and G. Estrada, “A comparison of reinforcement learning techniques for fuzzy cloud auto-scaling,” in 17th IEEE/ACM international symposium on cluster, cloud and grid computing (CCGRID), pp. 64–73, IEEE, 2017.
  31. C. Qu, R. N. Calheiros, and R. Buyya, “Auto-scaling web applications in clouds: A taxonomy and survey,” ACM Computing Surveys (CSUR), vol. 51, no. 4, pp. 1–33, 2018.
  32. Z. Ding and Q. Huang, “Copa: A combined autoscaling method for kubernetes,” in IEEE International Conference on Web Services (ICWS), pp. 416–425, IEEE, 2021.
  33. J. Tong, M. Wei, M. Pan, and Y. Yu, “A holistic auto-scaling algorithm for multi-service applications based on balanced queuing network,” in IEEE International Conference on Web Services (ICWS), pp. 531–540, IEEE, 2021.
  34. P. Kang and P. Lama, “Robust resource scaling of containerized microservices with probabilistic machine learning,” in IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC), pp. 122–131, IEEE, 2020.
  35. N. S. Joshi, R. Raghuwanshi, Y. M. Agarwal, B. Annappa, and D. Sachin, “Arima-pid: container auto scaling based on predictive analysis and control theory,” Multimedia Tools and Applications, pp. 1–18, 2023.
  36. L. Baresi, S. Guinea, A. Leva, and G. Quattrocchi, “A discrete-time feedback controller for containerized cloud applications,” in 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 217–228, 2016.
  37. L. Baresi and G. Quattrocchi, “A simulation-based comparison between industrial autoscaling solutions and cocos for cloud applications,” in IEEE International Conference on Web Services (ICWS), pp. 94–101, IEEE, 2020.
  38. L. Toka, G. Dobreff, B. Fodor, and B. Sonkoly, “Machine learning-based scaling management for kubernetes edge clusters,” IEEE Transactions on Network and Service Management, vol. 18, no. 1, pp. 958–972, 2021.
  39. J. Yang, C. Liu, Y. Shang, B. Cheng, Z. Mao, C. Liu, L. Niu, and J. Chen, “A cost-aware auto-scaling approach using the workload prediction in service clouds,” Information Systems Frontiers, vol. 16, pp. 7–18, 2014.
  40. S. Islam, J. Keung, K. Lee, and A. Liu, “Empirical prediction models for adaptive resource provisioning in the cloud,” Future Generation Computer Systems, vol. 28, no. 1, pp. 155–162, 2012.
  41. N. Roy, A. Dubey, and A. Gokhale, “Efficient autoscaling in the cloud using predictive models for workload forecasting,” in IEEE 4th International Conference on Cloud Computing, pp. 500–507, IEEE, 2011.
  42. S. Zhang, T. Wu, M. Pan, C. Zhang, and Y. Yu, “A-sarsa: A predictive container auto-scaling algorithm based on reinforcement learning,” in IEEE international conference on web services (ICWS), pp. 489–497, IEEE, 2020.
  43. S. M. R. Nouri, H. Li, S. Venugopal, W. Guo, M. He, and W. Tian, “Autonomic decentralized elasticity based on a reinforcement learning controller for cloud applications,” Future Generation Computer Systems, vol. 94, pp. 765–780, 2019.
  44. S. Horovitz and Y. Arian, “Efficient cloud auto-scaling with sla objective using q-learning,” in IEEE 6th International Conference on Future Internet of Things and Cloud (FiCloud), pp. 85–92, IEEE, 2018.
  45. F. Rossi, M. Nardelli, and V. Cardellini, “Horizontal and vertical scaling of container-based applications using reinforcement learning,” in IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 329–338, IEEE, 2019.
  46. Z. Yang, P. Nguyen, H. Jin, and K. Nahrstedt, “Miras: Model-based reinforcement learning for microservice resource allocation over scientific workflows,” in IEEE 39th international conference on distributed computing systems (ICDCS), pp. 122–132, IEEE, 2019.
  47. P. Arcaini, E. Riccobene, and P. Scandurra, “Modeling and analyzing mape-k feedback loops for self-adaptation,” in IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), pp. 13–23, IEEE, 2015.
  48. C. Barna, H. Khazaei, M. Fokaefs, and M. Litoiu, “Delivering elastic containerized cloud applications to enable devops,” in IEEE/ACM 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), pp. 65–75, IEEE, 2017.
  49. H. Khazaei, R. Ravichandiran, B. Park, H. Bannazadeh, A. Tizghadam, and A. Leon-Garcia, “Elascale: autoscaling and monitoring as a service,” arXiv preprint arXiv:1711.03204, 2017.
  50. Amazon Web Services. [Online]. https://www.amazon.com (Accessed: August 2, 2023).
  51. Amazon Elastic Kubernetes Service. [Online]. https://aws.amazon.com/eks (Accessed: August 10, 2023).
  52. Online Boutique. [Online]. https://github.com/GoogleCloudPlatform/microservices-demo.git (Accessed: August 23, 2023).
  53. C. M. Aderaldo, N. C. Mendonça, C. Pahl, and P. Jamshidi, “Benchmark requirements for microservices architecture research,” in IEEE/ACM 1st International Workshop on Establishing the Community-Wide Infrastructure for Architecture-Based Software Engineering (ECASE), pp. 8–13, IEEE, 2017.
  54. B. Choi, J. Park, C. Lee, and D. Han, “phpa: A proactive autoscaling framework for microservice chain,” in 5th Asia-Pacific Workshop on Networking (APNet), pp. 65–71, 2021.
  55. R. R. Karn, R. Das, D. R. Pant, J. Heikkonen, and R. Kanth, “Automated testing and resilience of microservice’s network-link using istio service mesh,” in 31st Conference of Open Innovations Association (FRUCT), pp. 79–88, IEEE, 2022.
  56. Locust - An open source load testing tool. [Online]. https://locust.io (Accessed: August 28, 2023).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Hussain Ahmad (16 papers)
  2. Christoph Treude (138 papers)
  3. Markus Wagner (90 papers)
  4. Claudia Szabo (8 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets