Q-MIND: Defeating Stealthy DoS Attacks in SDN with a Machine-learning based Defense Framework (1907.11887v2)

Abstract: Software Defined Networking (SDN) enables flexible and scalable network control and management. However, it also introduces new vulnerabilities that can be exploited by attackers. In particular, low-rate and slow or stealthy Denial-of-Service (DoS) attacks are recently attracting attention from researchers because of their detection challenges. In this paper, we propose a novel machine learning based defense framework named Q-MIND, to effectively detect and mitigate stealthy DoS attacks in SDN-based networks. We first analyze the adversary model of stealthy DoS attacks, the related vulnerabilities in SDN-based networks and the key characteristics of stealthy DoS attacks. Next, we describe and analyze an anomaly detection system that uses a Reinforcement Learning-based approach based on Q-Learning in order to maximize its detection performance. Finally, we outline the complete Q-MIND defense framework that incorporates the optimal policy derived from the Q-Learning agent to efficiently defeat stealthy DoS attacks in SDN-based networks. An extensive comparison of the Q-MIND framework and currently existing methods shows that significant improvements in attack detection and mitigation performance are obtained by Q-MIND.