Detection and mitigation algorithm for malicious TCP port scan attacks in software-defined networking

The vulnerabilities existing in the Internet such as insecure network architectures are exploited to gain un-authorized access to a network. A launch of an attack usually begins with a deliberate process of analyzing potential victims. This attack is called scan attack. The most common scan attack type is called TCP port scan attack as TCP is a connection oriented protocol. A TCP port scan attack misuses the process of establishing connection between two hosts for communication (TCP three-way handshake) by a half open sacn type (not completing the connection). The existing TCP port scan attack detection approaches have many issues such as detection is not done in real-time, no automated mitigation approach is available and many port scan detection approaches are hardware dependent. In this thesis, based on drawbacks of the existing TCP port scan detection approaches, the requirements for a TCP port scan attack detection and mitigation algorithm are derived and the algorithm is proposed to be developed on an architecture called OrchSec SDN architecture (an orchestrator based SDN architecture). Software-Defined Networking (SDN) is a new paradigm in networking where the data plane (consisting of routers, switches) is separated from the control plane (makes decisions on traffic in the network). The architecture used is called the OrchSec SDN architecture and it separates the monitoring and control functions of the network and uses an orchestrator to coordinate the communication between them. The OrchSec SDN architecture utilizes the network monitoring and SDN control functions and hence is used to develop the TCP port scan attack detection and mitigation algorithm. Finally, the behavior of the proposed algorithm is tested and validated. The most important future work would be to extend the algorithm to detect and mitigate all the existing port scan attack types.