Dataset generation and network intrusion detection based on flow-level information

Abstract

The growth of the Internet and networking has made securing networks against attacks a very challenging task. For high-speed networks, flow meta-data inspection can replace conventional Deep Packet Inspection but with the cost of low precision of identifying attacks since the former deals with an aggregated version of the traffic. The first part of this research addresses the problem of the lack in benchmarking datasets for developing new Network Intrusion Detection Systems (NIDSs) or comparing existing NIDSs. The aim in the second part is to design a near real-time NIDS without degrading the detection accuracy when compared to conventional misuse packet-based approaches. To achieve the first objective, a NIDS dataset creation framework had been developed. Based on that framework, a flow-level NIDS dataset had been created. The traces were collected from campus main routers in NetFlow format while malicious traffic of different attack scenarios was generated by Nmap and BoNesi tools. In the second part a flow-based software-based system were developed to detect and identify network volume-level attacks in near real-time. Attack detection is based on statistical time-aggregated features of the exported NetFlow version of the traffic to detect several scan and Denial-of-Service (DoS) attacks. A validation for the designed system is done using Defense Advanced Research Projects Agency (DARPA) datasets. The timeline performance outperformed all relevant software-based systems and suited for up to one gigabit per second links with an average detection delay of less than one minute. The proposed method achieved 95% True Positive Rate (TPR) and almost zero False Positive Rate (FPR). Compared to relevant methods when operated in the same conditions, the proposed method improved the TPR by 4% and improved FPR by 1%. In addition, the capability of flow-based approach in detecting packet-level attacks was experimentally demonstrated. The results against Snort were benchmarked and 75% TPR and almost zero FPR were achieved.