Intelligent clustering with PCA and unsupervised learning algorithm in intrusion alert correlation

Abstract

As security threats advance in a drastic way, most of the organizations implement multiple Network Intrusion Detection Systems (NIDSs) to optimize detection and to provide comprehensive view of intrusion activities. But NIDSs trigger a massive amount of alerts even for a day and overwhelmed security experts. Thus, automated and intelligent clustering is important to reveal their structural correlation by grouping alerts with common attributes. We propose a new hybrid clustering model based on Improved Unit Range (IUR), Principal Component Analysis (PCA) and unsupervised learning algorithm (Expectation Maximization) to aggregate similar alerts and to reduce the number of alerts. We tested against other unsupervised learning algorithms to validate the performance of the proposed model. Our empirical results show using DARPA 2000 dataset the proposed model gives better results in terms of the clustering accuracy and processing time.