Enhanced non-parametric sequence-based learning algorithm for outlier detection in the internet of things

Abstract

Although research on outlier detection methods has been an investigation area for long, few of those studies relate to an Internet of Things (IoT) domain. Several critical decisions taken on daily business operations depend on various data collected over time. Therefore, it is mandatory to guarantee its correctness, integrity, and accuracy before any further processing can commence. Outliers are often assumed to be Error by most algorithms in the past, which is always attributed to faulty sensors. Hence, this assumption has been investigated and results show that outliers can be classified into Error and Event types with the support of a Non-parametric sequence-based learning algorithm. The event type outlier is majorly caused by abnormality from sensor readings, which are very important and should not be ignored. However, the non-parametric sequence approach and other existing techniques still find it elusive to detect outliers in the global search space of a large dataset. Therefore, this paper proposes an Enhanced Non-parametric sequence learning algorithm based on Ensemble Clustering Techniques to detect Event and Error outliers in large datasets. Experiments are conducted on six different datasets from the UCL repository, except one collected from a laboratory testbed, to demonstrate the robustness and effectiveness of the proposed approach over the existing techniques. The results show a remarkable performance rate of 96.653% accuracy, 94.284% precision, and 98.112% for Error outlier detection. It also performs better in Event outlier detection with 87.611% accuracy, 71.141% precision and 85.755% specificity with 1291 s execution time.