Missing data imputation with hybrid feature selection for fertility dataset

Abstract

Missing values poses a great concern in medical analysis as it may alter the result of analysed data and cloud the judgement of the medical practitioner which ultimately affecting the precise treatment a patient should receive. Even though there are many imputation methods that have been developed, the main issues with missing values such as accuracy and bias in prediction remain unsolved. In this paper, Fuzzy c-means (FCM) is employed as the imputation method. However, FCM does not consider the factor of irrelevant features. Noise and redundant data in the irrelevant features can reduce the accuracy of imputation and increase the computational time of FCM. An approach to tackle this problem is by using a feature selection method. By removing features that are irrelevant, the accuracy of imputation can be increased. Therefore, in this study, a hybrid imputation model Principal Component Analysis-Support Vector Machines-FCM (PCA-SVM-FCM) is proposed. The effectiveness of the proposed model is tested on a medical dataset which is Fertility dataset. Its performance is then validated by comparing it with SVM-FCM. Experimental result demonstrated that the proposed model performs better than SVM-FCM by producing a much lower error in estimation when tested using RMSE and MAE. The proposed model was then further verified by using Thiel’s U test and producing lowU value that indicates it is sufficient and significant. Therefore, PCA-SVM-FCM can be a feasible imputation tool to assist medical practitioner to obtain a reliable and better data analysis result.