Universiti Teknologi Malaysia Institutional Repository

Evaluating imputation methods for rainfall data under high variability in Johor River Basin, Malaysia

Sa’adi, Zulfaqar and Yusop, Zulkifli and Alias, Nor Eliza and Chow, Ming Fai and Muhammad, Mohd. Khairul Idlan and Ramli, Muhammad Wafiy Adli and Iqbal, Zafar and Shiru, Mohammed Sanusi and Wira Rohmat, Faizal Immaddudin and Mohamad, Nur Athirah and Ahmad, Mohamad Faizal (2023) Evaluating imputation methods for rainfall data under high variability in Johor River Basin, Malaysia. Applied Computing and Geosciences, 20 (NA). pp. 1-16. ISSN 2590-1974

[img] PDF
513kB

Official URL: http://dx.doi.org/10.1016/j.acags.2023.100145

Abstract

Missing values in rainfall records might result in erroneous predictions and inefficient management practices with significant economic, environmental, and social consequences. This is particularly important for rainfall datasets in Peninsular Malaysia (PM) due to the high level of missingness that can affect the inherent pattern in the highly variable time series. In this work, 21 target rainfall stations in the Johor River Basin (JRB) with daily data between 1970 and 2015 were used to examine 19 different multiple imputation methods that were carried out using the Multivariate Imputation by Chained Equations (MICE) package in R. For each station, artificial missing data were added at rates of up to 5%, 10%, 20%, and 30% for different types of missingness, namely, Missing Completely At Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR), leaving the original missing data intact. The imputation quality was evaluated based on several statistical performance metrics, namely mean absolute error (MAE), root mean square error (RMSE), normalized root mean square error (NRMSE), Nash-Sutcliffe efficiency (NSE), modified degree of agreement (MD), coefficient of determination (R2), Kling-Gupta efficiency (KGE), and volumetric efficiency (VE), which were later ranked and aggregated by using the compromise programming index (CPI) to select the best method. The results showed that linear regression predicted values (norm.predict) consistently ranked the highest under all types and levels of missingness. For example, under MAR, MNAR, and MCAR, this method showed the lowest MAE values, ranging between 0.78 and 2.25, 0.93–2.57, and 0.87–2.43, respectively. It also consistently shows higher NSE and R2 values of 0.71–0.92, 0.6–0.92, and 0.66–0.91, and 0.77–0.92, 0.71–0.93, and 0.75–0.92 under MAR, MCAR, and MNAR, respectively. The methods of mean, rf, and cart also appear to be efficient. The incorporation of the compromise programming index (CPI) as a decision-support tool has enabled an objective assessment of the output from the multiple performance metrics for the ranking and selection of the top-performing method. During validation, the Probability Density Function (PDF) demonstrated that even with up to 30% missingness, the shape of the distribution was retained after imputation compared to the actual data. The methodology proposed in this study can help in choosing suitable imputation methods for other tropical rainfall datasets, leading to improved accuracy in rainfall estimation and prediction.

Item Type:Article
Uncontrolled Keywords:Daily rainfall, Johor river basin, Missing data, Multiple imputation methods, Peninsular Malaysia, Spatiotemporal variability
Subjects:T Technology > TA Engineering (General). Civil engineering (General)
Divisions:Civil Engineering
ID Code:105055
Deposited By: Widya Wahid
Deposited On:02 Apr 2024 06:43
Last Modified:02 Apr 2024 06:43

Repository Staff Only: item control page