Different learning functions for weighted kernel regression in solving small sample problem with noise

Abstract

Previously, weighted kernel regression (WKR) for solving small samples problem has been reported. In the original WKR, the simple iterative learning technique and the formulated learning function in estimating weight parameters are designed only to solve non-noisy and small training samples problem. In this study, an extension of WKR in solving noisy and small training samples is investigated. The objective of the investigation is to extend the capability and effectiveness of WKR when solving various problems. Therefore, four new learning functions are proposed for estimating weight parameters. In general, the formulated learning functions are added with a regularization term instead of error term only as in the existing WKR. However, one free parameter associated to the regularization term has firstly to be predefined. Hence, a simple cross-validation technique is introduced to estimate this free parameter value. The improvement, in terms of the prediction accuracy as compared to existing WKR is presented through a series of experiments.