Variable selection using least absolute shrinkage and selection operator

Abstract

Least Absolute Shrinkage and Selection Operator (LASSO) and Forward Selection are variable selection method that implement in this study. The objectives of this study are to apply forward selection method in variable selection for a regression model, to apply LASSO method in variable selection for a regression model using quadratic programming and leave one out cross validation and choosing the better model obtained from forward selection and LASSO method using least mean square error. The forward selection method implemented in the statistical package for social sciences (SPSS). Quadratic programming technique and leave one out cross validation from MATLAB software is applied to solve LASSO. The analyzed result showed forward selection and LASSO are chosen the same variable that should be included in the model. However the coefficient of the regression for both model differ. To choose between the two models, MSE is used as the criteria where the model with the smallest MSE is taken as the best model. The MSE for forward selection and LASSO are 0.4959 and 0.4765 respectively. Thus, LASSO is the better model compared to forward selection model.