Penalized regression method in high dimensional data

Abstract

In huge multivariate data set with a number of variables greater than the number of samples, the standard linear model (or ordinary least squares method) performs badly. In such situations, a better option is penalized regression, which allows you to design a linear regression model that is penalized for having too many variables by adding a constraint to the equation Shrinkage or regularization procedures are other names for this. The penalty has the effect of reducing (i.e. shrinking) the coefficient values towards zero. This permits the coefficients of the less important variables to be near to or equal to zero. By decreasing the number of coefficients and maintaining those with coefficients greater than zero, penalized regression models improve prediction in new data when compared to traditional methods. We demonstrate that the proposed regularizer is capable of achieving competitive results as well as exceedingly compact networks. Extensive tests are carried out on a number of benchmark datasets to demonstrate the effectiveness of the method.