Caprice 3-4
Comparison of Regularization and Dimension Reduction Methods (304810)
*Chathurangi Heshani Pathiravasan , Southern Illinois University, CarbondaleKeywords: Principal Component Analysis, Subset Selection, Ridge Regression, LASSO
Modern data sets are usually high dimensional. If the number of attributes are higher than the number of observations, it is very hard to fit a precise model and obtain accurate predictions. A detailed review and comparison of the current methods of dimension reduction is provided in this study. These methods are characterized into three categories namely: subset selection, projection techniques and regularization approaches. Best subset selection, forward step-wise selection and backward selection can be used to identify a subset of predictors which are needed for the model. Principal component analysis allows us to project all the predictors into low dimensional space. Ridge regression, LASSO (least absolute shrinkage and selection operator) and elastic net generate fitted models using all predictors, but the estimated coefficients are shrunken towards to zero relative to the least square estimates. The performance of these dimension reduction techniques is illustrated through challenging models and demonstrated on real data examples.