Keywords: Model Selection; Solution Path; lasso; penalized likelihood; tuning; crossvalidation
The performances of penalized likelihood approaches profoundly depend on the selection of the tuning parameter; however there has not been a common agreement on the criterion for choosing the tuning parameter. Moreover, penalized likelihood estimation based on a single value of the tuning parameter would suffer from several drawbacks. This article introduces a novel approach for feature selection based on the whole solution paths rather than choosing one single tuning parameter, which significantly improves the selection accuracy. Moreover, it allows for feature selection using ridge or other strictly convex penalties. Based on the proposed selection algorithm, we created an innovative score to rank the importance of the features, and further develop an inference procedure that includes a graphical display and a quantitative measure to measure the uncertainty of the selection results, a model selection confidence set, and confidence intervals for coefficients each variable.