A Compromise Between the Reduction of Collinearity Problems and Bias of Estimation in Ridge Regression: With the Perspective of Loss Function

• Activity Number: 378
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 11:35 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Xiyuan Liu*
• Companies: The George Washington University
• Keywords: ridge regression ; loss function ; mean square error ; variance inflation factor ; opimization

Abstract

Ridge regression is a well-known methods that often appears in dealing with the problem of collinearity. At present, there are many studies that involved the application of this method. However, due to the inherent weakness, the ridge model itself may suffer from bias estimation of parameter, leading to serious deviation. There are lots of researches focused on the consequences caused by bias estimation of ridge regression, but the compromise between the deviation of parameter and the effect of reducing collineaity of model gained less attention. In this paper, firstly, the data of violation rate and economy of the United States were available to exhibit the serious impact on the model caused by bias estimation on parameter according to the results of significant testing, secondly I calculated the mean square error (MSE) of estimated parameter, and then find the deviation of MSE between ridge regression and OLS (ordinal least square), thirdly optimized the loss function between the variance inflation factor (VIF) and the deviation of MSE of ridge regression. Ultimately, I obtained an acceptable bias range of estimation and optimal VIF in order to modify the ridge model.