Comparing the Performance of Prediction Model of Ridge and Elastic Net in Correlated Dataset

Abstract

Multicollinearity refers to a condition where high correlation between independent variables in linear regression model occurs.  In this case, using ordinary least squares (OLS) leads to unstable model. Some penalized regression approaches such as ridge and elastic-net regression can be applied to overcome the problem. Penalized regression estimates model by adding a constrain on the size of parameter regression. In this study, simulation dataset is generated, comprised of 100 observation and 95 independent variables with high correlation. This empirical study shows that elastic-net method outperforms the ridge regression and OLS.  In correlated dataset, the OLS is failed to produce a prediction model based on mean squared error (MSE)