A Comparison between Some Penalized Methods for Estimating Parameters : Simulation Study

Dlshad Mahmood Saleh; Dler Hussein Kadir; Dashty Ismil Jamil

doi:10.25212/lfu.qzj.8.1.44

Authors

Dlshad Mahmood Saleh Department of Statistics and Informatics, College of Administration and Economics, University of Salahaddin, Erbil, Kurdistan Region, Iraq
Dler Hussein Kadir Department of Statistics and Informatics, College of Administration and Economics, University of Salahaddin, Erbil, Kurdistan Region, Iraq Department of Business of Administration, Cihan University-Erbil, Kurdistan region, Iraq.
Dashty Ismil Jamil Department of Marketing, College of Administration and Economics, Lebanese French University, Erbil, Kurdistan Region, Iraq

DOI:

https://doi.org/10.25212/lfu.qzj.8.1.44

Keywords:

Penalized Method; Ridge Regression; Elastic-Net Regression; Elastic-Net; Bridge

Abstract

Regression analysis is one of the most popular statistical methods in various biological and economic studies where, frequently, the number of explanatory variables becomes large. Penalized methods have been adapted and have gained popularity as a key for simultaneously performing variable selection and model estimation. This paper proposes contamination procedure from the viewpoint of different types of penalized regression, aiming at identifying any types of penalized methods that are best to deal with contamination data. This paper demonstrates that the Lasso regression is the best method for contamination data depending on the heavy tail distribution behavior of the response variables and using simulation for (15%) data with contamination. The comparison between types of penalized methods based on the statistical criterion (MAE and MSE) and results shows that the Lasso regression is better than another type of penalized method

Downloads

Download data is not yet available.

References

- Breiman, L., 1995. Better subset regression using the nonnegative garrote. Technometrics, 37(4), pp.373-384.

- Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R., 2004. Least angle regression. The Annals of statistics, 32(2), pp.407-499.

- Fan, J. and Li, R., 2001. Variable selection via non concave penalized likelihood and its oracle properties. Journal of the American statistical Association, 96(456), pp.1348-1360.

- Frank, L.E. and Friedman, J.H., 1993. A statistical view of some chemometrics regression tools. Technometrics, 35(2), pp.109-135.

- Friedman, J. and Popescu, B.E., 2004. Gradient directed regularization. Unpublished manuscript, http://www-stat. Stanford.

- Fu, W. and Knight, K., 2000. Asymptotic for lasso-type estimators. The Annals of statistics, 28(5), pp.1356-1378.

- Fu, W.J., 1998. Penalized regressions: the bridge versus the lasso. Journal of computational and graphical statistics, 7(3), pp.397-416.

- Helwig, N.E., 2017. Regression with ordered predictors via ordinal smoothing splines. Frontiers in Applied Mathematics and Statistics, 3, p.15.

- Hoerl, A.E. and Kennard, R.W., 1970. Ridge regression: applications to non-orthogonal problems. Technometrics, 12(1), pp.69-82.

- Hoerl, A.E. and Kennard, R.W., 1970. Ridge regression: Biased estimation for non-orthogonal problems. Technometrics, 12(1), pp.55-67.

- Huang, J., Ma, S. and Zhang, C.H., 2008. Adaptive Lasso for sparse high-dimensional regression models. Statistica Sinica, pp.1603-1618.

- Tibshirani, R., 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), pp.267-288.

- Tutz, G. and Ulbricht, J., 2009. Penalized regression with correlation-based penalty. Statistics and Computing, 19(3), pp.239-253.

- Van der Kooij, A.J., 2007. Prediction accuracy and stability of regression with optimal scaling transformations. Leiden University.

- Wood, S.N., 2006. Generalized additive models: an introduction with R. chapman and hall/CRC.

- Zou, H. and Hastie, T., 2005. Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology), 67(2), pp.301-320.

- Zou, H. and Li, R., 2008. One-step sparse estimates in non-concave penalized likelihood models. The Annals of statistics, 36(4), pp.1509-1533.