A Comparison between Some Penalized Methods for Estimating Parameters
Simulation Study
DOI:
https://doi.org/10.25212/lfu.qzj.8.1.44الكلمات المفتاحية:
Penalized Method; Ridge Regression; Elastic-Net Regression; Elastic-Net; Bridgeالملخص
Regression analysis is one of the most popular statistical methods in various biological and economic studies where, frequently, the number of explanatory variables becomes large. Penalized methods have been adapted and have gained popularity as a key for simultaneously performing variable selection and model estimation. This paper proposes contamination procedure from the viewpoint of different types of penalized regression, aiming at identifying any types of penalized methods that are best to deal with contamination data. This paper demonstrates that the Lasso regression is the best method for contamination data depending on the heavy tail distribution behavior of the response variables and using simulation for (15%) data with contamination. The comparison between types of penalized methods based on the statistical criterion (MAE and MSE) and results shows that the Lasso regression is better than another type of penalized method
التنزيلات
المراجع
- Breiman, L., 1995. Better subset regression using the nonnegative garrote. Technometrics, 37(4), pp.373-384.
- Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R., 2004. Least angle regression. The Annals of statistics, 32(2), pp.407-499.
- Fan, J. and Li, R., 2001. Variable selection via non concave penalized likelihood and its oracle properties. Journal of the American statistical Association, 96(456), pp.1348-1360.
- Frank, L.E. and Friedman, J.H., 1993. A statistical view of some chemometrics regression tools. Technometrics, 35(2), pp.109-135.
- Friedman, J. and Popescu, B.E., 2004. Gradient directed regularization. Unpublished manuscript, http://www-stat. Stanford.
- Fu, W. and Knight, K., 2000. Asymptotic for lasso-type estimators. The Annals of statistics, 28(5), pp.1356-1378.
- Fu, W.J., 1998. Penalized regressions: the bridge versus the lasso. Journal of computational and graphical statistics, 7(3), pp.397-416.
- Helwig, N.E., 2017. Regression with ordered predictors via ordinal smoothing splines. Frontiers in Applied Mathematics and Statistics, 3, p.15.
- Hoerl, A.E. and Kennard, R.W., 1970. Ridge regression: applications to non-orthogonal problems. Technometrics, 12(1), pp.69-82.
- Hoerl, A.E. and Kennard, R.W., 1970. Ridge regression: Biased estimation for non-orthogonal problems. Technometrics, 12(1), pp.55-67.
- Huang, J., Ma, S. and Zhang, C.H., 2008. Adaptive Lasso for sparse high-dimensional regression models. Statistica Sinica, pp.1603-1618.
- Tibshirani, R., 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), pp.267-288.
- Tutz, G. and Ulbricht, J., 2009. Penalized regression with correlation-based penalty. Statistics and Computing, 19(3), pp.239-253.
- Van der Kooij, A.J., 2007. Prediction accuracy and stability of regression with optimal scaling transformations. Leiden University.
- Wood, S.N., 2006. Generalized additive models: an introduction with R. chapman and hall/CRC.
- Zou, H. and Hastie, T., 2005. Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology), 67(2), pp.301-320.
- Zou, H. and Li, R., 2008. One-step sparse estimates in non-concave penalized likelihood models. The Annals of statistics, 36(4), pp.1509-1533.
التنزيلات
منشور
كيفية الاقتباس
إصدار
القسم
الرخصة
هذا العمل مرخص بموجب Creative Commons Attribution 4.0 International License.
Qalaai Zanist Journal allows the author to retain the copyright in their articles. Articles are instead made available under a Creative Commons license to allow others to freely access, copy and use research provided the author is correctly attributed.
Creative Commons is a licensing scheme that allows authors to license their work so that others may re-use it without having to contact them for permission
