Statistical regression and classification: From linear models to machine learning

Statistical Regression and Classification: From Linear Models to Machine Learning

Volumn , Issue , 2017, Pages 1-493

  Matloff, Norman ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85031663784     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1201/9781315119588     Document Type: Book

References (138)

1. Aggarwal, C. Recommender Systems: The Textbook. Springer International Publishing, 2016.
2. Agresti, A. Categorical Data Analysis. Wiley Series in Probability and Statistics. Wiley, 2014.
3. Agresti, A. Foundations of Linear and Generalized Linear Models. Wiley Series in Probability and Statistics. Wiley, 2015.
4. Alfons, A. cvtools: Cross-validation tools for regression models. https://cran.r-project.org/web/packages/cvTools/index.html.
5. Ambler, G., et al. mfp: Multivariable fractional polynomials. https://cran.r-project.org/web/packages/mpf/index.html.
6. ASA. Data expo '09. http://stat-computing.org/dataexpo/2009/thedata.html, 2009.
7. Banerjee, S., and Roy, A. Linear Algebra and Matrix Analysis for Statistics. Chapman & Hall/CRC Texts in Statistical Science. Taylor & Francis, 2014.
8. Bartlett, P. L., and Traskin, M. Adaboost is consistent. Journal of Machine Learning Research 8 (2007), 2347-2368.
9. Bates, D., et al. lme4: Linear mixed-effects models using ‘eigen’ and s4. https://cran.r-project.org/web/packages/lme4/index.html.
10. Bates, D., and Maechler, M. Matrix: Sparse and dense matrix classes and methods. https://cran.r-project.org/web/packages/Matrix/index.html.
11. Baty, F., et al. nlstools: Tools for nonlinear regression analysis. https://cran.r-project.org/web/packages/nlstools/index.html.
12. Becker, R. The variance drain and Jensen’s inequality. AEPR Working Paper 2012-004, Indiana University, 2010.
13. Bellman, R. Computational Problems in the Theory of Dynamic Programming. Rand paper series. Rand Corporation, 1953.
14. Bendel, R. B., and Afifi, A. A. Comparison of stopping rules in forward stepwise regression. Journal of the American Statistical Association 72, 357 (1977), 46-53.
15. Bennett, K. P., and Campbell, C. Support vector machines: hype or hallelujah? ACM SIGKDD Explorations Newsletter 2, 2 (2000), 1-13.
16. Berk, R. Statistical Learning from a Regression Perspective. Springer Texts in Statistics. Springer International Publishing, 2016.
17. Berk, R., Brown, L., Buja, A., Zhang, K., and Zhao, L. Valid post-selection inference. Ann. Statist. 41, 2 (04 2013), 802-837.
18. Beyer, K. S., Goldstein, J., Ramakrishnan, R., and Shaft, U. When is “nearest neighbor” meaningful? In Proceedings of the 7th International Conference on Database Theory (London, UK, 1999), ICDT '99, Springer-Verlag, pp. 217-235.
19. Bhattacharya, R., and Holmes, S. An exposition of Götze’s estimation of the rate of convergence in the multivariate central limit theorem. arXiv 1003.4254, 2010.
20. Bickel, P. J., Hammel, E. A., and O’Connell, J. W. Sex Bias in Graduate Admissions: Data from Berkeley. Science 187, 4175 (1975), 398-404.
21. Braun, W., and Murdoch, D. A First Course in Statistical Programming with R. Cambridge University Press, 2016.
22. Breiman, L. Random forests. Machine Learning 45, 1 (2001), 5-32.
23. Breiman, L. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statist. Sci. 16, 3 (08 2001), 199-231.
24. Breiman, L., Friedman, J., Stone, C., and Olshen, R. Clas-sification and Regression Trees. The Wadsworth and Brooks-Cole statistics-probability series. Taylor & Francis, 1984.
25. Bretz, F., Hothorn, T., and Westfall, P. Multiple Comparisons Using R. CRC Press, 2016.
26. Brown, C. dummies: Create dummy/indicator variables exibly and efficiently. https://cran.r-project.org/web/packages/dummies/index.html.
27. Buhlmann, P. Statistical significance in high-dimensional linear models. Bernoulli 19, 4 (09 2013), 1212-1242.
28. Bühlmann, P., and van de Geer, S. Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer Series in Statistics. Springer Berlin Heidelberg, 2011.
29. Burgette, L., et al. twang: Toolkit for weighting and analysis of nonequivalent groups. https://cran.r-project.org/web/packages/twang/index.html.
30. Candes, E., Fan, Y., Janson, L., and Lv, J. Panning for gold: Model-free knockoffs for high-dimensional controlled variable selection. arXiv 1610.02351, 2016.
31. Card, D. Estimating the Return to Schooling: Progress on Some Persistent Econometric Problems. Econometrica 69, 5 (2001), 1127-1160.
32. Carroll, R., and Ruppert, D. Transformation and Weighting in Regression. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Taylor & Francis, 1988.
33. Chang, C.-C., and Lin, C.-J. Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 3 (May 2011), 27: 1-27: 27.
34. Chen, S. X., Qin, J., and Tang, C. Y. Mann-Whitney test with adjustments to pretreatment variables for missing values and observational study. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 75, 1 (2013), 81-102.
35. Christensen, R. Log-linear models and logistic regression, 2nd ed. Springer texts in statistics. Springer, New York, c1997. Earlier ed. published under title: Log-linear models. 1990.
36. Clarke, B., Fokoue, E., and Zhang, H. Principles and Theory for Data Mining and Machine Learning. Springer Series in Statistics. Springer New York, 2009.
37. Cleveland, W. S., and Loader, C. Smoothing by Local Regression: Principles and Methods. Physica-Verlag HD, Heidelberg, 1996, pp. 10-49.
38. Cristianini, N., and Shawe-Taylor, J. An Introduction to Support Vector Machines: And Other Kernel-based Learning Methods. Cambridge University Press, New York, NY, USA, 2000.
39. Davison, A., and Hinkley, D. Bootstrap Methods and Their Application. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1997.
40. Deng, H., and Wickham, H. Densty estimation in R. http://vita.had.co.nz/papers/density-estimation.pdf, 2011.
41. Dobson, A., and Barnett, A. An Introduction to Generalized Linear Models, Third Edition. Chapman & Hall/CRC Texts in Statistical Science. CRC Press, 2008.
42. Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. Least angle regression. Ann. Statist. 32, 2 (04 2004), 407-499.
43. Eicker, F. Limit theorems for regressions with unequal and dependent errors. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics. University of California Press, Berkeley, Calif., 1967, pp. 59-82.
44. Eldén, L. Matrix Methods in Data Mining and Pattern Recognition. Fundamentals of Algorithms. Society for Industrial and Applied MathematicsS, 2007.
45. Elzhov, T., et al. minpack.lm: R interface to the Levenberg-Marquardt nonlinear least-squares algorithm found in minpack, plus support for bounds. https://cran.r-project.org/web/packages/minpack.lm/index.html.
46. Emerson, J., and Kane, M. biganalytics: Utilities for ‘big.matrix’ objects from package ‘bigmemory’. https://cran.r-project.org/web/packages/biganalytics/index.html.
47. Faraway, J. Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models. Chapman & Hall/CRC Texts in Statistical Science. CRC Press, 2016.
48. Fong, W. M., and Ouliaris, S. Spectral tests of the martingale hypothesis for exchange rates. Journal of Applied Econometrics 10, 3 (1995), 255-71.
49. Fox, J., et al. car: Companion to applied regression. https://cran.r-project.org/web/packages/car/index.html.
50. Fox, J., and Weisberg, S. An R Companion to Applied Regression. SAGE Publications, 2011.
51. Friedman, J. Another approach to polychotomous classification. Tech. rep., Stanford University, 1996.
52. Galecki, A., and Burzykowski, T. Linear Mixed-Effects Models Using R: A Step-by-Step Approach. Springer Texts in Statistics. Springer New York, 2013.
53. Galilli, T. Is it harder to advertise to the more educated? https://www.r-statistics.com/tag/data-set/, 2010.
54. Gaujoux, R., and Seoighe, C. Nmf: Algorithms and frame-work for nonnegative matrix factorization (nmf). https://cran. r-project.org/web/packages/NMF/index.html.
55. Gelman, A. Why we hate stepwise regression. https://www.r-statistics.com/tag/data-set/, 2010.
56. Gentle, J. Matrix Algebra: Theory, Computations, and Applications in Statistics. Springer Texts in Statistics. Springer New York, 2010.
57. Gilbert, P., and Varadhan, R. numderiv: Accurate numerical derivatives. https://cran.r-project.org/web/packages/numDeriv/index.html.
58. Giles, D. The adjusted r-squared, again. http://davegiles.blogspot.com/2013/07/the-adjusted-r-squared-again.html, 2013.
59. Giles, J. A., and Giles, D. Pre-test estimation and testing in econometrics: Recent developments. Journal of Economic Surveys 7, 2 (1993), 145-97.
60. Goldenberg, A., Zheng, A., Fienberg, S., and Airoldi, E. A Survey of Statistical Network Models. Foundations and Trends(r) in Machine Learning. Now Publishers, 2010.
61. Harrell, F. Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. Springer Series in Statistics. Springer International Publishing, 2015.
62. Harville, D. Matrix Algebra From a Statistician’s Perspective. Springer New York, 2008.
63. Hastie, T., and Efron, B. lars: Least angle regression, lasso and forward stagewise? https://cran.r-project.org/web/packages/lars/index.html.
64. Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York, NY, USA, 2001.
65. Hastie, T., Tibshirani, R., and Wainwright, M. Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. CRC Press, 2015.
66. Hastie, T., Tibshirani, R., and Wainwright, M. Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman & Hall. Taylor & Francis, 2015.
67. Ho, T. K. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 8 (1998), 832-844.
68. Hoerl, A. E., and Kennard, R. W. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 42, 1 (Feb. 2000), 80-86.
69. Hornik, K., Stinchcombe, M., and White, H. Multilayer feed-forward networks are universal approximators. Neural Netw. 2, 5 (July 1989), 359-366.
70. Hothorn, T., et al. mvtnorm: Multivariate normal and t distributions. https://cran.r-project.org/web/packages/mvtnorm/index.html.
71. Hothorn, T., Hornik, K., and Zeileis, A. Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics 15, 3 (2006), 651-674.
72. Hothorn, T., and Zeileis, A. partykit: A toolkit for recursive partytioning. https://cran.r-project.org/web/packages/partykit/index.html.
73. Hsu, J. Multiple Comparisons: Theory and Methods. Taylor & Francis, 1996.
74. Inselberg, A. Parallel Coordinates: Visual Multidimensional Geometry and Its Applications. Advanced series in agricultural sciences. Springer New York, 2009.
75. Jackson, M. Social and Economic Networks. Princeton University Press. Princeton University Press, 2010.
76. James, G., Witten, D., Hastie, T., and Tibshirani, R. An Introduction to Statistical Learning: with Applications in R. Springer Texts in Statistics. Springer New York, 2013.
77. Jiang, J. Linear and Generalized Linear Mixed Models and Their Applications. Springer Series in Statistics. Springer, Dordrecht, 2007.
78. Jordan, M. I. Leo Breiman. Ann. Appl. Stat. 4, 4 (12 2010), 1642-1643.
79. Kang, H., et al. ivmodel: Statistical inference and sensitivity analysis for instrumental variables model. https://cran.r-project. org/web/packages/ivmodel/index.html.
80. Kaye, D., and Freedman, D. Reference guide on statistics. https://www.nap.edu/read/13163/chapter/7.
81. Koenker, R., et al. quantreg: Quantile regression. https:// cran.r-project.org/web/packages/quantreg/index.html.
82. Kraemer, N., and Schaefer, J. parcor: Regularized estimation of partial correlation matrices. https://cran.r-project.org/web/packages/parcor/index.html.
83. Kuhn, M., and Johnson, K. Applied Predictive Modeling. Springer New York, 2013.
84. Lange, K. Numerical Analysis for Statisticians. Springer, 2001.
85. Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., Dean, J., and Ng, A. Building high-level features using large scale unsupervised learning. In International Conference in Machine Learning (2012).
86. Lee, D. D., and Seung, H. S. Learning the parts of objects by nonnegative matrix factorization. Nature 401 (1999), 788-791.
87. Leisch, F., and Dimitriadou, E. mlbench: Machine learning benchmark problems. https://cran.r-project.org/web/packages/mlbench/index.html.
88. Li, S., et al. Fnn: Fast nearest neighbor search algorithms and applications. https://cran.r-project.org/web/packages/FNN/index.html.
89. Liaw, A. randomForest: Breiman and Cutler’s random forests for classification and regression. https://cran.r-project.org/web/packages/randomForest/index.html.
90. Lin, C.-J. On the convergence of multiplicative update algorithms for nonnegative matrix factorization. Trans. Neur. Netw. 18, 6 (Nov. 2007), 1589-1596.
91. Little, R., and Rubin, D. Statistical Analysis with Missing Data. Wiley Series in Probability and Statistics. Wiley, 2014.
92. Loh, W.-Y. Classification and regression trees. Data Mining and Knowledge Discovery 1 (2011), 14-23.
93. Luke, D. A Users Guide to Network Analysis in R. Use R! Springer International Publishing, 2015.
94. Lumley, T. biglm: bounded memory linear and generalized linear models. https://cran.r-project.org/web/packages/sandwich/index.html.
95. Lumley, T., and Zeileis, A. sandwich: Robust covariance matrix estimators. https://cran.r-project.org/web/packages/sandwich/index.html.
96. Matloff, N. partools: Tools for the ‘parallel’ package. https://cran.r-project.org/web/packages/partools/index.html.
97. Matloff, N. regtools: Regression tools. https://cran. r-project.org/web/packages/regtools/index.html.
98. Matloff, N. Estimation of internet file-access/modification rates from indirect data. ACM Trans. Model. Comput. Simul. 15, 3 (July 2005), 233-253.
99. Matloff, N. From Algorithms to Z Scores: Probabilistic and Statistical Modeling in Computer Science. http://heather.cs.ucdavis.edu/probstatbook, 2010.
100. Matloff, N. The Art of R Programming: A Tour of Statistical Software Design. No Starch Press Series. No Starch Press, 2011.
101. Matloff, N. Parallel Computing for Data Science: With Examples in R, C++ and CUDA. Chapman & Hall/CRC The R Series. CRC Press, 2015.
102. Matloff, N. Big-n and big-p in big data. In Handbook of Big Data, P. Bühlmann, P. Drineas, M. Kane, and M. J. van der Laan, Eds. CRC, 2016.
103. Matloff, N. Software alchemy: Turning complex statistical computations into embrassingly-parallel ones. Journal of Statistical Software 71, 1 (2016).
104. Matloff, N., and Xie, Y. freqparcoord: Novel methods for parallel coordinates. https://cran.r-project.org/web/packages/freqparcoord/index.html.
105. Matloff, N. S. Ergodicity conditions for a dissonant voting model. Ann. Probab. 5, 3 (06 1977), 371-386.
106. Matloff, N. S. Use of regression functions for improved estimation of means. Biometrika 68, 3 (Dec. 1981), 685-689.
107. Matloff, N. S. Use of covariates in randomized response settings. Statistics & Probability Letters 2, 1 (1984), 31-34.
108. McCann, L. Written Testimony of Laurie A. McCann, AARP Foundation Litigation, EEOC. 2016.
109. Milborrow, S. rpart.plot: Plot ‘rpart’ models: An enhanced version of ‘plot.rpart’. https://cran.r-project.org/web/packages/rpart.plot/index.html.
110. Miller, A. Subset Selection in Regression. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. CRC Press, 2002.
111. Miller, A. J. Selection of subsets of regression variables. Journal of the Royal Statistical Society. Series A (General) (1984), 389-425.
112. Mizon, G. Inferential procedures in nonlinear models: An application in a UK industrial cross section study of factor substitution and returns to scale. Econometrica 45, 5 (1977), 1221-42.
113. Müller, U. U. Estimating linear functionals in nonlinear regression with responses missing at random. Ann. Statist. 37, 5A (10 2009), 2245-2277.
114. Murphy, K. Machine Learning: A Probabilistic Perspective. Adaptive computation and machine learning series. MIT Press, 2012.
115. 2/n is large. Ann. Statist. 12, 4 (12 1984), 1298-1309.
116. Rao, C., and Toutenburg, H. Linear Models: Least Squares and Alternatives. Springer, 1999.
117. Rao, J., and Molina, I. Small Area Estimation. Wiley Series in Survey Methodology. Wiley, 2015.
118. Rao, R. B., and Fung, G. On the dangers of cross-validation. An experimental evaluation. SIAM, pp. 588-596.
119. Rifkin, R., and Klautau, A. In defense of one-vs-all classification. J. Mach. Learn. Res. 5 (Dec. 2004), 101-141.
120. Ripley, B. Pattern Recognition and Neural Networks. Cambridge University Press, 2007.
121. Ripley, B., and Venables, W. nnet: Feed-forward neural networks and multinomial log-linear models. https://cran. r-project.org/web/packages/nnet/index.html.
122. Rose, R. Nonparametric Estimation of Weights in Least-Squares Regression Analysis. PhD thesis, University of California, Davis, 1978.
123. Rudin, W. Principles of Mathematical Analysis. McGraw-Hill, 1964.
124. Rufibach, K., and Duembgen, L. logcondens: Estimate a logconcave probability density from iid observations. https://cran. r-project.org/web/packages/logcondens/index.html.
125. Sarkar, D. lattice: Trellis graphics for r. https://cran. r-project.org/web/packages/lattice/index.html.
126. Schoerker, B., et al. Ggally: Extension to ‘ggplot2'. https://cran.r-project.org/web/packages/GGally/index.html.
127. Scott, D. Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley Series in Probability and Statistics. Wiley, 2015.
128. Shao, J. Linear model selection by cross-validation. J. of the American Statistical Association 88 (1993), 486-494.
129. Shawe-Taylor, J., and Cristianini, N. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
130. Stone, C. Private communication, 2016.
131. Therneau, T., et al. Introduction to recursive partitioning using the rpart routines. https://cran.r-project.org/web/packages/rpart/vignettes/longintro.pdf.
132. Therneau, T., et al. rpart: Recursive partitioning and regression trees. https://cran.r-project.org/web/packages/rpart/index. html.
133. Valentini, G., Dietterich, T. G., and Cristianini, N. Biasvariance analysis of support vector machines for the development of svm-based ensemble methods. Journal of Machine Learning Research 5 (2004), 725-775.
134. Vinod, H. Hands-on Matrix Algebra Using R: Active and Motivated Learning with Applications. World Scientific, 2011.
135. Wasserman, L. All of Nonparametric Statistics. Springer Texts in Statistics. Springer New York, 2006.
136. Wasserman, L. All of Statistics: A Concise Course in Statistical Inference. Springer Texts in Statistics. Springer New York, 2013.
137. Zeileis, A. Object-oriented computation of sandwich estimators. Journal of Statistical Software 16, 1 (2006), 1-16.
138. Zou, H., Hastie, T., and Tibshirani, R. Sparse principal component analysis. Journal of Computational and Graphical Statistics 15 (2004), 2006.