An Elementary Introduction to Statistical Learning Theory

An Elementary Introduction to Statistical Learning Theory

Volumn , Issue , 2011, Pages 1-219

  Kulkarni, Sanjeev ^a   Harman, Gilbert ^b  

^a Princeton University   (United States)

^b PRINCETON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; DIAGNOSIS; IMAGE RECOGNITION; LEARNING SYSTEMS; MEDICAL IMAGING; PHILOSOPHICAL ASPECTS; PROBABILITY; SPEECH RECOGNITION; STATISTICS;

BAYES DECISION RULES; NEAREST NEIGHBOR RULE; PATTERN RECOGNITION PROBLEMS; STATISTICAL ARBITRAGE; STATISTICAL LEARNING; STATISTICAL LEARNING THEORY; STATISTICAL PATTERN RECOGNITION; SYSTEMATIC METHODOLOGY;

ENGINEERING EDUCATION;

EID: 84949789552     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118023471     Document Type: Book

References (144)

1. Aizerman MA, Braverman EM, Rozonoer LI. Theoretical foundations of the potential function method in pattern recognition learning. Automat Remote Control 1964; 25: 917-936.
2. Akaike H. An approximation to the density function. Ann Inst Stat Math 1954; 6: 127-132.
3. Akaike H. A new look at the statistical model identification. IEEE Trans Automat Control 1974; 19: 716-723.
4. Alon N, Ben-David S, Cesa-Bianchi N, Haussler D. Scale sensitive dimensions, uniform convergence, and learnability. Symposium on Foundations of Computer Science. Washington, D.C.: IEEE Computer Society Press; 1993.
5. Angluin D, Smith CH. Inductive inference: theory and methods. Comput Surv 1983; 15:237-269.
6. Anthony M, Bartlett PL. Neural network learning: theoretical foundations. Cambridge: Cambridge University Press; 1999.
7. Bartlett PL, Long PM, Williamson RC. Fat-shattering and the learnability of real-valued functions. J Comput Syst Sci 1996; 52(3): 434-452.
8. Bartlett P, Schölkopf B, Schuurmans D, Smola AJ, editors. Advances in large-margin classifiers. Cambridge (MA): MIT Press; 2000.
9. Bashkirov O, Braverman EM, Muchnik IE. Potential function algorithms for pattern recognition learning machines. Automat Remote Control 1964; 25: 692-695.
10. Bengson J, Moffett MA, Wright JC. The folk on knowing how. Phil Stud 2009; 142: 387-401.
11. Bertsekas D, Tsitsiklis J. Introduction to probability. 2nd ed. Belmont (MA): Athena Scientific; 2008.
12. Billingsly P. Probability and measure. 3rd ed. New York: Wiley Interscience; 1995.
13. Bishop C. Pattern recognition and machine learning. New York: Springer; 2006.
14. Blum M. A machine-independent theory of the complexity of recursive functions. J Assoc Comput Mach 1967; 14: 322-336.
15. Blum L, Blum M. Toward a mathematical theory of inductive inference. Inf Control 1975; 28: 125-155.
16. Blumer A, Ehrenfeucht A, Haussler D, Warmuth M. Learnability and the Vapnik-Chervonenkis dimension. J ACM 1989; 36(4): 929-965.
17. Bongard M. Pattern recognition. Washington (DC): Spartan Books; 1970.
18. Boser B, Guyon I, Vapnik VN. A training algorithm for optimal margin classifiers. Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory. New York: Association for Computing Machinery; 1992. pp. 144-152.
19. Braverman EM. The method of potential functions. Automat Remote Control 1965; 26: 2130-2138.
20. Burges CJC. A tutorial on support vector machines for pattern recognition. Data Mining Knowl Discov 1998; 2: 121-167.
21. Chung KL. A course in probability theory. 2nd ed. Amsterdam: Academic Press; 2000.
22. Corder GW, Foreman DI. Nonparametric statistics for non-statisticians: a step-by-step approach. Hoboken (NJ): Wiley; 2009.
23. Corfield D, Schölkopf B, Vapnik V. Falsificationism and statistical learning theory: comparing the popper and Vapnik-Chervonenkis dimensions. J Gen Philos Sci. 2009; 40: 51-58.
24. Cortes C, Vapnik VN. Support vector networks. Mach Learn 1995; 20: 1-25.
25. Cover TM. Estimation by the nearest neighbor rule. IEEE Trans Inf Theory 1968;IT-14:50-55.
26. Cover TM, Hart PE. Nearest neighbor pattern classification. IEEE Trans Inf Theory 1967; 13(1): 21-27.
27. Cristianini N, Shawe-Taylor J. An introduction to support vector machines. Cambridge University Press; 2000.
28. Cybenko G. Approximations by superpositions of sigmoidal functions. Math Control Signals Syst 1989; 2(4): 303-314.
29. Dasarathy BV, editor. Nearest Neighbor (NN) Norms: NN pattern classification techniques. Washington (DC): IEEE Computer Society; 1991.
30. Della Rocca M. Two spheres, twenty spheres, and the identity of indiscernibles. Pac Philos Q 2005; 86: 480-492.
31. Descartes R. Meditationes de Prima Philosophia. Paris; 1641.
32. Devijver PR, Kittler J. Pattern recognition: a statistical approach. Englewood Cliffs (NJ):Prentice-Hall; 1982.
33. Devroye L, Györfi L, Lugosi G. A probabilistic theory of pattern recognition. New York:Springer Verlag; 1996.
34. Dewey J. Logic: the theory of inquiry. New York: Holt; 1938.
35. Dietterich TG. Ensemble learning. The handbook of brain theory and neural networks. 2nd ed. Cambridge (MA): MIT Press; 2003
36. Duda RO, Hart PE, Pattern classification and Scene Analysis. New York: Wiley; 1973.
37. Duda RO, Hart PE, Stork DG. Pattern classification. 2nd ed. New York: Wiley; 2001.
38. Dudley RM. Real analysis and probability. 2nd ed. Cambridge: Cambridge University Press; 2002.
39. Dworkin R. Law's empire. Cambridge (MA): Harvard UP; 1986.
40. Everitt BS. Chance rules: an informal guide to probability, risk, and statistics. New York:Copernicus, Springer-Verlag; 1999.
41. Feller W. Volume 1, An introduction to probability theory and its applications. 3rd ed. New York: Wiley; 1968.
42. Fix E, Hodges JL. Discriminatory analysis: nonparametric discrimination: consistency properties. USAF Sch Aviat Med 1951; 4: 261-279.
43. Fix E, Hodges JL. Discriminatory analysis: nonparametric discrimination: small sample performance. USAF Sch Aviat Med 1952; 11: 280-322.
44. Forster MR, Sober E. How to tell when simpler, more unified, or less Ad Hoc theories will provide more accurate predictions. Br J Philos Sci 1994; 45: 1-35.
45. Freund Y. Boosting a weak learning algorithm by majority. Inf Comput 1995; 121(2): 256-285.
46. Freund Y, Schapire RE. A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 1997; 55(1): 119-139.
47. Freund Y, Schapire RE. A short introduction to boosting. J Jpn Soc Artif Intell 1999;14(5):771-780. (In Japanese, translation by Naoki Abe.)
48. Fukunaga K. Introduction to statistical pattern recognition. 2nd ed. San Diego (CA): Academic Press; 1990.
49. Gigerenzer G, et al., Simple heuristics that make us smart. Oxford: Oxford University Press; 1999.
50. Gold EM. Language identification in the limit. Inf Control 1967; 10: 447-474.
51. Goodman N. Fact, fiction, and forecast. 2nd ed. Indianapolis (IN): Bobbs-Merrill; 1965.
52. Györfi L, editor. Principles of nonparametric learning. New York: Springer; 2002.
53. Györfi L, Härdle W, Sarda P, Vieu P. Nonparametric curve estimation from time series. Lecture notes in statistics. Berlin: Springer-Verlag; 1989.
54. Györfi L, Kohler M, Krzyzak A, Walk H. A distribution-free theory of nonparametric regression. Springer; 2002.
55. Hacking I. The logic of statistical inference. Cambridge: Cambridge University Press; 1965.
56. Hacking I. The emergence of probability. Cambridge: Cambridge University Press; 1984.
57. Hájek A. What conditional probability could not be. Synthese 2003; 137: 273-323.
58. Hardle W. Applied nonparametric regression. Cambridge University Press; 1992.
59. Harman G. Change in view: principles of reasoning. Cambridge (MA): MIT Press; 1986.
60. Harman G. Simplicity as a pragmatic criterion for deciding what hypotheses to take seriously. In: Harman G, editor. Reasoning, meaning, and mind. Oxford: Oxford University Press; 1999.
61. Harman G, Kulkarni S. Reliable reasoning: induction and statistical learning theory. Cambridge (MA): MIT Press; 2007.
62. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. 2nd ed. Springer; 2009.
63. Haussler D. Decision theoretic generalizations of the PAC model for neural net and other learning applications. Inf Comput 1992; 100: 78-150.
64. Haykin S. Neural networks: a comprehensive foundation. New York: Macmillan Publishing Company; 1994.
65. Ho YC, Agrawala A. On pattern classification algorithms: introduction and survey. Proc IEEE 1968; 56: 2101-2114.
66. Holst M, Irle A. Nearest neighbor classification with dependent training sequences. Ann Stat 2001; 29(5): 1424-1442.
67. Kearns MJ, Schapire RE. Efficient distribution-free learning of probabilistic concepts. J Comput Syst Sci 1994; 48(3): 464-497.
68. Kearns M, Valiant LG. Learning Boolean formulae or finite automata is as hard as factoring Technical Report TR-14-88. Harvard University, Aiken Computation Laboratory; 1988.
69. Kearns MJ, Vazirani UV. An introduction to computational learning theory. Cambridge (MA): MIT Press; 1994.
70. Kolmogorov AN. Grundbegriffe der wahrscheinlichkeitrechnung. Ergebnisse der mathematik; translated as foundations of probability. New York: Chelsea Publishing Company; 1933; 1950.
71. Kugel P. Induction, pure and simple. Inf Control 1977; 35: 276-336.
72. Kulkarni SR, Lugosi G, Venkatesh S. Learning pattern classification-A survey. IEEE Trans Inf Theory 1998; 44(6): 2178-2206.
73. Lozano A, Kulkarni SR, Schapire R. Convergence and consistency of regularized boosting algorithms with stationary β-mixing observations. Volume 18, Advances in neural information processing systems; 2006.
74. Ludlow P. Simplicity and generative grammar. In Stainton R, Murasugi K, editors. Philosophy and linguistics. Boulder (CO): Westview Press; 1998.
75. Mach E. The science of mechanics. 6th ed. Chicago: Open Court; 1960.
76. McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 1943; 5: 115-133.
77. Meir R, Rätsch G. An introduction to boosting and leveraging. In: Mendelson S, Smola A, editors. Advanced lectures on machine learning, LNCS. New York: Springer; 2003. pp. 119-184.
78. Minsky M, Papert S. Perceptrons. Expanded edition. Cambridge (MA): MIT Press; 1988.
79. Mitchell TM. Machine learning. New York: McGraw-Hill; 1997. pp. 226-229.
80. Nadaraya EA. On estimating regression. Theory Probab Appl 1964; 9: 141-142.
81. Nadaraya EA. Remarks on nonparametric estimates for density functions and regression curves. Theory Probab Appl 1970; 15: 134-137.
82. Nadaraya EA. Nonparametric estimation of probability densities and regression curves. Dordrecht:Kluwer; 1989.
83. Nagy G. State of the art in pattern recognition. Proc IEEE 1968; 56: 836-862.
84. Nilsson NJ. Learning machines. New York: McGraw-Hill; 1965.
85. Nozick R. Simplicity as fall-out. In: Cauman L, Levi I, Parsons C, editors. How many questions: essays in honor of sydney morgenbesser. Indianapolis: Hackett Publishing; 1983. pp. 105-119.
86. Parzen E. On the estimation of a probability density function and the mode. Ann Math Stat 1962; 33: 1065-1076.
87. Peirce CS. In: Hartshorne C, Weiss P, Burks A, editors. Collected papers of Charles Sanders Peirce, 8 vols. Cambridge: Harvard University Press; 1931-58.
88. Poldrak RA, Halchenko YO, Hanson SJ. Decoding the large-scale structure of brain function by classifying mental states across individuals. Psychol Sci 2009; 20(11): 1364-1372.
89. Pollard D. Convergence of stochastic processes. New York: Springer; 1984.
90. Popper K. Objective knowledge: an evolutionary approach. Oxford: Clarendon Press; 1979.
91. Popper K. The logic of scientific discovery. London: Routledge; 2002.
92. Putnam H. "Degree of confirmation" and inductive logic. In: Schillp A, editor. The philosophy of Rudolph Carnap. LaSalle (IL): Open Court; 1963.
93. Rabinowitz L. Elementary probability with applications. Wellesley (MA): AK Peters; 2004.
94. Rawls J. A theory of justice. Cambridge (MA): Harvard University Press; 1971.
95. Rissanen J. Volume 15, Stochastic complexity in statistical inquiry, Singapore: World Scientific, Series in Computer Science; 1989.
96. Rosenblatt M. Remarks on some nonparametric estimates of a density function. Ann Math Stat 1956; 27: 832-837.
97. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 1958; 65: 386-408.
98. Rosenblatt F. Perceptron simulation experiments. Proc Inst Radio Eng 1960; 48: 301-309.
99. Rosenblatt F. Principles of neurodynamics. Washington (DC): Spartan Books; 1962.
100. Ross SM. A first course in probability. 8th ed. Upper Saddle River (NJ): Prentice-Hall; 2009.
101. Rumelhart DE, McClelland JL, editors. Volume 1, Parallel distributed processing: explorations in the microstructure of cognition. Cambridge (MA): MIT Press; 1986.
102. Rumelhart DE, Hinton GE, McClelland JL. Learning representations by back-propagating errors. Nature 1986a;323:533-536.
103. Rumelhart DE, Hinton GE, McClelland JL. Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL, editors. Volume 1, Parallel distributed processing. Cambridge (MA): MIT Press; 1986b, Chapter 8.
104. Russell S, Norvig P. Artificial intelligence: a modern approach. Learning from examples. Upper Saddle River (NJ): Prentice-Hall; 2010. pp. 645-767, Chapter 18.
105. Schalkoff RJ. Pattern recognition: statistical, structural, and neural approaches. New York:Wiley; 1992.
106. Schapire RE. The strength of weak learnability. Mach Learn 1990; 5(2): 197-227.
107. Schapire RE. A brief introduction to boosting. Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. Stockholm; 1999.
108. Schapire RE. The boosting approach to machine learning: an overview. MSRI Workshop on Nonlinear Estimation and Classification; 2001 Mar; Berkeley (CA).
109. Schölkopf B, Burges CJC, Smola AJ, editors. Advances in kernel methods: support vector learning. Cambridge (MA): MIT Press; 1999.
110. Schölkopf B, Smola AJ. Learning with kernels: support vector machines, regularization, optimization, and beyond (Adaptive Computation and Machine Learning). Cambridge (MA): MIT Press; 2001.
111. Scott DW. Multivariate density estimation: theory, practice and visualization. New York:Wiley; 1992.
112. Shawe-Taylor J, Cristianini N. Kernel methods for pattern analysis. Cambridge: Cambridge University Press; 2004.
113. Silverman BW. Density estimation. London: Chapman and Hall; 1986.
114. Sober E. Simplicity. Oxford: Oxford University Press; 1975.
115. Sober E. Reconstructing the past. Cambridge (MA): MIT Press; 1988.
116. Sober E. Let's razor Occam's razor. In: Knowles D, editor. Explanation and its limits. Cambridge: Cambridge University Press; 1990.
117. Solomonoff RJ. A formal theory of inductive inference. Inf Control 1964;7:1-22, 224-254.
118. Stalker D. Grue: the new riddle of induction. Peru (IL): Open Court; 1994.
119. Stanley J, Williamson T. Knowing how. J Philos 2001; 98: 411-444.
120. Steinwart I, Hush D, Scovel C. Learning from dependent observations. J Multivariate Anal 2009; 100: 175-194.
121. Stich SP. Moral philosophy and mental representation. In: Hechter M, Nadel L, Michod R, editors. The origin of values. Hawthorne (NY): Aldine de Gruyter; 1993. pp. 215-228.
122. Stone CJ. Consistent nonparametric regression. Ann Stat 1977; 5: 595-645.
123. Theodoridis S, Koutroumbas K. Pattern recognition. 4th ed. Amsterdam: Academic Press; 2008.
124. Theodoridis S, Pikrakis A, Koutroumbas K, Cavouras D. Introduction to pattern recognition:a matlab approach. Amsterdam: Academic Press; 2010.
125. Tijms H. Understanding probability: chance rules in everyday life. Cambridge: Cambridge University Press; 2007.
126. Turney PD. Inductive inference and stability [PhD dissertation]. Department of Philosophy, University of Toronto; 1988.
127. Valiant LG. The complexity of enumeration and reliability problems. SIAM J Comput 1979; 8: 410-421.
128. Valiant LG. A theory of the learnable. Commun ACM 1984; 27(11): 1134-1142.
129. Vapnik VN. Estimation of dependencies based on empirical data. New York: Springer-Verlag; 1982.
130. Vapnik VN. The nature of statistical learning theory. New York: Springer; 1991.
131. Vapnik VN. The nature of statistical learning theory. New York: Springer-Verlag; 1996.
132. Vapnik VN. The nature of statistical learning theory. New York: Springer; 1999.
133. Vapnik V. Statistical learning theory. New York: Wiley-Interscience; 1998.
134. Vapnik VN, Chervonenkis A. On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 1971; 16(2): 264-280.
135. Vickers J. The Problem of Induction, in The Stanford Encylopedia of Philosophy; 2010, http://plato.stanford.edu/entries/induction-problem/.
136. Vidyasagar M. A theory of learning and generalization. London: Springer-Verlag; 1997.
137. Vidyasagar M. Convergence of empirical means with α-mixing input sequences, and an application to PAC learning. Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference; 2005. pp. 560-565.
138. Watanabe MS. Knowing and guessing. New York: Wiley; 1969.
139. Watson GS. Smooth regression analysis. Sankhya Ser A 1964; 26: 359-372.
140. Werbos PJ. Beyond regression: new tools for prediction and analysis in the behavioral sciences [PhD thesis]. Cambridge (MA); Harvard University; 1974.
141. Widrow B, Generalization and information storage in networks of adaline 'neurons". In:Yovitz MC, editor. Self-organizing systems. Wahington (DC): Spartan; 1962. pp. 435-461.
142. Widrow B, Hoff ME. Adaptive switching circuits. IRE WESCON Convention Record; 1960; 4:96-104.
143. Winograd T, Flores F. Understanding computers and cognition. Norwood (NJ): Ablex; 1986.
144. Zellner A, Keuzenkamp H, McAleer M, editors. Simplicity, inference and econometric modelling. Cambridge: Cambridge University Press; 2002.