Algorithmic learning in a random world

Algorithmic Learning in a Random World

Volumn , Issue , 2005, Pages 1-324

  Vovk, Vladimir ^a   Gammerman, Alexander ^a   Shafer, Glenn ^b  

^a UNIVERSITY OF LONDON   (United Kingdom)

^b RUTGERS BUSINESS SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84892133506     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/b106715     Document Type: Book

References (252)

1. Martin ANTHONY (2003). Boolean functions and artificial neural networks. CDAM Research Report LSE-CDAM-2003-01, Centre for Discrete and Applicable Mathematics, London School of Economics and Political Science.
2. Martin ANTHONY and Peter L. BARTLETT (1999). Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge, England.
3. John ARBUTHNOTT (1710-1712). An argument for divine Providence, taken from the constant regularity observ'd in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27:186-190.
4. Raymond C. ARCHIBALD (1926). A rare pamphlet of Moivre and some of his discoveries. Isis, 8:671-683. A facsimile copy of De Moivre's 1733 pamphlet starts on p. 677.
5. Nachman ARONSZAJN (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68:337-404.
6. Eugene A. ASARIN (1987). Some properties of Kolmogorov δ-random finite sequences. Theory of Probability and its Applications, 32:507-508.
7. Eugene A. ASARIN (1988). On some properties of finite objects random in the algorithmic sense. Soviet Mathematics Doklady, 36:109-112. The Russian original published in 1987.
8. Kazuoki AZUMA (1967). Weighted sums of certain dependent random variables. Tohoku Mathematical Journal, 19:357-367.
9. George A. BAKER (1935). The probability that the mean of a second sample will differ from the mean of a first sample by less than a certain multiple of the standard deviation of the first sample. Annals of Mathematical Statistics, 6:197-201.
10. Tadeusz BANACHIEWICZ (1937a). Sur l'inverse d'un cracovien et une solution générale d'un système d'equations linéares. Comptes rendus mensuels des Séances de la Classe des Sciences Mathématiques et Naturelles de l'Academic Polonaise des Sciences et des Lettres, 4:3-4.
11. Tadeusz BANACHIEWICZ (1937b). Zur Berechnung der Determinanten, wie auch der Inversen, und zur darauf basierten Auflösung der Systeme linearer Gleichungen. Acta Astronomica, Series C, 3:41-67.
12. George A. BARNARD (1977). Pivotal inference and the Bayesian controversy. Bulletin of the International Statistical Institute, 47:543-551.
13. Thomas BAYES (1764). An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53:370-418.
14. James O. BERGER and Mohan DELAMPADY (1987). Testing precise hypotheses (with discussion). Statistical Science, 2:317-352.
15. Jose M. BERNARDO and Adrian F. M. SMITH (1994). Bayesian Theory. Wiley, Chichester, England. Paperback edition: 2000.
16. Jacob BERNOULLI (1713). Ars Conjectandi. Thurnisius, Basel. Russian translation (second edition, with commentaries by Oscar B. Sheynin and Yurii V. Prokhorov):0 3aKOHe, Nauka, Moscow, 1986.
17. Christopher M. BISHOP (1995). Neural Networks for Pattern Recognition. Oxford University Press, Oxford, England.
18. Nicolas BOURBAKI (1958). Eléments de mathématique, Livre IV, Fonctions d'une variable réelle (théorie élé mentaire). Hermann, Paris, second edition.
19. Nicolas BOURBAKI (1969). Élements de mathématique, Livre VI, Intégration, Chapitre IX, Intégration sur les espaces topologiques séparé, Note historique. Hermann, Paris.
20. Leo BREIMAN (1994). Bagging predictors. Technical Report 421, Department of Statistics, University of California, Berkley.
21. Lawrence D. BROWN, T. TONY CAI, and Anirban DAS GUPTA (2001). Interval estimation for a binomial proportion (with discussion). Statistical Science, 16:101-133.
22. Francesco P. CANTELLI (1933). Sulla determinazione empirica della leggi di probabilità. Giornale dell'Istituto Italiano degli Attuari, 4:421-424.
23. Aleksei Ya. CHERVONENKIS (2004). (Chervonenkis's recollections). Manuscript.
24. R. DENNIS COOK and Sanford WEISBERG (1982). Residuals and Influence in Regression. Chapman and Hall, New York.
25. Thomas H. CORMEN, Charles E. LEISERSON, Ronald L. RIVEST, and Clifford STEIN (2001). Introduction to Algorithms. MIT Press, Cambridge, MA, second edition.
26. Robert G. COWELL, A. PHILIP DAWID, Steffen L. LAURITZEN, and David J. SPIEGELHALTER (1999). Probabilistic Networks and Expert Systems. Springer, New York.
27. David R. COX (1958a). The regression analysis of binary sequences (with discussion). Journal of the Royal Statistical Society, Series B, 20:215-242.
28. David R. COX (1958b). Some problems connected with statistical inference. Annals of Mathematical Statistics, 29:357-372.
29. David R. COX (1970). The Analysis of Binary Data. Methuen, London. Second edition: 1989 (with E. J. Snell).
30. David R. COX and David V. HINKLEY (1974). Theoretical Statistics. Chapman and Hall, London.
31. Harald CRAMER (1946). Mathematical Methods of Statistics. Princeton University Press, Princeton, NJ.
32. Noel A. C. CRESSIE (1993). Statistics for Spatial Data. Wiley, New York, revised edition.
33. Nello CRISTIANINI and John SHAWE-TAYLOR (2000). An Introduction to Support Vector Machines and Other Kernel-based Methods. Cambridge University Press, Cambridge, England.
34. Anthony C. DAVISON and David V. HINKLEY (1997). Bootstrap Methods and their Applications. Cambridge University Press, Cambridge, England.
35. A. PHILIP DAWID (1982). Intersubjective statistical models. In George S. Koch and Fabio Spizzichino, editors, Exchangeability in Probability and Statistics, pages 217-232. North-Holland, Amsterdam.
36. A. PHILIP DAWID (1985). Self-calibrating priors do not exist: Comment. Journal of the American Statistical Association, 80:340-341. This is a contribution to the discussion in Oakes (1985).
37. A. PHILIP DAWID (1986). Probability forecasting. In Samuel Kotz, Norman L. Johnson, and Campbell B. Read, editors, Encyclopedia of Statistical Sciences, volume 7, pages 210-218. Wiley, New York.
38. A. PHILIP DAWID and Mervyn STONE (1982). The functional-model basis of fiducial inference (with discussion). Annals of Statistics, 10:1054-1067.
39. n in seriem expansi. Included in Archibald 1926.
40. Morris H. DE GROOT (1988). A conversation with George A. Barnard. Statistical Science, 3:196-212.
41. Philip DERBEKO, Ran EL-YANIV, and Ron MEIR (2004). Error bounds for transductive learning via compression and clustering. In Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA. The journal version: Explicit learning curves for transduction and application to clustering and compression algorithms; to appear in Journal of Artificial Intelligence Research.
42. Luc DEVROYE, László GYÖRFI, Adam KRZYZAK, and Gábor LUGOSI (1994). On the strong universal consistency of nearest neighbor regression function estimates. Annals of Statistics, 22:1371-1385.
43. Luc DEVROYE, László GYÖRFI, and Gábor LUGOSI (1996). A Probabilistic Theory of Pattern Recognition. Springer, New York.
44. Luc DEVROYE and Gábor LUGOSI (2001). Combinatorial Methods in Density Estimation. Springer, New York.
45. Persi DIACONIS and David A. FREEDMAN (1980). De Finetti's theorem for Markov chains. Annals of Probability, 8:115-130.
46. Thomas G. DIETTERICH and Ghulum BAKIRI (1995). Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2:263-286.
47. Jean DIEUDONNÉ (1948). Sur le théorème de Lebesgue-Nikodym. III. Annates de l'Institut Fourier, 23:25-53.
48. Joseph L. DOOB (1953). Stochastic Processes. Wiley, New York.
49. Norman R. DRAPER and Harry SMITH (1998). Applied Regression Analysis. Wiley, New York, third edition.
50. Harris DRUCKER, Chris J. C. BURGES, Linda KAUFMAN, Alex SMOLA, and Vladimir N. VAPNIK (1996). Support Vector regression machines. In Michael C. Mozer, Michael I. Jordan, and Thomas Petsche, editors, Advances in Neural Information Processing Systems 9, pages 155-161. MIT Press, Cambridge, MA.
51. Richard M. DUDLEY (2002). Real Analysis and Probability. Cambridge University Press, Cambridge, England. Original edition published in 1989 by Wadsworth.
52. Aryeh DVORETZKY, Jack C. KIEFER, and Jacob WOLFOWITZ (1956). Asymptotic minimax character of a sample distribution function and of the classical multinomial estimator. Annals of Mathematical Statistics, 27:642-669.
53. Bradley EFRON (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7:1-26.
54. Bradley EFRON (2003). Second thoughts on the bootstrap. Statistical Science, 18:135-140.
55. Bradley EFRON and Robert J. TIBSHIRANI (1993). An Introduction to the Bootstrap. Chapman and Hall, New York.
56. Bruno DE FINETTI (1930). Funzione caratteristica di un fenomeno aleatorio. Memorie della Reale Accademia Nazionale dei Lincei: Classe di scienze fisiche, matematiche, e naturali, Serie 6, 4:86-133.
57. Bruno DE FINETTI (1937). La prévision, ses lois logiques, ses sources subjectives. Annales de l'Institut Henri Poincaré, 7:1-68. An English translation of this article is included in Kyburg and Smokier 1980.
58. Bruno DE FINETTI (1938). Sur la condition d'équivalence partielle, volume 739 of Actualites Scientifiques et Industrielles. Hermann, Paris. An English translation is included in Jeffrey 1981, pp. 193-206.
59. Ronald A. FISHER (1922). The goodness of fit of regression formulae and the distribution of regression coefficients. Journal of the Royal Statistical Society, 85:597-612.
60. Ronald A. FISHER (1925). Applications of "Student's" distribution. Metron, 5:90-104.
61. Ronald A. FISHER (1930). Inverse probability. Proceedings of the Cambridge Philosophical Society, 26:528-535.
62. Ronald A. FISHER (1935). The fiducial argument in statistical inference. Annals of Eugenics, 6:391-398.
63. Ronald A. FISHER (1973a). Statistical Methods and Scientific Inference. Hafner, New York, third edition. Included in Fisher 1991. First edition: 1956.
64. Ronald A. FISHER (1973b). Statistical Methods for Research Workers. Hafner, New York, fourteenth (revised and enlarged) edition. Included in Fisher 1991. First edition: 1925.
65. Ronald A. FISHER (1991). Statistical Methods, Experimental Design, and Scientific Inference. Oxford University Press, Oxford, England. Edited by J. Henry Bennett.
66. Sally FLOYD and Manfred K. WARMUTH (1995). Sample compression, learnability, and the Vapnik-Chervonenkis dimension. Machine Learning, 21:269-304.
67. Donald A. S. FRASER (1951). Sequentially determined statistically equivalent blocks. Annals of Mathematical Statistics, 22:372-381.
68. Donald A. S. FRASER (1953). Nonparametric tolerance regions. Annals of Mathematical Statistics, 24:44-55.
69. Donald A. S. FRASER (1957). Nonparametric Methods in Statistics. Wiley, New York.
70. Donald A. S. FRASER and Irwin GUTTMAN (1956). Tolerance regions. Annals of Mathematical Statistics, 27:16-32.
71. David A. FREEDMAN (1962). Invariants under mixing which generalise de Finetti's theorem. Annals of Mathematical Statistics, 33:916-923.
72. David A. FREEDMAN (1963). Invariants under mixing which generalise de Finetti's theorem: Continuous time parameter. Annals of Mathematical Statistics, 34:1194-1216.
73. Yoav FREUND, Yishay MANSOUR, and Robert E. SCHAPIRE (2004). Generalization bounds for averaged classifiers. Annals of Statistics, 32:1698-1722.
74. Yoav FREUND and Robert E. SCHAPIRE (1996). Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, pages 148-156.
75. Yoav FREUND and Robert E. SCHAPIRE (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55:119-139. Conference version: EuroCOLT'1995.
76. Peter GA (1980). Exact expressions for some randomness tests. Zeitschrift fur Mathematische Logik und Grundlagen der Mathematik, 26:385-394.
77. Alex GAMMERMAN (1997). Machine learning: Progress and prospects. Royal Holloway, University of London, Egham, Surrey, TW20 OEX, England. This booklet is based on an inaugural lecture delivered on 11 December 1996.
78. Alex GAMMERMAN, Vladimir VAPNIK, and Vladimir VOVK (1997). Transduction in pattern recognition. Manuscript submitted to the Fifteenth International Joint Conference on Artificial Intelligence in January 1997. Extended version published as Gammerman et al. 1998. The algorithm proposed in this paper was described in a 1996 public lecture (Gammerman 1997).
79. Alex GAMMERMAN, Vladimir VOVK, and Vladimir VAPNIK (1998). Learning by transduction. In Gregory F. Cooper and Serafi'n Moral, editors, Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pages 148-155, San Francisco, CA. Morgan Kaufmann.
80. Carl F. GAUSS (1823). Theoria Combinationis Observationum Erroribus Minimis Obnoxiae, Pars Posterior. Dieterich, Gottingen.
81. Valery I. GLIVENKO (1933). Sulla determinazione empirica di probabilità. Giornale dell'Istituto Italiano degli Attuari, 4:92-99.
82. Irwin GUTTMAN (1970). Statistical Tolerance Regions: Classical and Bayesian. Griffin, London.
83. Jules HAAG (1928). Sur une probleme général de probabilités et ses diverses applications. In Proceedings of the International Congress of Mathematicians, Toronto, 1924, pages 659-674, Toronto. Toronto University Press.
84. David HARRISON and Daniel L. RUBINFELD (1978). Hedonic prices and the demand for clean air. Journal of Environmental Economics and Management, 5:81-102.
85. David HAUSSLER, Nick LITTLESTONE, and Manfred K. WARMUTH (1990). Predicting {0, 1}-functions on randomly drawn points. Technical Report UCSC-CRL-90-54, Computer Research Laboratory, University of California, Santa Cruz. Lemma 2.6 and Theorem 2.2 of this technical report are not stated explicitly in its final version Haussler et al. (1994).
86. David HAUSSLER, Nick LITTLESTONE, and Manfred K. WARMUTH (1994). Predicting {0,1}-functions on randomly drawn points. Information and Computation, 115:248-292. Preliminary versions in: Proceedings of the Twenty-Ninth Annual Symposium on Foundations of Computer Science, pp. 100-109 (1988); Proceedings of the First Workshop on Computational Learning Theory, pp. 280-296 (1988).
87. Harold V. HENDERSON and Shayle R. SEARLE (1981). On deriving the inverse of a sum of matrices. SIAM Review, 23:53-60.
88. Ralf HERBRICH and Robert C. WILLIAMSON (2002). Learning and generalization: Theoretical bounds. In Michael A. Arbib, editor, Handbook of Brain Theory and Neural Networks, pages 3140-3150. MIT Press, Cambridge, MA, second edition.
89. Mark HERBSTER and Manfred K. WARMUTH (1998). Tracking the best expert. Machine Learning, 32:151-178.
90. Edwin HEWITT and Leonard J. SAVAGE (1955). Symmetric measures on Cartesian products. Transactions of the American Mathematical Society, 80:470-501.
91. David C. HOAGLIN and Roy E. WELSCH (1978). The hat matrix in regression and ANOVA. American Statistician, 32:17-22.
92. Wassily HOEFFDING (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58:13-30.
93. Arthur E. HOERL (1959). Optimum solution of many variables equations. Chemical Engineering Progress, 55:69-78.
94. Arthur E. HOERL (1962). Applications of ridge analysis to regression problems. Chemical Engineering Progress, 58:54-59.
95. Arthur E. HOERL and Robert W. KENNARD (1970a). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12:55-67.
96. Arthur E. HOERL and Robert W. KENNARD (1970b). Ridge regression: Applications to nonorthogonal problems. Technometrics, 12:69-82.
97. Erratum, 12:723.
98. Roger W. HOERL (1985). Ridge analysis 25 years later. American Statistician, 39:186-192.
99. David W. HOSMER and Stanley LEMESHOW (2000). Applied Logistic Regression. Wiley, New York, second edition.
100. David HUME (1739-1740). A Treatise of Human Nature. Noon (vols. 1-2, 1739) and Longman (vol. 3, 1740), London.
101. Richard C. JEFFREY, editor (1981). Studies in Inductive Logic and Probability. University of California Press, Berkley, CA.
102. Finn V. JENSEN (1996). An Introduction to Bayesian Networks. UCL Press, London.
103. Marek KARPINSKI and Angus J. MACINTYRE (1995). Polynomial bounds for VC dimension of sigmoidal neural networks. In Proceeding of the Twenty-Seventh Annual ACM Symposium on the Theory of Computing, pages 200-208, New York. ACM Press.
104. Marek KARPINSKI and Angus J. MACINTYRE (1997). Polynomial bounds for VC dimension of sigmoidal and general Pfaffian neural networks. Journal of Computer and System Sciences, 54:169-176.
105. Johan H. B. KEMPERMAN (1956). Generalized tolerance limits. Annals of Mathematical Statistics, 27:180-486.
106. Aleksandr Ya. KHINCHIN (1932). Sur les classes d'événements équivalents. 39:40-43.
107. Berna E. KILINÇ (2001). The reception of John Venn's philosophy of probability. In Vincent F. Hendricks, Stig Andur Pedersen, and Klaus Frovin jørgensen, editors, Probability Theory: Philosophy, Recent History and Relations to Science, pages 97-121. Kluwer, Dordrecht.
108. John F. C. KINGMAN (1972). On random sequences with spherical symmetry. Biometrika, 59:492-493.
109. John F. C. KINGMAN (1978). Uses of exchangeability. Annals of Probability, 6:183-197.
110. Andrei N. KOLMOGOROV (1928). Sur une formule limite de M. A. Khintchine. Comptes rendus des Séances de l'Académie des Sciences, 186:824-825.
111. Andrei N. KOLMOGOROV (1929). Über das Gesetz des iterierten Logarithmus. Mathematische Annalen, 101:126-135.
112. Andrei N. KOLMOGOROV (1933a). Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer, Berlin. Published in English as Foundations of the Theory of Probability (Chelsea, New York). Translation edited by Nathan Morrison. First edition, 1950; second edition, 1956.
113. Andrei N. KOLMOGOROV (1933b). Sulla determinazione empirica di unna legge di distribuzione. Giornale dell'Istituto Italiano degli Attuari, 4:83-91.
114. Andrei N. KOLMOGOROV (1942). (The estimation of the mean and precision from a finite sample of observations). 6:3-32.
115. Andrei N. KOLMOGOROV (1963). On tables of random numbers. Sankhya, The Indian Journal of Statistics, Series A, 25:369-376.
116. Andrei N. KOLMOGOROV (1965). Three approaches to the quantitative definition of information. Problems of Information Transmission, 1:1-7.
117. Andrei N. KOLMOGOROV (1968). Logical basis for information theory and probability theory. IEEE Transactions on Information Theory, IT-14:662-664.
118. Andrei N. KOLMOGOROV (1983). Combinatorial foundations of information theory and the calculus of probabilities. Russian Mathematical Surveys, 38:29-40. This article was written in 1970 in connection with Kolmogorov's talk at the International Mathematical Congress in Nice.
119. Henry KYBURG and Henry SMOKLER, editors (1980). Studies in Subjective Probability. Krieger, New York, second edition.
120. John LANGFORD (2004). Tutorial on practical prediction theory for classification. Article based on an ICML'2003 tutorial and available from the author's web site. Accessed on March 28, 2004.
121. John LANGFORD and John SHAWE-TAYLOR (2003). PAC-Bayes and margins. In Suzanna Becker, Sebastian Thrun, and Klaus Obermayer, editors, Advances in Neural Information Processing Systems 15. MIT Press, Cambridge, MA.
122. Pierre Simon LAPLACE (1774). Mémoire sur la probabilité des causes par les événements. Mémoires de mathématique et de physique, presentés à l'Académie royale des sciences, par divers savans & lûs dans ses assemblées, 6:621-656.
123. English translation: Statistical Science, 1:364-378.
124. Steffen L. LAURITZEN (1982). Statistical Models as Extremal Families. Aalborg University Press, Aalborg, Denmark.
125. Steffen L. LAURITZEN (1988). Extremal Families and Systems of Sufficient Statistics, volume 49 of Lecture Notes in Statistics. Springer, New York.
126. Steffen L. LAURITZEN (1996). Graphical Models, volume 17 of Oxford Statistical Science Series. Clarendon Press, Oxford, England.
127. Yann LE CUN, Bernhard E. BOSER, John S. DENKER, Donnie HENDERSON, R. E. HOWARD, Wayne E. HUBBARD, and Lawrence D. JACKEL (1990). Handwritten digit recognition with a back-propagation network. In David S. Touretzky, editor, Advances in Neural Information Processing Systems 2, pages 396-404. Morgan Kaufmann, San Mateo, CA. The surname of the first author is spelt as Le Cun in this paper; we use the spelling that Prof. LeCun now prefers.
128. Adrien M. LEGENDRE (1805). Nouvellcs méthodes pour la détermination des orbites des cométes. Courcier, Paris.
129. Erich L. LEHMANN (1986). Testing Statistical Hypotheses. Wiley, New York, second edition. First edition: 1959.
130. Leonid A. LEVIN (1973). On the notion of a random sequence. Soviet Mathematics Doklady, 14:1413.
131. Leonid A. LEVIN (1976). Uniform tests of randomness. Soviet Mathematics Doklady, 17:337-340.
132. Leonid A. LEVIN (1984). Randomness conservation inequalities; information and independence in mathematical theories. Information and Control, 61:15-37.
133. Paul LÉVY (1922). Leçons d'Analyse fonctionnelle. Gauthier-Villars, Paris.
134. Ming LI and Paul VITÁNYI (1997). An Introduction to Kolmogorov Complexity and its Applications. Springer, New York, second edition.
135. Yi LI, Philip M. LONG, and Aravind SRINIVASAN (2001). The one-inclusion graph algorithm is near-optimal for the prediction model of learning. IEEE Transactions on Information Theory, 47:1257-1261.
136. Nick LITTLESTONE and Manfred K. WARMUTH (1986). Relating data compression and learnability. Technical report, University of California, Santa Cruz.
137. Andrei A. MARKOV (1906). (Extension of the law of large numbers to dependent events). Notices of the Physico-Mathematical Society of the University of Kazan, Second Series, 15:135-156.
138. Per MARTIN-LÖF (1966). The definition of random sequences. Information and Control, 9:602-619.
139. Per MARTIN-LÖF (1974). Repetitive structures. In Ole E. Barndorff-Nielsen, Preben Blaesild, and Geert Schou, editors, Proceedings of Conference on Foundational Questions in Statistical Inference, Aarhus. Memoirs 1.
140. Pascal MASSART (1990). The tight constant in the Dvoretzky-Kiefer- Wolfowitz inequality. Annals of Probability, 18:1269-1283.
141. Georges MATHERON (1963). Principles of geostatistics. Economic Geology, 58:1246-1266.
142. David A. McALLESTER (1998). Some PAC-Bayesian theorems. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pages 230-234, New York. ACM Press. Journal version: McAllester 1999b.
143. David A. McALLESTER (1999a). PAC-Bayesian model averaging. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, pages 164-170, New York. ACM Press.
144. David A. McALLESTER (1999b). Some PAC-Bayesian theorems. Machine Learning, 37:355-363.
145. Colin J. H. McDIARMID (1989). On the method of bounded differences. In J. Siemons, editor, Surveys in Combinatorics: Invited Papers at the Twelfth British Combinatorial Conference, volume 141 of London Mathematical Society Lecture Notes Series, pages 148-188. Cambridge University Press, Cambridge, England.
146. Thomas MELLUISH (2005). Transductive algorithms for finding confidence information for regression estimation in the typicalness framework. PhD thesis, Royal Holloway, University of London. Submitted.
147. Thomas MELLUISH, Craig SAUNDERS, Ilia NOURETDINOV, and Vladimir VOVK (2001a). Comparing the Bayes and typicalness frameworks. Technical Report CLRC-TR-01-05, Computer Learning Research Centre, Royal Holloway, University of London.
148. Thomas MELLUISH, Craig SAUNDERS, Ilia NOURETDINOV, and Vladimir VOVK (2001b). Comparing the Bayes and typicalness frameworks. In Luc De Raedt and Peter A. Flach, editors, Machine Learning: ECML'2001. Proceedings of the Twelfth European Conference on Machine Learning, volume 2167 of Lecture Notes in Computer Science, pages 360-371, Heidelberg. Springer. The formula for the variance of the predictive distribution in terms of a kernel (§4 of this paper) is wrong; this is corrected in Melluish et al. 2001a and Melluish 2005.
149. James MERCER (1909). Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London, 209:415-446.
150. Richard VON MISES (1919). Grundlagen der Wahrscheinlichkeitsrechnung. Mathematische Zeitschrift, 5:52-99.
151. Richard VON MISES (1928). Wahrscheinlichkeitsrechnung, Statistik und Wahrheit. Julius Springer, Wien. The second German edition appeared in 1936 and the third in 1951. English editions, under the title Probability, Statistics and Truth, appeared in 1939 and 1957.
152. Melanie MITCHELL (1996). An Introduction to Genetic Algorithms. MIT Press, Cambridge, MA.
153. Tom M. MITCHELL (1997). Machine Learning. McGrow-Hill, New York.
154. Douglas C. MONTGOMERY, Elizabeth A. PECK, and G. GEOFFREY VINING (2001). Introduction to Linear Regression Analysis. Wiley, New York, third edition.
155. NeuroCOLT (2002). Generalisation bounds less than 0.5. NeuroCOLT Workshop, 29 April - 2 May 2002, Windsor, England.
156. Ilia NOURETDINOV, Thomas MELLUISH, and Vladimir VOVK (2001a). Ridge Regression Confidence Machine. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 385-392, San Francisco, CA. Morgan Kaufmann.
157. Ilia NOURETDINOV and Vladimir VOVK (2003). Criterion of calibration for transductive confidence machine with limited feedback. In Ricard Gavaldà, Klaus P. Jantke, and Eiji Takimoto, editors, Proceedings of the Fourteenth International Conference on Algorithmic Learning Theory, volume 2842 of Lecture Notes in Artificial Intelligence, pages 259-267, Berlin. Springer. To appear in Theoretical Computer Science (Special Issue devoted to the ALT'2003).
158. Ilia NOURETDINOV, Vladimir VOVK, Michael VYUGIN, and Alex GAMMERMAN (2001b). Pattern recognition and density estimation under the general i.i.d. assumption. In David Helmbold and Bob Williamson, editors, Proceedings of the Fourteenth Annual Conference on Computational Learning Theory and Fifth European Conference on Computational Learning Theory, volume 2111 of Lecture Notes in Artificial Intelligence, pages 337-353. Springer.
159. Ilia NOURETDINOV, Vladimir V. VYUGIN, and Alex GAMMERMAN (2003). Transductive confidence machine is universal. In Ricard Gavaldá, Klaus P. Jantke, and Eiji Takimoto, editors, Proceedings of the Fourteenth International Conference on Algorithmic Learning Theory, volume 2842 of Lecture Notes in Artificial Intelligence, pages 283-297, Berlin. Springer.
160. David OAKES (1985). Self-calibrating priors do not exist (with discussion). Journal of the American Statistical Association, 80:339-342.
161. Anthony O'HAGAN (1994). Kendall's Advanced Theory of Statistics, volume 2b: Bayesian Inference. Arnold, London.
162. Harris PAPADOPOULOS (2004). Qualified predictions for large data sets. PhD thesis, Royal Holloway, University of London.
163. Harris PAPADOPOULOS, Konstantinos PROEDROU, Vladimir VOVK, and Alex GAMMERMAN (2002a). Inductive Confidence Machines for regression. In Machine Learning: ECML'2002. Proceedings of the Thirteenth European Conference on Machine Learning, volume 2430 of Lecture Notes in Computer Science, pages 345-356.
164. Harris PAPADOPOULOS, Vladimir VOVK, and Alex GAMMERMAN (2002b). Qualified predictions for large data sets in the case of pattern recognition. In Proceedings of the International Conference on Machine Learning and Applications (ICMLA'02), pages 159-163. CSREA Press.
165. Judea PEARL (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Fransisco, CA.
166. Egon S. PEARSON (1968). Studies in the history of probability and statistics. XX: Some early correspondence between W. S. Gosset, R. A. Fisher and Karl Pearson, with notes and comments. Biometrika, 55:445-457.
167. Karl PEARSON (1925). James Bernoulli's theorem. Biometrika, 17:201-210.
168. Richard R. PICARD and Kenneth N. BERK (1990). Data splitting. American Statistician, 44:140-147.
169. Robin L. PLACKETT (1972). Studies in the history of probability and statistics. XXIX: The discovery of the method of least squares. Biometrika, 59:239-251.
170. Karl R. POPPER (1999). The Logic of Scientific Discovery. Routledge, London. First published in German in 1934; first English edition 1959.
171. Konstantinos PROEDROU (2003). Rigorous measures of confidence for pattern recognition and regression. PhD thesis, Royal Holloway, University of London.
172. Yurii V. PROKHOROV (1986). This is Commentary II to the second Russian edition of Jacob Bernoulli's Ars conjectandi (Bernoulli 1713).
173. J. ROSS QUINLAN (1993). C4-5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA.
174. Nancy REID (1994). A conversation with Sir David Cox. Statistical Science, 9:439-455.
175. Brian D. RIPLEY (1996). Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge, England.
176. Ronald L. RIVEST and Robert H. SLOAN (1988). Learning complicated concepts reliably and usefully. In Proceedings of the First Annual Workshop on Computational Learning Theory, pages 69-79, San Mateo, CA. Morgan Kaufmann.
177. L. CHRIS G. ROGERS and David WILLIAMS (1994). Diffusions, Markov Processes, and Martingales, volume 1: Foundations. Wiley, Chichester, England, second edition. Reissued by Cambridge University Press in Cambridge Mathematical Library, 2000.
178. David E. RUMELHART and James L. MCCLELLAND, editors (1986). Parallel Distributed Processing: Explorations in the Micro structure of Cognition, volume 1: Foundations. MIT Press, Cambridge, MA.
179. Daniil RYABKO (2003). Relaxing i.i.d. assumption in online pattern recognition. Technical Report CS-TR-03-11, Department of Computer Science, Royal Holloway, University of London.
180. Daniil RYABKO, Vladimir VOVK, and Alex GAMMERMAN (2003). Online prediction with real teachers. Technical Report CS-TR-03-09, Department of Computer Science, Royal Holloway, University of London.
181. Craig SAUNDERS (2000). Efficient implementation and experimental testing of transductive algorithms for predicting with confidence. PhD thesis, Royal Holloway, University of London.
182. Craig SAUNDERS, Alex GAMMERMAN, and Vladimir VOVK (1998). Ridge regression learning algorithm in dual variables. In Jude W. Shavlik, editor, Machine Learning, Proceedings of the Fifteenth International Conference, pages 515-521, San Francisco, CA. Morgan Kaufmann.
183. Craig SAUNDERS, Alex GAMMERMAN, and Vladimir VOVK (1999). Transduction with confidence and credibility. In Thomas Dean, editor, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, volume 2, pages 722-726. Morgan Kaufmann.
184. Craig SAUNDERS, Alexander GAMMERMAN, and Vladimir VOVK (2000). Computationally efficient transductive machines. In Hiroki Arimura, Sanjay Jain, and Arun Sharma, editors, Proceedings of the Eleventh International Conference on Algorithmic Learning Theory (ALT 2000), volume 1968 of Lecture Notes in Artificial Intelligenece, pages 325-333, Berlin. Springer.
185. Robert E. SCHAPIRE (1990). The strength of weak learnability. Machine Learning, 5:197-227.
186. Robert E. SCHAPIRE, Yoav FREUND, Peter L. BARTLETT, and Wee Sun LEE (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26:1651-1686.
187. Mark J. SCHERVISH (1995). Theory of Statistics. Springer, New York.
188. Bernhard SCHOLKOPF and Alexander J. SMOLA (2002). Learning with Kernels. MIT Press, Cambridge, MA.
189. Hilary L. SEAL (1967). Studies in the history of probability and statistics. XV: The historical development of the Gauss linear model. Biometrika, 54:1-24.
190. Matthias SEEGER (2003). PAC-Bayesian generalization error bounds for Gaussian process classification. Journal of Machine Learning Research, 3:233-269.
191. Glenn SHAFER (1996a). The Art of Causal Conjecture. MIT Press, Cambridge, MA.
192. Glenn SHAFER (1996b). Probabilistic Expert Systems, volume 67 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia.
193. Glenn SHAFER and Vladimir VOVK (2001). Probability and Finance: It's only a Game! Wiley, New York.
194. Glenn SHAFER and Vladimir VOVK (2003). Sources of Kolmogorov's Grundbegriffe, The Game-Theoretic Probability and Finance Project, Working Paper #4, http://probabilityandf inance.com.
195. John SHAWE-TAYLOR and Nello CRISTIANINI (2004). Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, England.
196. Walter A. SHEWHART (1931). Economic Control of Quality of Manufactured Product. Van Nostrand, New York.
197. Oscar B. SHEYNIN (2003). (The History of Probability Theory before the Twentieth Century). St. Petersburg.
198. Albert N. SHIRYAEV (1996). Probability. Springer, New York, second edition.
199. Patrice SIMARD, Yann LECUN, and John DENKER (1993). Efficient pattern recognition using a new transformation distance. In Steven J. Hanson, Jack D. Cowan, and C. Lee Giles, editors, Advances in Neural Information Processing Systems 5. Morgan Kaufmann, San Mateo, CA.
200. Adrian F. M. SMITH (1981). On random sequences with centred spherical symmetry. Journal of the Royal Statistical Society, Series B, 43:208-209.
201. J. LAURIE SNELL (1997). A conversation with Joe Doob. Statistical Science, 12:301-311.
202. Stephen M. STIGLER (1981). Gauss and the invention of least squares. Annals of Statistics, 9:465-474. Revised version published as Chap. 17 of Stigler 1999.
203. Stephen M. STIGLER (1986a). The History of Statistics: The Measurement of Uncertainty before 1900. Harvard University Press, Cambridge, MA.
204. Stephen M. STIGLER (1986b). Laplace's 1774 memoir on inverse probability. Statistical Science, 1:359-378.
205. Stephen M. STIGLER (1999). Statistics on the Table: The History of Statistical Concepts and Methods. Harvard University Press, Cambridge, MA.
206. Robert A. STINE (1985). Bootstrap prediction intervals for regression. Journal of the American Statistical Association, 80:1026-1031.
207. Mark O. STETSON, Alex GAMMERMAN, Vladimir N. VAPNIK, Vladimir VOVK, Chris WATKINS, and Jason WESTON (1997). Support Vector ANOVA decomposition. Technical Report CSD-TR-97-22, Royal Holloway, University of London.
208. Charles J. STONE (1977). Consistent nonparametric regression (with discussion). Annals of Statistics, 5:595-645.
209. Mervyn STONE (1974). Cross-validatory choice and assessment of statistical predictions (with discussion). Journal of the Royal Statistical Society, Series B, 36:111-147. Barnard's comment (proposing the vote of thanks):133-135.
210. William F. STOUT (1970). A martingale analogue of kolmogorov's law of the iterated logarithm. Zeitschrift fur Wahrscheinlichkeitstheorie und verwandte Gebiete, 15:279-290.
211. Alan STUART, Keith J. ORD, and Steven ARNOLD (1999). Kendall's Advanced Theory of Statistics, volume 2a: Classical Inference and the Linear Model. Arnold, London, sixth edition.
212. William S. GOSSETT (Student) (1908). On the probable error of a mean. Biometrika, 6:1-25.
213. Richard S. SUTTON and Andrew G. BARTO (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.
214. Kei TAKEUCHI (1975). Statistical Prediction Theory. Baifukan, Tokyo. In Japanese.
215. John W. TUKEY (1947). Nonparametric estimation II: Statistically equivalent blocks and tolerance regions - the continuous case. Annals of Mathematical Statistics, 18:529-539.
216. Alan M. TURING (1950). Computing machinery and intelligence. Mind, 59:433-460.
217. William T. TUTTE (2001). Graph Theory. Cambridge University Press, Cambridge, England.
218. Leslie G. VALIANT (1984). A theory of the learnable. Communications of the ACM, 27:1134-1142.
219. Vladimir N. VAPNIK (1982). Estimation of Dependences Based on Empirical Data. Springer, New York. This is the English translation of: Nauka, Moscow, 1979.
220. Vladimir N. VAPNIK (1995). The Nature of Statistical Learning Theory. Springer, New York. Second edition: 2000.
221. Vladimir N. VAPNIK (1998). Statistical Learning Theory. Wiley, New York.
222. Vladimir N. VAPNIK and Aleksei Ya. CHERVONENKIS (1968). On the uniform convergence of relative frequencies of events to their probabilities. Soviet Mathematics Doklady, 9:915-918.
223. Vladimir N. VAPNIK and Aleksei Ya. CHERVONENKIS (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and Its Applications, 16:264-280.
224. Vladimir N. VAPNIK and Aleksei Ya. CHERVONENKIS (1974). (Theory of Pattern Recognition). Nauka, Moscow. German translation: W. Wapnik and A. Tscherwonenkis, Theorie der Zeichenerkennung, Akademie-Verlag, Berlin.
225. Vladimir N. VAPNIK and Alexander M. STERIN (1977). Ordered minimization of total risk in a pattern-recognition problem. Automation and Remote Conrol, 10:1495-1503. Russian original in: 10:83-92.
226. John VENN (1866). The Logic of Chance. Macmillan, London.
227. Jean VILLE (1939). Étude critique de la notion de collectif. Gauthier-Villars, Paris.
228. Vladimir VOVK (1986). On the concept of the Bernoulli property. Russian Mathematical Surveys, 41:247-248.
229. Vladimir VOVK (1990). Aggregating strategies. In Mark Fulk and John Case, editors, Proceedings of the Third Annual Workshop on Computational Learning Theory, pages 371-383, San Mateo, CA. Morgan Kaufmann.
230. Vladimir VOVK (1993). A logic of probability, with application to the foundations of statistics (with discussion). Journal of the Royal Statistical Society, Series B, 55:317-351.
231. Vladimir VOVK (1999). Derandomizing stochastic prediction strategies. Machine Learning, 35:247-282.
232. Vladimir VOVK (2001a). Competitive on-line statistics. International Statistical Review, 69:213-248.
233. Vladimir VOVK (2001b). Probability theory for the Brier game. Theoretical Computer Science, 261:57-79.
234. Vladimir VOVK (2002a). Asymptotic optimality of Transductive Confidence Machine. In Proceedings of the Thirteenth International Conference on Algorithmic Learning Theory, volume 2533 of Lecture Notes in Artificial Intelligence, pages 336-350, Berlin. Springer.
235. Vladimir VOVK (2002b). On-line Confidence Machines are well-calibrated. In Proceedings of the Forty-Third Annual Symposium on Foundations of Computer Science, pages 187-196, Los Alamitos, CA. IEEE Computer Society. This paper's main results were first reported at the NeuroCOLT workshop in May 2002.
236. Vladimir VOVK (2003a). Universal well-calibrated algorithm for on-line classification. In Bernhard Schölkopf and Manfred K. Warmuth, editors, Learning Theory and Kernel Machines: Sixteenth Annual Conference on Learning Theory and Seventh Kernel Workshop, COLT/Kernel 2003, volume 2777 of Lecture Notes in Artificial Intelligence, pages 358-372, Berlin. Springer.
237. Appears in revised form in the Special Issue on Advances in Learning Theory and Kernel Methods of Journal of Machine Learning Research devoted to COLT/Kernel'2003, 5:575-604.
238. Vladimir VOVK (2003b). Well-calibrated predictions from on-line compression models. In Ricard Gavalda, Klaus P. Jantke, and Eiji Takimoto, editors, Proceedings of the Fourteenth International Conference on Algorithmic Learning Theory, volume 2842 of Lecture Notes in Artificial Intelligence, pages 268-282, Berlin. Springer.
239. Vladimir VOVK (2004). Well-calibrated predictions from on-line compression models. This is the journal version (containing a section on hypergraphical models) of Vovk 2003b; to appear in the Special Issue of Theoretical Computer Science devoted to the ALT'2003 conference.
240. Vladimir VOVK, Alex GAMMERMAN, and Craig SAUNDERS (1999). Machine-learning applications of algorithmic randomness. In Proceedings of the Sixteenth International Conference on Machine Learning, pages 444-453, San Francisco, CA. Morgan Kaufmann.
241. Vladimir VOVK, David LINDSAY, Ilia NOURETDINOV, and Alex GAMMERMAN (2003a). Mondrian confidence machine, On-line Compression Modelling project, http://vovk.net/kp, Working Paper #4.
242. Vladimir VOVK, Ilia NOURETDINOV, and Alex GAMMERMAN (2003b). Testing exchangeability on-line. In Tom Fawcett and Nina Mishra, editors, Proceedings of the Twentieth International Conference on Machine Learning, pages 768-775, Menlo Park, CA. AAAI Press.
243. Vladimir VOVK and Glenn SHAFER (2003). Kolmogorov's contributions to the foundations of probability. Problems of Information Transmission, 39:21-31.
244. Vladimir VOVK and Vladimir V. V'YUGIN (1993). On the empirical validity of the Bayesian method. Journal of the Royal Statistical Society, Series B, 55:253-266.
245. Vladimir V. V'YUGIN (1994). Algorithmic entropy (complexity) of finite objects and its applications to defining randomness and amount of information. Selecta Mathematica Sovietica, 13:357-389.
246. Grace WAHBA (1990). Spline Models for Observational Data, volume 59 of CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia.
247. Abraham WALD (1943). An extension of Wilks' method for setting tolerance limits. Annals of Mathematical Statistics, 14:45-55.
248. Abraham WALD (1947). Sequential Analysis. Wiley, New York.
249. Abraham WALD and Jacob WOLFOWITZ (1948). Optimum character of the sequential probability ratio test. Annals of Mathematical Statistics, 19:326-339.
250. Chris WATKINS and Jason WESTON (1999). Support vector machines for multiclass pattern recognition. In Proceedings of the Seventh European Symposium on Artificial Neural Networks, pages 219-224.
251. Samuel S. WILKS (1941). Determination of samples sizes for setting tolerance limits. Annals of Mathematical Statistics, 12:91-96.
252. David WILLIAMS (1991). Probability with Martingales. Cambridge University Press, Cambridge, England.