How to use expert advice

Journal of the ACM

Volumn 44, Issue 3, 1997, Pages 427-485

  Cesa Bianchi, Nicolò ^a   Freund, Yoav ^b   Haussler, David ^c   Helmbold, David P ^c   Schapire, Robert E ^b   Warmuth, Manfred K ^c  

^a UNIVERSITY OF MILAN   (Italy)

^b AT AND T LABS RESEARCH   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Algorithms

Indexed keywords

ARTIFICIAL INTELLIGENCE; KNOWLEDGE ACQUISITION; LEARNING ALGORITHMS; PATTERN RECOGNITION;

AUTOMATIC PROGRAMMING;

EXPERT SYSTEMS;

EID: 0031140246     PISSN: 00045411     EISSN: None     Source Type: Journal    
DOI: 10.1145/258128.258179     Document Type: Article

References (47)

1. BIRGE, L., AND MASSART, P. 1993. Rates of convergence for minimum contrast estimators. Prob. Theory Rel. Fields 97, 113-150.
2. BLUMER, A., EHRENFEUCHT, A., HAUSSLER, D., AND WARMUTH, M. K. 1989. Learnability and the Vapnik-Chervonenkis dimension. J. ACM 36, 4, 929-965.
3. CESA-BIANCHI, N., FREUND, Y., HELMBOLD, D. P., HAUSSLER, D., SCHAPIRE, R. E., AND WARMUTH, M. K. 1993. How to use expert advice. In Proceedings of the 25th Annual ACM Symposium on the Theory of Computing (San Diego, Calif., May 16-18). ACM, New York, pp. 382-391.
4. CESA-BIANCHI, N., FREUND, Y., HELMBOLD, D. P., AND WARMUTH, M. K. 1996. On-line prediction and conversion strategies. Mach. Learn., to appear.
5. CHUNG, T. H. 1994. Approximate methods for sequential decision making using expert advice. In Proceedings of the 7th Annual ACM Conference on Computational Learning Theory (New Brunswick, N.J., July 12-15). ACM, New York, pp. 183-189.
6. COVER, T. M. 1965. Behaviour of sequential predictors of binary sequences. In Transactions of the 4th Prague Conference on Information Theory, Statistical Decision Functions, Random Processes. Publishing House of the Czechoslovak Academy of Sciences.
7. COVER, T. M., AND SHANAR, A. 1977. Compound Bayes predictors for sequences with apparent Markov structure. IEEE Trans. Syst. Man Cybernet. SMC-7, 6 (June), 421-424.
8. DAWID, A. P. 1984. Statistical theory: The prequential approach. J. Roy. Stat. Soc., Series A, 278-292.
9. DAWID, A. 1991. Prequential analysis, stochastic complexity and Bayesian inference. In Bayesian Statistics, vol. 4. Oxford University Press, pp. 109-125.
10. DAWID, A. P. 1996. Prequential data analysis. Curr. Iss. Stat. Inference, to appear.
11. DESANTIS, A., MARKOWSKI, G., AND WEGMAN, M. N. 1988. Learning probabilistic prediction functions. In Proceedings of the 1988 Workshop on Computational Learning Theory. Morgan-Kaufmann, San Mateo, Calif., pp. 312-328.
12. FEDER, M., MERHAV, N., AND GUTMAN, M. 1992. Universal prediction of individual sequences. IEEE Trans. Inf. Theory 38, 1258-1270.
13. FIAT, A., FOSTER, D., KARLOFF, H., RABANI, Y., RAVID, Y., AND VISWANATHAN, S. 1991a. Competitive algorithms for layered graph traversal. In Proceedings of the 32nd Annual Symposium on Foundations of Computer Science. IEEE, New York, pp. 288-297.
14. FIAT, A., KARP, R., LUBY, M., MCGEOCH, L., SLEATOR, D., AND YOUNG, N. 1991b. Competitive paging algorithms. J. Algorithms 12, 688-699.
15. FIAT, A., RABANI, Y., AND RAVID, Y. 1994. Competitive k-server algorithms. J. Comput Syst. Sci. 48, 3, 410-428.
16. GALAMBOS, J. 1987. The Asymptotic Theory of Extreme Order Statistics, 2nd ed. R. E. Kreiger.
17. HANNAN, J. 1957. Approximation to Bayes risk in repeated play. In Contributions to the Theory of Games, vol. 3. Princeton University Press, Princeton, N.J., pp. 97-139.
18. HAUSSLER, D., AND BARRON, A. 1992. How well do Bayes methods work for on-line prediction of {+1, -1} values? In Proceedings of the 3rd NEC Symposium on Computation and Cognition. SIAM.
19. HAUSSLER, D., KEARNS, M., LITTLESTONE, N., AND WARMUTH, M. K. 1991. Equivalence of models for polynomial learnability. Inf. Comput. 95, 129-161.
20. HAUSSLER, D., KEARNS, M., AND SCHAPIRE, R. 1994. Bounds on the sample complexity of Bayesian learning using information theory and the VC dimensions. Mach. Learn. 14, 84-114.
21. HAUSSLER, D., KIVINEN, J., AND WARMUTH, M. K. 1995. Tight worst-case loss bounds for predicting with expert advice. In Computational Learning Theory: Second European Conference, EuroCOLT '95. Springer-Verlag, New York, pp. 69-83.
22. HAUSSLER, D., LITTLESTONE, N., AND WARMUTH, M. K. 1994. Predicting {0, 1}-functions on randomly drawn points. Inf. Comput. 115, 2, 248-292.
23. HAYKIN, S. 1994. Neural Networks: A Comprehensive Foundation. MacMillan, New York.
24. HELMBOLD, D., AND WARMUTH, M. K. 1995. On weak learning. J. Comput. Syst. Sci. 50, 3 (June), 551-573.
25. KEARNS, M. J., AND SCHAPIRE, R. E. 1994. Efficient distribution-free learning of probabilistic concepts. J. Comput. Syst. Sci. 48, 3, 464-497.
26. KEARNS, M. J., SCHAPIRE, R. E., AND SELLIE, L. M. 1994. Toward efficient agnostic learning. Mach. Learn. 17, 115-141.
27. KIVINEN, J., AND WARMUTH, M. K. 1994. Using experts for predicting continuous outcomes. In Computational Learning Theory: EuroCOLT '93. Springer-Verlag, New York, pp. 109-120.
28. LITTLESTONE, N. 1989. From on-line to batch learning. In Proceedings of the 2nd Annual Workshop on Computational Learning Theory. Morgan-Kaufmann, San Mateo, Calif., pp. 269-284.
29. LITTLESTONE, N., LONG, P. M., AND WARMUTH, M. K. 1995. On-line learning of linear functions. Computat. Complexity 5, 1, 1-23.
30. LITTLESTONE, N., AND WARMUTH, M. K. 1994. The weighted majority algorithm. Inf. Comput. 108, 2, 212-261.
31. MERHAV, N., AND FEDER, M. 1993. Universal schemes for sequential decision for individual data sequences. IEEE Trans. Inf. Theory, 39, 4, 1280-1292.
32. RISSANEN, J. 1978. Modeling by shortest data description. Automatica 14, 465-471.
33. RISSANEN, J. 1986. Stochastic complexity and modeling. Ann. Stat. 14, 3, 1080-1100.
34. RISSANEN, J., AND LANGDON, G. G., JR. 1981. Universal modeling and coding. IEEE Trans. Inf. Theory IT-27, 1 (Jan.), 12-23.
35. SEUNG, H. S., SOMPOLINSKY, H., AND TISHBY, N. 1992. Statistical mechanics of learning from examples. Phys. Rev A 45, 8, 6056-6091.
36. SHTARKOV, Yu. M. 1975. Coding of discrete sources with unknown statistics. In Topics in Information Theory. North-Holland, Amsterdam, The Netherlands, pp. 559-574.
37. SHTARKOV, YU. M. 1987. Universal sequential coding of single messages. Prob. Inf. Transm. 23 (July-September), 175-186.
38. SOMPOLINSKY, H., TISHBY, N., AND SEUNG, H. S. 1992. Learning from examples in large neural networks. Phys. Rev. Lett. 65, 1683-1686.
39. STONE, C. J. 1977. Cross-validation: A review. Math. Operationforsch. Statist. Ser Statist. 9, 127-139.
40. TALAGRAND, M. 1994. Sharper bounds for Gaussian and empirical processes. Ann. Prob. 22, 1, 28-76.
41. VALIANT, L. G. 1984. A theory of the learnable. Commun. ACM 27, 11 (Nov.), 1134-1142.
42. VAPNIK, V. N. 1982. Estimation of Dependences Based on Empirical Data. Springer-Verlag, New York.
43. VAPNIK, V. 1992. Principles of risk minimization for learning theory. In Advances in Neural Information Processing Systems, vol. 4. John E. Moody, Steve J. Hanson, and Richard P. Lippman, eds. Morgan Kaufmann, San Mateo, Calif.
44. VOVK, V. G. 1990. Aggregating strategies. In Proceedings of the 3rd Annual Workshop on Computational Learning Theory. Morgan-Kaufmann, San Mateo, Calif., pp. 371-383.
45. VOVK, V. G. 1992. Universal forecasting algorithms. Inf. Comput. 96, 2 (Feb.), 245-277.
46. VOVK, V. G. 1993. A logic of probability, with application to the foundations of statistics. J. Roy. Statis Soc, Ser. B-Methodolical 55, 2, 317-351.
47. YAMANISHI, K. 1995. A loss bound model for on-line stochastic prediction algorithms. Inf. Comput. 119, 1, 39-54.