Universal schemes for learning the best nonlinear predictor given the infinite past and side information

IEEE Transactions on Information Theory

Volumn 45, Issue 4, 1999, Pages 1165-1185

  Algoet, Paul ^a  

^a NONE   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

LEAST SQUARES APPROXIMATIONS; REGRESSION ANALYSIS; TIME SERIES ANALYSIS;

REAL-VALUED TIME SERIES; UNIVERSAL PREDICTION SCHEMES;

NONLINEAR FILTERING;

EID: 0032676483     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.761258     Document Type: Article

References (43)

1. P. H. Algoet Universal schemes for prediction, gambling and portfolio selection," Ann. Probab., vol. 20, pp. 901-941, 1992. Correction: ibid., vol. 23, pp. 474-478, 1995.
2. _ The strong law of large numbers for sequential decisions under uncertainty," IEEE Trans. Inform. Theory, vol. 40, pp. 609-634, May 1994.
3. D. H. Bailey Sequential schemes for classifying and predicting ergodic processes," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1976.
4. A. R. Barron Complexity regularization with applications to artificial neural networks," in Nonparametric Functional Estimation and Related Topics. G. Roussas, Ed. Dordrecht, The Netherlands: Kluwer, 1991, pp. 561-576.
5. D. Bosq, Nonparametric Statistics for Stochastic Processes. New York: Springer-Verlag, 1996.
6. D. L. Burkholder and R. F. Gundy Extrapolation and interpolation of quasi-linear operators on martingales," Acta Math., vol. 124, pp. 249-304, 1970.
7. n, 0 < p < 2," Ann. Math. Statist., vol. 42, pp. 393-394, 1971.
8. Y. S. Chow and H. Teicher, Probability Theory: Independence, Interchangeability, Martingales, 2nd ed. New York: Springer-Verlag, 1978.
9. T. M. Cover Open problems in information theory," in 1975 IEEE Joint Workshop Information Theory. New York: IEEE Press, 1975, pp. 35-36.
10. L. Devroye and L. Györfi Distribution-free exponential bound for the LI error of partitioning estimates of a regression function," in Probability and Statistical Decision Theory, F. Konecny, J. Mogyorödi, and W. Wertz, Eds. Dordrecht, The Netherlands: Reidel, 1985, pp. 67-76.
11. L. Devroye, L. Gyorfi, A. Krzyzak, and G. Lugosi On the strong universal consistency of nearest neighbor regression function estimation," Ann. Statist., vol. 22, pp. 1371-1385, 1994.
12. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
13. J. L. Doob, Stochastic Processes. New York: Wiley, 1953.
14. G. A. Edgar and L. Sucheston, Stopping times and Directed Processes. Cambridge, U.K.: Cambridge Univ. Press, 1992.
15. R. H. Farrell Representation of invariant measures," Illinois J. Math., vol. 6, pp. 447-467, 1962.
16. L. Györfi Universal consistencies of a regression estimate for unbounded regression functions," in Nonparametric Functional Estimation and Related Topics, G. Roussas, Ed. Dordrecht, The Netherlands: Kluwer, 1991, pp. 329-338.
17. L. Györfi, W. Hardie, P. Sarda, and Ph. Vieu, Nonparametric Curve Estimation from Time Series. Berlin, Germany: Springer-Verlag, 1989.
18. L. Györfi, G. Morvai, and S. Yakowitz Limits to on-line forecasting for ergodic time series," IEEE Trans. Inform. Theory, vol. 44, pp. 886-892, Mar. 1998.
19. L. Gordon and R. A. Olshen Almost surely consistent nonparametric regression from recursive partitioning estimates," J. Multivariate Anal., vol. 15, pp. 147-163, 1984.
20. P. Hall and C. C. Heyde, Martingale Limit Theory and Its Application. New York: Academic, 1980.
21. M. Kac On the notion of recurrence in discrete stochastic sequences," Bull. Amer. Math. Soc., vol. 53, pp. 1002-1010, Oct. 1947.
22. K. Knopp, Theory and Application of Infinite Series. London, U.K.: Blackie and Son, 1928.
23. A. S. Kozek, J. R. Leslie, and E. F. Schuster On a universal strong law of large numbers for conditional expectations," preprint, Nov. 1996.
24. U. Krengel, Ergodic Theorems. Berlin, Germany: de Gruyter, 1985.
25. P. T. Maker The ergodic theorem for a sequence of functions," Duke Math. J., vol. 6, pp. 27-30, 1940.
26. P. Masani and N. Wiener Nonlinear prediction," in Probability and Statistics, the Harald Cramer volume, U. Grenander, Ed. New York: Wiley, 1959, pp. 190-212.
27. D. S. Modha and E. Masry Memory-universal prediction of stationary random processes," IEEE Trans. Inform. Theory, vol. 44, pp. 117-133, Jan. 1998.
28. G. Morvai Estimation of conditional distributions for stationary time series," Ph.D. dissertation, Tech. Univ. Budapest, Budapest, Hungary, 1994.
29. G. Morvai, S. Yakowitz, and L. Györfi Nonparametric inference for ergodic, stationary time series," A. Statist., vol. 24, pp. 370-379, Feb. 1996.
30. G. Morvai, S. Yakowitz, and P. Algoet Weakly convergent non-parametric forecasting of stationary time series," IEEE Trans. Inform. Theory, vol. 43, pp. 483-498, Mar. 1997.
31. J. Neveu, Martingales A Temps Discret. Paris, France: Masson, 1972.
32. D. S. Ornstein Guessing the next output of a stationary process," Israel J. Math., vol. 30, pp. 292-296, 1978.
33. D. S. Ornstein and B. Weiss Entropy and data compression schemes," IEEE Trans. Inform. Theory, vol. 39, pp. 78-83, Jan. 1993.
34. K. R. Parthasarathy, Probability Measures on Metric Spaces. New York: Academic, 1967.
35. K. Petersen, Ergodic Theory. Cambridge, U.K.: Cambridge Univ. Press, 1983.
36. J. B. Robertson On a problem of non-linear prediction theory," Ann. Math. Statist., vol. 36, pp. 1554-1555, 1965.
37. B. Ya. Ryabco Prediction of random sequences and universal coding," Probl. Inform. Transm., vol. 24, pp. 87-96, Apr.-June 1988.
38. B. Scharpellini Conditional expectations of stationary processes," Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 56, pp. 427-441, 1981.
39. C. J. Stone Consistent nonparametric regression," A. Statist., vol. 5, pp. 595-645, 1977.
40. W. F. Stout, Almost Sure Convergence. New York: Academic, 1974.
41. H. G. Tucker A generalization of the Glivenko-Cantelli theorem," Ann. Math. Statist., vol. 30, pp. 828-830, 1959.
42. A. D. Wyner and J. Ziv Some asymptotic properties of entropy of a stationary ergodic data source with applications to data compression," IEEE Trans. Inform. Theory, vol. 35, pp. 1250-1258, Nov. 1989.
43. S. Yakowitz, L. Györfi, J. Kieffer, and G. Morvai Strongly-consistent nonparametric estimation of smooth regression functions for stationary ergodic sequences," J. Multivariate Anal, submitted for publication.