Minimum complexity regression estimation with weakly dependent observations

IEEE Transactions on Information Theory

Volumn 42, Issue 6 PART 2, 1996, Pages 2133-2145

  Modha, Dharmendra S ^a,b   Masry, Elias ^a,b  

^a IEEE

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Bernstein inequality; Minimum complexity regression estimation; Mixing processes; Neural networks; Rates of convergence

Indexed keywords

EID: 0000973081     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.556602     Document Type: Article

References (34)

1. D. W. K. Andrews and D. Pollard, "An introduction to functional central limit theorems for dependent stochastic processes," Int. Statist. Rev., vol. 62, no. 1, pp. 119-132, 1994.
2. M. A. Arcones and B. Yu, "Central limit theorems for empirical and U-processes of stationary mixing sequences," J. Theor. Probab., vol. 7, no. 1, 1994.
3. A. R. Barron, "Complexity regularization," in Proc. NATO Advanced Study Institute on Nonparametric Functional Estimation, G. Roussas, Ed. Dordrecht, The Netherlands: Kluwer, 1991, pp. 561-576.
4. _, "Universal approximation bounds for superpositions of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, no. 3, pp. 930-945, May 1993.
5. _, "Approximation and estimation bounds for artificial neural networks," Machine Learning, vol. 14, pp. 115-133, 1994.
6. A. R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 37, no. 4, pp. 1034-1054, July 1991.
7. D. Bosq, "Inégalite de Bernstein pour les processus stationnaires et mélangeants. Applications," C. R. Seances de l'Acad. des Sci. Paris, ser. A, vol. 281, pp. 1095-1098, 1975.
8. M. Carbon, "Inégalite de Bernstein pour les processus fortement mélangeants, non nécessairement stationnaires. Applications," C. R. Seances de l'Acad. des Sci. Paris, ser. I, vol. 297, pp. 303-306, 1983.
9. C. C. Craig, "On the Tchebychef inequality of Bernstein," Ann. Math. Statist., vol. 4, pp. 94-102, 1933.
10. Y. A. Davydov, "Mixing conditions for Markov chains," Theory of Probability and Its Applications, vol. XVIII, no. 2, pp. 312-328, 1973.
11. P. Doukhan, Mixing: Properties and Examples. New York: Springer-Verlag, 1994.
12. P. Doukhan, P. Massart, and E. Rio, "Invariance principles for absolutely regular empirical processes," Ann. Inst. Henri Poincaré, Probab. Statist., vol. 31, no. 2, pp. 393-427, 1995.
13. A. Farago and G. Lugosi, "Strong universal consistency of neural network classifiers," IEEE Trans. Inform. Theory, vol. 39, no. 4, pp. 1146-1151, July 1993.
14. W. A. Fuller, Introduction to Statistical Time Series. New York: Wiley, 1976.
15. P. Hall and C. C. Heyde, Martingale Limit Theory and Its Application. New York: Academic Press, 1980.
16. W. Hoeffding, "Probability inequalities for sums of bounded random variables," J. Amer. Statist. Assoc., vol. 58, pp. 13-30, 1963.
17. K. Hornik, M. B. Stinchcombe, H. White, and P. Auer, "Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives," Neural Comput., vol. 6, pp. 1262-1275, 1994.
18. G. Lugosi and K. Zeger, "Nonparametric estimation via empirical risk minimization," IEEE Trans. Inform. Theory, vol. 41, no. 3, pp. 677-687, May 1995.
19. _, "Concept learning using complexity regularization," IEEE Trans. Inform. Theory, vol. 42, no. 1, pp. 48-54, Jan. 1996.
20. E. Masry, "Strong consistency and rates for deconvolution of multivariate densities of stationary processes," Stoch. Process. Appl., vol. 47, pp. 53-74, 1993.
21. D. F. McCaffrey and A. R. Gallant, "Convergence rates for single hidden layer feedforward networks," Neural Networks, vol. 7, no. 1, pp. 147-158, 1994.
22. D. S. Modha, "Universal prediction of stationary random processes," Ph.D. dissertation, Dept. Elec. Comput. Eng., Univ. California at San Diego, 1995.
23. D. Pollard, Convergence of Stochastic Processes. New York: Springer-Verlag, 1984.
24. J. Rissanen, Stochastic Complexity in Statistical Inquiry. Teaneck, NJ: World Scientific, 1989.
25. P. M. Robinson, "Nonparametric estimators for time series," J. Time Series Anal., vol. 4, pp. 185-297, 1983.
26. G. G. Roussas, "Nonparametric regression estimation under mixing conditions," Stoch. Process. Appl., vol. 36, pp. 107-116, 1990.
27. M. Rosenblatt, "A central limit theorem and strong mixing conditions," Proc. Nat. Acad. Sci., vol. 4, pp. 43-47, 1956.
28. G. R. Shorack and J. A. Wellner, Empirical Processes with Applications to Statistics. New York: Wiley, 1986.
29. V. Vapnik, Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag, 1982.
30. H. White, "Connectionist nonparametric regression: Multilayer feedforward networks can learn arbitrary mappings," Neural Networks, vol. 3, pp. 535-549, 1989.
31. H. White and J. M. Wooldridge, "Some results on sieve estimation with dependent observations," in Nonparametric and Semiparametric Methods in Econometrics and Statistics: Proc. 5th Int. Symp. on Economic Theory and Econometrics, W. A. Barnett, J. Powell, and G. Tauchen, Eds. New York: Cambridge Univ. Press, 1991.
32. C. S. Withers, "Conditions for linear processes to be strong-mixing," Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 57, pp. 477-480, 1981.
33. B. Yu, "Rates of convergence for empirical processes of stationary mixing sequences," Ann. Probab., vol. 22, no. 1, pp. 94-116, 1994.
34. J. E. Yukich, M. B. Stinchcombe, and H. White, "Sup-norm approximation bounds for networks through probabilistic methods," IEEE Trans. Inform. Theory, vol. 41, no. 4, pp. 1021-1027, 1995.