Consistency in models for distributed learning under communication constraints

IEEE Transactions on Information Theory

Volumn 52, Issue 1, 2006, Pages 52-63

  Predd, Joel B ^a   Kulkarni, Sanjeev R ^a   Poor, H Vincent ^a  

^a Princeton University   (United States)

Author keywords

Classsification; Consistency; Distributed learning; Nonparametric; Regression; Sensor networks; Statistical pattern recognition

Indexed keywords

CLASSIFICATION (OF INFORMATION); PATTERN RECOGNITION; REGRESSION ANALYSIS; SAMPLING;

BINARY CLASSIFICATION; COMMUNICATION CONSTRAINTS; DISTRIBUTED LEARNING; IDENTICALLY DISTRIBUTED; REGRESSION FRAMEWORKS; SAMPLING PROCESS; SENSOR NETWORKS; STATISTICAL PATTERN RECOGNITION;

INFORMATION THEORY;

EID: 33144479374     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.860420     Document Type: Article

References (40)

1. I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, "A survey on sensor networks," IEEE Commun. Mag., vol. 40, no. 8, pp. 102-114, Aug. 2002.
2. M. Barkat and P. K. Varshney, "Decentralized CFAR signal detection," IEEE Trans. Aerosp. Electron. Syst., vol. 25, pp. 141-149, Mar. 1989.
3. R. Blum, S. Kassam, and H. V. Poor, "Distributed detection with multiple sensors II: Advanced topics," Proc. IEEE, vol. 85, no. 1, pp. 64-79, Jan. 1997.
4. L. Breiman, "Bagging predictors," Mach. Learn., vol. 26, no. 2, pp. 123-140, 1996.
5. T. M. Cover, "Rates of convergence for nearest neighbor procedures," in Proc. 1st Annu. Hawaii Conf. Systems Theory, Honolulu, HI, Jan. 1968, pp. 413-415.
6. A. D'Costa and A. M. Sayeed, "Collaborative signal processing for distributed classification in sensor networks," in Proc. IPSN'03 (Lecture Notes in Computer Science), F. Zhao and L. Guibas, Eds. Berlin, Germany: Springer-Verlag, 2003.
7. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York, NY: Springer, 1996.
8. M. Dong, L. Tong, and B. M. Sadler, "Information retrieval and processing in sensor networks: Deterministic scheduling vs. random access," IEEE Trans. Inf. Theory, submitted for publication.
9. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," Comp. Syst. Sci., vol. 55, pp. 119-139, 1997.
10. Y. Freund, R. E. Schapire, Y. Singer, and M. K. Warmuth, "Using and combining predictors that specialize," in Proc. 29th Annual ACM Symp. Theory of Computing, El Paso, TX, May 1997, pp. 334-343.
11. W. Greblicki and M. Pawlak, "Necessary and sufficient conditions for bayes risk consistency of recursive kernel classification rule," IEEE Trans. Inf. Theory, vol. IT-33, no. 3, pp. 408-412, May 1987.
12. L. Györfi, M. Kohler, A. Krzyzak, and H. Walk, A Distribution-Free Theory of Nonparametric Regression. New York: Springer-Verlag, 2002.
13. R. Jacobs, M. I. Jordan, S. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Comput., vol. 3, no. 1, pp. 125-130, 1991.
14. M. Kearns and H. S. Seung, "Learning from a population of hypotheses," Mach. Learn., vol. 18, pp. 255-276, 1995.
15. J. Kittler, M. Hatef, P. W. Duin, and J. Matas, "On combining classifiers," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 3, pp. 226-239, Mar. 1998.
16. A. N. Kolmogorov and S. V. Fomin, Introductory Real Analysis. New York: Dover, 1975.
17. J. H. Kotecha, V. Ramachandran, and A. Sayeed, "Distributed multitarget classification in wireless sensor networks," IEEE J. Sel. Areas in Commun. (Special Issue on Self-Organizing Distributed Collaborative Sensor Networks), vol. 23, no. 4, pp. 703-713, Apr. 2005.
18. A. Krzyzak, "The rates of convergence of kernel regression estimates and classification rules," IEEE Trans. Inf. Theory, vol. IT-32, no. 5, pp. 668-679, Sep. 1986.
19. S. R. Kulkarni and S. E. Posner, "Rates of convergence of nearest neighbor estimation under arbitrary sampling," IEEE Trans. Inf. Theory, vol. 41, no. 4, pp. 1028-1039, Jul. 1995.
20. n-nn and kernel estimators consistent for arbitrary processes," IEEE. Trans. Inf. Theory, vol. 48, no. 10, pp. 2785-2788, Oct. 2002.
21. S. Kumar, F. Zhao, and D. Shephard, "Collaborative signal and information processing in microsensor networks," IEEE Signal Process. Mag., vol. 19, no. 2, pp. 13-14, 2002.
22. A. Lazarevic and Z. Obradovic, "The distributed boosting algorithm," in KDD'01: Proc. 7th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining. San Francisco, CA: ACM Press, 2001, pp. 311-316.
23. D. Li, K. Wong, Y. H. Hu, and A. Sayeed, "Detection, classification, and tracking of targets," IEEE Signal Process. Mag., vol. 19, no. 2, pp. 17-29, 2002.
24. G. Lugosi, "Learning with an unreliable teacher," Pattern Recogn., vol. 25, pp. 79-87, 1992.
25. G. Morvai, S. R. Kulkarni, and A. B. Nobel, "Regression estimation from an individual stable sequence," Statistics, vol. 33, pp. 99-119, 1999.
26. X. Nguyen, M. J. Wainwright, and M. I. Jordan, "Decentralized detection and classification using kernel methods," in Proc. 21st Int. Conf. Machine Learning, Banff, AB, Canada, 2004.
27. A. B. Nobel, "Limits to classification and regression estimation from ergodic processes," Ann. Statist., vol. 27, pp. 262-273, 1999.
28. A. B. Nobel and T. M. Adams, "On regression estimation from ergodic samples with additive noise," IEEE Trans. Inf. Theory, vol. 47, no. 7, pp. 2895-2902, Nov. 2001.
29. A. B. Nobel, G. Morvai, and S.Kulkarni, "Density estimation from an individual sequence," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 537-541, Mar. 1998.
30. J. B. Predd, S. R. Kulkarni, and H. V. Poor, "Consistency in a model for distributed learning with specialists," in Proc. IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 465.
31. J. B. Predd, S. R. Kulkarni, and H. V. Poor, "Consistency in models for communication constrained distributed learning," in Learning Theory, Proc. 17th Annu. Conference on Learning Theory, COLT 2004, J. Shawe-Taylor and Y. Singer, Eds. New York: Springer-Verlag, 2004.
32. J. B. Predd, S. R. Kulkarni, and H. V. Poor, "Distributed learning in wireless sensor networks," in Proc. 42nd Annu. Allerton Conf. Communication, Control, and Computing, Monticello, IL, Sep. 2004.
33. G. Roussas, "Nonparametric estimation in markov processes," Ann. Inst. Statist. Math., vol. 21, pp. 73-87, 1967.
34. S. N. Simic, "A learning theory approach to sensor networks," IEEE Pervasive Computing, vol. 2, no. 4, pp. 44-49, 2003.
35. C. J. Stone, "Consistent nonparametric regression," Ann. Statist., vol. 5, pp. 595-645, 1977.
36. J. N. Tsitsiklis, "Decentralized detection," in Advances in Statistical Signal Processing. Greenwich, CT: JAI Press, 1993, pp. 297-344.
37. P. K. Varshney, Distributed Detection and Data Fusion. New York: Springer-Verlag, 1996.
38. V. V. Veeravalli, "Decentralized quickest change detection," IEEE Trans. Inf. Theory, vol. 47, no. 4, pp. 1657-1656, May 2001.
39. S. Yakowitz, "Nonparametric density and regression estimation from Markov sequences without mixing assumptions," J. Multivar. Anal., vol. 30, pp. 124-136, 1989.
40. S. Yakowitz, "Nearest neighbor regression estimation for null-recurrent Markov time series," Stoch. Processes Appl., vol. 48, pp. 311-318, 1993.