Distributed learning in wireless sensor networks

IEEE Signal Processing Magazine

Volumn 23, Issue 4, 2006, Pages 56-69

  Predd, Joel B ^a   Kulkarni, Sanjeev R ^b   Poor, H Vincent ^b  

^a PRINCETON UNIVERSITY   (United States)

^b Princeton University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DISTRIBUTED DATABASE SYSTEMS; ESTIMATION; LEARNING SYSTEMS;

DISTRIBUTED LEARNING; WIRELESS SENSOR NETWORKS;

SENSOR DATA FUSION;

EID: 85032752026     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2006.1657817     Document Type: Review

References (105)

1. S.M. Aji and R.J. McEliece, "The generalized distributive law," IEEE Trans. Inform. Theory, vol. 46, no. 2, pp. 325-343, Mar. 2000.
2. I.F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, "A survey on sensor networks," IEEE Commun. Mag., vol. 40, no. 8, pp. 102-114, 2002.
3. M.M. Al-Ibrahim and P.K. Varshney, "Nonparametric sequential detection based on multisensor data," in Proc. 23rd Annu. Conf. Information Science Systems, Johns Hopkins Univ., Baltimore, MD, Mar. 1989, pp. 157-162.
4. M. Anthony and P. Bartlett, Neural Network Learning: Theoretical Foundations. Cambridge, U.K.: Cambridge Univ. Press, 1999.
5. S. Bandyopadhyay and E. Coyle, "An energy efficient hierarchical clustering algorithm for wireless sensor networks," in Proc. 22nd Annu. Joint Conf. IEEE Computer Communications Societies (Infocom), San Francisco, CA, Mar.-Apr. 2003.
6. M. Barkat and P.K. Varshney, "Decentralized CFAR signal detection," IEEE Trans. Aerosp. Electron. Syst., vol. 25, no. 2, pp. 141-149, 1989.
7. T. Berger, Z. Zhang, and H. Vishwanathan, "The CEO problem," IEEE Trans. Inform. Theory, vol. 42, no. 3, pp. 887-902, 1996.
8. D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods. Belmont, MA: Athena Scientific, 1997.
9. D. Blatt and A. Hero, "Distributed maximum likelihood estimation for sensor networks," in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Montreal, Quebec, Canada, May 2004, pp. 929-932.
10. R. Blum, S. Kassam, and H.V. Poor, "Distributed detection with multiple sensors: Part II - Advanced topics," Proc. IEEE, vol. 85, pp. 64-79, 1997.
11. A. Bordes, S. Ertekin, J. Weston, and L. Bottou, "Fast kernel classifiers with online and active learning," J. Mach. Learning Res., vol. 6, pp. 1579-1619, 2005.
12. L. Breiman, "Bagging predictors," Mach. Learning, vol. 26, no. 2, pp. 123-140, 1996.
13. Y. Censor and S.A. Zenios, Parallel Optimization: Theory, Algorithms, and Applications. New York: Oxford, 1997.
14. N. Cesa-Bianchi, Y. Freund, D. Haussler, D.P. Helmbold, R.E. Schapire, and M.K. Warmuth, "How to use expert advice," J. Assoc. Comput. Mach., vol. 44, no. 3, pp. 427-485, 1997.
15. M. Cetin, L. Chen, J.W. Fisher, III, A.T. Ihler, R.L. Moses, M.J. Wainwright, and A.S. Willsky, "Distributed fusion in sensor networks: A graphical models perspective," IEEE Signal Processing Mag., vol. 23, no. 4, pp. 42-55, June 2006.
16. J.-F. Chamberland and V.V. Veeravalli, "Asymptotic results for decentralized detection in power constrained wireless sensor networks," IEEE J. Select. Areas Commun., vol. 22, no. 6, pp. 1007-1015, 2004.
17. N.V. Chawla, L.O. Hall, K.W. Bowyer, and W.P. Kegelmeyer, "Learning ensembles from bites: A scalable and accurate approach," J. Mach. Learning Res., vol. 5, pp. 421-451, 2004.
18. B. Chen, L. Tong, and P.K. Varshney, "Channel aware distributed detection in wireless sensor networks," IEEE Signal Processing Mag., vol. 23, no. 4, pp. 16-25, June 2006.
19. T.M. Cover, "Rates of convergence for nearest neighbor procedures," in Proc. Hawaii Int. Conf. Systems Sciences, Honolulu, Jan. 1968, pp. 413-415.
20. A. D'Costa, V. Ramachadran, and A.M. Sayeed, "Distributed classification of Gaussian space-time sources in wireless sensor networks," IEEE J. Select. Areas Commun, vol. 22, no. 6, pp. 1026-1036, 2004.
21. A. D'Costa and A.M. Sayeed, "Collaborative signal processing for distributed classification in sensor networks," in Proc. 2nd Int. Workshop Information Processing Sensor Networks, Palo Alto, CA, Apr. 2003, pp. 193-208.
22. V. Delouille, R. Neelamani, and R. Baraniuk, "Robust distributed estimation in sensor networks using the embedded polygons algorithm," in Proc. 3rd Int. Symp. Information Processing Sensor Networks, Berkeley, CA, Apr. 2004.
23. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
24. R. Duda, P. Hart, and D. Stork, Pattern Classification, 2nd ed. New York: Wiley-Interscience, 2001.
25. Y. Freund and R.E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," Comput. Syst. Scis., vol. 55, no. 1, pp. 119-139, 1997.
26. Y. Freund, R.E. Schapire, Y. Singer, and M.K. Warmuth, "Using and combining predictors that specialize," in Proc. 29th Annu. ACM Symp. Theory Computing, El Paso, Texas, pp. 334-343, May 1997.
27. S. Gambs, B. Kégl, and E. Aïmeur, "Privacy-preserving boosting," submitted for publication.
28. M. Gastpar, M. Vetterli, and P.L. Dragotti, "Sensing reality and communicating bits: A dangerous liaison," IEEE Signal Processing Mag., vol. 23, no. 4, pp. 70-83, June 2006.
29. S. Gezici, Z. Tian, G.B. Giannakis, H. Kobayashi, A.F. Molisch, H.V. Poor, and Z. Sahinoglu, "Localization via ultra-wideband radios," IEEE Signal Processing Mag., vol. 22, no. 4, pp. 70-84, 2005.
30. G. Golub and C.V. Loan, Matrix Computations. Baltimore, MD: Johns Hopkins Univ. Press, 1989.
31. H.P. Graf, E. Cosatto, L. Bottou, I. Dourdanovic, and V. Vapnik, "Parallel support vector machines: The Cascade SVM," in Advances in Neural Information Processing Systems, vol. 17, L. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, pp. 521-528.
32. W. Greblicki and M. Pawlak, "Necessary and sufficient conditions for Bayes risk consistency of recursive kernel classification rule," IEEE Trans. Inform. Theory, vol. 33, no. 3, pp. 408-412, 1987.
33. C. Guestrin, P. Bodi, R. Thibau, M. Paskin, and S. Madde, "Distributed regression: An efficient framework for modeling sensor network data," in Proc. 3rd Int. Symp. Information Processing Sensor Networks, Berkeley, CA, Apr. 2004, pp. 1-10.
34. L. Györfi, M. Kohler, A. Krzyzak, and H. Walk, A Distribution-Free Theory of Nonparametric Regression. New York: Springer-Verlag, 2002.
35. J. Han, P.K. Varshney, and V.C. Vannicola, "Some results on distributed nonparametric detection," in Proc. 29th Conf. Decision Control, Honolulu, Hawaii, Dec. 1990, pp. 2698-2703.
36. T.S. Han and S. Amari, "Statistical inference under multiterminal data compression," IEEE Trans. Inform. Theory, vol. 44, no. 6, pp. 2300-2324, 1998.
37. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer-Verlag, 2001.
38. E.K. Hussaini, A.A.M. Al-Bassiouni, and Y.A. El-Far, "Decentralized CFAR signal detection," Signal Process., vol. 44, no. 3, pp. 299-307, 1995.
39. R. Jacobs, M.I. Jordan, S. Nowlan, and G.E. Hinton, "Adaptive mixtures of local experts," Neural Computation, vol. 3, no. 1, pp. 125-130, 1991.
40. M. Jordan, Ed., Learning in Graphical Models. Cambridge, MA: MIT Press, 1999.
41. M. Kearns and H.S. Seung, "Learning from a population of hypotheses," Mach. Learning, vol. 18, no. 2-3, pp. 255-276, 1995.
42. G. Kimeldorf and G. Wahba, "Some results on Tchebycheffian spline functions," J. Math. Anal. Appl., vol. 33, no. 1, pp. 82-95, 1971.
43. J. Kittler, M. Hatef, P.W. Duin, and J. Matas, "On combining classifiers," IEEE Trans. Pattern Anal. Machine Intell., vol. 20, no. 3, pp. 226-239, 1998.
44. J.H. Kotecha, V. Ramachandran, and A. Sayeed, "Distributed multi-target classification in wireless sensor networks," IEEE J. Select. Areas Commun., vol. 23, no. 4, pp. 703-713, 2005.
45. A. Krzyzak, "The rates of convergence of kernel regression estimates and classification rules," IEEE Trans. Inform. Theory, vol. 32, no. 5, pp. 668-679, 1986.
46. F.R. Kschischang, B.J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 498-519, 2001.
47. S.R. Kulkarni, G. Lugosi, and S.S. Venkatesh, "Learning pattern classification: A survey," IEEE Trans. Inform. Theory, vol. 44, no. 6, pp. 2178-2206, 1998.
48. S.R. Kulkarni and S.E. Posner, "Rates of convergence of nearest neighbor estimation under arbitrary sampling," IEEE Trans. Inform. Theory, vol. 41, no. 4, pp. 1028-1039, 1995.
49. n-nn and kernel estimators consistent for arbitrary processes," IEEE Trans. Inform. Theory, vol. 48, no. 10, pp. 2785-2788, 2002.
50. A. Lazarevic and Z. Obradovic, "The distributed boosting algorithm," in Proc. 7th ACM SIGKDD Int. Conf. Knowledge Discovery Data Mining, San Francisco, California, 2001, pp. 311-316.
51. D. Li, K. Wong, Y.H. Hu, and A. Sayeed, "Detection, classification, and tracking of targets," IEEE Signal Processing Mag., vol. 19, no. 2, pp. 17-29, 2002.
52. N. Littlestone and M. Warmuth, "The weighted majority algorithm," Inform. Computation, vol. 108, no. 2, pp. 212-261, 1994.
53. H.A. Loeliger, "An introduction to factor graphs," IEEE Signal Processing Mag., vol. 21, no. 1, pp. 28-41, 2004.
54. Z.-Q. Luo, "An isotropic universal decentralied estimation scheme for a bandwidth constrained ad hoc sensor network," IEEE J. Select. Areas Commun., vol. 23, no. 4, pp. 735-744, 2005.
55. Z.Q. Luo, "Universal decentralized estimation in a bandwidth constraint sensor network," IEEE Trans. Inform. Theory, vol. 51, no. 6, pp. 2210-2219, 2005.
56. T. Mitchell, Machine Learning. New York: McGraw Hill, 1997.
57. C.C. Moallemi and B.V. Roy, "Distributed optimization in adaptive networks," in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schlkopf, Eds. Cambridge, MA: MIT Press, 2004.
58. G. Morvai, S.R. Kulkarni, and A.B. Nobel, "Regression estimation from an individual stable sequence," Statistics, vol. 33, pp. 99-118, 1999.
59. A. Nakamura, J. Takeuchi, and M. Abe, "Efficient distribution-free learning of simple concepts," Ann. Math. Artificial Intell, vol. 23, no. 1-2, pp. 53-82, 1998.
60. A. Nasipuri and S. Tantaratana, "Nonparametric distributed detection using Wilcoxin statistics," Signal Process., vol. 57, no. 2, pp. 139-146, 1997.
61. A. Nedic and D. Bertsekas, "Incremental subgradient methods for nondifferentiable optimization," MIT, Cambridge, MA, Tech. Rep. LIDS-P-2460, 1999.
62. "Convergence rate of incremental subgradient algorithms," in Stochastic Optimization: Algorithms and Applications, S. Uryasev and P. M. Pardalos, Eds. Dordrecht, The Netherlands: Kluwer, 2000, pp. 263-304.
63. X. Nguyen, M.I. Jordan, and B. Sinopoli, "A kernel-based learning approach to ad hoc sensor network localization," ACM Trans. Sensor Networks, vol. 1, pp. 134-152, 2005.
64. X. Nguyen, M.J. Wainwright, and M.I. Jordan, "Nonparametric decentralied detection using kernel methods," IEEE Trans. Signal Processing, vol. 53, no. 11, pp. 4053-4066, 2005.
65. A.B. Nobel, "Limits to classification and regression estimation from ergodic processes," Ann. Statist, vol. 27, no. 1, pp. 262-273, 1999.
66. A.B. Nobel and T.M. Adams, "Estimating a function from ergodic samples with additive noise," IEEE Trans. Inform. Theory, vol. 47, no. 7, pp. 2895-2902, 2001.
67. A.B. Nobel, G. Morvai, and S. Kulkarni, "Density estimation from an individual sequence," IEEE Trans. Inform. Theory, vol. 44, no. 2, pp. 537-541, 1998.
68. R. Nowak and U. Mitra, "Boundary estimation in sensor networks: Theory and methods," in Proc. 2nd Int. Workshop Information Processing Sensor Networks, Palo Alto, CA, 22-23 Apr. 2003, pp. 80-95.
69. R.D. Nowak, "Distributed EM algorithms for density estimation and clustering in sensor networks," IEEE Trans. Signal Processing, vol. 51, no. 8, pp. 2245-2253, 2003.
70. M.A. Paskin and G.D. Lawrence, "Junction tree algorithms for solving sparse linear systems," EECS Dept., UC, Berkeley, Tech. Rep. UCB/ CSD-03-1271, 2003.
71. M.A. Paskin, C.E. Guestrin, and J. McFadden, "A robust architecture for inference in sensor networks," in Proc. 4th Int. Symp. Information Processing Sensor Networks, UCLA, Apr. 2005, pp. 55-62.
72. N. Patwari, J.N. Ash, S. Kyperountas, A.O. Hero, R.L. Moses, and N.S. Correal, "Locating the nodes: Cooperative localization in wireless sensor networks," IEEE Signal Processing Mag., vol. 22, no. 4, pp. 54-69, 2005.
73. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Francisco, CA: Morgan Kaufmann, 1988.
74. K. Plarre and P.R. Kumar, "Extended message passing algorithm for inference in loopy Gaussian graphical models," Ad Hoc Networks, vol. 2, pp. 153-169, 2004.
75. J.B. Predd, S.R. Kulkarni, and H.V. Poor, "Consistency in a model for distributed learning with specialists," in Proc. 2004 IEEE Int. Symp. Information Theory, Chicago, IL, June 2004, p. 465.
76. J.B. Predd, S.R. Kulkarni, and H.V. Poor, "Regression in sensor networks: Training distributively with alternating projections," in Proc. SPIE Conf. Advanced Signal Processing Algorithms, Architectures, and Implementations XV (invited), San Diego, CA, July-Aug. 2005.
77. J.B. Predd, S.R. Kulkarni, and H.V. Poor, "Consistency in models for distributed learning under communication constraints," IEEE Trans. Inform. Theory, vol. 52, no. 1, pp. 52-63, 2006.
78. J.B. Predd, S.R. Kulkarni, and H.V. Poor, "Distributed kernel regression: An algorithm for training collaboratively," in Proc. 2006 IEEE Information Theory Workshop, Punta del Este, Uruguay, Mar. 2006.
79. F. Provost and D.N. Hennessy, "Scaling up: Distributed machine learning with cooperation," in Proc. 13th Nat. Conf. Artificial Intelligence, Portland, OR, 1996, pp. 74-79.
80. J.R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kauffman, 1992.
81. M.G. Rabbat and R.D. Nowak, "Quantized incremental algorithms for distributed optimization," IEEE J. Special Areas Commun., vol. 23, no. 4, pp. 798-808, 2006.
82. M. Rabbat and R. Nowak, "Distributed optimization in sensor networks," in Proc. 3rd Int. Symp. Information Processing Sensor Networks, Berkeley, CA, Apr. 2004.
83. A. Ribeiro and G.B. Giannakis, "Bandwidth-constrained distributed estimation for wireless sensor networks, part I: Gaussian PDF," IEEE Trans. Signal Processing, vol. 54, no. 3, pp. 1131-1143, 2006.
84. A. Ribeiro and G.B. Giannakis, "Bandwidth-constrained distributed estimation for wireless sensor networks, part II: Unknown PDF," IEEE Trans. Signal Processing, to be published.
85. G. Roussas, "Nonparametric estimation in Markov processes," Ann. Inst. Statist. Math., vol. 21, pp. 73-87, 1967.
86. B. Schölkopf and A. Smola, Learning with Kernels, 1st ed. Cambridge, MA: MIT Press, 2002.
87. S.D. Servetto, "On the feasibility of large scale wireless sensor networks," in Proc. 40th Annu. Allerton Conf. Communication, Control, Computing, Monticello, Illinois, Oct. 2002.
88. S.-H. Son, M. Chiang, S.R. Kulkarni, and S.C. Schwartz, "The value of clustering in distributed estimation for sensor networks," in Proc. IEEE Int. Conf. Wireless Networks, Communications, Mobile Computing, Maui, Hawaii, June 2005, vol. 2, pp. 969-974.
89. S.-H. Son, S.R. Kulkarni, S.C. Schwartz, and M. Roan, "Communication-estimation tradeoffs in wireless sensor networks," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, Philadelphia, PA, Mar. 2005, vol. 5, pp. 1065-1068.
90. C.J. Stone, "Consistent nonparametric regression," Ann. Statist., vol. 5, no. 4, pp. 595-645, 1977.
91. Y. Sung, L. Tong, and H.V. Poor, "Neyman-Pearson detection of Gauss-Markov signals in noise: Closed-form error exponent and properties," IEEE Trans. Inform. Theory, vol. 52, no. 4, pp. 1354-1365, 2006.
92. Y. Sung, L. Tong, and H.V. Poor, "A large deviations approach to sensor scheduling for detection of correlated random fields," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, Philadelphia, PA, 19-23 Mar. 2005.
93. J.N. Tsitsiklis, "Decentralized detection," in Advances in Statistical Signal Processing, H.V. Poor and J.B. Thomas, Eds. Greenwich, CT: JAI Press, 1993, pp. 297-344.
94. L.G. Valiant, "A theory of the learnable," Commun. ACM, vol. 27, no. 11, pp. 1134-1142, 1984.
95. V.N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1991.
96. P.K. Varshney, Distributed Detection and Data Fusion. New York: Springer-Verlag, 1996.
97. R. Vishwanathan and A. Ansari, "Distributed detection of a signal in generalized Gaussian noise," IEEE Trans. Acoustic, Speech, Signal Processing, vol. 37, no. 5, pp. 775-778, 1989.
98. R. Vishwanathan and P.K. Varshney, "Distributed detection with multiple sensors: Part I - Fundamentals," Proc. IEEE, vol. 85, no. 1, pp. 54-63, 1997.
99. H. Viswanathan and T. Berger, "The quadratic Gaussian CEO problem," IEEE Trans. Inform. Theory, vol. 43, no. 5, pp. 1549-1559, 1997.
100. J.-J. Xiao and Z.-Q. Luo, "Universal decentralized detection in a bandwidth constraint sensor network," IEEE Trans. Signal Processing, vol. 53, no. 8, pp. 2617-2624, 2005.
101. J.-J. Xiao and Z.-Q. Luo, "Universal decentralized estimation in an inhomogeneous sensing environment," IEEE Trans. Inform. Theory, vol. 51, no. 10, pp. 3564-3575, 2005.
102. J.-J. Xiao, A. Ribeiro, Z.-Q. Luo, and G.B. Giannakis, "Distributed compression-estimation using wireless sensor networks," IEEE Signal Processing Mag., vol. 23, no. 4, pp. 26-41, June 2006.
103. S. Yakowitz, "Nonparametric density and regression estimation from Markov sequences without mixing assumptions," J. Multivariate Anal., vol. 30, no. 1, pp. 124-136, 1989.
104. S. Yakowitz, "Nearest neighbor regression estimation for null-recurrent Markov time series," Stochastic Processes Their Applicat., vol. 48, no. 2, pp. 311-318, 1993.
105. K. Yamanishi, "Distributed cooperative Bayesian learning," in Proc. Conf. Learning Theory, Nashville, Tennessee, July 1997, pp. 250-262.