Distributed unsupervised Gaussian mixture learning for density estimation in sensor networks

IEEE Transactions on Instrumentation and Measurement

Volumn 59, Issue 9, 2010, Pages 2250-2260

  Safarinejadian, Behrooz ^a,b   Menhaj, Mohammad B ^a   Karrari, Mehdi ^a  

^a SHIRAZ UNIVERSITY OF TECHNOLOGY   (Iran)

^b AMIRKABIR UNIVERSITY OF TECHNOLOGY   (Iran)

Author keywords

Data clustering; density estimation; expectation maximization (EM) algorithm; sensor networks; unsupervised learning

Indexed keywords

DATA CLUSTERING; DENSITY ESTIMATION; DIFFUSION SPEED; DISTRIBUTED AVERAGING; DISTRIBUTED TARGET; ENVIRONMENTAL MONITORING; EXPECTATION MAXIMIZATION; EXPECTATION-MAXIMIZATION ALGORITHMS; GAUSSIAN MIXTURE MODEL; GAUSSIAN MIXTURES; MODEL ORDER; NEIGHBORING NODES; SIMULATION RESULT; SUFFICIENT STATISTICS;

CLUSTERING ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; FAULT TOLERANCE; IMAGE SEGMENTATION; LEARNING ALGORITHMS; MAXIMUM PRINCIPLE; OPTIMIZATION; QUALITY ASSURANCE; SENSOR NETWORKS; UNSUPERVISED LEARNING;

PARAMETER ESTIMATION;

EID: 77955712146     PISSN: 00189456     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIM.2009.2036348     Document Type: Article

References (38)

1. S. Datta, K. Bhaduri, C. Giannella, R. Wolff, and H. Kargupta, "Distributed data mining in peer-to-peer networks," IEEE Internet Comput., vol.10, no.4, pp. 18-26, Jul. 2006.
2. C. Vincent and B. Wüthrich, "Distributed mining of classification rules," Knowl. Inf. Syst., vol.4, no.1, pp. 1-30, Jan. 2002.
3. R. Wolff and A. Schuster, "Association rule mining in peer-to-peer systems," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.34, no.6, pp. 2426-2438, Dec. 2004.
4. C. Giannella, H. Dutta, S. Mukherjee, and H. Kargupta, "Efficient kernel density estimation over distributed data," in Proc. 9th Int. Workshop High Performance Distrib. Mining, SIAM Int. Conf. Data Mining, Bethesda, MD, Apr. 2006.
5. R. Chen, K. Sivakumar, and H. Kargupta, "Collective mining of Bayesian networks from distributed heterogeneous data," Knowl. Inf. Syst., vol.6, no.2, pp. 164-187, Mar. 2004.
6. X. Nguyen, M. I. Jordan, and B. Sinopoli, "A kernel-based learning approach to ad hoc sensor network localization," ACM Trans. Sensor Netw., vol.1, no.1, pp. 134-152, Aug. 2005.
7. S. Simic, "A learning-theory approach to sensor networks," Pervasive Comput., vol.2, no.4, pp. 44-49, Oct. 2003.
8. M. Paskin, C. Guestrin, and J. Mc Fadden, "A robust architecture for inference in distributed sensor networks," in Proc. Int. Conf. IPSN, 2005, pp. 55-62.
9. M. Paskin and C. Guestrin, "Robust probabilistic inference in distributed systems," in Proc. Uncertainty Artif. Intell., 2004, pp. 436-445.
10. B. Safarinejadian, M. B. Menhaj, and M. Karrari, "A distributed variational Bayesian algorithm for density estimation in sensor networks," IEICE Trans. Inf. Syst., vol.E92-D, no.5, pp. 1037-1048, 2009.
11. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood estimation from incomplete data via the EM algorithm," J. R. Stat. Soc., Ser. B, vol.39, no.1, pp. 1-38, 1977.
12. C. Ordonez and E. Omiecinski, "Accelerating EM clustering to find highquality solutions," Knowl. Inf. Syst., vol.7, no.2, pp. 135-157, Feb. 2005.
13. J. J. Verbeek, N. Vlassis, and J. R. J. Nunnink, "A variational EMapproach for large-scale mixture modeling," in Proc. 8th Annu. Conf. Adv. School Comput. Imaging, Heijen, The Netherlands, 2003.
14. S. Roweis and Z. Ghahramani, "A unifying review of linear Gaussian models," Neural Comput., vol.11, no.2, pp. 305-345, Feb. 1999.
15. R. Neal and G. Hinton, "A view of the EM algorithm that justifies incremental, sparse, and other variants," in Learning in Graphical Models, M. I. Jordan, Ed. Cambridge, MA: MIT Press, 1999, pp. 355-368.
16. C. Ordonez and E. Omiecinski, "FREM: Fast and robust EM clustering for large data sets," in Proc. ACM CIKM Conf., 2002, pp. 590-599.
17. G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions. New York: Wiley, 1997, pp. 120-211.
18. R. D. Nowak, "Distributed EM algorithms for density estimation and clustering in sensor networks," IEEE Trans. Signal Process., vol.51, no.8, pp. 2245-2253, Aug. 2003.
19. B. Safarinejadian, M. B. Menhaj, and M. Karrari, "Distributed data clustering using expectation maximization algorithm," J. Appl. Sci., vol.9, no.5, pp. 854-864, 2009.
20. B. Safarinejadian, M. B. Menhaj, and M. Karrari, "A distributed EM algorithm to estimate the parameters of a finite mixture of components," Knowl. Inf. Syst., 2009, DOI: 10.1007/s10115-009-0218-y, to be published.
21. X. Lin, C. Clifton, and M. Zhu, "Privacy-preserving clustering with distributed EM mixture modeling," Knowl. Inf. Syst., vol.8, no.1, pp. 68-81, Jul. 2005.
22. S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, "Gossip algorithms: Design, analysis and applications," in Proc. IEEE Infocom, 2005, pp. 1653-1664.
23. P. Eugster, R. Guerraoui, A.-M. Kermarrec, and L. Massoulie, "From epidemics to distributed computing," Computer, vol.37, no.5, pp. 60- 67, May 2004.
24. D. Kemp, A. Dobra, and J. Gehrke, "Computing aggregate information using gossip," in Proc. FoCS, 2003, pp. 482-491.
25. W. Zajdel, "Bayesian visual surveillance-From object detection to distributed cameras," Ph.D. dissertation, Universiteit van Amsterdam, Amsterdam, The Netherlands, 2006.
26. H. Bensmail, G. Celeux, A. Raftery, and C. Robert, "Inference in modelbased cluster analysis," Stat. Comput., vol.7, no.1, pp. 1-10, 1997.
27. K. Mengersen and C. Robert, "Testing for mixtures: A Bayesian entropic approach," in Proc. 5th Valencia Int. Meeting Bayesian Stat., J. Bernardo, J. Berger, A. Dawid, and F. Smith, Ed.s, 1996, pp. 255-276.
28. K. Roeder and L. Wasserman, "Practical Bayesian density estimation using mixtures of normals," J. Amer. Stat. Assoc., vol.92, pp. 894-902, 1997.
29. R. Neal, "Bayesian mixture modeling," in Proc. 11th Int. Workshop Maximum Entropy Bayesian Methods Stat. Anal., 1992, pp. 197-211.
30. C. Rasmussen, "The infinite Gaussian mixture model," in Advances in Neural Information Processing Systems 12, S. Solla, T. Leen, and K.-R. Muller, Eds. Cambridge, MA: MIT Press, 2000, pp. 554-560.
31. S. Richardson and P. Green, "On Bayesian analysis of mixtures with unknown number of components," J. R. Stat. Soc., Ser. B, vol.59, pp. 731- 792, 1997.
32. C. Wallace and D. Dowe, "Minimum message length and Kolmogorov complexity," Comput. J., vol.42, no.4, pp. 270-283, Apr. 1999.
33. C. Wallace and P. Freeman, "Estimation and inference via compact coding," J. R. Stat. Soc., Ser. B, vol.49, no.3, pp. 241-252, 1987.
34. J. Rissanen, Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific, 1989.
35. T. Cover and J. Thomas, Elements of Information Theory. NewYork: Wiley, 1991.
36. M. A. T. Figueiredo and A. K. Jain, "Unsupervised learning of finite mixture models," IEEE Trans. Pattern Anal. Mach. Intell., vol.24, no.3, pp. 381-396, Mar. 2002.
37. J. Besg, "On the statistical analysis of dirty pictures," J. R. Stat. Soc., Ser. B Methodol., vol.48, no.3, pp. 259-302, 1986.
38. S. Dasgupta, "Learning mixtures of Gaussians," in Proc. 40th Annu. Symp. FOCS, Washington, DC, 1999, p. 634.