Distributed information theoretic clustering

IEEE Transactions on Signal Processing

Volumn 62, Issue 13, 2014, Pages 3442-3453

  Shen, Pengcheng ^a   Li, Chunguang ^a  

^a ZHEJIANG UNIVERSITY   (China)

Author keywords

Diffusion cooperation; discriminative clustering; distributed clustering; information theory; mutual information

Indexed keywords

CLUSTER ANALYSIS; IMAGE SEGMENTATION; INFORMATION THEORY; SENSOR NODES;

CENTRALIZED INFORMATIONS; DISCRIMINATIVE CLUSTERING; DISTRIBUTED CLUSTERING; DISTRIBUTED CLUSTERING ALGORITHM; DISTRIBUTED DATA ANALYSIS; INFORMATION THEORETIC MEASURE; MUTUAL INFORMATIONS; WIRELESS SENSOR NETWORK (WSNS);

CLUSTERING ALGORITHMS;

EID: 84903288977     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2014.2327010     Document Type: Article

References (43)

1. A. K. Jain, "Data clustering: 50 years beyond K-means Pattern Recogn. Lett., vol. 31, no. 8, pp. 651-666, 2010
2. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York, NY, USA:Wiley, 2002
3. A. K. Jain, A. Topchy, M. H. C. Law, and J. M. Buhmann, "Landscape of clustering algorithms in Proc. Int. Conf. Pattern Recogniy, 2004, vol. 1, pp. 260-263
4. R. Xu and D. Wunsch-II, "Survey of clustering algorithms IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 645-678, May 2005
5. A. K. Jain, M. N. Murty, and P. J. Flynn, "Data clustering: A review ACM Comput. Surveys, vol. 31, no. 3, pp. 265-323, 1999
6. J.Grabmeier andA. Rudolph, "Techniques of cluster algorithms in Data mining Data Mining and Knowl. Discov., vol. 6, no. 4, pp. 303-360, 2002
7. P. A. Forero, A. Cano, and G. B. Giannakis, "Distributed clustering using wireless sensor networks IEEE J. Sel. Topics Signal Process., vol. 5, no. 4, pp. 707-724, Aug. 2011
8. I. Dhillon and D. Modha, "A data-clustering algorithm on Distributed memory multiprocessors Large-Scale Parallel Data Mining, Lecture Notes in Artif. Intell., pp. 245-260, 2000
9. P. A. Forero, A. Cano, and G. B. Giannakis, "Consensus-based k-means algorithm for distributed learning using wireless sensor networks in Proc. Workshop Sens., Signal, Inf. Process., Sedona, AZ, USA, May 11-14, 2008
10. P. A. Forero, A. Cano, and G. B. Giannakis, "Consensus-based distributed expectation-maximization algorithm for density estimation and classification using wireless sensor networks in Proc. Int. Conf. Acoust., Speech, Signal Process., Las Vegas, NV, USA, 2008, pp. 1989-1992
11. J. Wolfe, A. Haghighi, and D. Klein, "Fully distributed EM for very large datasets in Proc. 25th Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 1184-1191
12. R. D. Nowak, "Distributed EM algorithms for density estimation and clustering in sensor networks IEEE Trans. Signal Process., vol. 51, no. 8, pp. 2245-2253, Aug. 2003
13. D. Gu, "Distributed EMalgorithm for Gaussian mixtures in sensor networks IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1154-1166, Jul. 2008
14. E. Januzaj, H. P. Kriegel, and P. Martin, "Scalable density-based distributed clustering in Proc. Knowl. Discov. Databases: PKDD 2004, 2004, vol. 19, no. 7, pp. 231-244
15. M. Klusch, S. Lodi, and G. Moro, "Distributed clustering based on sampling local density estimates in Proc. Int. Joint Conf. Artif. Intell., 2003, pp. 485-490
16. H. Kargupta, W. Huang, S. Krishnamoorthy, and E. Johnson, "Distributed clustering using collective principal component analysis Knowl. Inf. Syst. J., vol. 3, no. 4, pp. 422-448, 2001
17. S. Bandyopadhyay, C. Giannella, U. Maulik, H. Kargupta, K. Liu, and S. Datta, "Clustering distributed data streams in peer-To-peer environments Inf. Sci., vol. 176, no. 14, pp. 1952-1985, 2006
18. E. Gokcay and J. C. Principe, "Information theoretic clustering IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 2, pp. 158-170, Apr. 2002
19. R. Jenssen, T. Eltoft, and J. C. Principe, "Information theoretic clustering: A unifying review of three recent algorithms in Proc. 6th Nordic Signal Process. Symp., Espoo, Finland, 2004, pp. 292-295. (Pubitemid 40087965
20. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, "Clustering with Bregman Divergences J. Mach. Learn. Res., vol. 6, pp. 1705-1749, 2005
21. N. Slonim, G. S. Atwal, G. Tkacik, and W. Bialek, "Information based clustering Proc. Nat. Acad. Sci. USA, vol. 105, no. 51, pp. 18297-18302, 2005
22. X. Bai, S. W. Luo, and Y. B. Zhao, "Entropy based soft K-means clustering in Proc. EEE Int. Conf. Granular Comput., 2008, pp. 107-110
23. Q. Song, "Arobust information clustering algorithm Neural Comput., pp. 2672-2698, 2005
24. C. Bohm, C. Faloutsos, J. Y. Pan, and C. Plant, "Robust informationtheoretic clustering in Proc. 12th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2005, vol. 17, no. 12, pp. 65-75
25. S. Gordon, H. Greenspan, and J. Goldberger, "Applying the information bottleneck principle to unsupervised clustering of discrete and continuous image representations in Proc. 9th IEEE Int. Conf. Comput. Vision, 2003, pp. 370-377
26. A. Bardera, J. Rigau, I. Boada, M. Feixas, and M. Sbert, "Image segmentation using information bottleneck method IEEE Trans. Image Process., vol. 18, no. 7, pp. 1601-1612, Jul. 2009
27. N. Tishby, F. Pereira, and W. Bialek, "The information bottleneck method in Proc. 37th Ann. Allerton Conf. Commun., Contr. Comput., 1999, pp. 368-377
28. A. Krause, P. Perona, and R. G. Gomes, "Discriminative clustering by regularized information maximization in Proc. Adv. Neural Inf. Process. Syst., 2010, pp. 775-783
29. S. Merugu and J. Ghosh, "A privacy-sensitive approach to distributed clustering Pattern Recognit. Lett., vol. 26, no. 4, pp. 399-410, 2005
30. A. M. Elmisery and H. Fu, "Privacy preserving distributed learning clustering of healthcare data using cryptography protocols in Proc. IEEE 34th Ann. Comput. Software and Appl. Conf. Workshops, 2010, pp. 140-145
31. T. M. Cover and J.A. Thomas, Elements of Information Theory. New York, NY, USA: Wiley, 1991
32. J. C. Principe, Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives. New York, NY, USA: Springer-Verlag, 2010
33. E. Parzen, "On estimation of a probability density function and mode in Time Series Analysis Papers. San Francisco, CA: Holden-Day, 1967
34. F. S. Cattivelli, C. G. Lopes, and A. H. Sayed, "Diffusion LMS strategies for distributed estimation IEEE Trans. Signal Process., vol. 58, no. 3, pp. 1035-1048, Mar. 2010
35. C. G. Lopes and A. H. Sayed, "Distributed processing over adaptive networks in Proc. Adapt. Sens. Array Process. Workshop, Lexington, MA, USA, Jun. 2006
36. C. Li, P. Shen, Y. Liu, and Z. Zhang, "Diffusion information theoretic learning for distributed estimation over network IEEE Trans. Signal Process., vol. 61, no. 16, pp. 4011-4024, Aug 15, 2013
37. F. S. Cattivelli, C. G. Lopes, and A. H. Sayed, "Diffusion recursive least-squares for distributed estimation over adaptive networks IEEE Trans. Signal Process., vol. 56, no. 5, pp. 1865-1877, May 2008
38. Y. Liu, C. Li, and Z. Zhang, "Diffusion sparse least-mean squares over networks IEEE Trans. Signal Process., vol. 60, no. 8, pp. 4480-4485, Aug. 2012
39. Z. Liu, Y. Liu, and C. Li, "Distributed sparse recursive least-squares over networks IEEE Trans. Signal Process., vol. 62, no. 6, pp. 1386-1395, Jun. 2014
40. P. A. Forero, A. Cano, and G. B. Giannakis, "Consensus-based distributed support vector machines J. Mach. Learn. Res., vol. 11, pp. 1663-1707, 2010
41. X. He, D. Cai, Y. Shao, H. Bao, and J. Han, "Laplacian regularized Gaussian mixture model for data clustering IEEE Trans. Knowl. Data Eng., vol. 23, no. 9, pp. 1406-1418, 2011
42. R. Jenssen, D. Erdogmus, K. E. Hild, J. C. Principe, and T. Eltoft, "Optimizing the Cauchy-Schwarz PDF divergence for information theoretic, non-parametric clustering in Proc. Int. Workshop on Energy Minimiz. Methods in Compute. Vision Pattern Recogn., St. Augustine, FL, USA, Nov. 2005, pp. 34-45. (Pubitemid 43722924
43. S. Datta, C. R. Giannella, and H. Kargupta, "Approximate distributed k-means clustering over a peer-To-peer network IEEE Trans. Knowl. Data Eng., vol. 21, no. 10, pp. 1372-1388, Oct. 2009