A generalized maximum entropy approach to Bregman Co-clustering and matrix approximation

Journal of Machine Learning Research

Volumn 8, Issue , 2007, Pages 1919-1986

  Banerjee, Arindam ^a   Dhillon, Inderjit ^b   Ghosh, Joydeep ^c   Merugu, Srujana ^d   Modha, Dharmendra S ^e  

^a UNIVERSITY OF MINNESOTA   (United States)

^b University of Texas at Austin   (United States)

^c The University of Texas at Austin   (United States)

^d YAHOO RESEARCH   (United States)

^e IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

Bregman divergences; Bregman information; Co clustering; Matrix approximation; Maximum entropy

Indexed keywords

APPROXIMATION THEORY; DATA REDUCTION; ERROR ANALYSIS; INFORMATION ANALYSIS; MICROARRAYS;

BREGMAN DIVERGENCES; BREGMAN INFORMATION; MATRIX APPROXIMATION; MAXIMUM ENTROPY;

MATRIX ALGEBRA;

EID: 34548691246     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (50)

1. K. S. Azoury and M. K. Warmuth. Relative loss bounds for on-line density estimation with the exponential family of distributions. Machine Learning, 43(3):211-246, 2001.
2. A. Banerjee, X. Guo, and H. Wang. On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory, 51(7):2664-2669, July 2005a.
3. A. Banerjee, S. Merugu, I. Dhillon, and J. Ghosh. Clustering with Bregman divergences. Journal of Machine Learning Research, 6:1705-1749, 2005b.
4. H. H. Bauschke and J. M. Borowein. Legendre functions and the method of random Bregman projections. Journal of Convex Analysis, 4(1):27-67, 1997.
5. R. Bekkerman, R. El-Yaniv, and A. McCallum. Multi-way distributional clustering via pairwise interactions. In Proceedings of the International Conference on Machine Learning (ICML), pages 41-48, 2005.
6. L. M. Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Physics, 7:200-217, 1967.
7. R. Cai, L. Lu, and L. Cai. Unsupervised auditory scene categorization via key audio effects and information-theoretic co-clustering. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP05), pages 1073-1076, 2005.
8. J. J. M. Carrasco, D. Fain, K. Lang, and L. Zhukov. Clustering of bipartite advertiser-keyword graph. In Proceedings of the Workshop on Large Scale Clustering, ICDM, 2003.
9. Y. Censor and S. Zenios. Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, 1998.
10. D. Chakrabarti, S. Papadimitriou, D. S. Modha, and C. Faloutsos. Fully automatic cross-associations. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD), pages 79-88, 2004.
11. Y. Cheng and G. M. Church. Biclustering of expression data. In Proceedings of the 8th International Conference on Intelligent Systems for Molecular Biology (ISMB), pages 93-103, 2000.
12. H. Cho, I. S. Dhillon, Y. Guan, and S. Sra. Minimum sum-squared residue co-clustering of gene expression data. In Proceedings of the 4th SIAM International Conference on Data Mining (SDM), pages 114-125, 2004.
13. M. Collins, R. E. Schapire, and Y Singer. Logistic regression, adaboost and bregman distances. In Proceedings of the 13th Annual Conference on Computational Learing Theory (COLT), pages 158-169, 2000.
14. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley-Interscience, 1991.
15. I. Csiszár. Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. The Annals of Statistics, 19:2032-2066, 1991.
16. S. Della Pietra, V. Della Pietra, and J. Lafferty. Duality and auxiliary functions for Bregman distances. Technical Report CMU-CS-01-109, School of Computer Science, Carnegie Mellon University, 2001.
17. I. Dhillon, S. Mallela, and R. Kumar. A divisive information-theoretic feature clustering algorithm for text classification. Journal of Machine Learning Research, 3(4): 1265-1287, 2003a.
18. I. Dhillon, S. Mallela, and D. Modha. Information-theoretic co-clustering. In Proceedings of the 9th International Conference on Knowledge Discovery and Data Mining (KDD), pages 89-98, 2003b.
19. I. S. Dhillon. Co-clustering documents and words using bipartite spectral graph partitioning. In Proceedings of the 7th International Conference on Knowledge Discovery and Data Mining, pages 269-274, 2001.
20. I. S. Dhillon and D. S. Modha. Concept decompositions for large sparse text data using clustering. Machine Learning, 42(1): 143-175, January 2001.
21. D. Freitag. Trained named entity recognition using distributional clusters. In EMNLP, pages 262-269, 2004.
22. B. Gao, T. Liu, X. Zheng, Q. Cheng, and W. Ma. Consistent bipartite graph co-partitioning for star-structured high-order heterogeneous data co-clustering. In Proceedings of the 11th International Conference on Knowledge Discovery and Data Mining (KDD), pages 41-50, 2005.
23. T. George and S. Merugu. A scalable collaborative filtering framework based on co-clustering. In Proceedings of the IEEE Conference on Data Mining, pages 625-628, 2005.
24. J. Ghosh. Scalable clustering. In Nong Ye, editor, The Handbook of Data Mining, pages 247-277. Lawrence Erlbaum Assoc., 2003.
25. GroupLens. Movielens data set. http://www.cs.umn.edu/Research/GroupLens/ data/ml-data.tar.gz.
26. P. D. Grünwald and A. Dawid. Game theory, maximum entropy, minimum discrepancy, and robust Bayesian decision theory. Annals of Statistics, 32(4), 2004.
27. J. Guan, G. Qiu, and X. Y. Xue. Spectral images and features co-clustering with application to content-based image retrieval. In IEEE Workshop on Multimedia Signal Processing, 2005.
28. J. A. Hartigan. Direct clustering of a data matrix. Journal of the American Statistical Association, 67(337): 123-129, 1972.
29. T. Hofmann. Latent semantic models for collaborative filtering. ACM Transactions on Information Systems, 22(1):89-115, 2004.
30. T. Hofmann and J. Puzicha. Unsupervised learning from dyadic data. Technical Report ICSI TR-98-042, International Computer Science Institute (ICSI), Berkeley, 1998.
31. A. K. Jain and R. C. Dubes. Algorithms for Clustering Data. Prentice Hall, New Jersey, 1988.
32. E. T. Jaynes. Information theory and statistical mechanics. Physical Reviews, 106:620-630, 1957.
33. Y. Kluger, R. Basri, J. T. Chang, and M. Gerstein. Spectral biclustering of microarray data: Coclustering genes and conditions. Genome Research, 13(4):703-716, 2003.
34. J. Lafferty. Additive models, boosting, and inference for generalized divergences. In Proceedings of the 13th Annual Conference on Computational Learing Theory (COLT), 1999.
35. D. L. Lee and S. Seung. Algorithms for non-negative matrix factorization. In Proceedings of the 14th Annual Conference on Neural Information Processing Systems (NIPS), pages 556-562, 2001.
36. H. Li and N. Abe. Word clustering and disambiguation based on co-occurence data. In COLING-ACL, pages 749-755, 1998.
37. T. Li. A general model for clustering binary data. In Proceedings of the 11th International Conference on Knowledge Discovery and Data Mining (KDD), pages 188-197, 2005.
38. Z. Lin and R.B. Altman. Finding haplotype tagging snps by use of principal components analysis. The American Journal of Human Genetics, 75:850-861, 2004.
39. S. C. Madeira and A. L. Oliveira. Biclustering algorithms for biological data analysis: A survey. IEEE Trans. Computational Biology and Bioinformatics, 1(1):24-45, 2004.
40. C. H. Papadimitriou, P. Raghavan, H. Tamaki, and S. Vempala. Latent semantic indexing: A probabilistic analysis. In Proceedings of the 16th Annual ACM Symposium on Principles of Distributed Computing (PODC), pages 159-168, 1998.
41. L. Parsons, E. Haque, and H. Liu. Subspace clustering for high dimensinal data: A review. ACM SIGKDD Explorations, 6(1):90-105, 2004.
42. G. Qiu. Image and feature co-clustering. In Proceedings of the International Conference on Pattern Recognition, pages 991-994, 2004.
43. P. Resnick, N. Iacovou, M. Suchak, P. Bergstorm, and J. Riedl. GroupLens: An Open Architecture for Collaborative Filtering of Netnews. In Proceedings of the ACM Conference on CSCW, pages 175-186, 1994.
44. R. T. Rockafellar. Convex Analysis. Princeton Landmarks in Mathematics. Princeton University Press, 1970.
45. R. Rohwer and D. Freitag. Towards full automation of lexicon construction. In Dan Moldovan and Roxana Girju, editors, HLT-NAACL 2004: Workshop on Computational Lexical Semantics, pages 9-16, 2004.
46. B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Application of dimensionality reduction in recommender systems - a case study. In WebKDD Workshop., 2000.
47. J. Shore and R. Johnson. Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross entropy. IEEE Transactions on Information Theory, 26(1):26-37, 1980.
48. A. Strehl and J. Ghosh. Cluster ensembles - a knowledge reuse framework for combining partitionings. Journal of Machine Learning Research, 3(3):583-617, 2002.
49. H. Takamura and Y Matsumoto. Co-clustering for text categorization. Information Processing Society of Japan Journal, 2003.
50. H. Zhong, J. Shi, and M. Visontai. Detecting unusual activity in video. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pages 819-826, 2004.