Multivariate information bottleneck

Neural Computation

Volumn 18, Issue 8, 2006, Pages 1739-1789

  Slonim, Noam ^a   Friedman, Nir ^b   Tishby, Naftali ^b  

^a PRINCETON UNIVERSITY   (United States)

^b HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; BAYES THEOREM; BIOLOGY; ELECTROPHYSIOLOGY; GENOMICS; LINGUISTICS; METHODOLOGY; MULTIVARIATE ANALYSIS; PROTEOMICS; SIGNAL PROCESSING; STATISTICS;

ALGORITHMS; BAYES THEOREM; COMPUTATIONAL BIOLOGY; ELECTROPHYSIOLOGY; GENOMICS; LINGUISTICS; MULTIVARIATE ANALYSIS; PROTEOMICS; SIGNAL PROCESSING, COMPUTER-ASSISTED; STATISTICS;

EID: 33745827787     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2006.18.8.1739     Document Type: Article

References (33)

1. Apweiler, R., and Attwood, T. K., Bairoch, A., Bateman, A., Birney, E., Biswas, M., Bucher, P., Cerutti, L., Corpet, F., Croning, M. D., Durbin, R., Falquet, L., Fleischmann, W., Gouzy, J., Hermjakob, H., Hulo, N., Jonassen, I., Kahn, D., Kanapin, A., Karavidopoulou, Y., Lopez, R., Marx, B., Mulder, N. J., Oinn, T. M., Pagni, M., Servant, F., Sigrist, C. J., & Zdobnov, E. M. (2000). InterPro-an integrated documentation resource for protein families, domains and functional sites, Biainformatics, 16, 1145-1150.
2. Attwood, T. K., Croning, M. D. R., Flower, D. R., Lewis, A. P., Mabey, J. E., Scordis, P., Selley, J., & Wright, W. (2000). PRINTS-S: The database formerly known as PRINTS. Nucl. Acids Res., 28, 225-227.
3. Atwal, C. S., & Slonim, N. (2005). Information bottleneck with finite, samples. Unpublished manuscript.
4. Baker, L. D., & McCallum, A. K. (1998). Distributional clustering of words for text classification. In Proc. of the 21th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM.
5. Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Camp., 7, 1129-1159.
6. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley.
7. Csiszár, I., & Tusnády, G. (1984). Information geometry and alternating minimization procedures. Statistics and Decisions, suppl. 1, 205-237.
8. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39, 1-38.
9. Elidan, G., & Friedman, N. (2003). The information bottleneck expectation maximization algorith. In Proc. of the 19th Conf. on Uncertainty in Artificial Intelligence (UAI-19). San Mateo, CA: Morgan Kaufmann.
10. Friedman, N., Mosenzon, O., Slonim, N., & Tishby, N. (2001). Multivariate information bottleneck. In Proc. of Uncertainty in Artificial Intelligence (UAI-17). San Mateo, CA: Morgan Kaufmann.
11. Gilad-Bachrach, R., Navot, A., & Tishby, N. (2003). An information theoretic tradeoff between complexity and accuracy. In The Sixteenth Annual Conference on learning Theory (COLT). New York: Springer.
12. Globerson, A., & Tishby, N. (2004). The minimum information principle in discriminative learning. In C. Meek, M. Chickering, & J. Halpem (Eds.), Uncertainty in artificial intelligence. Banff, Canada: AUAI Press.
13. Golub, T., Slonim, D., Tamayo, P., Huard, C. M., Caasenbeek, J. M., Coller, H., Loh, M., Downing, J., Caligiuri, M., Bloomfield, C., & Lander, E. (1999). Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science, 286, 531-537.
14. Gutman, M. (1989). Asymptotically optimal classification for multiple tests with empirically observed statistics, IEEE Trans. Inf. Theory, 35(2), 401-408.
15. Hofmann, T. (1999). Probabilistic latent semantic indexing. In Proc. of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 50-57). New York: ACM.
16. Lang, K. (1995). Learning to filter netnews. In Proc. of the 12th International Conf. on Machine Learning (ICML). San Mateo, CA: Morgan Kaufmann.
17. Parker, E. A., Gedeon, T., & Dimitrov, A. G. (2002). Annealing and the rate distortion problem. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in neural information processing systems, 15 (pp. 969-976). Cambridge, MA: MIT Press.
18. Pearl, J. (1988). Probabilistic reasoning in intelligent systems. San Francisco: Morgan Kaufmann.
19. Pereira, F. C., Tishby, N., & Lee, L. (1993). Distributional clustering of English words. In 30th Annual Meeting of the Association for Computational Linguistics. New York: ACM.
20. Rose, K. (1998). Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proceedings of the IEEE, 86, 2210-2239.
21. Schriebman, A. (2000). Stochastic modeling for efficient computation of information theoretic quantities. Unpublished doctoral dissertation, Hebrew University, Jerusalem, Israel. Available online at http://www.cs.huji.ac.il/labs/learning/Theses/theses_list.html.
22. Sebastiani, F. (2002). Machine learning in automated text categorization. ACM. Computing Surveys, 34(1), 1-47.
23. Shannon, C. E. (1948). A mathematical theory of communication. Bell Systems Technical Journal, 27, 379-123, 623-656.
24. Slonim, N. (2002). The information bottleneck: Theory and applications. Unpublished doctoral dissertation, Hebrew University, Jerusalem, Israel.
25. Slonim, N., Friedman, N., & Tishby, N. (2002). Unsupervised document classification using sequential information maximization. In Proc. of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM.
26. Slonim, N., Somerville, R., Tishby, N., & Lahav, O. (2001). Objective classification of galaxy spectra using the information bottleneck method. Monthly Notes of the Royal Astronomical Society, 323, 270-284.
27. Slonim, N., & Tishby, N. (1999). Agglomerative information bottleneck. In S. A. Solla, T. K. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems, 12 (pp. 617-623). Cambridge, MA: MIT Press.
28. Slonim, N., & Tishby, N. (2000). Document clustering using word clusters via the information bottleneck method. In Proc. of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 208-215). New York: ACM.
29. Slonim, N., & Weiss, Y. (2002). Maximum likelihood and the information bottleneck. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in neural information processing systems, 15. Cambridge, MA: MIT Press.
30. Still, S., & Bialek, W. (2003). How many clusters? Physics/030101.
31. Studeny, M., & Vejnarova, J. (1998). The Multi-information function as a tool for measuring stochastic dependence. In M. I. Jordan (Ed.), Learning in graphical models (pp. 261-298). Dordrecht: Kluwer.
32. Tishby, N., Pereira, F., & Bialek, W. (1999). The information bottleneck method. In Proc. of 37th Allerton Conference on Communication and Computation.
33. Tishby, N., & Slonim, N. (2000). Data clustering by Markovian relaxation and the information bottleneck method. In T. K. Leen, T. G. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems, 13. Cambridge, MA: MIT Press.