Approximate maximum entropy joint feature inference consistent with arbitrary lower-order probability constraints: Application to statistical classification

Neural Computation

Volumn 12, Issue 9, 2000, Pages 2175-2207

  Miller, David J ^a   Yan, Lian ^a  

^a The Pennsylvania State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0001435073     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976600300015105     Document Type: Article

References (38)

1. Ahmad, S., & Tresp, V. (1993). Classification with missing and uncertain inputs. In Proc. of IEEE Intl. Conf. on Neural Networks, (pp. 1949-1954).
2. Berger, A. L., Della Pietra, S. A., & Della Pietra, V. J. (1996). A maximum entropy approach to natural language processing. Computational Linguistics, 22(1), 39-68.
3. Berman, A., & Plemmons, R. J. (1979). Nonnegative matrices in the mathematical sciences. New York: Academic Press.
4. Bilbro, G. L., & Van den Bout, D. E. (1992). Maximum entropy and learning theory. Neural Computation, 4, 839-853.
5. Buhmann, J., & Kuhnel, H. (1994). Vector quantization with complexity costs. IEEE Trans. on Info. Theory, 39, 1133-1145.
6. Buntine, W. (1996). A guide to the literature on learning probabilistic networks from data. IEEE Trans. on Knowl. and Data Eng., 8, 195-210.
7. Burg, J. P. (1975). Maximum entropy spectral analysis. Unpublished doctoral dissertation, Stanford University, Stanford, CA.
8. Cheeseman, P. (1983). A method of computing generalized Bayesian probability values for expert systems. In Proc. of the Eighth Intl. Joint Conf. on AI (Vol. 1, pp. 198-202).
9. Cheeseman, P., Kelly, J., Self, M., Stutz, J., Taylor, W., & Freeman, D. (1988). AutoClass: A Bayesian classification system. In Proc. of the Fifth Intl. Conf. on Machine Learning.
10. Chow, C., & Liu, C. (1968). Approximating discrete probability distributions with dependence trees. IEEE Trans. on Info. Theory, 14, 462-467.
11. Cooper, G. F., & Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9, 309-347.
12. Cover, T. M., & Thomas, J. A. (1991). Elements of Information Theory. New York: Wiley.
13. Edwards, D., & Havranek, T. (1985). A fast procedure for model search in multidimensional contingency tables. Biometrica, 72, 339-351.
14. Frey, B. J. (1998). Graphical models for machine learning and digital communication. Cambridge, MA: MIT Press.
15. Friedman, N., & Goldszmidt, M. (1996). Building classifiers using Bayesian networks. In Proc. of the National Conf. on AI (Vol. 2, pp. 1277-1284).
16. Friedman, N., Goldszmidt, M., & Lee, T. J. (1998). Bayesian network classification with continuous attributes: getting the best of both discretization and parametric fitting. In Proc. of the Intl. Conf. on Machine Learning (pp. 179-187).
17. Gevarter, W. B. (1986). Automatic probabilistic knowledge acquisition from data. (Tech. Memo. No. 88224). Mountain View, CA: NASA Ames Research Center.
18. Ghahramani, Z., & Jordan, M. I. (1994). Supervised learning from incomplete data via an EM approach. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 120-127). San Mateo, CA: Morgan Kauffman.
19. Haykin, S. (1999). Neural networks: A comprehensive foundation. Englewood Cliffs, NJ: Prentice Hall.
20. Heckerman, D., Geiger, A., & Chickering, D. M. (1995). Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20, 197-243.
21. Herskovits, E., & Cooper, G. (1991). Kutato: An entropy-driven system for construction of probabilistic expert systems from databases. In P. P. Bonissone, M. Henrione, L. N. Kanal, & J. F. Lemmer (Eds.), Uncertainty in AI 6(pp. 117-125). Amsterdam: Elsevier.
22. Jaynes, E. T. (1982). Papers on probability, statistics and statistical physics. Dordrecht: Reidel.
23. Jirousek, R., & Preucil, S. (1995). On the effective implementation of the iterative proportional fitting procedure. Comp. Stat. and Data Anal., 19, 177-189.
24. Kang, H., Kim, K., & Kim, J. H. (1997). Optimal approximation of discrete probability distribution with kth-order dependency and its application to com-bining multiple classifiers. Patt. Rec. Letters, 18, 515-523.
25. Ku, H. H., & Kullback, S. (1969). Approximating discrete probability distributions. IEEE Trans. on Info. Theory, 15(4), 444-447.
26. Little, R. J. A. (1993). Pattern-mixture models for multivariate incomplete data. Journal of the Amer. Stat. Assoc., 88, 125-134.
27. Meila, M., & Jordan, M. I. (1998). Estimating dependency structure as a hidden variable. In M. Kearns, M. Jordan, & S. Solla (Eds.), Advances in neural information processing systems, 10 (pp. 584-590). Cambridge, MA: MIT Press.
28. Miller, D. J., Rao, A. V., Rose, K., & Gersho, A. (1996). A global optimization technique for statistical classifier design. IEEE Trans. on Sig. Proc., 44, 3108-3122.
29. Miller, D. J. & Yan, L. (1999). Critic-driven ensemble classification. IEEE Trans. on Sig. Proc., 47, 2833-2844.
30. Neal, R. M. (1992). Connectionist learning of belief networks. Art. Intell., 56, 71-113.
31. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks for plausible inference. San Mateo, CA: Morgan Kauffman.
32. Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Mateo, CA: Morgan Kauffman.
33. Ripley, B. D. (1996). Pattern recognition and neural networks. Cambridge: Cambridge University Press.
34. Rose, K., Gurewitz, E., & Fox, G. C. (1992). Vector quantization by deterministic annealing. IEEE Trans. on Info. Theory, 38, 1249-1257.
35. Shore, J. E., & Johnson, R. W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans. on Info. Theory, 26, 26-37.
36. Yan, L., & Miller, D. J. (1999a). General statistical inference by an approximate application of the maximum entropy principle. In Proc. of IEEE Workshop on Neural Networks for Sig. Proc. (pp. 112-121).
37. Yan, L., & Miller, D. J. (1999b). General statistical inference for discrete and mixed spaces by an approximate application of the maximum entropy principle. Manuscript submitted for publication.
38. Zhu, S. C., Wu, Y. N., & Mumford, D. (1997). Minimax entropy principle and its application to texture modeling. Neural Computation, 9(8), 1627-1660.