Lower and upper bounds for misclassification probability based on Renyi's information

Journal of VLSI Signal Processing Systems for Signal, Image, and Video Technology

Volumn 37, Issue 2-3, 2004, Pages 305-317

  Erdogmus, Deniz ^a   Principe, Jose C ^a  

^a UNIVERSITY OF FLORIDA   (United States)

Author keywords

Classifier performance; Error bounds; Fano's inequality; Renyi's entropy

Indexed keywords

CLASSIFICATION (OF INFORMATION); CONVERGENCE OF NUMERICAL METHODS; DATA TRANSFER; DIGITAL COMMUNICATION SYSTEMS; ERROR ANALYSIS; PROBABILITY;

CLASSIFIER PERFORMANCE; ERROR BOUNDS; FANO'S INEQUALITY; RENYI'S ENTROPY;

SIGNAL PROCESSING;

EID: 3543121719     PISSN: 13875485     EISSN: None     Source Type: Journal    
DOI: 10.1023/B:VLSI.0000027493.48841.39     Document Type: Conference Paper

References (20)

1. T. Cover and J. Thomas, Elements of Information Theory, New York: John Wiley, 1991.
2. R. Linsker, "Towards an Organizing Principle for a Layered Perceptual Network," Neural Information Processing Systems, 1988, pp. 485-494.
3. K. Torkkola, "Visualizing Class Structure in Data Using Mutual Information," in Proceedings of Neural Networks for Signal Processing X, Sydney, Australia, 2000, pp. 376-385.
4. K. Fu, "Statistical Pattern Recognition," in Adaptive, Learning and Pattern Recognition Systems, Mendel and Fu (Eds.), New York: Academic Press, 1970, pp. 35-76.
5. K. Fukunaga, An Introduction to Statistical Pattern Recognition, New York: Academic Press, 1972.
6. G. Deco and D. Obradovic, An Information Theoretic Approach to Neural Computing, New York: Springer, 1996.
7. J. Fisher, "Nonlinear Extensions to the MACE Filter," Ph.D. Dissertation, University of Florida, 1997.
8. B. Ripley, Pattern Recognition and Neural Networks, New York: Cambridge University Press, 1996.
9. V. Vapnik, The Nature of Statistical Learning Theory, New York: Springer Verlag, 1995.
10. R.M. Fano, Transmission of Information: A Statistical Theory of Communications, New York: MIT Press and John Wiley & Sons Inc., 1961.
11. C.E. Shannon, "A Mathematical Theory of Communications," Bell Systems Technical Journal, vol. 27, 1948, pp. 379-423, 623-656.
12. A. Renyi, Probability Theory, New York: American Elsevier Publishing Company Inc., 1970.
13. R.G. Gallager, Information Theory and Reliable Communication, New York: John Wiley & Sons Inc., 1968.
14. M.B. Bassat and J. Raviv, "Renyi's Entropy and the Prbability of Error," IEEE Transactions on Information Theory, vol. 24, no. 3, 1978, pp. 324-330.
15. M. Feder and N. Merhav, "Relations Between Entropy and Error Probability," IEEE Transactions on Information Theory, vol. 40, no. 1, 1994, pp. 259-266.
16. T.S. Han and S. Verdu, "Generalizing the Fano Inequality," IEEE Transactions on Information Theory, vol. 40, no. 4, 1994, pp.1247-1251.
17. H.V. Poor and S. Verdu, "A Lower Bound on the Probability of Error in Multihypothesis Testing," IEEE Transactions on Information Theory, vol. 41, no. 6, 1995, pp. 1992-1994.
18. S. Kullback, Information Theory and Statistics, New York: Dover Publications Inc., 1968.
19. D. Erdogmus and J.C. Principe, "Information Transfer Through Classifiers and its Relation to Probability of Error," in Proceedings of International Joint Conference on Neural Networks (IJCNN'01), Washington, DC, 2001, pp. 50-54.
20. C. Bishop, Neural Networks for Pattern Recognition, Oxford: Clarendon Press, 1995.