Evaluation criteria based on mutual information for classifications including rejected class

Zidonghua Xuebao/ Acta Automatica Sinica

Volumn 34, Issue 11, 2008, Pages 1396-1403

  Hu, Bao Gang ^a,b   Wang, Yong ^a,b  

^a INSTITUTE OF AUTOMATION   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

Author keywords

Classification; Confusion matrix; Entropy; Evaluation criteria; Machine learning; Mutual information

Indexed keywords

CLASSIFICATION; CONFUSION MATRIX; EVALUATION CRITERIA; MACHINE LEARNING; MUTUAL INFORMATION;

IMAGE RESOLUTION; INFORMATION THEORY; LEARNING SYSTEMS; SCHEMATIC DIAGRAMS;

ROBOT LEARNING;

EID: 57249085800     PISSN: 02544156     EISSN: None     Source Type: Journal    
DOI: 10.3724/SP.J.1004.2008.01396     Document Type: Article

References (31)

1. Haykin S. Neural Networks: A Comprehensive Foundation (Second Edition). New York: Printice Hall, 1999
2. Xu D X. Energy, Entropy and information potential for neural computation [Ph. D. dissertation], University of Florida, 1998
3. Cover T M, Thomas J A. Elements of Information Theory. New York: John Wiley, 1991
4. Mitchell T M. Machine Learning. New York: McGraw-Hill, 1997
5. Mackay D J C. Information Theory, Inference and Learning Algorithms. Cambridge: Cambridge University Press, 2003. 140, 532-533
6. Linsker R. Self-organization in a perceptual network. Computer, 1988, 21(3): 105-117
7. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 1994, 5(4): 537-550
8. Kwak N, Choi C H. Input feature selection for classification problems. IEEE Transactions on Neural Networks, 2002, 13(1): 143-159
9. Peng H C, Long F H, Ding C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(8): 1226-1238
10. Huang Jin-Jie, Lv Ning, Li Shuang-Quan, Cai Yun-Ze. Feature selection for classificatory analysis based on information-theoretic criteria. Acta Automatica Sinica, 2008, 34(3): 383-392
11. Comon P. Independent component analysis: a new concept? Signal Processing, 1994, 36(3): 287-314
12. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P. Multimodality image registration by maximization of mutual information. IEEE Transactions on Medical Imaging, 1997, 16(2): 187-198
13. Tang Min. A novel image registration method combining morphological gradient mutual information with multiresolution optimizer. Acta Automatica Sinica, 2008, 34(3): 246-250 (in Chinese)
14. Principe J C, Xu D X, Zhao Q, Fisher J W. Learning from examples with information-theoretic criteria. Journal of VLSI Signal Processing Systems, 2000, 26(1-2): 61-77
15. Sridhar D V, Bartlett E B, Seagrave R C. Information theoretic subset selection for neural network models. Computers and Chemical Engineering, 1998, 22(4-5): 613-626
16. Press W H, Flannery B P, Teukolsky S A, Vetterling W T. Numerical Recipes in C. Cambridge: Cambridge University Press, 1988
17. Strehl A, Ghosh J. Cluster ensembles-a knowledge reuse framework for combining multiple partitions. The Journal of Machine Learning Research, 2003, 3: 583-617
18. Quinlan J R. Introduction of decision trees. Machine Learning, 1986, 1(1): 81-106
19. Quinlan J R. C4.5: Programs for Machine Learning. San Francisco: Morgan Kaufmann, 1993
20. Ding Xiao-Qing, Wu You-Shou. Information entropy theory in pattern recognition. Acta Electronica Sinica, 1993, 21(8): 1-8 (in Chinese)
21. Fano R M. Transmission of Information: A Statistical Theory of Communication. New York: MIT, 1961
22. Hellman M, Raviv J. Probability of error, equivocation, and the Chernoff bound. IEEE Transactions on Information Theory, 1970, 16(4): 368-372
23. Eriksson T, Kim S, Kang H G, Lee C. An information-theoretic perspective on feature selection in speaker recognition. IEEE Signal Processing Letters, 2005, 12(7): 500-503
24. Wang Yong, Hu Bao-Gang. A study on integrated evaluating kernel classification performance using statistical methods. Chinese Journal of Computers, 2008, 31(6): 64-74 (in Chinese)
25. Duda R O, Hart P E, Stork D G. Pattern Classification (Second Edition). New York: John Wiley, 2001
26. Wagner R F, Metz C E, Campbell G. Assessment of medical imaging systems and computer aids: a tutorial review. Academic Radiology, 2007, 14(6): 723-748
27. Wallach H. Evaluation metrics for hard classifiers [Online]. available: http://www.inference.phy.cam.ac.uk/hmw26/, November 3, 2007
28. Hu Bao-Gang, Wang Yong, Yang Shuang-Hong, Qu Han-Bing. How to add transparency to artificial neural networks? Pattern Recognition and Artificial Intelligence, 2007, 20(1): 72-84 (in Chinese)
29. Yang S H, Hu B G. A stagewise least square loss function for classification. In: Proceedings of the 2008 SIAM International Conference on Data Mining. Japan: IEEE, 2008. 120-131
30. Miao Dou-Qian, Wang Jue. On the relationships between information entropy and roughness of knowledge in rough set theory. Pattern Recognition and Artificial Intelligence, 1998, 11(1): 34-40 (in Chinese)
31. Borga M. Learning multidimensional signal processing [Ph. D. dissertation], Linkoping University, 1998