Minimum error entropy classification

Studies in Computational Intelligence

Volumn 420, Issue , 2013, Pages 1-278

  Marques De Sá, Joaquim P ^a   Silva, Luís M A ^b   Santos, Jorge M F ^c   Alexandre, Luís A ^d  

^a INEB INSTITUTO DE ENGENHARIA BIOMÉDICA   (Portugal)

^b UNIVERSITY OF AVEIRO   (Portugal)

^c INSTITUTO SUPERIOR DE ENGENHARIA DO PORTO   (Portugal)

^d UNIVERSITY OF BEIRA INTERIOR   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84867489006     PISSN: 1860949X     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-642-29029-9     Document Type: Article

References (254)

1. Ahmad, I.A., Lin, P.: A Nonparametric Estimation of the Entropy for Absolutely Continuous Distributions. IEEE Trans. Information Theory 22(3), 372-375 (1976)
2. Alexandre, L.A.: Maximizing the zero-error density for RTRL. In: 8th IEEE Int. Symposium on Signal Processing and Information Technology, IEEE Press, Sarajevo (2008)
3. Alexandre, L.A.: Single Layer Complex Valued Neural Network with Entropic Cost Function. In: Honkela, T. (ed.) ICANN 2011, Part I. LNCS, vol. 6791, pp. 331-338. Springer, Heidelberg (2011)
4. Alexandre, L.A., Campilho, A., Kamel, M.: A Probabilistic Model for the Cooperative Modular Neural Network. In: Perales, F.J., Campilho, A.C., Perez, N., Sanfeliu, A. (eds.) IbPRIA 2003. LNCS, vol. 2652, pp. 11-18. Springer, Heidelberg (2003)
5. Alexandre, L.A., Campilho, A., Kamel, M.: Bounds for the Average Generalization Error of the Mixture of Experts Neural Network. In: Fred, A., Caelli, T.M., Duin, R.P.W., Campilho, A.C., de Ridder, D. (eds.) SSPR&SPR 2004. LNCS, vol. 3138, pp. 618-625. Springer, Heidelberg (2004)
6. Alexandre, L.A., Marques de Sa, J.: Error Entropy Minimization for LSTM Training. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 244-253. Springer, Heidelberg (2006)
7. Alexandre, L.A., Silva, L., Santos, J., Marques de Sa, J.: Experimental evaluation of multilayer perceptrons with entropic risk functionals on realworld datasets. Technical Report 2/2011, INEB-Instituto de Engenharia Biomedica, Porto, Portugal (2011)
8. Allwein, E., Schapire, R., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. In: Int. Conf. on Machine Learning, pp. 9-16. Morgan Kaufmann (2000)
9. Amin, M., Murase, K.: Single-layered complex-valued neural network for realvalued classification problems. Neurocomputing 72, 945-955 (2009)
10. Andreev, A., Kanto, A., Malo, P.: Simple Approach for Distribution Selection in the Pearson System, Working Paper W388, Helsinki School of Economics (2005)
11. Anthony, M., Bartlett, P.: Neural network learning: theoretical foundations. Cambridge University Press (1999)
12. Argentiero, P., Chin, R., Beaudet, P.: An automated approach to the design of decision tree classifiers. IEEE Trans. on Pattern Analysis and Machine Intelligence 4(1), 51-57 (1982)
13. Asuncion, A., Newman, D.: UCI machine learning repository, University of California, School of Information and Computer Science (2010), http://www.ics.uci.edu/~mlearn/MLRepository.html
14. Auda, G., Kamel, M.: Modular neural network classifiers: A comparative study. Intel. Robotic Systems 21, 117-129 (1998)
15. Balakrishnan, N., Nevzorov, V.: A Primer on Statistical Distributions. John Wiley & Sons, Inc. (2003)
16. Banarer, V., Perwass, C., Sommer, G.: Design of a Multilayered Feed-Forward Neural Network Using Hypersphere Neurons. In: Petkov, N., Westenberg, M.A. (eds.) CAIP 2003. LNCS, vol. 2756, pp. 571-578. Springer, Heidelberg (2003)
17. Banarer, V., Perwass, C., Sommer, G.: The hypersphere neuron. In: European Symposium on Artificial Neural Networks, pp. 469-474 (2003)
18. Batchelor, B.: Practical Approach to Pattem Classification. Plenum Press, Plenum Pub. Co. Ltd. (1974)
19. Batchelor, B.: Classification and data analysis in vector spaces. In: Batchelor, B. (ed.) Pattern Recognition: Ideas in Practice, Plenum Press, Plenum Pub. Co. Ltd. (1978)
20. Battiti, R.: Accelerated backpropagation learning: Two optimization methods. Complex Systems 3, 331-342 (1989)
21. Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36(1-2), 105-139 (1999)
22. Baum, E., Wilczek, F.: Supervised learning of probability distributions by neural networks. In: NIPS 1987, pp. 52-61 (1987)
23. Beirlant, J., Dudewicz, E.J., Gyorfi, L., van der Meulen, E.C.: Nonparametric Entropy Estimation: An Overview. Int. J. Math. Stat. Sci. 6(1), 17-39 (1997)
24. Beiu, V., Pauw, T.: Tight Bounds on the Size of Neural Networks for Classification Problems. In: Cabestany, J., Mira, J., Moreno-Diaz, R. (eds.) IWANN 1997. LNCS, vol. 1240, pp. 743-752. Springer, Heidelberg (1997)
25. Berkhin, P.: Survey of clustering data mining techniques. Tech. rep., Accrue Software, San Jose, CA (2002)
26. Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press (1995)
27. Bishop, C.: Pattern Recognition and Machine Learning, 2nd edn. Springer (2007)
28. Blakrishnan, N., Navzorov, V.B.: A Primer on Statistical Distributions. John Wiley & Sons (2003)
29. Blum, A., Rivest, R.: Training a 3-node Neural Network is NP-Complete. In: Hanson, S.J., Rivest, R.L., Remmele, W. (eds.) MIT-Siemens 1993. LNCS, vol. 661, pp. 9-28. Springer, Heidelberg (1993)
30. Bousquet, O., Boucheron, S., Lugosi, G.: Introduction to statistical learning theory. In: Bousquet, O., von Luxburg, U., Ratsch, G. (eds.) Machine Learning 2003. LNCS (LNAI), vol. 3176, pp. 169-207. Springer, Heidelberg (2004)
31. Bowman, A., Azzalini, A.: Applied Smooting Techniques for Data Analysis. Oxford University Press (1997)
32. Brady, M., Raghavan, R., Slawny, J.: Gradient descent fails to separate. In: IEEE Int. Conf. on Neural Networks, vol. 1, pp. 649-656 (1988)
33. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Chapman & Hall/CRC (1993)
34. Buntine, W.: A theory of learning classification rules. Ph.D. thesis, Univ. of Technology, Sidney (1990)
35. Buntine, W., Niblett, T.: A further comparison of splitting rules for decisiontree induction. Machine Learning 8, 75-85 (1992)
36. Chan, L., Fallside, F.: An adaptive training algorithm for backpropagation networks. Computer Speech and Language 2, 205-218 (1987)
37. Chen, B., Hu, J., Zhu, Y., Sun, Z.: Information theoretic interpretation of error criteria. Acta Automatica Sinica 35(10), 1302-1309 (2009)
38. Chen, T., Geman, S.: On the minimum entropy of a mixture of unimodal and symmetric distributions. IEEE Trans. Information Theory 54(7), 1366-1374 (2008)
39. Cheng, C., Fu, A., Zhang, Y.: Entropy-based subspace clustering for mining numerical data. In: Int. Conf. on Knowledge Discovery and Data Mining, pp. 84-93 (1999)
40. Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence 17(8), 790-799 (1995)
41. Cherkassky, V., Mulier, F.: Learning from data: concepts, theory and methods. John Wiley & Sons (1998)
42. Chung, F.: Spectral Graph Theory, vol. 92. American Mathematical Society, Providence (1997)
43. Collobert, R., Bengio, S.: Links between perceptrons, MLPs and SVMs. In: Proc. of the 21st Int. Conf. on Machine Learning (2004)
44. Comaniciu, D., Meer, P.: Mean shift analysis and applications. In: IEEE Int. Conf. on Computer Vision, pp. 1197-1203 (1999)
45. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analisys. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(5), 603-619 (2002)
46. Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273-297 (1995)
47. Costa, J., Hero, A., Vignat, C.: On Solutions to Multivariate Maximum alpha-Entropy Problems. In: Rangarajan, A., Figueiredo, M., Zerubia, J. (eds.) EMMCVPR 2003. LNCS, vol. 2683, pp. 211-226. Springer, Heidelberg (2003)
48. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley Interscience (1991)
49. Deco, G., Obradovic, D.: An Information-Theoretic Approach to Neural Computing. Springer (1996)
50. Dembo, A., Cover, T.M., Thomas, J.A.: Information theoretic inequalities. IEEE Trans. on Information Theory 37(6), 1501-1518 (1991)
51. Demar, J.: Statistical comparisons of classifiers over multiple data sets. J. of Machine Learning Research 7, 1-30 (2006)
52. Devroye, L., Gyorfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer (1996)
53. Devroye, L., Lugosi, G.: Combinatorial Methods in Density Estimation. Springer (2001)
54. Dietterich, T.: An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40(2), 139-157 (2000)
55. Ding, C., He, X., Zha, H., Gu, M., Simon, H.: A min-max cut algorithm for graph partitioning and data clustering. In: Int. Conf. on Data Mining, pp. 107-114 (2001)
56. Dixon, W., Massey, F.: Introduction to Statistical Analysis, 4th edn. McGraw-Hill Companies (1983)
57. Draghici, S., Beiu, V.: Entropy based comparison of neural networks for classification. In: Italian Conf. on Neural Networks (1997)
58. Du, J.X., Zhai, C.M.: Structure optimization algorithm for radial basis probabilistic neural networks based on the moving median center hyperspheres algorithm. Advances in Neural Networks 55, 136-143 (2009)
59. Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley Interscience (2001)
60. Duin, R.: Dutch handwritten numerals, http://www.ph.tn.tudelft.nl/~duin
61. Dwass, M.: Probability: Theory and Applications.W. A. Benjamin, Inc. (1970)
62. Ebrahimi, N., Soofi, E., Soyer, R.: Information measures in perspective. Int. Statistical Review 78, 383-412 (2010)
63. Elman, J.: Finding structure in time. Tech. Rep. CRL 8801, University of California, Center for Research in Language, San Diego (1988)
64. Ennaji, A., Ribert, A., Lecourtier, Y.: From data topology to a modular classifier. Int. Journal on Document Analysis and Recognition 6(1), 1-9 (2003)
65. Erdogmus, D., Hild II, K., Principe, J.: Blind source separation using Renyi's marginal entropies. Neurocomputing 49, 25-38 (2002)
66. Erdogmus, D., Principe, J.: Comparison of entropy and mean square error criteria in adaptive system training using higher order statistics. In: Int. Conf. on ICA and Signal Separation, Helsinki, Finland, pp. 75-80 (2000)
67. Erdogmus, D., Principe, J.: An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems. IEEE Trans. on Signal Processing 50(7), 1780-1786 (2002)
68. Esposito, F., Malerba, D., Semeraro, G., Kay, J.: A comparative analysis of methods for pruning decision trees. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(5), 476-493 (1997)
69. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: 2nd Int. Conf. on Knowledge Discovery and Data Mining, pp. 226-231. AAAI Press (1996)
70. Alvarez Estevez, D., Principe, J., Moret-Bonillo, V.: Neuro-fuzzy classification using the correntropy criterion: Application to sleep depth estimation. In: Arabnia, H., de la Fuente, D., Kozerenko, E., Olivas, J., Chang, R., LaMonica, P., Liuzzi, R., Solo, A. (eds.) IC-AI, pp. 9-15. CSREA Press (2010)
71. Faivishevsky, L., Goldberger, J.: A nonparametric information theoretic clustering algorithm. In: Proc. of the 27th Int. Conf. on Machine Learning, pp. 351-358 (2010)
72. Fhalman, S.: Faster-learning variations on back-propagation: An empirical study. In: Touretzky, D., Hinton, G., Sejnowski, T. (eds.) Connectionist Models Summer School, pp. 38-51. Morgan Kaufmann (1988)
73. Fiedler, M.: A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory. Czechoslovak Mathematical Journal 25(100), 619-633 (1975)
74. Foley, D.: Considerations of sample and feature size. IEEE Trans. Information Theory 18(5), 618-626 (1972)
75. Forina, M., Armanino, C.: Eigenvector projection and simplified non-linear mapping of fatty acid content of italian olive oils. Ann. Chim. 72, 127-155 (1981)
76. Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press Professional, Inc. (1990)
77. Fukunaga, K., Hostetler, L.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. in Information Theory 21, 32-40 (1975)
78. Funahashi, K.: Multilayer neural networks and Bayes decision theory. Neural Networks 11(2), 209-213 (1998)
79. Garcia, S., Fernandez, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Information Sciences 180, 2044-2064 (2010)
80. Gelfand, S., Ravishankar, C., Delp, E.: An iterative growing and pruning algorithm for classification tree design. IEEE Trans. on Pattern Analysis and Machine Intelligence 13(2), 163-174 (1991)
81. Gers, F., Schmidhuber, J.: Recurrent nets that time and count. In: Int. Joint Conf. on Neural Networks, Como, Italy, pp. 189-194 (2000)
82. Gers, F., Schmidhuber, J., Cummins, F.: Learning to forget: Continual prediction with LSTM. Neural Computation 12(10), 2451-2471 (2000)
83. Gish, H.: A probabilistic approach to the understanding and training of neural network classifiers. In: Int. Conf. on Acoustics, Speech, and Signal Processing, vol. 3, pp. 1361-1364 (1990)
84. Gokcay, E., Principe, J.: Information theoretic clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(2), 158-171 (2002)
85. Guha, S., Rastogi, R., Shim, K.: CURE: an efficient clustering algorithm for large databases. In: Int. Conf. on Management of Data, pp. 73-84 (1998)
86. Guha, S., Rastogi, R., Shim, K.: ROCK: A robust clustering algorithm for categorical attributes. Information Systems 25(5), 345-366 (2000)
87. Gut, A.: Probability: A Graduate Course. Springer (2005)
88. Hall, E., Wise, G.: On optimal estimation with respect to a large family of cost functions. IEEE Trans. Information Theory 37(3), 691-693 (1991)
89. Hampshire II, J., Pearlmutter, B.: Equivalence proofs for multilayer perceptron classifiers and the Bayesian discriminant function. In: Touretzky, D., Elman, J., Sejnowsky, T., Hinton, G. (eds.) Proc. 1990 Connectionist Models Summer School. Morgan Kaufmann, San Mateo (1990)
90. Hampshire II, J.,Waibel, A.: A novel objective function for improved phoneme recognition using time-delay neural networks. IEEE Trans. on Neural Networks 1(2), 216-228 (1990)
91. Harrison, M.: Introduction to Formal Language Theory. Addison-Wesley Series in Computer Science (1978)
92. Hartuv, E., Schmitt, A., Lange, J., Meier-Ewert, S., Lehrachs, H., Shamir, R.: An algorithm for clustering cDNAs for gene expression analysis. In: Annual Conf. on Research in Computational Molecular Biology, pp. 188-197 (1999)
93. Hartuv, E., Shamir, R.: A clustering algorithm based on graph connectivity. Information Processing Letters 76(4-6), 175-181 (2000)
94. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer (2001)
95. Haykin, S.: Neural Networks: A Comprehensive Foundation, 2nd edn. Prentice-Hall, New Jersey (1999)
96. Haykin, S.: Neural Networks and Learning Machines, 3rd edn. Prentice-Hall (2009)
97. He, Z., Xu, X., Deng, S.: K-ANMI: A mutual information based clustering algorithm for categorical data. Computing Research Repository abs/cs/051 (2005)
98. Hero, A., Ma, B., Michel, O., Gorman, J.: Applications of entropic spanning graphs. IEEE Signal Processing Magazine 19(5), 85-95 (2002)
99. Hild II, K., Erdogmus, D., Principe, J.: Blind source separation using Renyi's mutual information. IEEE Signal Processing Letters 8, 174-176 (2001)
100. Hirose, A.: Complex-Valued Neural Networks. Springer (2006)
101. Hirose, A.: Complex-valued neural networks: The merits and their origins. In: Int. Joint Conf. on Neural Networks, pp. 1209-1216 (2009)
102. Hochberg, Y., Tamhane, A.: Multiple Comparison Procedures. John Wiley & Sons, Inc. (1987)
103. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735-1780 (1997)
104. Hsin, H., Li, C., Sun, M., Sclabassi, R.: An adaptive training algorithm for back-propagation neural networks. IEEE Trans. on Systems, Man, and Cybernetics 25, 512-514 (1995)
105. Huang, G.B., Babri, H.: Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions. IEEE Trans. on Neural Networks 9(1), 224-229 (1998)
106. Huber, P.: Robust estimation of a location parameter. The Annals of Mathematical Statistics 35(1), 73-101 (1964)
107. Hubert, L., Arabie, P.: Comparing partitions. Journal of Classification 2(1), 193-218 (1985)
108. Hyafil, L., Rivest, R.L.: Constructing Optimal Binary Decision Trees is NPcomplete. Information Processing Letters 5(1), 15-17 (1976)
109. Jacobs, R.: Increased rates of convergence through learning rate adaptation. Neural Networks 1, 295-307 (1988)
110. Jacobs, R., Jordan, M., Nowlan, S., Hinton, G.: Adaptive mixtures of local experts. Neural Computation (3), 79-87 (1991)
111. Jacobs, R., Peng, F., Tanner, M.: A bayesian approach to model selection in hierarchical mixtures-of-experts architectures. Neural Networks 10(2), 231-241 (1997)
112. Jain, A., Dubes, R.: Algorithms for Clustering Data. Prentice Hall Int. (1988)
113. Jain, A., Murty, M., Flynn, P.: Data clustering: a review. ACM Computing Surveys 31(3), 264-323 (1999)
114. Jain, A., Topchy, A., Law, M., Buhmann, J.: Landscape of clustering algorithms. In: 17th Int. Conf. on Pattern Recognition, vol. 1, pp. 260-263 (2004)
115. Janzura, M., Koski, T., Otahal, A.: Minimum entropy of error principle in estimation. Information Sciences (79), 123-144 (1994)
116. Jensen, D., Oates, T., Cohen, P.: Building simple models: A case study with decision trees. In: Liu, X., Cohen, P., Berthold, M. (eds.) IDA 1997. LNCS, vol. 1280, pp. 211-222. Springer, Heidelberg (1997)
117. Jenssen, R., Hild II, K., Erdogmus, D., Principe, J., Eltoft, T.: Clustering using Renyi's entropy. In: Int. Joint Conf. on Neural Networks, pp. 523-528 (2003)
118. Jeong, K.H., Liu, W., Han, S., Hasanbelliu, E., Principe, J.: The correntropy MACE filter. Pattern Recognition 42(5), 871-885 (2009)
119. Jirina Jr., M., Jirina, M.: Neural network classifier based on growing hyperspheres. Neural Network World 10(3), 417-428 (2000)
120. Johnson, E., Mehrotra, A., Nemhauser, G.: Min-cut clustering. Mathematical Programming 62, 133-151 (1993)
121. Johnson, N.L., Kotz, S., Blakrishnan, N.: Continuous Univariate Distributions, vol. 1. John Wiley & Sons (1994)
122. Johnson, N.L., Kotz, S., Blakrishnan, N.: Continuous Univariate Distributions, vol. 2. John Wiley & Sons (1995)
123. Jordan, M., Jacobs, R.: Hierarchical mixture of experts and the EM algorithm. Neural Computation 6, 181-214 (1994)
124. Kannan, R., Vempala, S., Vetta, A.: On clusterings: Good, bad, and spectral. In: Annual Symposium on the Foundation of Computer Science, pp. 367-380 (2000)
125. Kapur, J.: Maximum-Entropy Models in Science and Engineering. John Wiley & Sons, New York (1993)
126. Karypis, G.: Cluto: Software package for clustering high-dimensional datasets, Version 2.1.1 (2003)
127. Karypis, G., Han, E.H., Kumar, V.: Chameleon: Hierarchical clustering using dynamic modeling. IEEE Computer 32(8), 68-75 (1999)
128. Kaufman, L., Rousseeuw, P.: Finding Groups in Data: An Introduction to Cluster Analysis. John Wiley & Sons, New York (1990)
129. Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Trans. Pattern Analysis and Machine Intelligence 20(3), 226-239 (1998)
130. Kulkarni, A.: On the mean accuracy of hierarchical classifiers. IEEE Trans. on Computers C-27(8), 771-776 (1978)
131. Kullback, S.: Information Theory and Statistics. Dover Pub. (1968)
132. Lagarias, J., Reeds, J., Wright, M., Wright, P.: Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM Journal of Optimization 9, 112-147 (1998)
133. Lazo, A.V., Rathie, P.N.: On the Entropy of Continuous Probability Distributions. IEEE Trans. on Information Theory 24(1), 120-122 (1978)
134. Lee, Y., Choi, S.: Minimum entropy, k-means, spectral clustering. In: IEEE Int. Joint Conf. on Neural Networks, vol. 1, pp. 117-122 (2004)
135. Lee, Y., Choi, S.: Maximum within-cluster association. Pattern Recognition Letters 26(10), 1412-1422 (2005)
136. Lewis, F.: Optimal Estimation. With an Introduction to Stochastic Control Theory. John Wiley & Sons (1986)
137. Li, H., Zhang, K., Jiang, T.: Minimum entropy clustering and applications to gene expression analysis. In: IEEE Computational Systems Bioinformatics Conf., pp. 142-151 (2004)
138. Li, J., Liu, H.: Ensembles of cascading trees. In: Int. Conf. Data Mining, pp. 585-588 (2003)
139. Li, R.H., Belford, G.: Instability of decision tree classification algorithms. In: Proc. of the 8th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, KDD 2002, p. 570 (2002)
140. Linsker, R.: Self-organization in a perceptual network. IEEE Computer 21, 105-117 (1988)
141. Liu, W., Pokharel, P., Principe, J.: Correntropy: Properties and applications in non-Gaussian signal processing. IEEE Trans. on Signal Processing 55(11), 5286-5298 (2007)
142. Looney, C.: Pattern recognition using neural networks: theory and algorithms for engineers and scientists. Oxford University Press, New York (1997)
143. Mackey, M., Glass, L.: Oscillation and chaos in physiological control systems. Science 197, 287-289 (1977)
144. Maghsoodloo, S., Huang, C.: Computing probability integrals of a bivariate normal distribution. Interstat. (334) (1995)
145. Magoulas, G., Vrahatis, M., Androulakis, G.: Effective back-propagation with variable stepsize. Neural Networks 10, 69-82 (1997)
146. Magoulas, G., Vrahatis, M., Androulakis, G.: Improving the convergence of the backpropagation algorithm using learning rate adaptation methods. Neural Computation 11(7), 1769-1796 (1999)
147. Maimon, O., Rokach, L.: Decomposition Methodology for Knowledge Discovery and Data Mining: Theory and Applications. World Scientific Pub. Co. (2005)
148. Mandic, D., Goh, V.: Complex valued nonlinear adaptive filters. John Wiley & Sons (2009)
149. Marques de Sa, J.: Applied Statistics using SPSS, STATISTICA and MATLAB. Springer (2003)
150. Marques de Sa, J.: MSE, CE and MEE. Tech. Rep. 5/2008, Instituto de Engenharia Biomedica (PSI/NNIG) (2008)
151. Marques de Sa, J., Gama, J., Sebastiao, R., Alexandre, L.A.: Decision Trees Using the Minimum Entropy-of-Error Principle. In: Jiang, X., Petkov, N. (eds.) CAIP 2009. LNCS, vol. 5702, pp. 799-807. Springer, Heidelberg (2009)
152. Marques de Sa, J., Sebastiao, R., Gama, J.: Tree classifiers based on minimum error entropy decisions. Canadian Journal on Artificial Intelligence, Machine Learning and Pattern Recognition 2(3), 41-55 (2011)
153. Marques de Sa, J., Sebastiao, R., Gama, J., Fontes, T.: New results on minimum error entropy decision trees. In: San Martin, C., Kim, S.-W. (eds.) CIARP 2011. LNCS, vol. 7042, pp. 355-362. Springer, Heidelberg (2011)
154. Matula, D.: Cluster analysis via graph theoretic techniques. In: Mullin, R., Reid, K., Roselle, D. (eds.) Proc. Luisiana Conf. on Combinatorics, Graph Theory and Computing, pp. 199-212 (1970)
155. Matula, D.: k-components, clusters and slicings in graphs. SIAM J. Appl. Math. 22(3), 459-480 (1972)
156. McCulloch, W., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 5, 115-133 (1943)
157. Meila, M., Shi, J.: A random walks view of spectral segmentation. In: 8th Int. Workshop on Artificial Intelligence and Statistics, pp. 8-11 (2001)
158. Mergel, V.: Test of goodness of fit for the inverse-Gaussian distribution. Math. Commun. 4(2), 191-195 (1999)
159. Mokkadem, A.: Estimation of the entropy and information of absolutely continuous random variables. IEEE Trans. Information Theory 35(1), 193-196 (1989)
160. Moller, M.: Efficient training of feed-forward neural networks. Ph.D. thesis, Computer Science Department, Aarhus University (1993)
161. Murthy, S.: Automatic construction of decision trees from data: A multidisciplinary survey. Data Mining and Knowledge Discovery 2(4), 345-389 (1998)
162. Nadarajah, S., Zografos, K.: Formulas for Renyi information and related measures for univariate distributions. Inf. Sci. 155(1-2), 119-138 (2003)
163. NCI60: Stanford NCI60 cancer microarray project, http://genome-www. stanford.edu/nci60/ (2000)
164. Neelakanta, P., Abusalah, S., Groff, D., Sudhakar, R., Park, J.: Csiszar's generalized error measures for gradient-descent-based optimizations in neural networks using the backpropagation algorithm. Connection Science 8(1), 79-114 (1996)
165. Nelder, J., Mead, R.: A simplex method for function minimization. Computer Journal 7, 308-313 (1965)
166. Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Advances in Neural Information Processing Systems, vol. 14 (2001)
167. Ng, G., Wahab, A., Shi, D.: Entropy learning and relevance criteria for neural network pruning. Int. Journal of Neural Systems 13(5), 291-305 (2003)
168. Papoulis, A.: Probability, Random Variables and Stochastic Processes. McGraw-Hill Co. Inc. (1991)
169. Parzen, E.: On the estimation of a probability density function and the mode. Annals of Mathematical Statistics 33, 1065-1076 (1962)
170. Pelagotti, A., Piuri, V.: Entropic analysis and incremental synthesis of multilayered feedforward neural networks. Int. Journal of Neural Systems 8(5-6), 647-659 (1997)
171. Perez-Ortiz, J., Gers, F., Eck, D., Schmidhuber, J.: Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets. Neural Networks 16(2), 241-250 (2003)
172. Pipberger, H., Arms, R., Stallmann, F.: Automatic screening of normal and abnormal electrocardiograms by means of digital electronic computer. In: Proc. Soc. Exp. Biol. Med., pp. 106-130 (1961)
173. Porter, M.: An algorithm for suffix stripping. Program 14(3), 130-137 (1980)
174. Principe, J.: Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives. Springer (2010)
175. Principe, J., Xu, D.: Learning from examples with information theoretic criteria. Journal of VLSI Signal Processing Systems 26(1-2), 61-77 (2000)
176. Prokhorov, A.V.: Pearson Curves. In: Hazewinkel, M. (ed.) Encyclopaedia of Mathematics. Kluwer Academic Pub. (2002)
177. Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, Inc. (1993)
178. Raileanu, L., Stoffel, K.: Theoretical comparison between the Gini index and information gain criteria. Annals of Mathematics and Artificial Intelligence 41, 77-93 (2004)
179. Rama, B., Jayashree, P., Jiwani, S.: A survey on clustering: Current status and challenging issues. Int. Journal on Computer Science and Engineering 2(9), 2976-2980 (2010)
180. Rand, W.: Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 66, 846-850 (1971)
181. Rao, B.L.S.P.: Nonparametric Functional Estimation. Academic Press, Inc. (1983)
182. Raudys, S., Pikelis, V.: On dimesionality, sample size, classification error, and complexity of classification algorithm in pattern recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 2(3), 242-252 (1980)
183. Renyi, A.: Probability Theory. Elsevier Science Pub. Co. Inc. (1970)
184. Reza, F.: An Introduction to Information Theory. Dover Pub. (1994)
185. Richard, M., Lippmann, R.: Neural network classifiers estimate Bayesian a posteriori probabilities. Neural Computation 3, 461-483 (1991)
186. Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The Rprop algorithm. In: IEEE Int. Conf. on Neural Networks, pp. 586-591 (1993)
187. Robinson, A., Fallside, F.: The utility driven dynamic error propagation network. Tech. Rep. CUED/F-INFENG/TR.1, Cambridge University, Engineering Department (1987)
188. Rokach, L., Maimon, O.: Decision trees. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook. Springer (2005)
189. Rosasco, L., De Vito, E., Caponnetto, A., Piana, M.: Are loss functions all the same? Neural Computation 16(5), 1063-1076 (2004)
190. Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review 65, 368-408 (1958)
191. Rueda, L.: A one-dimensional analysis for the probability of error of linear classifiers for normally distributed classes. Pattern Recognition 38(8), 1197-1207 (2005)
192. Rumelhart, D., Hinton, G., Williams, R.: Learning representations by backpropagation errors. Nature 323, 533-536 (1986)
193. Saerens, M., Latinne, P., Decaestecker, C.: Any reasonable cost function can be used for a posteriori probability approximation. IEEE Trans. on Neural Networks 13(5), 1204-1210 (2002)
194. Safavian, S., Landgrebe, D.: A survey of decision tree classifier methodology. IEEE Trans. on Systems Man and Cybernetics 21(3), 660-674 (1991)
195. Salzberg, S.: On comparing classifiers: Pitfalls to avoid and a recommended approach. Data Mining and Knowledge Discovery 1, 317-327 (1997)
196. Santamaria, I., Pokharel, P., Principe, J.: Generalized correlation function: definition, properties, and application to blind equalization. IEEE Trans. on Signal Processing 54(6-1), 2187-2197 (2006)
197. Santos, J.: Repository of data sets used on technical report Human Clustering on Bi-dimensional Data: An Assessment (2005), http://www.dema.isep. ipp.pt/~jms/datasets
198. Santos, J.: Data classification with neural networks and entropic criteria. Ph.D. thesis, University of Porto (2007)
199. Santos, J., Alexandre, L., Marques de Sa, J.: The error entropy minimization algorithm for neural network classification. In: Lofti, A. (ed.) Int. Conf. on Recent Advances in Soft Computing, pp. 92-97 (2004)
200. Santos, J., Alexandre, L., Marques de Sa, J.: Modular neural network task decomposition via entropic clustering. In: 6th Int. Conf. on Intelligent Systems Design and Applications, pp. 62-67. IEEE Computer Society Press (2006)
201. Santos, J., Marques de Sa, J.: Human clustering on bi-dimensional data: An assessment. Technical Report 1/2005, INEB-Instituto de Engenharia Biomedica, Porto, Portugal (2005)
202. Santos, J., Marques de Sa, J., Alexandre, L.: Batch-Sequential Algorithm for Neural Networks Trained with Entropic Criteria. In: Duch, W., Kacprzyk, J., Oja, E., Zadro?ny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 91-96. Springer, Heidelberg (2005)
203. Santos, J., Marques de Sa, J., Alexandre, L.: Neural networks trained with the EEM algorithm: Tuning the smoothing parameter. WSEAS Trans. on Systems 4(4), 295-300 (2005)
204. Santos, J., Marques de Sa, J., Alexandre, L.: LEGClust-a clustering algorithm based on layered entropic subgraphs. IEEE Trans. on Pattern Analysis and Machine Intelligence 30(1), 62-75 (2008)
205. Santos, J., Marques de Sa, J., Alexandre, L., Sereno, F.: Optimization of the error entropy minimization algorithm for neural network classification. In: Dagli, C.H., Buczak, A., Enke, D., Embrechts, M., Ersoy, O. (eds.) Intelligent Engineering Systems Through Artificial Neural Networks, vol. 14, pp. 81-86. ASME Press (2004)
206. Savitha, R., Suresh, S., Sundararajan, N., Saratchandran, P.: A new learning algorithm with logarithmic performance index for complex-valued neural networks. Neurocomputing 72, 3771-3781 (2009)
207. Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge University Press (2004)
208. Sherman, S.: Non-mean-square error criteria. IRE Trans. Information Theory 4(3), 125-126 (1958)
209. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(8), 888-905 (2000)
210. Silva, F., Almeida, L.: Speeding up backpropagation. In: Eckmiller, R. (ed.) Advanced Neural Computers, pp. 151-158 (1990)
211. Silva, J.E., Marques de Sa, J., Jossinet, J.: Classification of breast tissue by electrical impedance spectroscopy. Medical & Biological Engineering & Computing 38(1), 26-30 (2000)
212. Silva, L.M.: Neural networks with error-density risk functionals for data classification. Ph.D. thesis, University of Porto, Portugal (2008)
213. Silva, L.M., Alexandre, L.A., Marques de Sa, J.: Neural Network Classification: Maximizing Zero-Error Density. In: Singh, S., Singh, M., Apte, C., Perner, P. (eds.) ICAPR 2005. LNCS, vol. 3686, pp. 127-135. Springer, Heidelberg (2005)
214. Silva, L.M., Alexandre, L., Marques de Sa, J.: New developments of the ZEDM algorithm. In: 6th Int. Conf. on Intelligent Systems Design and Applications, vol. 1, pp. 1067-1072. IEEE Computer Society Press (2006)
215. Silva, L.M., Embrechts, M.J., Santos, J.M., Marques de Sa, J.: The Influence of the Risk Functional in Data Classification with MLPs. In: K?rkova, V., Neruda, R., Koutnik, J. (eds.) ICANN 2008, Part I. LNCS, vol. 5163, pp. 185-194. Springer, Heidelberg (2008)
216. Silva, L.M., Felgueiras, C., Alexandre, L., Marques de Sa, J.: Error entropy in classification problems: a univariate data analysis. Neural Computation 18(9), 2036-2061 (2006)
217. Silva, L.M., Marques de Sa, J., Alexandre, L.: Neural network classification using Shannon's entropy. In: European Symposium on Artificial Neural Networks, pp. 217-222 (2005)
218. Silva, L.M., Marques de Sa, J., Alexandre, L.: Data classification with multilayer perceptrons using a generalized error function. Neural Networks 21(9), 1302-1310 (2008)
219. Silva, L.M., Marques de Sa, J., Alexandre, L.: The MEE principle in data classification: A perceptron-based analysis. Neural Computation 22(10), 2698-2728 (2010)
220. Silva, L.M., Santos, J., Marques de Sa, J.: Classification performance of multilayer perceptrons with different risk functionals. Technical Report 1/2011, INEB-Instituto de Engenharia Biomedica, Porto, Portugal (2011)
221. Silverman, B.W.: Density Estimation for Statistics and Data Analysis, vol. 26. Chapman & Hall (1986)
222. Stam, A.: Some inequalities satisfied by the quantities of information of Fisher and Shannon. Information and Control 2, 101-112 (1959)
223. Stoller, D.: Univariate two-population distribution free discrimination. Journal of the American Statistical Association 49, 770-777 (1954)
224. Tapia, R.A., Thompson, J.R.: Nonparametric Probability Density Estimation. The John Hopkins University Press (1978)
225. Theodoridis, S., Koutroumbas, K.: Pattern Recognition. Academic Press (2009)
226. Tsai, H., Lee, S.: Entropy-based generation of supervised neural networks for classification of structured patterns. IEEE Trans. on Neural Networks 15(2), 283-297 (2004)
227. Vapnik, V.: Statistical Learning Theory. John Wiley & Sons (1998)
228. Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer (2010)
229. Verma, D., Meila, M.: A comparison of spectral clustering algorithms. Tech. Rep. UW-CSE-03-05-01, Washington University (2003)
230. Vidyasagar, M.: A Theory of Learning and Generalization, with Applications to Neural Networks and Control Systems. Springer (1997)
231. Vilalta, R., Achari, M., Eick, C.: Class decomposition via clustering: a new framework for low-variance classifiers. In: IEEE Int. Conf. on Data Mining, pp. 673-676 (2003)
232. Vogl, T., Mangis, J., Rigler, J., Zink, W., Alkon, D.: Accelerating the convergence of the back-propagation method. Biological Cybernetics 59(4-5), 257-263 (1988)
233. von Luxburg, U., Scholkopf, B.: Statistical learning theory: Models, concepts, and results. In: Gabbay, S.H.D., Woods, J. (eds.) Handbook for the History of Logic. Inductive Logic, vol. 10, Elsevier (2011)
234. Vretos, N., Solachildis, V., Pitas, I.: A mutual information based face clustering algorithm for movies. In: Int. Conf. on Multimedia and Expo, pp. 1013-1016. IEEE (2006)
235. Wand, M., Jones, M.: Kernel Smoothing. Chapman and Hall (1995)
236. Wang, X.F., Du, J.X., Zhang, G.J.: Recognition of Leaf Images based on Shape Features using a Hypersphere Classifier. In: Huang,D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 87-96. Springer,Heidelberg (2005)
237. Watanabe, S.: Pattern recognition as a quest for minimum entropy. Pattern Recognition 13(5), 381-387 (1981)
238. Watanabe, S.: Pattern Recognition: Human and Mechanical. John Wiley & Sons, Inc. (1985)
239. Weisstein, E.W.: Pearson System (2010), http://mathworld.wolfram.com/ PearsonSystem.html
240. Werbos, P.: Generalization of backpropagation with application to a recurrent gas market model. Neural Networks 1(4), 339-356 (1988)
241. Williams, R., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1, 270-280 (1989)
242. Williams, R., Zipser, D.: Gradient-based learning algorithms for recurrent networks and their computational complexity. In: Chauvin, Y., Rumelhart, D.E. (eds.) Back-propagation: Theory, Architectures and Applications. Erlbaum, Hillsdale (1995)
243. Woodward, W.A.: Approximation of Type IV Probability Integral. Master's thesis, Texas Tech University (1971)
244. Wu, Z., Leahy, R.: An optimal graph theoretic approach to data clustering: Theory and its application to image segmentation. IEEE Trans. on Pattern Analysis and Machine Learning 15(11), 1101-1113 (1993)
245. Xu, D.: Entropy and Information Potential for Neural Computation. Ph.D. thesis, University of Florida, USA (1999)
246. Xu, D., Principe, J.C.: Learning from Examples with Quadratic Mutual Information. In: IEEE Signal Processing Society Workshop Neural Networks for Signal Processing VIII, pp. 155-164 (1998)
247. Xu, R.,Wunsch II, D.: Survey of clustering algorithms. IEEE Trans. on Neural Networks 16(3), 645-678 (2005)
248. Xu, X.: Dbscan, http://ifsc.ualr.edu/xwxu
249. Hochberg, Y., Tamhane, A.C.: Multiple Comparison Procedures. John Wiley & Sons (1997)
250. Yuan, H., Xiong, F., Huai, X.: A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy. Computers and Electronics in Agriculture 40(1-3), 57-64 (2003)
251. Yuan, L., Kesavan, H.: Minimum entropy and information measure. IEEE Trans. Systems, Man, and Cybernetics 28(3), 488-491 (1998)
252. Zhang, G.: Neural networks for classification: a survey. IEEE Trans. on Systems, Man and Cybernetics, Part C (Applications and Reviews) 30(4), 451-462 (2000)
253. Zhang, T., Ramakrishnan, R., Livny, M.: BIRCH: An efficient clustering method for very large databases. In: ACM SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery, Montreal, Canada, pp. 103-114 (1996)
254. Zhang, T., Ramakrishnan, R., Livny, M.: BIRCH: A new data clustering algorithm and its applications. Data Mining and Knowledge Discovery 1(2), 141-182 (1997)