Feature selection and extraction using an unsupervised biologically- suggested approximation to gebelein's maximal correlation

International Journal of Pattern Recognition and Artificial Intelligence

Volumn 24, Issue 3, 2010, Pages 337-358

  Kursun, Olcay ^a   Favorov, Oleg V ^b  

^a ISTANBUL UNIVERSITY   (Turkey)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

eigen faces; face recognition; Kernel canonical correlation analysis; mutual information; principal component analysis; SINBAD; Statistical dependence; support vector machine

Indexed keywords

CEREBRAL CORTEX; CRITICAL STEPS; DEPENDENCY RELATION; EXPLICIT FORMULA; EXTRACTION METHOD; FEATURE SELECTION; KERNEL CANONICAL CORRELATION ANALYSIS; MAXIMAL CORRELATION; MUTUAL INFORMATIONS; PATTERN RECOGNITION TECHNIQUES; REAL DATA SETS; SINBAD; STATISTICAL DEPENDENCE;

CORRELATION METHODS; FACE RECOGNITION; FEATURE EXTRACTION; SUPPORT VECTOR MACHINES;

PRINCIPAL COMPONENT ANALYSIS;

EID: 77954208132     PISSN: 02180014     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218001410008007     Document Type: Article

References (58)

1. A. Asuncion and D. J. Newman, UCI machine learning repository [http://www.ics. uci.edu/~mlearn/MLRepository.html], University of California at Irvine (2007).
2. The ORL face database by Olivetti Research Laboratory [http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html], AT&T Laboratories, Cambridge.
3. F. Bach and M. I. Jordan, Kernel independent component analysis, J. Mach. Learn. Res. 3 (2002) 1-48.
4. S. Baker and T. Kanade, Super-resolution: Reconstruction or recognition? IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, IEEE (Baltimore, Maryland, 2001).
5. S. Becker, Implicit learning in 3D object recognition: The importance of temporal context, Neural Comput. 11 (1999) 347-374.
6. S. Becker, Mutual information maximization: Models of cortical self-organization, Network 7 (1996) 7-31.
7. S. Becker, JPMAX: Learning to recognize moving objects as a model-̄tting problem, Advances in Neural Information Processing Systems, Vol.7 (Morgan Kaufmann Publishers, San Mateo, CA, 1995), pp. 933-940.
8. S. Becker and G. E. Hinton, A self-organizing neural network that discovers surfaces in random-dot stereograms, Nature 355 (1992) 161-163.
9. C. B. Bell, Mutual information and maximal correlation as measures of dependence, Ann. Math. Stat. 33 (1962) 587-595.
10. L. Breiman and J. H. Friedman, Estimating optimal transformations for multiple regression and correlation, J. Amer. Stat. Assoc. 80 (1985) 580-598.
11. W. Bryc, A. Dembo and A. Kagan, On the maximum correlation coe±cient, Th. Probab. Appl. 49 (2005) 132-138.
12. S. Chopra, R. Hadsell and Y. LeCun, Learning a similarity metric discriminatively, with application to face verīcation, Proc. CVPR 1 (2005) 539-546.
13. D. Endres and P. Foldiak, Bayesian bin distribution inference and mutual information, IEEE Trans. Inform. Th. 51 (2005) 3766-3779.
14. O. V. Favorov and D. Ryder, SINBAD: A neocortical mechanism for discovering environmental variables and regularities hidden in sensory input, Biol. Cybern. 90 (2004) 191-202.
15. O. V. Favorov, D. Ryder, J. T. Hester, D. G. Kelly and M. Tommerdahl, The cortical pyramidal cell as a set of interacting error backpropagating networks: A mechanism for discovering nature's order, Theories of the Cerebral Cortex, eds. R. Hecht-Nielsen and T. McKenna (Springer, London, 2003), pp. 25-64.
16. W. T. Freeman, T. R. Jones and E. C. Pasztor, Example-based super-resolution, Mitsubishi Electronic Research Laboratory, TR-2001-2030 (2001).
17. J. Friedman, Another approach to polychotomous classīcation, Stanford University, Technical report (1996).
18. H. Gebelein, Das statistische problem der correlation also variations und eigenwert problem, Zeitschrift fur Ange. Mathematik und Mechanik 21 (1941) 364-379.
19. A. Gretton, R. Herbrich, A. Smola, O. Bousquet and B. Scholkopf, Kernel methods for measuring independence, J. Mach. Learn. Res. 6 (2005) 2075-2129.
20. M. D. C. Gurrera, Construction of bivariate distributions and statistical dependence operations, PhD Dissertation, University of Barcelona, Spain (2005).
21. H. A. Guvenir, B. Acar, G. Demiroz and A. Cekin, A supervised machine learning algorithm for arrhythmia analysis, Proc. Comput. Cardiol. Conf. 24 (Lund, Sweden), 1997, pp. 433-436.
22. D. Hardoon, S. Szedmak and J. Shawe-Taylor, Canonical correlation analysis: An overview with application to learning methods, Neural Comput. 16 (2004) 2639-2664.
23. T. Hastie and R. Tibshirani, Classīcation by pairwise coupling, Neural Information Processing Systems, Vol.10 (MIT Press, Cambridge, MA, 1997), pp. 507-513.
24. C. Havran, L. Hupet, J. Czyz, J. Lee, L. Vandendorpe and M. Verleysen, Independent component analysis for face authentication, Proceedings of Knowledge-Based Intelligent Information and Engineering Systems (Crema, Italy, 2002), pp. 1207-1211.
25. L. B. Holder, I. Russell, Z. Markov, A. G. Pipe and B. Carse, Current and future trends in feature selection and extraction for classīcation problems, Int. J. Patt. Recogn. Artif. Intell. 19 (2005) 133-142.
26. H. Hotelling, Relations between two sets of variates, Biometrika 28 (1936) 312-377.
27. C. W. Hsu and C. J. Lin, A comparison of methods for multi-class support vector machines, IEEE Trans. Neural Networks 13 (2002) 415-425.
28. J. P. A. Ioannidis, Why most published research ̄ndings are false? PLoS Med 2 (2005) e124.
29. T. Joachims, Making large-scale SVM learning practical, Advances in Kernel Methods-Support Vector Learning, eds. B. Scholkopf, C. Burges and A. Smola (MIT-Press, Cambridge, MA, 1999).
30. K. Jogdeo, Concepts of dependence, Encyclopedia of Statistical Sciences, eds. S. Kotz and N. L. Johnson, Vol.1 (John Wiley & Sons, New York, 1982), pp. 324-334.
31. K. P. Kording and P. Konig, Learning with two sites of synaptic integration, Network: Comput. Neural Syst. 11 (2000) 25-39.
32. O. Kursun, A generalization of Shannon's mutual information for improved feature selection in databases involving possibly rare but well-predictable classes, Proc. Int. Conf. Artīcial Intelligence and Pattern Recognition (Orlando, FL, 2008).
33. O. Kursun and O. Favorov, SINBAD automation of scientīc discovery: From factor analysis to theory synthesis, Natural Computing 3(2) (2004) 207-233.
34. O. Kursun and O. V. Favorov, Single-frame super-resolution by inference from learned features, Istanbul University J. Electr. Electron. Eng. 3 (2003) 673-681.
35. O. Kursun and O. Favorov, Single-frame super-resolution by a cortex based mechanism using high level visual features in natural images, Proc. Sixth IEEE Workshop on Applications of Computer Vision (2002), pp. 112-117.
36. N. Kwak, Feature extraction based on direct calculation of mutual information, Int. J. Patt. Recogn. Artif. Intell. 21 (2007) 1213-1231.
37. N. Kwak and C. H. Choi, Input feature selection by mutual information based on Parzen window, IEEE Trans. Patt. Anal. Mach. Intell. 24 (2002) 1667-1671.
38. Y. Lee, Y. Lin and G. Wahba, Multicategory support vector machines, Department of Statistics, University of Wisconsin, Madison, WI, Technical Report 1043 (2001).
39. D. T. Levin and B. L. Angelone, Categorical perception of race, Perception 31 (2001) 567-578.
40. D. S. Lindsay, P. C. Jack and M. A. Christian, Other-race face perception, J. Appl. Psychol. 76 (1991) 587-589.
41. E. Mjolsness and D. DeCoste, Machine learning for science: State of the art and future prospects, Science 293 (2001) 2051-2055.
42. J. T. Morgan, J. Ham, M. M. Crawford, A. Henneguelle and J. Ghosh, Adaptive feature spaces for land cover classīcation with limited ground truth data, Int. J. Patt. Recogn. Artif. Intell., Special Issue on Fusion of Multiple Classīers, eds. J. Kittler and F. Roli, 18 (2004) 777-799.
43. S. Y. Novak, On Gebelein's correlation coe±cient, Stat. Probab. Lett. 69 (2004) 299-303.
44. L. I. Perlovsky, Conundrum of combinatorial complexity, IEEE Trans. Patt. Anal. Mach. Intell. 20 (1998) 666-670.
45. W. A. Phillips, J. Kay and D. Smyth, The discovery of structure by multi-stream networks of local processors with contextual guidance, Network: Comput. Neural Syst. 6 (1995) 225-246.
46. J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAGs for multiclass classīcation, Neural Information Processing Systems, Vol.12 (MIT Press, Cambridge, MA, 2000), pp. 547-553.
47. A. Renyi, On measures of dependence, Acta Mathematica Academiae Scieniarum Hungaricae 10 (1959) 441-451.
48. D. E. Rumelhart, G. E. Hinton and R. J. Williams, Learning internal representations by error propagation, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, eds. D. E. Rumelhart and J. L. McClelland, PDP Research Group, Vol.1 (MIT Press, Cambridge, MA, 1986), pp. 318-362.
49. D. Ryder, SINBAD neurosemantics: A theory of mental representation, Mind & Language 19 (2004) 211-240.
50. D. Ryder and O. V. Favorov, The new associationism: A neural explanation for the predictive powers of cerebral cortex, Brain and Mind 2 (2001) 161-194.
51. B. Scholkopf and A. Smola, Learning with Kernels (MIT Press, Cambridge, MA, 2002).
52. B. Scholkopf, A. Smola and K. Muller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput. 10 (1998) 1299-1319.
53. C. E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 27 (1948) 379-423, 623-656.
54. F. Y. Shih and K. Zhang, Support vector machine networks for multi-class classīcation, Int. J. Patt. Recogn. Artif. Intell. 19 (2005) 775-786. (Pubitemid 41374103)
55. J. Stone, Learning perceptually salient visual parameters using spatiotemporal smoothness constraints, Neural Comput. 8 (1996) 1463-1492.
56. M. A. Turk and A. Pentland, Eigenfaces for recognition, J. Cogn. Neurosci. 3 (1991) 71-86.
57. V. Vapnik, Statistical Learning Theory (Wiley, NY, 1998).
58. P. M. Walker and J. W. Tanaka, A perceptual encoding advantage for own-versus other-race faces, Perception 32 (2003) 1117-1125.