Mixture subclass discriminant analysis link to restricted Gaussian model and other generalizations

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 1, 2013, Pages 8-21

  Gkalelis, Nikolaos ^a,b   Mezaris, Vasileios ^a   Kompatsiaris, Ioannis ^b   Stathaki, Tania ^b  

^a Information Technologies Institute Centre for Research and Technology Hellas (CERTH)   (Greece)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Classification; Clustering; Discriminant analysis; Feature extraction; Machine learning; Mixture of gaussians; Pattern recognition; Probabilistic algorithms

Indexed keywords

CLUSTERING; EXPECTATION MAXIMIZATION; ITERATIVE ALGORITHM; LINEAR DISCRIMINANT ANALYSIS; MAXIMUM LIKELIHOOD ESTIMATE; MIXTURE OF GAUSSIANS; PROBABILISTIC ALGORITHM; SUBCLASS DISCRIMINANT ANALYSES (SDA);

ALGORITHMS; CLASSIFICATION (OF INFORMATION); FACE RECOGNITION; FEATURE EXTRACTION; GAUSSIAN DISTRIBUTION; IMAGE SEGMENTATION; LEARNING SYSTEMS; MAXIMUM LIKELIHOOD ESTIMATION; MIXTURES; PATTERN RECOGNITION;

DISCRIMINANT ANALYSIS;

EID: 84882879142     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2216545     Document Type: Article

References (58)

1. Y. Ephraim and H. L. V. Trees, "A signal subspace approach for speech enhancement," IEEE Trans. Speech Audio Process., vol. 3, no. 4, pp. 251-266, Jul. 1995.
2. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. San Diego, CA: Academic, 1990.
3. R. Duda, P. Hart, and D. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
4. S. Ji and J. Ye, "Generalized linear discriminant analysis: A unified framework and efficient model selection," IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1768-1782, Oct. 2008.
5. C. B. Moler and G. W. Stewart, "An algorithm for generalized matrix eigenvalue problems," SIAM J. Numer. Anal., vol. 10, no. 2, pp. 241-256, Apr. 1973.
6. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs. fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711-720, Jul. 1997.
7. H. Li, T. Jiang, and K. Zhang, "Efficient and robust feature extraction by maximum margin criterion," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 157-165, Jan. 2006. (Pubitemid 43263948)
8. C. S. Dhir and S.-Y. Lee, "Discriminant independent component analysis," IEEE Trans. Neural Netw., vol. 22, no. 6, pp. 845-857, Jun. 2011.
9. K. Muller, S. Mika, G. Ratsch, S. Tsuda, and B. Scholkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 181-202, Mar. 2001.
10. L. Wang, K. L. Chan, P. Xue, and L. Zhou, "A kernel-induced space selection approach to model selection in KLDA," IEEE Trans. Neural Netw., vol. 19, no. 12, pp. 2116-2131, Dec. 2008.
11. Z. Li, D. Lin, and X. Tang, "Nonparametric discriminant analysis for face recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 4, pp. 755-761, Apr. 2009.
12. J. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, "Face recognition using kernel direct discriminant analysis algorithms," IEEE Trans. Neural Netw., vol. 14, no. 1, pp. 117-126, Jan. 2003.
13. S. Zafeiriou, G. Tzimiropoulos, M. Petrou, and T. Stathaki, "Regularized kernel discriminant analysis with a robust kernel for face recognition and verification," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 3, pp. 526-534, Mar. 2012.
14. Z. Fan, Y. Xu, and D. Zhang, "Local linear discriminant analysis framework using sample neighbors," IEEE Trans. Neural Netw., vol. 22, no. 7, pp. 1119-1132, Jul. 2011.
15. B.-C. Kuo and K.-Y. Chang, "Feature extractions for small sample size classification problem," IEEE Trans. Geosci. Remote Sens., vol. 45, no. 3, pp. 756-764, Mar. 2007. (Pubitemid 46375747)
16. T. Hastie and R. Tibshirani, "Discriminant analysis by Gaussian mixtures," J. Royal Stat. Soc., Ser. B, vol. 58, no. 1, pp. 155-176, Jul. 1996.
17. M. Zhu and A. Martinez, "Subclass discriminant analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 8, pp. 1274-1286, Aug. 2006.
18. S.-W. Kim and R. P. W. Duin, "On using a pre-clustering technique to optimize LDA-based classifiers for appearance-based face recognition," in Proc. 12th Iberoamer. Congr. Pattern Recognit., Vina del Mar-Valparaiso, Chile, Nov. 2007, pp. 466-476.
19. M. H. Yang, D. J. Kriegman, and N. Ahuja, "Face detection using multimodal density models," Comput. Vis. Image Understand., vol. 84, no. 2, pp. 264-284, Oct. 2001. (Pubitemid 34222298)
20. A. Pnevmatikakis and L. Polymenakos, "Subclass linear discriminant analysis for video-based face recognition," J. Visual Commun. Image Represent., vol. 20, no. 8, pp. 543-551, Nov. 2009.
21. L. Clemmensen, T. Hastie, D. Witten, and B. Ersboll, "Sparse discriminant analysis," Technometrics, vol. 53, no. 4, pp. 406-413, Nov. 2011.
22. D. Wu and K. L. Boyer, "Resilient subclass discriminant analysis," in Proc. IEEE Int. Conf. Comput. Vis., Kyoto, Japan, Sep.-Oct. 2009, pp. 389-396.
23. F. Oveisi, "Subclass discriminant analysis using dynamic cluster formation for EEG-based brain-computer interface," in Proc. IEEE/EMBS 4th Int. Conf. Neural Eng., Antalya, Turkey, May 2009, pp. 303-306.
24. S.-W. Kim, "A pre-clustering technique for optimizing subclass discriminant analysis," Pattern Recognit. Lett., vol. 31, no. 6, pp. 462-468, Apr. 2010.
25. N. Gkalelis, V. Mezaris, and I. Kompatsiaris, "High-level event detection in video exploiting discriminant concepts," in Proc. 9th Int. Workshop Content-Based Multimedia Index., Madrid, Spain, Jun. 2011, pp. 85-90.
26. N. Gkalelis, V. Mezaris, and I. Kompatsiaris, "Mixture subclass discriminant analysis," IEEE Signal Process. Lett., vol. 18, no. 5, pp. 319-332, May 2011.
27. K. V. Mardia, J. T. Kent, and J. M. Bibby, Multivariate Analysis. New York: Academic, 1979.
28. N. A. Campbell, "Canonical variate analysis - A general model formulation," Austral. New Zealand J. Stat., vol. 26, no. 1, pp. 86-96, 1984.
29. D. Tao, X. Li, X. Wu, and S. J. Maybank, "Geometric mean for subspace selection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 260-274, Feb. 2009.
30. R. Lotlikar and R. Kothari, "Fractional-step dimensionality reduction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 6, pp. 623-627, Jun. 2000.
31. M. Loog, R. P. W. Duin, and R. Haeb-Umbach, "Multiclass linear dimension reduction by weighted pairwise Fisher criteria," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 7, pp. 762-766, Jul. 2001. (Pubitemid 32680763)
32. B. Chen, L. Yuan, H. Liu, and Z. Bao, "Kernel subclass discriminant analysis," Neurocomputing, vol. 71, nos. 1-3, pp. 455-458, Dec. 2007.
33. D. You, O. C. Hamsici, and A. M. Martinez, "Kernel optimization in discriminant analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 3, pp. 631-638, Mar. 2011.
34. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
35. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Royal Stat. Soc., Ser. B, Methodol., vol. 39, no. 1, pp. 1-38, 1977.
36. G. Mclachlan and D. Peel, Finite Mixture Models. New York: Wiley, 2000.
37. C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics). Secaucus, NJ: Springer-Verlag, 2006.
38. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
39. N. Vlassis and A. Likas, "A kurtosis-based dynamic approach to Gaussian mixture modeling," IEEE Trans. Syst., Man, Cybern. A, Syst. Humans, vol. 29, no. 4, pp. 393-399, Jul. 1999.
40. L. Wang and J. Ma, "A kurtosis and skewness based criterion for model selection on Gaussian mixture," in Proc. 2nd Int. Conf. Biomed. Eng. Inf., Tianjin, China, Oct. 2009, pp. 1-5.
41. I. T. Jolliffe, Principal Component Analysis . New York: Springer-Verlag, Oct. 2002.
42. A. Frank and A. Asuncion. (2010). UCI Machine Learning Repository [Online]. Available: http://archive.ics.uci.edu/ml
43. Gunnar Raetsch's Benchmark Datasets . (2012, May 1) [Online]. Availale: http://theoval.cmp.uea.ac.uk/~gcc/matlab/default.html#benchmarks
44. B. Leibe and B. Schiele, "Analyzing appearance and contour based methods for object categorization," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Madison, WI, Jun. 2003, pp. 409-415.
45. N. Gkalelis, V. Mezaris, and I. Kompatsiaris, "Automatic event-based indexing of multimedia content using a joint content-event model," in Proc. ACM Multimedia, Firenze, Italy, Oct. 2010, pp. 15-20.
46. D. B. Graham and N. M. Allinson, "Characterizing virtual eigensignatures for general purpose face recognition," in Face Recognition: From Theory to Applications, Computer and Systems Sciences, vol. 163, H. Wechsler, P. Phillips, V. Bruce, F. Fogelman-Soulie, and T. S. Luang, Eds. Berlin, Germany: Springer-Verlag, 1998, pp. 446-456.
47. F. Samaria and A. Harter, "Parameterisation of a stochastic model for human face identification," in Proc. 2nd IEEE Workshop Appl. Comput. Vis., Sarasota, FL, Dec. 1994, pp. 138-142.
48. K. Lee, J. Ho, and D. Kriegman, "Acquiring linear subspaces for face recognition under variable lighting," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 684-698, May 2005. (Pubitemid 40608402)
49. T. Sim, S. Baker, and M. Bsat, "The CMU pose, illumination, and expression database," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
50. D. Cai, X. He, Y. Hu, J. Han, and T. S. Huang, "Learning a spatially smooth subspace for face recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Minneapolis, MN, Jun. 2007, pp. 138-142.
51. Four Face Databases in MATLAB Format. (2012, May 1) [Online]. Available: http://http://www.cad.zju.edu.cn/home/dengcai/Data/FaceData.html
52. G. Rätsch, T. Onoda, and K.-R. Müller, "Soft margins for AdaBoost," Mach. Learn., vol. 42, no. 3, pp. 287-320, Mar. 2001.
53. C. E. Thomaz, D. F. Gillies, and R. Q. Feitosa, "A new covariance estimate for Bayesian classifiers in biometric recognition," IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 2, pp. 214-223, Feb. 2004.
54. Q. McNemar, "Note on the sampling error of the difference between correlated proportions or percentages," Psychometrika, vol. 12, no. 2, pp. 153-157, Jun. 1947.
55. S. Zafeiriou, G. Tzimiropoulos, M. Petrou, and T. Stathaki, "Regularized kernel discriminant analysis with a robust kernel for face recognition and verification," IEEE Trans. Neural Netw., vol. 23, no. 3, pp. 526-534, Mar. 2012.
56. Y. Aksu, D. J. Miller, G. Kesidis, and Q. X. Yang, "Margin- maximizing feature elimination methods for linear and nonlinear kernel-based discriminant functions," IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 701-717, May 2010.
57. F. Song, D. Mei, and H. Li, "Feature selection based on linear discriminant analysis," in Proc. IEEE Int. Conf. Intell. Syst. Design Eng. Appl., vol. 1. Changsha, China, Oct. 2010, pp. 746-749.
58. S. Huh and D. Lee, "Linear discriminant analysis for signatures," IEEE Trans. Neural Netw., vol. 21, no. 12, pp. 1990-1996, Dec. 2010.