Class-dependent PCA, MDC and LDA: A combined classifier for pattern classification

Pattern Recognition

Volumn 39, Issue 7, 2006, Pages 1215-1229

  Sharma, Alok ^a   Paliwal, Kuldip K ^a   Onwubolu, Godfrey C ^b  

^a GRIFFITH UNIVERSITY   (Australia)

^b UNIVERSITY OF THE SOUTH PACIFIC   (Fiji)

Author keywords

Class dependent PCA; Classification accuracy; LDA; Processing time; Total parameter requirement

Indexed keywords

DATA PROCESSING; LEARNING SYSTEMS; PRINCIPAL COMPONENT ANALYSIS;

LINEAR DISCRIMINANT ANALYSIS (LDA); MINIMUM DISTANCE CLASSIFIER (MDC); PATTERN CLASSIFIERS; PROCESSING TIME;

PATTERN RECOGNITION;

EID: 33748695776     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2006.02.001     Document Type: Article

References (60)

1. Jain A.K., Duin R.P.W., and Mao J. Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. 22 (2000) 4-37
2. Di Maio V., and Marciano F. Automatic classification of neural spike activity: an application of minimum distance classifiers. Cybern. Syst. 34 3 (2003) 173-192
3. Paclik P., and Duin R.P.W. Dissimilarity-based classification of spectra: computational issues. Real-Time Imaging 9 (2003) 237-244
4. F. Sahin, A radial basis function approach to a color image classification problem in a real time industrial application, Ph.D. Thesis, State University, Virginia, 2000.
5. P. Datta, D. Kibler, Symbolic nearest mean classifiers, Proceedings of the 14th National Conference on Artificial Intelligence, San Mateo, CA, 1997, pp. 82-87.
6. S. Griguolo, Pixel-by-pixel clustering for vegetation monitoring, International Conference on Alerte précoce et suivi de l'Environment, Niamey, Niger, 1994.
7. Lewenstein K., and Chojnacki M. Minimum distance classifiers in coronary artery disease diagnosing (2004), Modelling in Mechatronics, Kazimierz Dolny, Poland
8. T. Lambrou, A.D. Linney, R.D. Speller, A. Todd-Pokropek, Statistical classification of digital mamograms using features from the spatial and wavelet domains, Medical Image Understanding and Analysis, Portsmouth, UK, 22-23 July, 2002.
9. D. Toth, A. Condurache, T. Aach, A two-stage-classifier for defect classification in optical media inspection, 16th International Conference on Pattern Recognition (ICPR'02), vol. 4, 2002, pp. 373-376.
10. Oja E. Subspace Methods of Pattern Recognition (1983), Research Studies Press, New York
11. E. Oja, J. Parkkinen, On subspace clustering, Seventh International Conference on Pattern Recognition, vol. 2, 1984, pp. 692-695.
12. Fukunaga K. Introduction to Statistical Pattern Recognition (1990), Academic Press Inc., Hartcourt, Brace and Jovanovich, New York
13. Dony R.D., and Haykin S. Image segmentation using a mixture of principal components representation. IEE Proc. Vision, Image Signal Process. 144 2 (1997) 73-80
14. Duda R.O., and Hart P.E. Pattern Classification and Scene Analysis (1973), Wiley, New York
15. Swets D.L., and Weng J. Using discriminative eigenfeatures for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 18 8 (1996) 831-836
16. Belhumeur P.N., Hespanha J.P., and Kriegman D.J. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19 7 (1997) 711-720
17. W. Zhao, R. Chellappa, P.J. Phillips, Subspace linear discriminant analysis for face recognition, Technical Report CAR-TR-914, Center for Automation Research, University of Maryland, College Park, 1999.
18. W. Zhao, R. Chellappa, N. Nandhakumar, Empirical performance analysis of linear discriminant classifiers, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1998, pp. 164-169.
19. Yang J., Frangi A.F., Yang J.-y., Zhang D., and Zhong J. KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 27 2 (2005) 230-244
20. Mika S., Rätsch G., Weston J., Schölkopf B., Smola A., and Müller K.-R. Constructing descriptive and discriminative nonlinear features: Rayleigh coefficients in kernel feature spaces. IEEE Trans. Pattern Anal. Mach. Intell. 25 5 (2003) 623-628
21. X.-J. Wu, J. Kittler, J.-Y. Yang, K. Messerm S. Wang, A new direct LDA (D-LDA) algorithm for feature extraction in face recognition, Proceedings of the International Conference on Pattern Recognition vol. 4, 2004, pp. 545-548.
22. H. Jian, P.C. Yuen, C. Wen-Sheng, A novel subspace LDA algorithms for recognition of face images with illumination and pose variations, Proceedings of the International Conference on Machine Learning and Cybernetics, vol. 6, 2004, pp. 3589-3594.
23. Xiaogang W., and Xiaoou T. A unified framework for subspace face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26 9 (2004) 1222-1228
24. Ye J., Janardan R., Cheong H.P., and Haesun P. An optimization criterion for generalized discriminant analysis on undersampled problems. IEEE Trans. Pattern Anal. Mach. Intell. 26 8 (2004) 982-994
25. W. Xiaogang, T. Xiaoou, Using random subspace to combine multiple features for face recognition, Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004, pp. 284-289.
26. K. Tae-Kyun, K. Hyunwoo, H. Wonjun, K. Seok-Cheol, J. Kittler, Face description based on decomposition and combining of a facial space with LDA, Proceedings of the International Conference on Image Processing, vol. 3, 2003, pp. III-877-III-880.
27. R. Haeb-Umbach, H. Ney, Linear discriminant analysis for improved large vocabulary continuous speech recognition, Proceedings of the International Conference on Acoustics, Speech and Signal Processing, vol. 1, 1992, pp. 13-16.
28. O. Siohan, On the robustness of linear discriminant analysis as a preprocessing step for noisy speech recognition, Proceedings of the International Conference on Acoustics, Speech and Signal Processing, vol. 1, 1995, pp. 125-128.
29. M. Lieb, R. Haeb-Umbach, LDA derived cepstral trajectory filters in adverse environment conditions, Proceedings of the International Conference on Acoustics, Speech and Signal Processing, vol. 2, 2000, pp. II1105-II1108.
30. G. Potamianos, H.P. Graf, Linear discriminant analysis for speech reading, IEEE Second Workshop on Multimedia Signal Processing, 1998, pp. 221-226.
31. Chen X., Yang J., Zhang J., and Waibel A. Automatic detection and recognition of signs from natural scenes. IEEE Trans. Image Process. 13 1 (2004) 87-99
32. Xu L., Krzyżak A., and Suen C.Y. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Sys. Man Cybern. 22 3 (1992) 418-435
33. Jacobs R.A., Jordan M.I., Nowlan S.J., and Hinton G.E. Adaptive mixtures of local experts. Neural Comput. 3 (1991) 79-87
34. Ho T.K., Hull J.J., and Srihari S.N. Decision combination in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 16 1 (1994) 66-75
35. Tresp V., and Taniguchi M. Combining estimators using non-constant weighting functions. In: Tesauro G., Touretzky D.S., and Leen T.K. (Eds). Advances in Neural Information Processing Systems vol. 7 (1995), MIT Press, Cambridge, MA
36. K. Woods, K. Bowyer, W.P. Kegelmeyer, Combination of multiple classifiers using local accuracy estimates, IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR '96, 1996, pp. 391-396.
37. Woods K. Combination of multiple classifiers using local accuracy estimates. IEEE Trans. Pattern Anal. Mach. Intell. 19 4 (1997) 405-410
38. L. Zhou, S. Imai, Chinese all syllables recognition using combination of multiple classifiers, ICASSP, vol. 6, 1996, pp. 3494-3497.
39. F. Alimoglu, E. Alpaydin, Combining multiple representations and classifiers for pen-based handwritten digit recognition, International Conference on Document Analysis and Recognition, vol. 2, 1997, pp. 637-640.
40. J. Kittler, M. Hatef, R.P.W. Duin, J. Matas, On combining classifiers, International Conference on Pattern Recognition, vol. 2, 1996, pp. 897-901.
41. Kittler J., Hatef M., Duin R.P.W., and Matas J. On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20 3 (1998) 226-239
42. M. Breukelen van, R.P.W. Duin, Neural network initialization by combined classifiers, International Conference on Pattern Recognition, vol. 1, 1998, pp. 215-218.
43. L.A. Alexandre, A.C. Campilho, M. Kamel, Combining independent and unbiased classifiers using weighted average, International Conference on Pattern Recognition, vol. 2, 2000, pp. 495-498.
44. Turner K., and Gosh J. Linear and order statistics combiners for pattern classification. In: Sharkey A. (Ed). Combining Artificial Neural Nets (1999), Springer, Berlin 127-161
45. Ueda N. Optimal linear combination of neural networks for improving classification performance. IEEE Trans. Pattern Anal. Mach. Intell. 22 2 (2000) 207-215
46. Senior A. Combination fingerprint classifier. IEEE Trans. Pattern Anal. Mach. Intell. 23 10 (2001) 1165-1174
47. L. Lei, W. Xiao-Long, L. Bing-Quan, Combining multiple classifiers based on statistical method for handwritten Chinese character recognition, International Conference on Machine Learning and Cybernetics, vol. 1, 2002, pp. 252-255.
48. M. Yao, X. Pan, T. He, R. Zhang, An improved combination method of multiple classifiers based on fuzzy integrals, World Congress on Intelligent Control and Automation, vol. 3, 2002, pp. 2445-2447.
49. C.L. Blake, C.J. Merz, UCI repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, CA, 1998 〈http://www.ics.uci.edu/ ∼mlearn 〉.
50. D. Michie, D.J. Spiegelhalter, C.C. Taylor (Eds.), Machine Learning, Neural and Statistical Classification, Ellis Horwood, Chichester, 1994.
51. X.S. Zhou, T.S. Huang, Small sample learning during multimedia retrieval using BiasMap, Proceedings of the IEEE CS Conference on Computer Vision and Pattern Recognition, vol. 1, 2001, pp. 11-17.
52. R.P.W. Duin, Small sample size generalization, in: G. Borgefors (Ed.), SCIA'95, Proceedings of the Ninth Scandinavian Conference on Image Analysis, vol. 2, 1995, pp. 957-964.
53. M. Skurichina, R.P.W. Duin, Stabilizing classifiers for very small sample size, Proceedings of the International Conference on Pattern Recognition, vol. 2, 1996, pp. 891-896.
54. Skurichina M., and Duin R.P.W. Regularization of linear classifiers by adding redundant features. Pattern Anal. Appl. 2 1 (1999) 44-52
55. Raudys S., and Duin R.P.W. Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix. Pattern Recognition Lett. 19 (1998) 385-392
56. C. Feng, A. Suntherland, S. King, S. Muggleton, R. Henry, Comparison of machine learning classifiers to statistics and neural networks, AI & Stats Conference, 1993, pp. 41-52.
57. S.G. Garofalo, L.F. Lori, F.M. William, F.G. Jonathan, P.S. David, D.L. Nancy, The DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus CDROM, NIST, 1986.
58. Sharma A., Paliwal K.K., and Onwubolu G.C. Pattern classification: an improvement using combination of VQ and PCA based techniques. Am. J. Appl. Sci. 2 10 (2005) 1445-1555
59. N. Kambhatla, Local models and Gaussian mixture models for statistical data processing, Ph.D. Thesis, Oregon Graduate Institute of Science and Technology, 1996.
60. Young S., Evermann G., Hain T., Kershaw D., Moore G., Odell J., Ollason D., Povey D., Valtchev V., and Woodland P. The HTK Book Version 3.2 (2002), Cambridge University, Cambridge, England