A unification of component analysis methods

Handbook of Pattern Recognition and Computer Vision, Fourth Edition

Volumn , Issue , 2009, Pages 3-22

  De la Torre, F ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLUSTERING ALGORITHMS; DISCRIMINANT ANALYSIS; LEAST SQUARES APPROXIMATIONS; PROBLEM SOLVING;

CANONICAL CORRELATION ANALYSIS; LEAST SQUARES ESTIMATION; LEAST-SQUARES FORMULATION; LINEAR DISCRIMINANT ANALYSES (LDA); NORMALIZATION FACTORS; ON-LINE LEARNING METHODS; OPTIMIZATION ALGORITHMS; REDUCED RANK REGRESSION;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84969718516     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1142/9789814273398_001     Document Type: Chapter

References (80)

1. I. T. Jolliffe, Principal Comportent Analysis. (New York: Springer-Verlag, 1986).
2. B. Sehölkopf, A. Smola, and K. Muller, Nonlinear component analysis as a kernel eigenvalue problem., Neural Computation. 10, 1299-1319, (1998).
3. R. A. Fisher, The statistieal utilization of multiple measurements. Armais of Eugénies. 8, 376-386, (1938).
4. H. Hotellmg, Relations between two sets of variâtes, Biometrika. 28, 321-377, (1936).
5. B. Mohär, So me applications of Laplace eigenvalues of grap/is. In G. Hahn and G. Sabidussi (Eds.), Graph Symmetry: Alge braie Methods and Applications (Vol. NATO ASI Ser. C 497, p. 225-275). (Kluwer, 1997).
6. F. de la Torre. A least-squares nnified view of pea, Ida, eea and spectral graph methods. In te ch. repon CMU-RI-TR-08-29, Robotics Institute, Carnegie Mellon University, May, (2008).
7. M. Borga. Leaming multidimensional signal processing. In PhD Dissertation. Linkping University, Sweden, (1998).
8. S. Roweis and Z. Ghahramani, A unifying review of linear gaussian models.. Neural Computa-tion. 11(2), 305-345, (1999).
9. S. Yan, D. Xu, B. Zhang, and H. Zhang, Graph embedding: A general framework for dimen-sionality réduction, PAMI. 29(1), 40-51, (2007).
10. K. Fukunaga, Introduction to Statistieal Pattem Récognition, Second Edition. (Academic Press.Boston, MA, 1990).
11. F. de la Toire and T. Kanade. Multimodal oriented discriminant analysis. In International Conference on Machine Learning, pp. 177-184, (2005).
12. T. W. Anderson, Estimating linear restrictions on régression coefficients for multivariate normal distnbutions, Ann. Math. Statist. 12, 327-351, (1951).
13. T. W. Anderson, An Introduction to Multivariate Statistieal Analysis. (2nd ed. Wiley, New York. 1984).
14. S. S. Haykin, Adaptive ñlter theory. (Prentice-Hall, 1996).
15. L. Scharf, TheSVD and reduced rank signa!processing. S\T and SignaI Processing, II. (Else-vier, 1991).
16. K. I. Diamantaras, Principal Component Neural Networks (Therory and Applications). (John Wiley & Sons, 1996).
17. F. de la Tone and M. J. Black. Dynamic coupled component analysis. In Computer Vision and Pattern Récognition, pp. 643-650, (2001).
18. B. Scholkopf and A. Smola, Leaming with Kernels: Support Vector Machines, Regular ization, Optimization, and Beyond. (MIT Press, 2002).
19. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. (Cambridge University Press, 2004).
20. P. Baldi and K. Homik, Neural networks and principal component analysis: Leaming from examples without local minima, Neural Networks. 2, 53-58, (1989).
21. H. Murase and S. K. Nayar, Visual leaming and récognition of 3D objects from appearance. International Journal of Computer vision. 1(14), 5-24, (1995).
22. K. J. Bathe and E. Wilson, Numerical Methods in Finite Element. (Prentice-Hall. Englewood Cliffs, NJ., 1976).
23. G. Golub and C. F. V. Loan. Matrix Computations. (2nd ed. The Johns Hopkins University Press, 1989).
24. A. Buchanan and A. Fitzgibbon. Damped newton algorithms for matrix factorization with miss-ing data. In IEEE Conference on Computer Vision and Pattern Récognition, (2005).
25. F. de la Ton e and M. J. Black, A framework for robust subspace leaming. Liternational Journal of Computer Vision. 54, 117-142, (2003).
26. R. Fletcher, Practica! methods of optimization. (John Wiley and Sons., 1987).
27. A. Blake and A. Zissennan, Visual Reconstruction. (MIT Press senes, Massachusetts, 1987).
28. K. Pearson, On lines and planes of closest fit to systems of pomts in space, The London. Edinburgh and Dublin Philosoph ical Magazine and Journal. 6, 559-572, (1901).
29. H. Hotelling, Analysis of a complex of statistieal variables into principal components, Journal of Educational Psychology. 24, (1933).
30. E. Oja, A simplified neuron model as principal component analyzer, Journal of Mathematical Biology. 15, 267-273, (1982).
31. S. Roweis. EM algorithms for PC'A and SPC'A. In NWS, pp. 626-632, (1997).
32. K. R. Gabriel and S. Zamir, Lower rank approximation of matrices by least squares with any choice of weights, Technometrics, Vol. 21, pp. 21, 489-498, (1979).
33. H. Shum, K. Ikeuchi, and R. Reddv. Principal component analysis with missing data and its application to polyhedral object modeling, Pattem Analysis and Machine Intelligence. 17(9), 855-867, (1995).
34. M. Tippmg and C. M. Bishop, Probabilistic prmeipal component analysis, Journal of the Royal Statistical Society B. 61, 611-622, (1999).
35. M. Iraní and P. Anandan. Factorization with uncertamty. In European Conference on Computer Vision, pp. 539-553, (2000).
36. M. J. Greenacre, Tlieory and Applications of Correspondence Analysis. (Acadennc Press, London. 1984).
37. I. Tsang and J. Kwok. Distance metne leaming with kemels. In Proceedings of the Liternational Conference on Artificial Neural Networks, pp. 126-129 (June, 2003).
38. R. Hartley and F. Sehaffalitzkv. Powerfactonzation: an approach to affine reconstniction with missing and imcertain data. In Australia-Japan Advance Wbrkshop on Computer Vision, (2003).
39. D. Skocaj and A. Leonardis. Weighted and robust incrémental method for subspace leaming. In International Conference on Computer Vision ICCl. pp. 1494-1501, (2003).
40. S. Fidler, D. Skocaj, and A. Leonardis, Combming reconstmetive and discriminative subspace methods for robust classification and régression by subsampling, IEEE Transactions on Pattern Analysis and Machine Litelligence. 28(3), 337-350, (2006).
41. P. Aguiar, M. Stosic, and J. Xavier. Spectrally optimal factonzation of mcomplete matrices. In IEEE Computer Vision and Pattern Récognition, (2008).
42. A. Martmez and A. Kak, Pca versus Ida. IEEE Transactions on Pattern Analysis and Machine Intelligence. 23(2), 228-233, (2003).
43. P. Belhumeur, J. Hespanha, and D. Knegman. Eigenfaces vs. fisherfaces: Récognition using dass specific linear projection, IEEE Transactions on Pattern Analysis and Machine Litelligence. (19), 711-720, (1997).
44. T. Hastie, R. Tibshiram, and J. Fnedman, The Elements of Statistical Leaming: Data Mining. Inferen ce and Prédiction. (Springer, 2001).
45. W. Zhao. Discriminant component analysis for face récognition. In ICPR, pp. 818-821, (2000).
46. J. Hoffbeck and D. Landgrebe, Covariance matrix estimation and classification with limited training data. IEEE Transactions on Pattern Analysis and Machine Intelligence. 18(7), 763-767, (1996).
47. J. Ye, Characterization of a family of algorithms for generalized discriminant analysis on un-dersampled problems, The Journal of Machine Learning Research. 6(1), 483-502 (September, 2005).
48. S. Zhang and T. Sim. Discriminant subspace analysis: A fukunaga-koontz approach., PAMÍ 29, 1732-1745, (2007).
49. R. O. Duda, P. E. Hait, and D. G. Stork, Pattern Classification. (John Wiley and Sons Inc., 2001).
50. P. Gallinari, S. Thiria, F. Badran, and F. Fogelman-Soulie, On the relations between discriminant analysis and multilayer perceptrons. Neural Networks. 4, 349-360, (1991).
51. J. Ye. Least squares linear discriminant analysis. In ICML, pp. 1087-1093, (2007).
52. F. de la Torre and T. Kanade. Discriminative cluster analysis. In Liternational Conference on Machine Learning; vol. 148, pp. 241-248, New York, NY, USA (June, 2006). ACM Press.
53. S. Mika. Kernel fisher discriminants. In PhD thesis, University of Technology; Berlin, (2002)
54. T.-K. Kim, J. Kittler, and R. Cipolla., Discriminative leaming and récognition of image set classes using canonical corrélations, IEEE Trans, on PAMI. 29(6), 1005-1018, (2007).
55. D. Skocaj and A. Leonardis. Appearance-based localization using cca. In Computer Vision Winter Wbrkshop, (2000).
56. T. Melzer, M. Reiter, and H. Bischof, Appearance models based on kernel canonical corrélation analysis, Pattern Récognition. 36(9), 1961-1971, (2003).
57. K. Mardia, J. Kent, and J. Bibby, Multivariate Analysis. (Academie Press. London, 1979).
58. S. Cherry, Singular value décomposition analysis and canonical corrélation analysis., J. Clima te. (9), 2003-2009, (1997).
59. V. J. Yohai and M. S. Garcia, Canonical variables as optimal predictors., The Armais ofStatistics. 8(4), 865-869, (1980).
60. M. Tso, Reduced-rank régression and canonical analyis., Journal of the Royal Statistieal Society. Series B. 43(2), 183-189, (1981).
61. M. Loog, R. Duin, and R. Hacb-Umbach, Multiclass linear dimension réduction by weighted pairwise fisher entena, IEEE Transactions on Pattern Analysis and Machine Intelligence. 7(23), 762-766, (2001).
62. J. B. MacQueen. Some methods for classification and analysis of multivariate observations. In 5-th Berkeley Symposium on Mathematical Statistics and Probability. Berkeley University of California Press., pp. 1:281-297, (1967).
63. A. K. Jain, Algorithms For Clustering Data. (Prentice Hall, 1988).
64. H. Zha, C. Ding, M. Gu, X. He, and H. Simon. Spectral relaxation for k-means clustering. In Neural Information Processing Systems, pp. 1057-1064, (2001).
65. C. Ding and X. He. K-means clustering via principal component analysis. In International Con-ference on Machine Learning, vol. 1, pp. 225-232, (2004).
66. R. Zass and A. Shashua. A unifying approach to hard and probabilistic clustering. In International Conference on Computer Vision. Beijing, (2005).
67. J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence. 22(8), 888-905 (Aug., 2000).
68. A. Y. Ng, M. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In Neural Infonnation Processing Systems, number 14, (2002).
69. S. Yu and J. Shi. Multiclass spectral clustering. In Proceedings of International Conference on Computer Vision, pp. 335-336 (Nice, France, October, 2003).
70. F. K. Chung, Spectral Graph Theory. (C'BMS Regional Conference Series in Mathemtics, vol 92, American Mathematical Society, Providence, 1997).
71. L. Hagen and A. Kahng, New spectral methods for ratio eut partitioning and clustering., IEEE. Trans, on Computed Aided Desgin. (11), 1074-1085, (1992).
72. D. Verma and M. Meila. C'ompanson of spectral clustermg methods. In NIPS, (2003).
73. R. Zass and A. Shashua. Doubly stochastic nonnalization for spectral clustering. In Neural In-fonnation Processing Systems, (2006).
74. M. Filippone, F. Camastra, F. Masulli, and S. Rovetta, A survey of kemel and spectral methods for clustering. Pattem Récognition. 41(1), 176-190, (2008).
75. I. S. Dhilloil, Y. Guan, and B. Kulis. A imified view of kemel k-means, spectral clustering and graph partitioning. In UTCS Tech. Report TR-04-25, (2004).
76. Y. Weiss. Segmentation using eigenvectors: a unifying view. In ICCV, (1999).
77. D. Tolliver. Spectral roimdmg and image segmentation. Technical report, doctoral dissertation, tech. report CMU-RI-TR-06-44, Robotics Institute, Carnegie Mellon University (August, 2006).
78. A. Rahimi and B. Recht. Clustermg with nonnalized cuts îs clustering with a hyperplane. In Statistical Leaming in Computer Vision, (2004).
79. Y. Bengio, P. Vincent, J. Paiement, P. Vincent, and M. Oumiet, Leaming eigenfunctions links spectral embedding and kemel pea. Neural Computation. (16), 2197-2219, (2004).
80. C. Ding, X. He, H. Zlia, M. Gu, and H. Simon. A min-max eut algontlun for graph partitioning and data clustering. In Proc. IEEE International Conference on Data Mining, (2001).