Regression reformulations of LLE and LTSA with locally linear transformation

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 41, Issue 5, 2011, Pages 1250-1262

  Xiang, Shiming ^a   Nie, Feiping ^b   Pan, Chunhong ^a   Zhang, Changshui ^b  

^a INSTITUTE OF AUTOMATION   (China)

^b TSINGHUA UNIVERSITY   (China)

Author keywords

Improved locally linear embedding (LLE) (ILLE); LLE; local tangent space alignment (LTSA); regression reformulation

Indexed keywords

LINEAR ALIGNMENTS; LINEAR METHODS; LINEAR RECONSTRUCTION; LLE; LOCAL TANGENT SPACE ALIGNMENT; LOCALLY LINEAR EMBEDDING; LOW-DIMENSIONAL SPACES; MANIFOLD LEARNING; REGRESSION REFORMULATION; TANGENT SPACE; UNIFIED FRAMEWORK;

ALIGNMENT; LEARNING ALGORITHMS; MATHEMATICAL TRANSFORMATIONS; REGRESSION ANALYSIS;

LINEAR TRANSFORMATIONS;

EID: 80052903676     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2011.2123886     Document Type: Article

References (65)

1. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples," J. Mach. Learn. Res., vol. 7, pp. 2399-2434, 2006. (Pubitemid 44708005)
2. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput., vol. 15, no. 6, pp. 1373-1396, Jun. 2003. (Pubitemid 37049796)
3. M. Brand, "Charting a manifold," in Advances in Neural Information Processing Systems 15. Cambridge,MA: MIT Press, 2003, pp. 961-968.
4. A. Brun, H. J. Park, H. Kuntsson, and C. F. Westin, "Coloring of DT-MRI fiber traces using Laplacian eigenmaps," in Proc. Int. Conf. Comput. Aided Syst. Theory, Las Palmas, Spain, 2003, pp. 518-529. (Pubitemid 37342488)
5. D. Cai, "Spectral regression: A regression framework for efficient regularized subspace learning," Ph.D. dissertation, Dept. Comput. Sci., Univ. Illinois Urbana-Champaign, Urbana, IL, May 2009.
6. D. Cai, X. He, and J. Han, "Spectral regression for efficient regularized subspace learning," in Proc. Int. Conf. Comput. Vis., Rio de Janeiro, Brazil, 2007, pp. 1-8.
7. H. Chang and D.-Y. Yeung, "Robust locally linear embedding," Pattern Recognit., vol. 39, no. 6, pp. 1053-1065, Jun. 2006.
8. H. Chang, D. Yeung, and Y. Xiong, "Super-resolution through neighbor embedding," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., Washington, DC, 2004, pp. 275-282.
9. C. Chen, J. Zhang, and R. Fleischer, "Distance approximating dimension reduction of Riemannian manifolds," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 1, pp. 208-217, Feb. 2010.
10. V. de Silva and J. B. Tenenbaum, "Global versus local methods in nonlinear dimensionality reduction," in Advances in Neural Information Processing Systems 15. Cambridge,MA: MIT Press, 2003, pp. 721-728.
11. P. Dollar, V. Rabaud, and S. Belongie, "Learning to traverse image manifolds," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 361-368.
12. P. Dollar, V. Rabaud, and S. Belongie, "Non-isometric manifold learning: Analysis and an algorithm," in Proc. Int. Conf. Mach. Learn., Corvallis, OR, 2007, pp. 241-248.
13. D. L. Donoho and C. Grimes, "Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data," Proc. Nat. Acad. Arts Sci., vol. 100, no. 10, pp. 5591-5596, 2003. (Pubitemid 36576852)
14. A. M. Farahmand, C. Szepesvari, and J.-Y. Audibert, "Manifold- adaptive dimension estimation," in Proc. Int. Conf. Mach. Learn., Corvallis, OR, 2007, pp. 265-272.
15. X. Geng, D.-C. Zhan, and Z.-H. Zhou, "Supervised nonlinear dimensionality reduction for visualization and classification," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 6, pp. 1098-1107, Dec. 2005. (Pubitemid 41800411)
16. S. Gerber, T. Tasdizen, and R. Whitaker, "Robust non-linear dimensionality reduction using successive 1-dimensional Laplacian eigenmaps," in Proc. Int. Conf. Mach. Learn., Corvallis, OR, 2007, pp. 281-288.
17. Y. Goldberg, A. Zakai, D. Kushnir, and Y. Ritov, "Manifold learning: The price of normalization," J. Mach. Learn. Res., vol. 9, pp. 1909-1939, 2008.
18. A. Gorban, B. Kegl, D. Wunsch, and A. Zinovyev, Principal Manifolds for Data Visualization and Dimension Reduction. Berlin, Germany: Springer-Verlag, 2007.
19. J. Ham, D. D. Lee, S. Mika, and B. Schokopf, "A kernel view of the dimensionality reduction of manifolds," in Proc. Int. Conf. Mach. Learn., Banff, AB, Canada, 2004, pp. 369-376. (Pubitemid 40290830)
20. X. He and P. Niyogi, "Locality preserving projections," in Proc. Annu. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, 2003, pp. 585-591.
21. M. Hein and M. Maier, "Manifold denoising," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 1-8.
22. D. Huang, Z. Yi, and X. Pu, "Manifold-based learning and synthesis," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 3, pp. 592-606, Jun. 2009.
23. O. C. Jenkins and M. J. Mataric, "A spatio-temporal extension to Isomap nonlinear dimension reduction," in Proc. Int. Conf. Mach. Learn., Banff, AB, Canada, 2004, pp. 441-448. (Pubitemid 40290839)
24. O. Kouropteva, O. Okun, and M. Pietikainen, "Selection of the optimal parameter value for the locally linear embedding algorithm," in Proc. 1st Int. Conf. Fuzzy Syst. Knowl. Discov., Singapore, 2007, pp. 359-363.
25. J. A. Lee and M. Verleysen, Nonlinear Dimensionality Reduction. New York: Springer-Verlag, 2007.
26. A. Levin, D. Lischinski, and Y. Weiss, "A closed form solution to natural image matting," in Proc. Int. Conf. Comput. Vis. Pattern Recog., New York, 2006, pp. 61-68. (Pubitemid 44931313)
27. H. Li and X. Li, "Improved LLE algorithm for motion analysis," in Proc. 7th Int. Symp. Adv. Parallel Process. Technol., Guangzhou, China, 2007, pp. 733-742.
28. X. Li, S. Lin, S. Yan, and D. Xu, "Discriminant locally linear embedding with high-order tensor data," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 2, pp. 342-352, Apr. 2008. (Pubitemid 351479557)
29. T. Lin and H. B. Zha, "Riemannian manifold learning," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 5, pp. 796-809, May 2008. (Pubitemid 351486749)
30. J. Lu and Y.-P. Tan, "Regularized locality preserving projections and its extensions for face recognition," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 3, pp. 958-963, Jun. 2010.
31. W. Min, K. Lu, and X. He, "Locality pursuit embedding," Pattern Recognit., vol. 37, no. 4, pp. 781-788, Apr. 2004. (Pubitemid 38301055)
32. P. Mordohai and G. Medioni, "Dimensionality estimation, manifold learning and function approximation using tensor voting," J. Mach. Learn. Res., vol. 11, pp. 411-450, 2010.
33. J. Nilsson, F. Sha, andM. I. Jordan, "Regression on manifolds using kernel dimension reduction," in Proc. Int. Conf. Mach. Learn., Corvallis, OR, 2007, pp. 697-704.
34. R. Penrose, "A generalized inverse for matrices," Proc. Cambridge Philos. Soc., vol. 51, pp. 406-413, 1995.
35. M. Polito and P. Perona, "Grouping and dimensionality reduction by locally linear embedding," in Advances in Neural Information Processing Systems 14. Cambridge, MA: MIT Press, 2002, pp. 1255-1262.
36. H. Qiao, P. Zhang, B. Zhang, and S. Zheng, "Learning an intrinsicvariable preserving manifold for dynamic visual tracking," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 3, pp. 868-880, Jun. 2010.
37. S. Roweis and L. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, Dec. 2000. (Pubitemid 32041578)
38. L. K. Saul and S. T. Roweis, "Thinking globally, fit locally: Unsupervised learning of low dimensional manifolds," J. Mach. Learn. Res., vol. 4, pp. 119-155, 2003.
39. G. Seber, Multivariate Observations. New York: Wiley, 1984.
40. F. Sha and L. K. Saul, "Analysis and extension of spectral methods for nonlinear dimensionality reduction," in Proc. Int. Conf. Mach. Learn., Bonn, Germany, 2005, pp. 784-791.
41. Y.W. Teh and S. Roweis, "Automatic alignment of local representations," in Advances in Neural Information Processing Systems 15. Cambridge, MA: MIT Press, 2003, pp. 841-848.
42. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, Dec. 2000. (Pubitemid 32041577)
43. M. Vlachos, C. Domeniconi, and D. Gunopulos, "Non-linear dimensionality reduction techniques for classification and visualization," in Proc. Int. Conf. Knowl. Discov. Data Mining, Edmonton, AB, Canada, 2002, pp. 645-651.
44. H.Wang, J. Zheng, Z. Yao, and L. Li, "Improved locally linear embedding through new distance computing," in Proc. 3rd Int. Symp. Neural Netw., Chengdu, China, 2006, pp. 1326-1333. (Pubitemid 44046274)
45. K. Q. Weinberger and L. K. Saul, "Unsupervised learning of image manifolds by semidefinite programming," in Proc. Comput. Vis. Pattern Recog., Washington, DC, 2004, pp. 988-995.
46. K. Q. Weinberger, F. Sha, and L. K. Saul, "Learning a kernel matrix for nonlinear dimensionality reduction," in Proc. Int. Conf. Mach. Learn., Banff, AB, Canada, 2004, pp. 888-905.
47. M. Wu, K. Yu, S. Yu, and B. Schölkopf, "Local learning projections," in Proc. Int. Conf. Mach. Learn., Corvallis, OR, 2007, pp. 1039-1046.
48. M. R. Wu and B. Schölkopf, "A local learning approach for clustering," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 1529-1536.
49. M. R.Wu and B. Schölkopf, "Transductive classification via local learning regularization," in Proc. Int. Conf. Artif. Intell. Statist., San Juan, Puerto Rico, 2007, pp. 628-635.
50. S. Xiang, F. Nie, Y. Song, C. Zhang, and C. Zhang, "Embedding new data points for manifold learning via coordinate propagation," Knowl. Inf. Syst., vol. 19, no. 2, pp. 159-184, May 2009.
51. S. Xiang, F. Nie, and C. Zhang, "Semi-supervised classification via local spline regression," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 2039-2053, Nov. 2010.
52. S. Xiang, F. Nie, C. Zhang, and C. Zhang, "Nonlinear dimensionality reduction with local spline embedding," IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1285-1298, Sep. 2009.
53. S. Yan, D. Xu, B. Zhang, and H. Zhang, "Graph embedding: A general framework for dimensionality reduction," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., San Diego, CA, 2005, pp. 830-837.
54. Y. Yang, F. Nie, S. Xiang, Y. Zhuang, and W. Wang, "Local and global regressive mapping for manifold learning with out-of-sample extrapolation," in Proc. AAAI Conf. Artif. Intell., Atlanta, GA, 2010, pp. 649-654.
55. D. Zhan and Z. Zhou, "Neighbor line-based locally linear embedding," in Proc. Pacific-Asia Conf. Knowl. Discov. Data Mining, Singapore, 2006, pp. 606-615.
56. C. Zhang, S. Xiang, F. Nie, and Y. Song, "Nonlinear dimensionality reduction with relative distance comparison," Neurocomputing, vol. 72, no. 7-9, pp. 1719-1731, Mar. 2009.
57. J. Zhang, J. Pu, C. Chen, and R. Fleischer, "Low-resolution gait recognition," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 4, pp. 986-996, Aug. 2010.
58. T. Zhang, K. Huang, X. Li, J. Yang, and D. Tao, "Discriminative orthogonal neighborhood-preserving projections for classification," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 1, pp. 253-263, Feb. 2010.
59. X. Zhang and G. Fan, "Dual gait generative models for human motion estimation from a single camera," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 4, pp. 1034-1049, Aug. 2010.
60. Z. Zhang and J. Wang, "MLLE: Modified locally linear embedding using multiple weights," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007, pp. 1593-1600.
61. Z. Zhang and H. Zha, "Principal manifolds and nonlinear dimensionality reduction via tangent space alignment," SIAM J. Sci. Comput., vol. 26, no. 1, pp. 313-338, 2004.
62. Z. Zhang and L. Zhao, "Probability-based locally linear embedding for classification," in Proc. 4th Int. Conf. Fuzzy Syst. Knowl. Discov., Haikou, China, 2007, vol. 3, pp. 243-247. (Pubitemid 351710897)
63. D. Zhao, "Formulating LLE using alignment technique," Pattern Recognit., vol. 39, no. 11, pp. 2233-2235, Nov. 2006. (Pubitemid 44175455)
64. D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Scholkopf, "Learning with local and global consistency," in Proc. Adv. Neural Inf. Process. Syst. 16, Vancouver, BC, Canada, 2003, pp. 321-328.
65. D. Zhou, J.Weston, A. Gretton, O. Bousquet, and B. Schölkopf, "Ranking on data manifolds," in Proc. Annu. Conf. Neural Inf. Process. Syst., 2003, pp. 169-176.