Adaptive data embedding framework for multiclass classification

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 8, 2012, Pages 1291-1303

  Mu, Tingting ^a   Jiang, Jianmin ^a,b   Wang, Yan ^a   Goulermas, John Y ^c  

^a UNIVERSITY OF BRADFORD   (United Kingdom)

^b UNIVERSITY OF SURREY   (United Kingdom)

^c UNIVERSITY OF LIVERPOOL   (United Kingdom)

Author keywords

Bilevel evolutionary optimization; classification; embedding; proximity information

Indexed keywords

AUTOMATIC GENERATION; EMBEDDING; EVOLUTIONARY OPTIMIZATIONS; MODEL IDENTIFICATION; MULTI-CLASS CLASSIFICATION; NON-LINEAR DISTRIBUTION; PROXIMITY INFORMATION; REAL-WORLD DATASETS;

CLASSIFICATION (OF INFORMATION); OPTIMIZATION;

DIGITAL STORAGE;

EID: 84875882648     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2200693     Document Type: Article

References (50)

1. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
2. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
3. L. J. P. van der Maaten, E. O. Postma, and H. J. van den Herik, "Dimensionality reduction: A comparative review," Tilburg Centre Creative Comput., Tilburg Univ., Tilburg, The Netherlands, Tech. Rep. TiCC-TR 2009-005, 2009.
4. T. Mu, J. Y. Goulermas, S. Ananiadou, and J. Tsujii, "Proximity- based frameworks for generating embeddings from multi-output data," IEEE Trans. Pattern Anal. Mach. Intell., DOI:10.1109/TPAMI.2012.20.
5. I. T. Jolliffe, Principal Component Analysis. New York: Springer-Verlag, 1986.
6. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, "Indexing by latent semantic analysis," J. Amer. Soc. Inf. Sci., vol. 41, no. 6, pp. 391-407, 1990.
7. W. S. Torgerson, "Multidimensional scaling: I. Theory and method," J. Psychometrika, vol. 17, no. 4, pp. 401-419, 1952.
8. E. Kokiopoulou and Y. Saad, "Orthogonal neighborhood preserving projections: A projection-based dimensionality reduction technique," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 12, pp. 2143-2156, Dec. 2007.
9. X. He and P. Niyogi, "Locality preserving projections," in Proc. Neural Inf. Process. Syst. 16, 2003, pp. 1-8.
10. P. K. Chan, M. D. F. Schlag, and J. Y. Zien, "Spectral K-way ratio-cut partitioning and clustering," IEEE Trans. Comput.-Aided Design Integr. Circuits Syst., vol. 13, no. 9, pp. 1088-1096, Sep. 1994.
11. Y. Pan, S. Ge, and A. A. Mamun, "Weighted locally linear embedding for dimension reduction," Pattern Recognit., vol. 42, no. 5, pp. 798-811, 2009.
12. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
13. A. Ng, M. Jordan, and Y. Weiss, "On spectral clustering: Analysis and algorithm," in Advances in Neural Information Processing Systems 14. Cambridge, MA: MIT Press, 2001.
14. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput., vol. 15, no. 6, pp. 1373-1396, 2003.
15. C. Hou, C. Zhang, Y. Wu, and Y. Jiao, "Stable local dimensionality reduction approaches," Pattern Recognit., vol. 42, no. 9, pp. 2054-2066, 2009.
16. J. Wang and Z. Zhang, "Nonlinear embedding preserving multiple local-linearities," Pattern Recognit., vol. 43, no. 4, pp. 1257-1268, 2010.
17. R. A. Fisher, "The use of multiple measurements in taxonomic problems," Ann. Eugen., vol. 7, no. 2, pp. 179-188, 1936.
18. H. Li, T. Jiang, and K. Zhang, "Efficient and robust feature extraction by maximum margin criterion," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 157-165, Jan. 2006.
19. W. Zhang, X. Xue, Z. Sun, Y. Guo, and H. Lu, "Optimal dimensionality of metric space for classification," in Proc. 24th Int. Conf. Mach. Learn., vol. 227. 2007, pp. 1135-1142.
20. S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, "Graph embedding and extensions: A general framework for dimensionality reduction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 40-51, Jan. 2007.
21. P. Belhumeur, J. Hespanda, and D. Kriegman, "Eigenfaces versus fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711-720, Jul. 1997.
22. T. Hastie and R. Tibshirani, "Discriminant adaptive nearest neighbor classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 6, pp. 607-616, Jun. 1996.
23. M. Sugiyama, "Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis," J. Mach. Learn. Res., vol. 8, pp. 1027-1061, May 2007.
24. H. Wang, S. Chen, Z. Hu, and W. Zheng, "Locality-preserved maximum information projection," IEEE Trans. Neural Netw., vol. 19, no. 4, pp. 571-585, Apr. 2008.
25. S. Ji and J. Ye, "Generalized linear discriminant analysis: A unified framework and efficient model selection," IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1768-1782, Oct. 2008.
26. E. Kokiopouloua and Y. Saadb, "Enhanced graph-based dimensionality reduction with repulsion Laplaceans," Pattern Recognit., vol. 42, no. 11, pp. 2392-2402, 2009.
27. T. Zhang, D. Tao, X. Li, and J. Yang, "Patch alignment for dimensionality reduction," IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1299-1313, Sep. 2009.
28. Q. Gao, H. Xu, Y. Li, and D. Xie, "Two-dimensional supervised local similarity and diversity projection," Pattern Recognit., vol. 43, no. 410, pp. 3359-3363, 2010.
29. W. K. Wong and H. T. Zhao, "Supervised optimal locality preserving projection," Pattern Recognit., vol. 45, no. 1, pp. 186-197, Jan. 2012.
30. T. Zhang, K. Huang, X. Li, J. Yang, and D. Tao, "Discriminative orthogonal neighborhood-preserving projections for classification," IEEE Trans. Syst., Man, Cybern., Part B, Cybern., vol. 40, no. 1, pp. 253-263, Feb. 2010.
31. E. Rodriguez-Martinez, J. Y. Goulermas, T. Mu, and J. F. Ralph, "Automatic induction of projection pursuit indices," IEEE Trans. Neural Netw., vol. 21, no. 8, pp. 1281-1295, Aug. 2010.
32. Y. Zhang and D. Yeung, "Semisupervised generalized discriminant analysis," IEEE Trans. Neural Netw., vol. 22, no. 8, pp. 1207-1217, Aug. 2011.
33. E. Kokiopoulou, J. Chen, and Y. Saad, "Trace optimization and eigenproblems in dimension reduction methods," Numer. Linear Algebra Appl., vol. 18, no. 3, pp. 565-602, 2011.
34. H. Wang, S. Yan, T. Huang, and X. Tang, "Trace ratio versus ratio trace for dimensionality reduction," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2007, pp. 1-8.
35. Y. Jia, F. Nie, and C. Zhang, "Trace ratio problem revisited," IEEE Trans. Neural Netw., vol. 20, no. 4, pp. 729-735, Apr. 2009.
36. C. Shen, H. Li, and M. J. Brooks, "Supervised dimensionality reduction via sequential semidefinite programming," Pattern Recognit., vol. 41, no. 12, pp. 3644-3652, 2008.
37. R. Jenatton, J. Mairal, G. Obozinski, and F. Bach, "Proximal methods for sparse hierarchical dictionary learning," in Proc. Int. Conf. Mach. Learn., 2010, pp. 1-8.
38. P. Piro, R. Nock, F. Nielsen, and M. Barlaud, "Leveraging k-NN for generic classification boosting," Neurocomputing, vol. 18, no. 15, pp. 3-9, 2011
39. A. Ng, J. Ngiam, C. Foo, Y. Mai, and C. Suen. (2011). Unsupervised Feature Learning and Deep Learning (UFLDL) Tutorial [Online]. Available: http://ufldl.stanford.edu/wiki/index.php/Data-Preprocessing
40. I. Borg and J. Lingoes, Multidimensional Similarity Structure Analysis. New York: Springer-Verlag, 1987.
41. E. Pekalska, R. Duin, and P. Paclik, "Prototype selection for dissimilarity-based classifiers," Pattern Recognit., vol. 39, no. 2, pp. 189-208, 2006.
42. T. Mu and S. Ananiadou, "Proximity-based graph embeddings for multi-label classification," in Proc. Int. Conf. Knowl. Discovery Inf. Retri., 2010, pp. 74-84.
43. R. Yan, J. Zhang, J. Yang, and A. G. Hauptmann, "A discriminative learning framework with pairwise constraints for video object classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 4, pp. 578-593, Apr. 2006.
44. T. Mu, A. K. Nandi, and R. Rangayyan, "Analysis of breast tumors in mammograms using the pairwise rayleigh quotient classifier," J. Electron. Imaging, vol. 16, no. 4, p. 043004, 2007.
45. F. Bach and M. Jordan, "Kernel independent component analysis," J. Mach. Learn. Res., vol. 3, pp. 1-48, Jul. 2002.
46. L. Sun, S. Ji, and J. Ye, "A least squares formulation for canonical correlation analysis," in Proc. 25th Int. Conf. Mach. Learn., 2008, pp. 1024-1031.
47. B. Colson, P. Marcotte, and G. Savard, "An overview of bilevel optimization," Ann. Oper. Res., vol. 153, no. 1, pp. 235-256, 2007.
48. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
49. S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K. Muller, "Fisher discriminant analysis with kernels," in Proc. IEEE Neural Netw. Signal Process. Workshop, Aug. 1999, pp. 41-48.
50. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.