Generalized iterative RELIEF for supervised distance metric learning

Pattern Recognition

Volumn 43, Issue 8, 2010, Pages 2971-2981

  Chang, Chin Chun ^a  

^a NATIONAL TAIWAN OCEAN UNIVERSITY   (Taiwan)

Author keywords

Distance metric learning; Feature weighting; Iterative RELIEF

Indexed keywords

CLASSIFICATION RATES; DATA SETS; DISTANCE FUNCTIONS; DISTANCE METRIC LEARNING; FEATURE WEIGHTING; GRAPH EMBEDDINGS; LEAVE-ONE-OUT; NEAREST-NEIGHBORS; NEIGHBOURHOOD; OBJECTIVE FUNCTIONS; RELIEF ALGORITHM;

EID: 77951229399     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2010.02.024     Document Type: Article

References (39)

1. K. Kira, L.A. Rendell, A practical approach to feature selection, in: Proceedings of the Ninth International Conference on Machine Learning, 1992, pp. 249-256.
2. Dietterich T.G. Machine-learning research-four current directions. AI Magazine 18 (1997) 97-136
3. Kohavi R., and John G. Wrappers for feature selection. Artificial Intelligence 97 (1997) 273-324
4. Wettschereck D., Aha D.W., and Mohri T. A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artificial Intelligence Review 11 (1997) 273-314
5. I. Kononenko, Estimating attributes: analysis and extensions of RELIEF, in: European Conference on Machine Learning, 1994, pp. 171-182.
6. Sun Y. Iterative RELIEF for feature weighting: algorithms, theories, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 29 6 (2007) 1035-1051
7. Kononenko I., Simec E., and Robnik-Sikonja M. Overcoming the myopia of inductive learning algorithms with RELIEFF. Applied Intelligence 7 (1997) 39-55
8. Robnik-Šikonja M., and Kononenko I. Theoretical and empirical analysis of ReliefF and RReliefF. Machine Learning 53 (2003) 23-69
9. Dempster A.P., Laird N.M., and Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B 39 1 (1977) 1-38
10. J. Yang, Y.-P. Li, Orthogonal RELIEF algorithm for feature selection, in: Proceedings of 2006 International Conference on Intelligent Computing, Springer, Berlin/Heidelberg, 2006.
11. Y. Sun, D. Wu, A RELIEF based feature extraction algorithm, in: Proceedings of the SIAM International Conference on Data Mining, 2008, pp. 188-195.
12. J. Bins, B. Draper, Feature selection from huge feature sets, in: Proceedings of the Eighth IEEE International Conference on Computer Vision, vol. 2, 2001, pp. 159-165.
13. R. Flórez-López, Reviewing RELIEF and its extensions: a new approach for estimating attributes considering high-correlated features, in: Proceedings of IEEE International Conference on Data Mining, 2002, pp. 605-608.
14. L. Yang, R. Jin, Distance metric learning: a comprehensive survey, Technical Report 24, Department of Computer Science and Engineering, Michigan State University, 2006.
15. Fukunaga K., and Flick T.E. An optimal global nearest neighbor metric. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI 6 3 (1984) 314-318
16. Hastie T., and Tibshirani R. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 18 6 (1996) 607-616
17. Goldberger J., Roweis S., Hinton G., and Salakhutdinov R. Neighbourhood components analysis. In: Saul L.K., Weiss Y., and Bottou L. (Eds). Advances in Neural Information Processing Systems, vol. 17 (2005), MIT Press, Cambridge, MA 513-520
18. L. Yang, R. Jin, R. Sukthankar, Y. Liu, An efficient algorithm for local distance metric learning, in: Proceedings of the 21st National Conference on Artificial Intelligence, 2006.
19. Xing E.P., Ng A.Y., Jordan M.I., and Russell S. Distance metric learning with application to clustering with side-information. In: Becker S.T.S., and Obermayer K. (Eds). Advances in Neural Information Processing Systems, vol. 15 (2003), MIT Press, Cambridge, MA 505-512
20. Weinberger K., Blitzer J., and Saul L. Distance metric learning for large margin nearest neighbor classification. In: Weiss Y., Schölkopf B., and Platt J. (Eds). Advances in Neural Information Processing Systems, vol. 18 (2006), MIT Press, Cambridge, MA 1473-1480
21. Yan S., Liu J., Tang X., and Huang T.S. A parameter-free framework for general supervised subspace learning. IEEE Transactions on Information Forensics and Security 2 1 (2007) 69-76
22. S. Chopra, R. Hadsell, Y. LeCun, Learning a similarity metric discriminatively, with application to face verification, in: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2005, pp. 539-546.
23. J.V. Davis, B. Kulis, P. Jain, I.S. Dhillon, Information-theoretic metric learning, in: Proceedings of the 24th international conference on Machine learning, 2007, pp. 209-216.
24. Yan S., Xu D., Zhang B., Zhang H.J., Yang Q., and Lin S. Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence 29 1 (2007) 40-51
25. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), John & Wiley Sons, Inc, New York
26. Bazaraa M.S., Sherali H.D., and Shetty C.M. Nonlinear Programming Theory and Algorithms. second ed. (1993), John & Wiley sons, Inc, New York
27. Chung F.R.K. Spectral Graph Theory (1997), American Mathematical Society, New York
28. Horn R.A., and Johnson C.R. Matrix Analysis (1985), Cambridge University Press, New York
29. Sugiyama M. Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. Journal of Machine Learning Research 8 (2007) 1027-1061
30. Zelnik-Manor L., and Perona P. Self-tuning spectral clustering. In: Saul L.K., Weiss Y., and Bottou L. (Eds). Advances in Neural Information Processing Systems vol. 17 (2005), MIT Press, Cambridge, MA 1601-1608
31. Xu D., Yan S., Tao D., Lin S., and Zhang H.J. Marginal Fisher analysis and its variants for human gait recognition and content-based image retrieval. IEEE Transactions on Image Processing 16 11 (2007) 2811-2821
32. A. Bar-Hillel, T. Hertz, N. Shental, D. Weinshall, Learning distance functions using equivalence relations, in: Proceedings of the 20th International Conference on Machine Learning, 2003, pp. 11-18.
33. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification. second ed. (2001), John Wiley & Sons, Inc
34. A. Asuncion, D. Newman, UCI machine learning repository, 2007. URL 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.
35. Singh D., Febbo P.G., Ross K., Jackson D.G., Manola J., Ladd C., Tamayo P., Renshaw A.A., D'Amico A.V., Richie J.P., Lander E.S., Loda M., Kantoff P.W., Golub T.R., and Sellers W.R. Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 1 (2002) 203-209
36. Ramaswamy S., Tamayo P., Rifkin R., Mukherjee S., Yeang C., Angelo M., Ladd C., Reich M., Latulippe E., Mesirov J., Poggio T., Gerald W., Loda M., Lander E., and Golub R. Multiclass cancer diagnosis using tumor gene expression signatures. Proceedings of the National Academy of Science 98 26 (2001) 15149-15154
37. Lee K., Ho J., and Kriegman D. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions Pattern Analysis and Machine Intelligence 27 5 (2005) 684-698
38. A.M. Martinez, R. Benavente, The AR face database, Technical Report 24, CVC, June 1998.
39. Fukunaga K. Introduction to Statistical Pattern Recognition. second ed. (1990), Academic Press, Boston