A unified feature and instance selection framework using optimum experimental design

IEEE Transactions on Image Processing

Volumn 21, Issue 5, 2012, Pages 2379-2388

  Zhang, Lijun ^a   Chen, Chun ^a   Bu, Jiajun ^a   He, Xiaofei ^a  

^a ZHEJIANG UNIVERSITY   (China)

Author keywords

Active learning; experimental design; feature selection; instance selection

Indexed keywords

ACTIVE LEARNING; BENCHMARK DATA; COMPACT REPRESENTATION; DATA MATRICES; GREEDY ALGORITHMS; INSTANCE SELECTION; LEARNING FUNCTIONS; OPTIMIZATION PROBLEMS; OPTIMUM EXPERIMENTAL DESIGN; PARAMETER COVARIANCE; PARAMETER ESTIMATE; TRAINING DATA;

COVARIANCE MATRIX; DESIGN OF EXPERIMENTS; PARAMETER ESTIMATION; PROBLEM SOLVING; STATISTICS;

FEATURE EXTRACTION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; INFORMATION RETRIEVAL; METHODOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 84860150282     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2012.2183879     Document Type: Article

References (56)

1. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, ser. Springer Series in Statistics. New York: Springer-Verlag, 2009.
2. Z. Xu, R. Jin, J. Ye, M. R. Lyu, and I. King, "Non-monotonic feature selection," in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 1145-1152.
3. L. Wolf and A. Shashua, "Feature selection for unsupervised and supervised inference: The emergence of sparsity in a weight-based approach," J. Mach. Learn. Res., vol. 6, pp. 1855-1887, Dec. 2005.
4. P. Mitra, C. A. Murthy, and S. K. Pal, "Unsupervised feature selection using feature similarity," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 301-312, Mar. 2002.
5. A. Destrero, C. De Mol, F. Odone, and A. Verri, "A sparsity-enforcing method for learning face features," IEEE Trans. Image Process., vol. 18, no. 1, pp. 188-201, Jan. 2009.
6. Z. Zhao and H. Liu, "Spectral feature selection for supervised and unsupervised learning," in Proc. 24th Int. Conf. Mach. Learn., 2007, pp. 1151-1157.
7. X. He, D. Cai, and P. Niyogi, "Laplacian score for feature selection," Adv. Neural Inf. Process. Syst., vol. 18, pp. 507-514, 2006.
8. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, Mar. 2003.
9. J. Zhao, K. Lu, and X. He, "Locality sensitive semi-supervised feature selection," Neurocomputing, vol. 71, no. 10-12, pp. 1842-1849, Jun. 2008.
10. A. Jain and D. Zongker, "Feature selection: evaluation, application, and small sample performance," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 2, pp. 153-158, Feb. 1997.
11. D. Liu, G. Hua, P. A. Viola, and T. Chen, "Integrated feature selection and higher-order spatial feature extraction for object categorization," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
12. P. K. Mallapragada, R. Jin, and A. K. Jain, "Online visual vocabulary pruning using pairwise constraints," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 3073-3080.
13. L. Zhang, C. Chen, J. Bu, Z. Chen, S. Tan, and X. He, "Discriminative codeword selection for image representation," in Proc. 18th ACM Int. Conf. Multimedia, 2010, pp. 173-182.
14. A. Atkinson, A. Donev, and R. Tobias, Optimum Experimental Designs, with SAS. New York: Oxford Univ. Press, 2007.
15. D. A. Cohn, Z. Ghahramani, and M. I. Jordan, "Active learning with statistical models," J. Artif. Intell. Res., vol. 4, no. 1, pp. 129-145, Jan. 1996.
16. B. Settles, Active learning literature survey Univ.Wisconsin-Madison, Dept. Comput. Sci., Madison, WI, Tech. Rep. 1648, 2009.
17. S. Tong and E. Chang, "Support vector machine active learning for image retrieval," in Proc. 9th ACM Int. Conf. Multimedia, 2001, pp. 107-118.
18. L. Zhang, C. Chen, W. Chen, J. Bu, D. Cai, and X. He, "Convex experimental design using manifold structure for image retrieval," in Proc. 17th ACM Int. Conf. Multimedia, 2009, pp. 45-54.
19. X. Tian, D. Tao, X.-S. Hua, and X. Wu, "Active reranking for web image search," IEEE Trans. Image Process., vol. 19, no. 3, pp. 805-820, Mar. 2010.
20. A. Kapoor, G. Hua, A. Akbarzadeh, and S. Baker, "Which faces to tag: Adding prior constraints into active learning," in Proc. 12th IEEE Int. Conf. Comput. Vis., 2009, pp. 1058-1065.
21. L. Zhang, C. Chen, J. Bu, D. Cai, X. He, and T. S. Huang, "Active learning based on locally linear reconstruction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 10, pp. 2026-2038, Oct. 2011.
22. K. Yu, S. Zhu, W. Xu, and Y. Gong, "Non-greedy active learning for text categorization using convex transductive experimental design," in Proc. 31st Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2008, pp. 635-642.
23. I. S. Dhillon, "Co-clustering documents and words using bipartite spectral graph partitioning," in Proc. 7th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2001, pp. 269-274.
24. Y. Yang and J. O. Pedersen, "A comparative study on feature selection in text categorization," in Proc. 14th Int. Conf. Mach. Learn., 1997, pp. 412-420.
25. K. Kira and L. A. Rendell, "The feature selection problem: Traditional methods and a new algorithm," in Proc. 10th Nat. Conf. Artif. Intell., 1992, pp. 129-134.
26. R. Kohavi and G. H. John, "Wrappers for feature subset selection," Artif. Intell., vol. 97, no. 1/2, pp. 273-324, Dec. 1997.
27. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik, "Feature selection for SVMs," Adv. Neural Inf. Process. Syst., vol. 13, pp. 668-674, 2001.
28. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc., Ser. B, Methodological, vol. 58, no. 1, pp. 267-288, 1996.
29. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Statist., vol. 32, no. 2, pp. 407-451, Apr. 2004.
30. G. Fung and O. L. Mangasarian, "Data selection for support vector machine classifiers," in Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2000, pp. 64-70.
31. T. Helleputte and P. Dupont, "Partially supervised feature selection with regularized linear models," in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 409-416.
32. C. Boutsidis, M. W. Mahoney, and P. Drineas, "Unsupervised feature selection for principal components analysis," in Proc. 14th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2008, pp. 61-69.
33. C. Boutsidis, M. Mahoney, and P. Drineas, "Unsupervised feature selection for the α-means clustering problem," in Proc. Adv. Neural Inf. Process. Syst., 2009, vol. 22, pp. 153-161.
34. J. G. Dy and C. E. Brodley, "Feature selection for unsupervised learning," J. Mach. Learn. Res., vol. 5, pp. 845-889, Dec. 2004.
35. M. H. C. Law, M. A. T. Figueiredo, and A. K. Jain, "Simultaneous feature selection and clustering using mixture models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, pp. 1154-1166, Sep. 2004.
36. V. Roth and T. Lange, "Feature selection in clustering problems," in Proc. Adv. Neural Inf. Process. Syst., 2004, vol. 16, pp. 473-480.
37. D. Cai, C. Zhang, and X. He, "Unsupervised feature selection for multicluster data," in Proc. 16th ACM SIGKDD, 2010, pp. 333-342.
38. F. R. Bach, B. Schölkopf, J. Platt, and T. Hoffman, Eds., "Active learning for misspecified generalized linear models," in Proc. Adv. Neural Inf. Process. Syst., 2007, vol. 19, pp. 65-72.
39. P. Gosselin and M. Cord, "Active learning methods for interactive image retrieval," IEEE Trans. Image Process., vol. 17, no. 7, pp. 1200-1211, Jul. 2008.
40. X. Zhu, J. Lafferty, and Z. Ghahramani, "Combining active learning and semi-supervised learning using gaussian fields and harmonic functions," in Proc. ICML-Workshop on Continuum Labeled to Unlabeled Data in Machine Learning and Data Mining, 2003, pp. 58-65.
41. N. Roy and A. McCallum, "Toward optimal active learning through sampling estimation of error reduction," in Proc. 18th Int. Conf. Machine Learn., 2001, pp. 441-448.
42. H. S. Seung, M. Opper, and H. Sompolinsky, "Query by committee," in Proc. 5th Annu.Workshop Comput. Learn. Theory, 1992, pp. 287-294.
43. Y. Guo and R. Greiner, "Optimistic active learning using mutual information," in Proc. 20th Int. Joint Conf. Artif. Intell., Hyderabad, India, 2007, pp. 823-829.
44. X. He, W. Min, D. Cai, and K. Zhou, "Laplacian optimal design for image retrieval," in Proc. 30th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2007, pp. 119-126.
45. X. He, "Laplacian regularized D-optimal design for active learning and its application to image retrieval," IEEE Trans. Image Process., vol. 19, no. 1, pp. 254-263, Jan. 2010.
46. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
47. K. Yu, J. Bi, and V. Tresp, "Active learning via transductive experimental design," in Proc. 23rd Int. Conf. Mach. Learn., 2006, pp. 1081-1088.
48. S. P. Asprey and S. Macchietto, "Designing robust optimal dynamic experiments," J. Process Control, vol. 12, no. 4, pp. 545-556, Jun. 2002.
49. H. Liu, H. Motoda, and L. Yu, "A selective sampling approach to active feature selection," Artif. Intell., vol. 159, no. 1/2, pp. 49-74, Nov. 2004.
50. G. Strang, Introduction to Linear Algebra, 3rd ed. Wellesley, MA: Wellesley-Cambridge Press, 2003.
51. X. He, S. Yan, Y. Hu, P. Niyogi, and H.-J. Zhang, "Face recognition using Laplacianfaces," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 3, pp. 328-340, Mar. 2005.
52. D. Cai, X. He, and J. Han, Using graph model for face analysis Dept. Comput. Sci., UIUC, Champaign, IL, Tech. Rep. UIUCDCS-R-2005-2636, 2005.
53. , O. Chapelle, B. Schölkopf, and A. Zien, Eds., Semi-Supervised Learning. Cambridge, MA: MIT Press, 2006.
54. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Discovery, vol. 2, no. 2, pp. 121-167, Jun. 1998.
55. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines 2001 [Online]. Available: http://www.csie.ntu.edu.tw/ cjlin/ libsvm.
56. T. Yang, R. Jin, and A. K. Jain, "Learning from noisy side information by generalized maximum entropy model," in Proc. 27th Int. Conf. Mach. Learn., 2010, pp. 1199-1206.