Feature selection for unsupervised and supervised inference: The emergence of sparsity in a weight-based approach

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages

  Wolf, Lior ^a   Shashua, Amnon ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

DATA REDUCTION; FUNCTIONS; LEAST SQUARES APPROXIMATIONS; MATRIX ALGEBRA; OPTIMIZATION; SET THEORY; WORLD WIDE WEB;

BIO-INFORMATICS; FEATURE SELECTION; SUBSET; SUPERVISED INFERENCE;

FEATURE EXTRACTION;

EID: 27844550205     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (50)

1. H. Almuallim and T .G Dietterich. Learning boolean concepts in the presence of many irrelevant features. AI, 69(1-2):279-305, 1991.
2. A. Ben-Dor, N. Friedman, and Z. Yakhini. Class discovery in gene expression data. In RECOMB, 2001.
3. A. Blum and P. Langley. Selection of relevant features and examples in machine learning. A1, 97 (1-2):245-271, 1997.
4. O. Bousquet and D. J. L. Herrmann. On the complexity of learning the kernel matrix. In NIPS, 2003.
5. P. S. Bradley and O. L. Mangasarian. Feature selection via concave minimization and support vector machines. In ICML, 1998.
6. M. Brand and K. Huang. A unifying theorem for spectral embedding and clustering. In Ninth Int. Workshop on AI and Statistics, 2003.
7. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3):131-159, 2002.
8. F. R. K. Chung. Spectral Graph Theory. AMS, 1998.
9. Z. Furedi and J. Komlos. The eigenvalues of random symmetric matrices. Combinatorica, 1(3): 233-241, 1981.
10. L. E. Gibbons, D. W. Hearn, P. M. Pardalos, and M. V. Ramana. Continuous characterizations of the maximum clique problem. Math. Oper. Res, 22:754-768, 1997.
11. G. Golub and C. F. V. Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, MD, 1996.
12. T. R. Golub, D. K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J. P. Mesirov, H. Coller, M. L. Loh, J. R. Downing, M. A. Caligiuri, C. D. Bloomfield, and E. S. Lander. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science, 286:531-537, 1999.
13. I. Guyon and A. Elissef. An introduction to variable and feature selection. Journal of Machine Learning Research, Special issue on special feature, 3:389-422, 2003.
14. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46:389-422, 2002.
15. K. Hall. An r-dimensional quadratic placement algorithm. Management Science, 17(3):219-229, 1970.
16. K. Hall, 'gene shaving' as a method for identifying distinct sets of genes with similar expression patterns. Genome Biology, 1(2):1-21, 2000.
17. W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
18. K. Kira and L. Rendell. A practical approach to feature selection. In Ninth Int, Workshop on Machine Learning, 1992.
19. J. Kivinen and M. K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Journal of Information and Computation, 132(1):1-63, 1997.
20. R. Kohavi and G. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1-2):273-324, 1997.
21. P. Langley and W. Iba. Average-case analysis of a nearest neighbor algorithm. In 13th Int. Joint Conf. on Artificial Intelligence, 1993.
22. D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788-791, 1999.
23. D. D. Lewis. Feature selection and feature extraction for text categorization. In Speech and Natural Language Workshop, 1992.
24. R. Linsker. Self-organization in a perceptual network. Computer, 21(3):105-117, 1988. ISSN 0018-9162. doi: http://dx.doi.org/10.1109/2.36.
25. P. M. Long and V. B. Vega. Boosting and microarray data. Machine Learning, 52:31-44, 2003.
26. M. L. Mehta. Random Matrices. Academic Press, 1991.
27. B. Mohar. The laplacian spectrum of graphs. In Y. Alavi et al., editor, Graph Theory, Combinatorics and Applications. Wiley, New York, 1991.
28. T. S. Motzkin and E. G. Straus. Maxima for graphs and a new proof of a theorem by turan. Canadian Journal of Math., 17:533-540, 1965.
29. A. Y. Ng, M. I. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In NIPS, 2001.
30. B. A. Olshausen and D. J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(13):607-609, 1996.
31. M. Pavan and M. Pelillo. A new graph-theoretic approach to clustering and segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2003.
32. P. Perona and W. T. Freeman. A factorization approach to grouping. In European Conference of Computer Vision (ECCV), 1998.
33. S. L. Pomeroy, P. Tamayo, M. Gaasenbeek, L. M. Sturla, M. Angelo, M. E. McLaughlin, J. Y. H. Kim, L. C. Goumnerova, P.McL. Black, C. Lau, J. C. Allen, D. Zagzag, J. M.Olson, T. Curran, C. Wetmore, J. A. Biegel, T. Poggio, S. Mukherjee, R. Rifkin, A.Califano, G.Stolovitzky, D. N. Louis, J. P. Mesirov, E. S. Lander, and T. R. Golub. Gene expression-based classification and outcome prediction of central nervous system embryonal tumors. Nature, 415(24):436-442, 2002.
34. S. Ramaswamy. Personal communication.
35. R. Rifkin, G. Yeo, and T. Poggio. Regularized Least Squares Classification, volume 190 of NATO Science Series III: Computer and Systems Sciences. IOS Press, Amsterdam, 2003.
36. S. Sarkar and K. L. Boyer. Quantitative measures of change based on feature organization: eigenvalues and eigenvectors. CVIU, 71(1): 110-136, 1998.
37. B. Scholkopf and A. J. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
38. A. Shashua and L. Wolf. Kernel feature selection with side data using a spectral approach. In T. Pajdla and J. Matas, editors, ECCV (3), volume 3023 of Lecture Notes in Computer Science, pages 39-53. Springer, 2004. ISBN 3-540-21982-X.
39. J. Shi and J. Malik. Normalized cuts and image segmentation. PAMI, 22(8):888-905, 2000.
40. M. A. Shipp, K. N. Ross, P. Tamayo, A. P. Weng, J.L Kutok, R.C Aguiar, M. Gaasenbeek, M. Angelo, M. Reich, G. S. Pinkus, T. S. Ray, M.A Koval, K.W Last, A. Norton, T. A. Lister, J. Mesirov, D. S. Neuberg, E. S. Lander, J. C. Aster, and T. R. Golub. Diffuse large b-cell lymphoma outcome prediction by gene expression profiling and supervised machine learning. Nature Medicine, 8(1):68-74, 2002.
41. N. Tishby, F. Pereira, and W. Bialek. The information bottleneck method. In Proceedings of the 37-th Annual Allerton Conference on Communication, Control and Computing, pages 368-377, 1999.
42. L. J. van 't Veer, H. Dai, M. J. van de Vijver, Y. D. He, A.A Hart, M. Mao, H. L. Peterse, K. van der Kooy, M. J. Marton, A. T. Witteveen, G. J. Schreiber, R. M. Kerkhoven, C. Roberts, P. S. Linsley, R. Bernards, and S. H. Friend. Gene expression profiling predicts clinical outcome of breast cancer. Nature, 415(6871):530-536, 2002.
43. V. N. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, 1998.
44. N. Vasconcelos. Feature selection by maximum marginal diversity: optimality and implications for visual recognition. In CVPR (1), pages 762-772. IEEE Computer Society, 2003. ISBN 0-7695-1900-8.
45. P. Viola and M. Jones. Robust real-time object detection. Technical Report CRL-2001-1, Compaq Cambridge Research Lab, 2001.
46. Y. Weiss. Segmentation using eigenvectors: A unifying view. In ICCV, 1999.
47. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik. Feature selection for svms. NIPS, 2001.
48. E. P. Wigner. On the distribution of the roots of certain symmetric matrices. Ann. of Math.(2), 67:325-327, 1958.
49. L. Wolf and A. Shashua. Kernel principal angles for classification machines with applications to image sequence interpretation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2003.
50. L. Wolf, A. Shashua, and S. Mukherjee. Gene selection via a spectral approach. In post CVPR IEEE Workshop on Computer Vision Methods for Bioinformatics (CVMB), 2005.