Statistical challenges with high dimensionality: Feature selection in knowledge discovery

International Congress of Mathematicians, ICM 2006

Volumn 3, Issue , 2006, Pages 595-622

  Fan, Jianqing ^a   Li, Runze ^b  

^a Princeton University   (United States)

^b PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

AIC; BIC; Bioinformatics; Financial econometrics; LASSO; Model selection; Oracle property; Penalized likelihood; Persistent; SCAD; Statistical learning

Indexed keywords

AIC; BIC; FINANCIAL ECONOMETRICS; LASSO; MODEL SELECTION; ORACLE PROPERTIES; PENALIZED LIKELIHOOD; PERSISTENT; SCAD; STATISTICAL LEARNING;

BIOINFORMATICS; FEATURE EXTRACTION; RISK MANAGEMENT;

STATISTICS;

EID: 84878031768     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (90)

1. Antoniadis, A., and Fan, J., Regularization of wavelets approximations (with discussions). J. Amer. Statist. Assoc. 96 (2001), 939-967.
2. Benjamini, Y., and Yekutieli, D., The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 (2001), 1165-1188.
3. Bickel, P. J., Minimax estimation of a normal mean subject to doing well at a point. In Recent Advances in Statistics (ed. by M. H. Rizvi, J. S. Rustagi, and D. Siegmund), Academic Press, New York 1983, 511-528.
4. Bickel, P. J., and Levina, E., Some theory for Fisher's linear discriminant, 'naive Bayes', and some alternatives when there are many more variables than observations. Bernoulli 10 (2004), 989-1010.
5. Brown, P. O., and Botstein, D., Exploring the new world of the genome with microarrays. Nat. Genet. 21 (suppl. 1) (1999), 33-37.
6. de Boor, C., A Practical Guide to Splines.Appl. Math. Sci. 27, Springer-Verlag, New York 1978.
7. Bregman, L. M., A relaxation method of finding a common points of convex sets and its application to the solulation of problems in convex programming. U.S.S.R. Comput. Math. Math. Phys. 7 (1967), 620-631.
8. Breiman, L., Better subset regression using the nonnegative garrote. Technometrics 37 (1995), 373-384.
9. Cai, J., Fan, J., Li, R., and Zhou, H., Variable selection for multivariate failure time data. Biometrika 92 (2005), 303-316.
10. Campbell, J. Y., Lo, A., and MacKinlay, A. C., The Econometrics of Financial Markets. Princeton University Press, Princeton, NJ, 1997.
11. Craven, P., and Wahba, G., Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 31 (1979), 377-403.
12. Cochrane, J. H., Asset Pricing. Princeton University Press, Princeton, NJ, 2001.
13. Coifman, R. R., and Saito, N., Constructions of local orthonormal bases for classification and regression. C. R. Acad. Sci. Paris Sér. I Math. 319 (1994), 191-196.
14. Coifman, R. R., and Wickerhauser, M.V., Entropy-based algorithms for best-basis selection. IEEE Trans. Inform. Theory 38 (1992), 713-718.
15. Chamberlain, G., and Rothschild, M., Arbitrage, factor structure, and mean-variance analysis on large asset markets. Econometrica 51 (1983), 1281-1304.
16. Diaconis, P., and Sturmfels, B., Albegraic algorithms for sampling from conditional distributions. Ann. Statist. 26 (1998), 363-397.
17. Daubechies, I., Ten Lectures on Wavelets. SIAM, Philadelphia 1992.
18. Dempster, A. P., Laird, N. M., and Rubin, D. B., Maximum likelihood from incomplete data via the EM algorithm. J. Royal Statist. Soc. B 39 (1977), 1-38
19. Donoho, D. L., High-dimensional data analysis: the curses and blessings of dimensionality. Aide-Memoire of the lecture in AMS conference "Math challenges of 21st Centrury (2000). Available at http://www-stat.stanford.edu/ ~donoho/Lectures.
20. Donoho, D. L., and Elad, E., Maximal sparsity representation via l1 Minimization. Proc. Nat. Aca. Sci. 100 (2003), 2197-2202.
21. Donoho, D. L., and Huo, X., Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory 47 (2001), 2845-2862.
22. Donoho, D. L., and Jin, J., Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32 (2004), 962-994.
23. Donoho, D. L., and Johnstone, I. M., Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 (1994), 425-455.
24. Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D., Wavelet shrinkage: asymptopia? J. Royal Statist. Soc. B 57 (1995), 301-369.
25. Dudoit, S., Shaffer, J. P., and Boldrick, J. C., Multiple hypothesis testing in microarray experiments. Statist. Sci. 18 (2003), 71-103.
26. Dudoit, Y., Yang, Y. H., Callow, M. J., and Speed, T. P., Statistical methods for identifying differentially expressed genes in replicated cDNA microarray experiments. Statist. Sinica 12 (2002), 111-139.
27. Efron, B., Large-scale simultaneous hypothesis testing: the choice of a null hypothesis. J. Amer. Statist. Assoc. 99 (2004), 96-104.
28. Eilers, P. H. C., and Marx, B. D., Flexible smoothing with B-splines and penalties. Statist. Sci. 11 (1996), 89-121.
29. Fan, J., A selective overview of nonparametric methods in financial econometrics (with discussion). Statist. Sci. 20 (2005), 316-354.
30. Fan, J., Chen, Y., Chan, H. M., Tam, P., and Ren, Y., Removing intensity effects and identifying significant genes for Affymetrix arrays in MIF-suppressed neuroblastoma cells. Proc. Natl Acad. Sci. USA 103 (2005), 17751-17756.
31. Fan, J., and Gijbels, I. Local Polynomial Modelling and Its Applications. Chapman and Hall, London 1996.
32. Fan, J., Fan, Y., and Lv, J., Large dimensional covariance matrix estimation using a factor model. Manuscript, 2005.
33. Fan, J., and Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 (2001), 1348-1360.
34. Fan, J., and Li, R., Variable selection for Cox's proportional hazards model and frailty model. Ann. Statist. 30 (2002), 74-99.
35. Fan, J., and Peng, H., Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 (2004), 928-961.
36. Fan, J., Peng, H., and Huang, T., Semilinear high-dimensional model for normalization of microarray data: a theoretical analysis and partial consistency (with discussion). J. Amer. Statist. Assoc. 100 (2005), 781-813.
37. Fan, J., Tam, P., Vand e Woude, G., and Ren, Y., Normalization and analysis of cDNA micro-arrays using within-array replications applied to neuroblastoma cell response to a cytokine. Proc. Natl Acad. Sci. USA 101 (2004), 1135-1140.
38. Foster, D. P., and George, E. I., The risk inflation criterion for multiple regression. Ann. Statist. 22 (1994), 1947-1975.
39. Frank, I. E., and Friedman, J. H., A statistical view of some chemometrics regression tools. Technometrics 35 (1993), 109-148.
40. Freund, Y., Schapire, R. E., A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Systems Sci. 55 (1997), 119-139.
41. Friedman, J., Hastie, T. Rosset, S., Tibshirani, R. and Zhu, J., Discussion of boosting papers. Ann. Statist. 32 (2004), 102-107.
42. 1-constraint. Ann. Statist. 34 (5) (2006), to appear.
43. Greenshtein, E., and Ritov, Y., Persistence in high-dimensional predictor selection and the virtue of overparametrization. Bernoulli 10 (2004), 971-988.
44. Genovese, C., and Wasserman, L., A stochastic process approach to false discovery control. Ann. Statist. 32 (2004), 1035-1061.
45. Hall, P., Edgeworth expansion for Student's t statistic under minimal moment conditions. Ann. Probab. 15 (1987), 920-931.
46. Hall, P., Some contemporary problems in statistical sciences. The Madrid Intelligencer (2006), to appear.
47. Hand, D. J., Mannila, H., and Smyth, P., Principles of Data Mining.MIT Press, Cambridge, MA, 2001.
48. Hastie, T., Tibshirani, R., and Friedman, J., The Elements of Statistical Learning; Data Mining, Inference and Prediction, Springer Ser. Statist., Springer-Verlag, New York 2001.
49. Huang, J. Z., Liu, N., Pourahmadi, M., and Liu, L., Covariance selection and estimation via penalised normal likelihood. Biometrika (2006), 85-98.
50. Huang, J., Wang, D., and Zhang, C., A two-way semi-linear model for normalization and significant analysis of cDNA microarray data. J. Amer. Statist. Assoc. 100 (2005), 814-829.
51. Hull, J. Options, Futures, and Other Derivatives. 5th ed., Prentice Hall, Upper Saddle Review, NJ, 2003.
52. Hunter, D. R., and Li, R., Variable selection using MM algorithms. Ann. Statist. 33 (2005), 1617-1642.
53. Jing, B. Y., Shao, Q.-M., and Wang, Q. Y., Self-normalized Cramér type large deviations for independent random variables. Ann. Probab. 31 (2003), 2167-2215.
54. Johnstone, I. M., On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 (2001), 295-327.
55. Korosok, M. R., and Ma, S., Marginal asymptotics for the "large p, small n" paradigm: With applications to micorarray data. Ann. Statist., to appear.
56. Lange, K., A gradient algorithm locally equivalent to the EM algorithm. J. Royal Statist. Soc. B 57 (1995), 425-437.
57. Lehmann, E. L., Romano, J. P., and Shaffer, J. P., On optimality of stepdown and stepup multiple test procedures. Ann. Statist. 33 (2005), 1084-1108.
58. Li, H., and Gui, J., Gradient directed regularization for sparse Gaussian concentration graphs, with applications of inference of genetic networks. Biostatistics (2006), to appear.
59. Li, R., Dziak, J., and Ma, H. Y., Nonconvex penalized least squares: characterizations, algorithm and application. Manuscript, 2006.
60. Li, R., Root, T., and Shiffman, S., A local linear estimation procedure for functional multilevel modeling. In Models for Intensive Longitudinal Data (ed. by T.Walls and J. Schafer), Oxford University Press, New York 2006, 63-83.
61. Lipschutz, R. J., Fodor, S., Gingeras, T., Lockhart, D. J., High density synthetic oloigonucleotide arrays. Nat. Genet. 21 (1999), 20-24.
62. Lin, Y., Support vector machine and the Bayes rule in classification. Data Mining and Knowledge Discovery 6 (2002), 259-275.
63. Ma, S, Kosorok, M. R., Huang, J., Xie, H., Manzella, L., and Soares, M. B., Robust semi-parametric cDNA microarray normalization and significance analysis. Biometrics (2006), to appear.
64. 2(R). Trans. Amer. Math. Soc. 315 (1989a), 69-87.
65. Mallat, S. G., Atheory for multiresolution signal decomposition: thewavelet representation. IEEE Trans. Pattn Anal. Mach. Intell. 11 (1989b), 674-693.
66. Meinshausen, N., Lasso with relaxation. Manuscript, 2005.
67. Meinshausen, N., and Bühlmann, P., High dimensional graphs and variable selection with the Lasso. Ann. Statist. 34 (3) (2006), to appear.
68. Meyer, Y., Ondelettes. Hermann, Paris 1990.
69. Miller, A. J., Subset Selection in Regression. Chapman&Hall/CRC, London 2002.
70. Moffatt, H. K., Risk Management: Value at Risk and Beyond. Cambridge University Press, New York 2003.
71. Nikolova, M., Local strong homogeneity of a regularized estimator. SIAM J. Appl. Math. 61 (2000), 633-658.
72. Pachter, L., and Sturmfels, B., Parametric inference for biological sequence analysis. Proc. Natl. Acad. Sci. USA 101 (2004), 16138-16143.
73. Pistone, G., Riccomagno, E., and Wynn, H. P., Algebraic Statistics: Computational Commutative Algebra in Statistics. Chapman & Hall / CRC, London 2000.
74. Ross, S., The arbitrage theory of capital asset pricing. J. Economic Theory 13 (1976), 341-360.
75. Ruppert, D., Wand, M. P., Carroll, R. J., Semiparametric regression. Cambridge University Press, Cambridge 2003.
76. Stone, C. J., Hansen, M., Kooperberg, C., and Truong, Y. K., Polynomial splines and their tensor products in extended linear modeling (with discussion). Ann. Statist. 25 (1997), 1371-1470.
77. Storey J. D., Taylor J. E., and Siegmund D., Strong control, conservative point estimation, and simultaneous conservative consistency of false discovery rates: A unified approach. J. Royal Statist. Soc. B 66 (2004), 187-205.
78. Sturmfels, B., and Sullivant, S., Toric ideals of phylogenetic invariants. J. Comput. Biol. 12 (2005), 204-228
79. Svrakic, N. M., Nesic, O., Dasu, M. R. K., Herndon, D., and Perez-Polo, J. R., Statistical approach to DNA chip analysis. Recent Prog. Hormone Res. 58 (2003), 75-93.
80. Tai, Y. C., and Speed, T. P., Amultivariate empirical Bayes statistic for replicated microarray time course data. Ann. Statist. 34 (5) (2006), to appear.
81. Tibshirani, R., Regression shrinkage and selection via the LASSO. J. Royal Statist. Soc. B 58 (1996), 267-288.
82. Tibshirani, R., Hastie, T., Narasimhan, B., and Chu, G., Class prediction by nearest shrunken centroids, with applications to DNA microarrays. Statist. Sci. 18 (2003), 104-117.
83. Tseng, G. C., Oh, M. K., Rohlin, L., Liao, J. C., and Wong, W. H., Issues inc DNA micro array analysis: quality filtering, channel normalization, models of variations and assessment of gene effects. Nucleic Acids Res. 29 (2001), 2549-2557.
84. Tusher, V. G., Tibshirani, R., and Chu, G., Significance analysis of microarrays applied to the ionizing radiation response. Proc. Natl. Acad. Sci. USA 98 (2001), 5116-5121.
85. Vapnik, V., The Nature of Statistical Learning. Springer-Verlag, New York 1995.
86. Walls, T., and Schater, J., Models for Intensive Longitudinal Data, Oxford University Press, New York 2006.
87. Wang, Y., Selective review on wavelets in Statistics. In Frontiers of Statistics (ed. by J. Fan and H. Koul), Imperial College Press, 2006.
88. Zhang, H. P., Yu, C.Y., and Singer, B., Cell and tumor classification using gene expression aata: Construction of forests. Proc. Natl. Acad. Sci. USA 100 (2003), 4168-4172.
89. Zhu, J., Rosset, S., Hastie, T., and Tibshirani, R., 1-norm support vector machines. Neural Information Processing Systems 16 (2003).
90. Zou, H., The adaptive Lasso and its oracle properties. Manuscript, 2005.