Treelets-an adaptive multi-scale basis for sparse unordered data

Annals of Applied Statistics

Volumn 2, Issue 2, 2008, Pages 435-471

  Lee, Ann B ^a   Nadler, Boaz ^a   Wasserman, Larry ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Dimensionality reduction; Feature selection; Hierarchical clustering; Local best basis; Multi resolution analysis; Principal component analysis; Small sample sizes; Sparsity

Indexed keywords

EID: 66549093404     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/07-AOAS137     Document Type: Article

References (53)

1. Ahn, J. and Marron, J. S. (2008). Maximal data piling in discrimination., Biometrika. To appear.
2. Angeletti, C., Harvey, N. R., Khomitch, V., Fischer, A. H., Levenson, R. M. and Rimm, D. L. (2005). Detection of malignancy in cytology specimens using spectral-spatial analysis., Laboratory Investigation 85 1555-1564.
3. Asimov, D. (1985). The Grand Tour: A tool for viewing multidimensional data., SIAM J. Sci. Comput. 6 128-143. Mathematical Reviews (MathSciNet): MR773286 Digital Object Identifier: doi:10.1137/0906011
4. Bair, E., Hastie, T., Paul, D. and Tibshirani, R. (2006). Prediction by supervised principal components., J. Amer. Statist. Assoc. 101 119-137. Mathematical Reviews (MathSciNet): MR2252436 Zentralblatt MATH: 1118.62326 Digital Object Identifier: doi:10.1198/016214505000000628
5. Belkin, M. and Niyogi, P. (2005). Semi-supervised learning on Riemannian manifolds., Machine Learning 56 209-239.
6. Beran, R. and Srivastava, M. (1985). Bootstrap tests and confidence regions for functions of a covariance matrix., Ann. Statist. 13 95-115. Mathematical Reviews (MathSciNet): MR773155 Digital Object Identifier: doi:10.1214/aos/1176346579 Project Euclid: euclid.aos/1176346579
7. Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices., Ann. Statist. 36 199-227. Mathematical Reviews (MathSciNet): MR2387969 Zentralblatt MATH: 1132.62040 Digital Object Identifier: doi:10.1214/009053607000000758 Project Euclid: euclid.aos/1201877299
8. Buckheit, J. and Donoho, D. (1995). Improved linear discrimination using time frequency dictionaries. In, Proc. SPIE 2569 540-551.
9. Candès, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when, p is much larger than n (with discussion). Ann. Statist. 35 2313-2404. Mathematical Reviews (MathSciNet): MR2382644 Zentralblatt MATH: 1139.62019 Digital Object Identifier: doi:10.1214/009053606000001523 Project Euclid: euclid.aos/1201012958
10. Coifman, R., Lafon, S., Lee, A., Maggioni, M., Nadler, B., Warner, F. and Zucker, S. (2005). Geometric diffusions as a tool for harmonics analysis and structure definition of data: Diffusion maps., Proc. Natl. Acad. Sci. 102 7426-7431.
11. Coifman, R. and Saito, N. (1996). The local Karhunen-Loève basis. In, Proc. IEEE International Symposium on Time-Frequency and Time-Scale Analysis 129-132. IEEE Signal Processing Society.
12. Coifman, R. and Wickerhauser, M. (1992). Entropy-based algorithms for best basis selection. In, Proc. IEEE Trans. Inform. Theory. 32 712-718.
13. Dettling, M. and Bühlmann, P. (2004). Finding predictive gene groups from microarray data., J. Multivariate Anal. 90 106-131.
14. Donoho, D. and Elad, M. (2003). Maximal sparsity representation via, l1 minimization. Proc. Natl. Acad. Sci. USA 100 2197-2202. Mathematical Reviews (MathSciNet): MR1963681 Zentralblatt MATH: 1064.94011 Digital Object Identifier: doi:10.1073/pnas.0437847100
15. Donoho, D. and Johnstone, I. (1995). Adapting to unknown smoothness via wavelet shrinkage., J. Amer. Statist. Assoc. 90 1200-1224. Mathematical Reviews (MathSciNet): MR1379464 Zentralblatt MATH: 0869.62024 Digital Object Identifier: doi:10.2307/2291512 JSTOR: links.jstor.org
16. Eisen, M., Spellman, P., Brown, P. and Botstein, D. (1998). Cluster analysis and display of genome-wide expression patterns., Proc. Natl. Acad. Sci. USA 95 14863-14868.
17. Fraley, C. and Raftery, A. E. (2002). Model-based clustering, discriminant analysis, and density estimation., J. Amer. Statist. Assoc. 97 611-631. Mathematical Reviews (MathSciNet): MR1951635 Zentralblatt MATH: 1073.62545 Digital Object Identifier: doi:10.1198/016214502760047131 JSTOR: links.jstor.org
18. Golub, G. and van Loan, C. F. (1996)., Matrix Computations, 3rd ed. Johns Hopkins Univ. Press. Mathematical Reviews (MathSciNet): MR1417720
19. Golub, T. R., Slonim, D. K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P. Coller, H., Lob, M. L., Downing, J. R., Caliguiri, M., Bloomfield, C. and Lander, E. (1999). Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring., Science 286 531-537. Zentralblatt MATH: 1047.65504
20. Gruber, M. (1998)., Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estimators. Dekker, New York. Mathematical Reviews (MathSciNet): MR1608582 Zentralblatt MATH: 0920.62085
21. Guyon, I., Weston, J., Barnhill, S. and Vapnik, V. (2002). Gene selection for cancer classification using support vector machines., Machine Learning 46 389-422.
22. Hall, P., Marron, J. S. and Neeman, A. (2005). Geometric representation of high dimension, low sample size data., J. R. Stat. Soc. Ser. B Stat. Methodol. 67 427-444. Mathematical Reviews (MathSciNet): MR2155347 Zentralblatt MATH: 1069.62097 Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00510.x JSTOR: links.jstor.org
23. Hastie, T., Tibshirani, R., Botstein, D. and Brown, P. (2001). Supervised harvesting of expression trees., Genome Biology 2 research0003.1-0003.12.
24. Hastie, T., Tibshirani, R. and Friedman, J. (2001)., The Elements of Statistical Learning. Springer, New York. Mathematical Reviews (MathSciNet): MR1851606
25. Jain, A. K., Murty, M. N. and Flynn, P. J. (1999). Data clustering: A review., ACM Computing Surveys 31 264-323.
26. Johnstone, I. and Lu, A. (2008). Sparse principal component analysis., J. Amer. Statist. Assoc. To appear.
27. Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal component analysis., Ann. Statist. 29 295-327. Mathematical Reviews (MathSciNet): MR1863961 Zentralblatt MATH: 1016.62078 Digital Object Identifier: doi:10.1214/aos/1009210544 Project Euclid: euclid.aos/1009210544
28. Jolliffe, I. T. (2002)., Principal Component Analysis, 2nd ed. Springer, New York. Mathematical Reviews (MathSciNet): MR2036084
29. Kalisch, M. and Bühlmann, P. (2007). Estimating high-dimensional directed acyclic graphs with the pc-algorithm., J. Machine Learning Research 8 613-636.
30. Kushmerick, N. (1999). Learning to remove internet advertisements. In, Proceedings of the Third Annual Conference on Autonomous Agents 175-181.
31. Lee, A. and Nadler, B. (2007). Treelets-a tool for dimensionality reduction and multi-scale analysis of unstructured data. In, Proc. of the Eleventh International Conference on Artificial Intelligence and Statistics (M. Meila and Y. Shen, eds.).
32. Levina, E. and Zhu, J. (2007). Sparse estimation of large covariance matrices via a hierarchical lasso penalty., Submitted.
33. Lorber, A., Faber, K. and Kowalski, B. R. (1997). Net analyte signal calculation in multivariate calibration., Anal. Chemometrics 69 1620-1626.
34. Mallat, S. (1998)., A Wavelet Tour of Signal Processing. Academic Press, San Diego, CA. Mathematical Reviews (MathSciNet): MR1614527
35. Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso., Ann. Statist. 34 1436-1462.
36. Murtagh, F. (2004). On ultrametricity, data coding, and computation., J. Classification 21 167-184. Mathematical Reviews (MathSciNet): MR2100389 Zentralblatt MATH: 1084.62052 Digital Object Identifier: doi:10.1007/s00357-004-0015-y
37. Murtagh, F. (2007). The Haar wavelet transform of a dendrogram., J. Classification 24 3-32. Mathematical Reviews (MathSciNet): MR2370773 Zentralblatt MATH: 1141.65094 Digital Object Identifier: doi:10.1007/s00357-007-0007-9
38. Murtagh, F., Starck, J.-L. and Berry, M. W. (2000). Overcoming the curse of dimensionality in clustering by means of the wavelet transform., Computer J. 43 107-120.
39. Nadler, B. (2007). Finite sample approximation results for principal component analysis: A matrix perturbation approach., Submitted. Mathematical Reviews (MathSciNet): MR2485013 Zentralblatt MATH: 1168.62058 Digital Object Identifier: doi:10.1214/08-AOS618 Project Euclid: euclid.aos/1231165185
40. Nadler, B. and Coifman, R. (2005a). Partial least squares, Beer's law and the net analyte signal: Statistical modeling and analysis., J. Chemometrics 19 45-54.
41. Nadler, B. and Coifman, R. (2005b). The prediction error in CLS and PLS: The importance of feature selection prior to multivariate calibration., J. Chemometrics 19 107-118.
42. Ogden, R. T. (1997)., Essential Wavelets for Statistical Applications and Data Analysis. Birkhäuser, Boston. Mathematical Reviews (MathSciNet): MR1420193 Zentralblatt MATH: 0868.62033
43. Saito, N. and Coifman, R. (1995). On local orthonormal bases for classification and regression. In, On Local Orthonormal Bases for Classification and Regression 1529-1532. IEEE Signal Processing Society.
44. Saito, N., Coifman, R., Geshwind, F. B. and Warner, F. (2002). Discriminant feature extraction using empirical probability density estimation and a local basis library., Pattern Recognition 35 2841-2852.
45. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso., J. Roy. Statist. Soc. Ser. B 58 267-288. Mathematical Reviews (MathSciNet): MR1379242 JSTOR: links.jstor.org
46. Tibshirani, R., Hastie, T., Eisen, M., Ross, D., Botstein, D. and Brown, P. (1999). Clustering methods for the analysis of DNA microarray data. Technical report, Dept. Statistics, Stanford, Univ.
47. Tibshirani, R., Hastie, T., Narasimhan, B. and Chu, G. (2002). Diagnosis of multiple cancer types by shrunken centroids of gene expression., Proc. Natl. Acad. Sci. 99 6567-6572.
48. Whittaker, J. (2001)., Graphical Models in Multivariate Statistics. Wiley, New York.
49. Xu, R. and Wunsch, D. (2005). Survey of clustering algorithms., IEEE Trans. Neural Networks 16 645-678.
50. Zhao, Z. and Liu, H. (2007). Searching for interacting features. In, Proceedings of the 20th International Joint Conference on AI (IJCAI-07).
51. Zhu, J. and Hastie, T. (2004). Classification of gene microarrays by penalized logistic regression., Biostatistics 5 427-444.
52. Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net., J. Roy. Statist. Soc. Ser. B 67 301-320. Mathematical Reviews (MathSciNet): MR2137327 Zentralblatt MATH: 1069.62054 Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00503.x JSTOR: links.jstor.org
53. Zou, H., Hastie, T. and Tibshirani, R. (2006). Sparse principal component analysis., J. Comput. Graph. Statist. 15 265-286. Mathematical Reviews (MathSciNet): MR2252527 Digital Object Identifier: doi:10.1198/106186006X113430