Regression on manifolds: Estimation of the exterior derivative

Annals of Statistics

Volumn 39, Issue 1, 2011, Pages 48-81

  Aswani, Anil ^a   Bickel, Peter ^a   Tomlin, Claire ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Collinearity; Manifold; Model selection; Nonparametric regression; Regularization

Indexed keywords

EID: 79551608900     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/10-AOS823     Document Type: Article

References (61)

1. AITCHISON, P. W. (1982). Generalized inverse matrices and their applications. Internat. J. Math. Ed. Sci. Tech. 13 99-109. MR0646552
2. ANDERSSON, M. (2009). A comparison of nine PLS1 algorithms. J. Chemometrics 23 518-529.
3. ASWANI, A., BICKEL, P. and TOMLIN, C. (2009). Statistics for sparse, high-dimensional, and nonparametric system identification. In IEEE International Conference on Robotics and Automation 2133-2138. IEEE Press, Piscataway, NJ.
4. ASWANI, A., KERÄNEN, S., BROWN, J., FOWLKES, C., KNOWLES, D., BIGGIN, M., BICKEL, P. and TOMLIN, C. (2010). Nonparametric identification of regulatory interactions from spatial and temporal gene expression data. BMC Bioinformatics 11 413.
5. BELKIN, M., NIYOGI, P. and SINDHWANI, V. (2006). Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7 2399-2434. MR2274444
6. BHATIA, R. (2007). Perturbation Bounds for Matrix Eigenvalues. Classics in Applied Mathematics 53. SIAM, Philadelphia, PA. MR2325304
7. BICKEL, P. and LEVINA, E. (2008). Covariance regularization by thresholding. Ann. Statist. 36 2577-2604. MR2485008
8. BICKEL, P. and LI, B. (2007). Local polynomial regression on unknown manifolds. In Complex Datasets and Inverse Problems: Tomography, Networks and Beyond. Institute of Mathematical Statistics Lecture Notes-Monograph Series 54 177-186. Inst. Math. Statist., Beachwood, OH. MR2459188
9. BICKEL, P. and FREEDMAN, D. (1981). Some asymptotic theory for the bootstrap. Ann. Statist. 9 1196-1217. MR0630103
10. CANDES, E. and TAO, T. (2007). The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist. 35 2313-2351. MR2382644
11. CARROLL, R., MACA, J. and RUPPERT, D. (1999). Nonparametric regression in the presence of measurement error. Biometrika 86 541-554. MR1723777
12. COSTA, J. and HERO, A. (2004). Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Trans. Signal Process. 52 2210-2221. MR2085582
13. FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. MR1946581
14. FAN, J. and TRUONG, Y. (1993). Nonparametric regression with errors in variables. Ann. Statist. 21 1900-1925. MR1245773
15. FAN, J. and GIJBELS, I. (1996). Local Polynomial Modelling and its Applications. Monographs on Statistics and Applied Probability 66. Chapman and Hall, London. MR1383587
16. FRANK, I. E. and FRIEDMAN, J. H. (1993). A statistical view of some chemometrics regression tools. Technometrics 35 109-135.
17. FRIEDMAN, J., HASTIE, T., HÖFLING, H. and TIBSHIRANI, R. (2007). Pathwise coordinate optimization. Ann. Appl. Statist. 1 302-332. MR2415737
18. FU, W. (2000). Ridge estimator in singular design with application to age-period-cohort analysis of disease rates. Comm. Statist. Theory Methods 29 263-278.
19. FU, W. (2008). A smooth cohort model in age-period-cohort analysis with applications to homicide arrest rates and lung cancer mortality rates. Sociol. Methods Res. 36 327-361. MR2422770
20. GEYER, C. (1994). On the asymptotics of constrained m-estimation. Ann. Statist. 22 1993-2010. MR1329179
21. GOLDBERG, Y., ZAKAI, A., KUSHNIR, D. and RITOV, Y. (2008).Manifold learning: The price of normalization. J.Mach. Learn. Res. 9 1909-1939. MR2438829
22. GOLUB, G. and VAN LOAN, C. (1996). Matrix Computations, 3rd ed. Johns Hopkins Univ. Press, Baltimore, MD. MR1417720
23. HEIN, M. and AUDIBERT, J.-Y. (2005). Intrinsic dimensionality estimation of submanifolds in Rd. In International Conference on Machine Learning 289-296. ACM, New York.
24. HELLAND, I. (1988). On the structure of partial least squares regression. Comm. Statist. Simulation Comput. 17 581-607. MR0955342
25. HOCKING, R. (1976). The analysis and selection of variables in linear regression. Biometrics 32 431-453. MR0398008
26. HOERL, A. E. and KENNARD, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 8 27-51.
27. HUFFEL, S.V. and VANDEWALLE, J. (1991). The Total Least Squares Problem: Computational Aspects and Analysis. SIAM, Philadelphia, PA. MR1118607
28. JOHNSTONE, I.M. and LU, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682-693.
29. KAROUI, N. E. (2008). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. 36 2717-2756. MR2485011
30. KNIGHT, K. and FU, W. (2000). Asymptotics for lasso-type estimators. Ann. Statist. 28 1356-1378. MR1805787
31. KRITCHMAN, S. and NADLER, B. (2008). Determining the number of components in a factor model from limited noisy data. Chemometrics and Intelligent Laboratory Systems 94 19-32.
32. LAFFERTY, J. and WASSERMAN, L. (2006). Rodeo: Sparse nonparametric regression in high dimensions. In Advances in Neural Information Processing Systems (NIPS) 18 707-714. MIT Press, Cambridge, MA.
33. LEDOIT, O. andWOLF, M. (2003). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365-411. MR2026339
34. LEE, J. (2003). Introduction to Smooth Manifolds. Springer, New York. MR1930091
35. LEVINA, E. and BICKEL, P. (2005). Maximum likelihood estimation of intrinsic dimension. In Advances in NIPS 17 777-784. MIT Press, Cambridge, MA.
36. LUGOSI, G. (2006). Concentration-of-measure inequalities. Technical report, Pompeu Fabra Univ.
37. MASSY, W. F. (1965). Principal components regression in exploratory statistical research. J. Amer. Statist. Assoc. 60 234-246.
38. MEINSHAUSEN, N. and YU, B. (2009). Lasso-type recovery of sparse representations for highdimensional data. Ann. Statist. 37 246-270. MR2488351
39. MEINSHAUSEN, N. and BUEHLMANN, P. (2010). Stability selection. J. Roy. Statist. Soc. Ser. B 72 417-473.
40. MISNER, C.W., THORNE, K. S. andWHEELER, J. A. (1973). Gravitation. W. H. Freeman and Co., San Francisco, CA. MR0418833
41. NADLER, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791-2817. MR2485013
42. NG, A. (1997). Preventing overfitting of cross-validation data. In 14th International Conference on Machine Learning 245-253. Morgan Kaufmann, San Francisco, CA.
43. NIYOGI, P. (2008). Manifold regularization and semi-supervised learning: Some theoretical analyses. Technical Report TR-2008-01, Univ. Chicago, Computer Science Dept.
44. RAO, R., FUNG, G. and ROSALES, R. (2008). On the dangers of cross-valdation: An experimental evaluation. In SIAM Data Mining. SIAM, Philadelphia, PA.
45. REUNANEN, J. (2003). Overfitting in making comparisons between variable selection methods. J.Mach. Learn. Res. 3 1371-1382.
46. RUPPERT, D. and WAND, M. (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 1346-1370. MR1311979
47. SASTRY, S. (1999). Nonlinear Systems. Springer, New York. MR1693648
48. SHAO, J. (1993). Linear model selection by cross-validation. J. Amer. Statist. Assoc. 88 486-494. MR1224373
49. SHAO, J. (1994). Bootstrap sample size in nonregular cases. Proc. Amer. Math. Soc. 122 1251-1262. MR1227529
50. SHAO, J. (1996). Bootstrap model selection. J. Amer. Statist. Assoc. 91 655-665. MR1395733
51. SPIVAK, M. (1965). Calculus on Manifolds. A Modern Approach to Classical Theorems of Advanced Calculus. W. A. Benjamin, Inc., New York.
52. STEWART, G. and SUN, J. (1990). Matrix Perturbation Theory. Academic Press, Boston, MA. MR1061154
53. TENENBAUM, J.B., DE SILVA, V. and LANGFORD, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science 290 2319-2323.
54. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. MR1379242
55. WOLD, H. (1975). Soft modeling by latent variables: The nonlinear iterative partial least squares approach. In Perspectives in Probability and Statistics, Papers in Honour of M. S. Bartlett (J. Gani, ed.) 117-142. Univ. Sheffield, Sheffield. MR0431486
56. WRIGHT, J., YANG, A., GANESH, A., SASTRY, S. andMA, Y. (2009). Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31 210-227.
57. WU, T. T. and LANGE, K. (2008). Coordinate descent algorithms for lasso penalized regression. Ann. Appl. Statist. 2 224-244. MR2415601
58. YANG, Y., FU, W. and LAND, K. (2004). A methodological comparison of age-period-cohort models: The intrinsic estimator and conventional generalized linear models. Sociological Methodology 34 75-110.
59. ZOU, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418-1429. MR2279469
60. ZOU, H. and HASTIE, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 301-320. MR2137327
61. ZOU, H., HASTIE, T. and TIBSHIRANI, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265-286. MR2252527