Novel high intrinsic dimensionality estimators

Machine Learning

Volumn 89, Issue 1-2, 2012, Pages 37-65

  Rozza, A ^a   Lombardi, G ^a   Ceruti, C ^a   Casiraghi, E ^a   Campadelli, P ^a  

^a UNIVERSITY OF MILAN   (Italy)

Author keywords

Dimensionality reduction; Intrinsic dimensionality estimation; Manifold learning

Indexed keywords

DATA SETS; DIMENSIONALITY ESTIMATION; DIMENSIONALITY REDUCTION; DIMENSIONALITY REDUCTION TECHNIQUES; ESTIMATION TECHNIQUES; GENERALIZATION CAPABILITY; HIDDEN UNITS; INFORMATION LOSS; MANIFOLD LEARNING; MIDDLE LAYER; MODEL ORDER; NEURAL NETWORK DESIGNS; NOISY DATA; OBJECTIVE EVALUATION; REAL DATA SETS; RELIABLE ESTIMATES; STATE-OF-THE-ART ALGORITHMS; STATISTICAL PROPERTIES; TIME SERIES PREDICTION;

ALGORITHMS; DATA PROCESSING; NEURAL NETWORKS; TIME SERIES;

ESTIMATION;

EID: 84865202317     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-012-5294-7     Document Type: Article

References (57)

1. A. F. Abate M. Nappi D. Riccio G. Sabatino 2007 2d and 3d face recognition: a survey Pattern Recognition Letters 28 1885 1906 10.1016/j.patrec.2006.12.018 (Pubitemid 47259204)
2. Baraniuk, R., & Wakin, R. (2006). Random projections of smooth manifolds. In Foundations of computational mathematics.
3. Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford: Oxford University Press.
4. C. M. Bishop 1998 Bayesian PCA Advances in Neural Information Processing Systems 11 382 388
5. F. Camastra 2003 Data dimensionality estimation methods: a survey Pattern Recognition 36 12 2945 2954 1059.68100 10.1016/S0031-3203(03)00176-6 (Pubitemid 37293405)
6. F. Camastra M. Filippone 2009 A comparative evaluation of nonlinear dynamics methods for time series prediction Neural Computing & Applications 18 8 1021 1029 10.1007/s00521-009-0266-y
7. F. Camastra A. Vinciarelli 2002 Estimating the intrinsic dimension of data with a fractal-based method IEEE Transactions on Pattern Analysis and Machine Intelligence 24 1404 1407 10.1109/TPAMI.2002.1039212 (Pubitemid 35327462)
8. L. Chen H. Liao M. Ko J. Lin G. Yu 2000 A new LDA-based face recognition system which can solve the small sample size problem Pattern Recognition 30 1713 1726 10.1016/S0031-3203(99)00139-9
9. J. Chen H. R. Fang Y. Saad 2009 Fast approximate kNN graph construction for high dimensional data via recursive Lanczos bisection Journal of Machine Learning Research 10 1989 2012 1235.68137
10. L. Chua M. Komuro T. Matsumoto 1985 The double scroll IEEE Transactions on Circuits and Systems 32 797 818 0578.94023 10.1109/TCS.1985.1085791
11. Clarkson, K. L. (2008). Tighter bounds for random projections of manifolds. In M. Teillaud (Ed.), Symposium on computational geometry (pp. 39-48). New York: ACM.
12. T. F. Coleman Y. Li 1996 An interior, trust region approach for nonlinear minimization subject to bounds SIAM Journal on Optimization 6 418 445 0855.65063 10.1137/0806023
13. M. Connor P. Kumar 2010 Fast construction of k-nearest neighbor graphs for point clouds IEEE Transactions on Visualization and Computer Graphics 16 4 599 608 10.1109/TVCG.2010.9
14. J. A. Costa A. O. Hero 2004 Geodesic entropic graphs for dimension and entropy estimation in manifold learning IEEE Transactions on Signal Processing 52 8 2210 2221 10.1109/TSP.2004.831130
15. Costa, J. A., & Hero, A. O. (2004). Learning intrinsic dimension and entropy of high-dimensional shape spaces. In Proceedings of European signal processing conference (EUSIPCO).
16. Costa, J. A., & Hero, A. O. (2005). Learning intrinsic dimension and entropy of shapes. In H. Krim & T. Yezzi (Eds.), Statistics and analysis of shapes. Basel: Birkhäuser.
17. J. P. Eckmann D. Ruelle 1992 Fundamental limitations for estimating dimensions and Lyapunov exponents in dynamical systems Physica D: Nonlinear Phenomena 56 2-3 185 187 0759.58030 10.1016/0167-2789(92)90023-G
18. Farahmand, A. M., Szepesvari, C., & Audibert, J. Y. (2007). Manifold-adaptive dimension estimation. In Proceedings of international conference on machine learning (ICML).
19. Fishman, G. S. (1996). Monte Carlo: concepts, algorithms, and applications. Springer series in operations research. New York: Springer.
20. Frank, A., & Asuncion, A. (2010). UCI machine learning repository.
21. J. H. Friedman 1989 Regularized discriminant analysis Journal of the American Statistical Association 84 165 175 10.1080/01621459.1989.10478752
22. Friedman, J. H., Hastie, T., & Tibshirani, R. (2009). The elements of statistical learning-data mining, inference and prediction. Berlin: Springer.
23. K. Fukunaga 1971 An algorithm for finding intrinsic dimensionality of data IEEE Transactions on Computers 20 176 183 0216.50201 10.1109/T-C.1971. 223208
24. Fukunaga, K. (1982). Intrinsic dimensionality extraction. In P. R. Krishnaiah & L. N. Kanal (Eds.), Classification, pattern recognition and reduction of dimensionality. Amsterdam: North-Holland.
25. P. Grassberger I. Procaccia 1983 Measuring the strangeness of strange attractors Physica D: Nonlinear Phenomena 9 189 208 0593.58024 10.1016/0167-2789(83)90298-1
26. Y. Guan J. G. Dy 2009 Sparse probabilistic principal component analysis Journal of Machine Learning Research-Proceedings Track 5 185 192
27. Hegde, C., Wakin, M. B., & Baraniuk, R. G. (2007). Random projections for manifold learning. In Advances in neural information processing systems.
28. Hein, M. (2005). Intrinsic dimensionality estimation of submanifolds in Euclidean space. In Proceedings of international conference on machine learning (ICML) (pp. 289-296). (Pubitemid 43183345)
29. Jollife, I. T. (1961). Adaptive control processes: a guided tour. Princeton: Princeton University Press.
30. Jollife, I. T. (1986). Principal component analysis. Springer series in statistics. New York: Springer.
31. Kégl, B. (2002). Intrinsic dimension estimation using packing numbers. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Proceedings of neural information processing systems (NIPS) (pp. 681-688). Cambridge: MIT Press.
32. Kirby, M. (1998). Geometric data analysis: an empirical approach to dimensionality reduction and the study of patterns. New York: Wiley.
33. Kivimäki, I., Lagus, K., Nieminen, I., Väyrynen, J., & Honkela, T. (2010). Using correlation dimension for analysing text data. In Proceedings of the 20th international conference on Artificial neural networks: Part I (ICANN2010) (pp. 368-373). Berlin: Springer.
34. Y. LeCun L. Bottou Y. Bengio P. Haffner 1998 Gradient-based learning applied to document recognition Proceedings of the IEEE 86 11 2278 2324 10.1109/5.726791
35. Lee, J. A., & Veleysen, M. (2007). Nonlinear dimensionality reduction. Springer series in information science and statistics. Berlin: Springer.
36. E. Levina P. J. Bickel 2005 Maximum likelihood estimation of intrinsic dimension Advances in Neural Information Processing Systems 17 1 777 784
37. Li, J., & Tao, D. (2010). Simple exponential family PCA. In Proceedings of international conference on artificial intelligence and statistics (AISTATS) (pp. 453-460).
38. Lombardi, G., Casiraghi, E., & Campadelli, P. (2009). The neighbors voting algorithm and its applications. In Studies in computational intelligence: Vol. 245. Applications of supervised and unsupervised ensemble methods (SUEMA) (pp. 151-173).
39. Lombardi, G., Rozza, A., Ceruti, C., Casiraghi, E., & Campadelli, P. (2011). Minimum neighbor distance estimators of intrinsic dimension. In Proceedings of European conference on machine learning (ECML) (Vol. 6912, pp. 374-389).
40. D. J. C. MacKay 1995 Probable networks and plausible predictions-a review of practical Bayesian methods for supervised neural networks Network: Computation in Neural Systems 6 3 469 505 0834.68098 10.1088/0954-898X/6/3/011
41. MacKay, D. J. C., & Ghahramani, Z. (2005). Comments on maximum likelihood estimation of intrinsic dimension by Elizaveta Levina and Peter Bickel. http://www.inference.phy.cam.ac.uk/mackay/dimension/.
42. P. Mordohai G. Medioni 2010 Dimensionality estimation, manifold learning and function approximation using tensor voting Journal of Machine Learning Research 11 411 450 1242.68239
43. Ott, E. (1993). Chaos in dynamical systems. Cambridge: Cambridge University Press.
44. Paredes, R., Chávez, E., Figueroa, K., & Navarro, G. (2006). Practical construction of k-nearest neighbor graphs in metric spaces. In C. Àlvarez & M. J. Serna (Eds.), Lecture notes in computer science: Vol. 4007. WEA (pp. 85-97). Berlin: Springer. (Pubitemid 44070929)
45. K. Pettis T. Bailey A. Jain R. Dubes 1979 An intrinsic dimensionality estimator from near-neighbor information IEEE Transactions on Pattern Analysis and Machine Intelligence 1 1 25 37 0418.68074 10.1109/TPAMI.1979.4766873 (Pubitemid 9468506)
46. Pineda, F. J., & Sommerer, J. C. (1994). Estimating generalized dimensions and choosing time delays: a fast algorithm. In Time series prediction. Forecasting the future and understanding the past (pp. 367-385).
47. S. T. Roweis L. K. Saul 2000 Nonlinear dimensionality reduction by locally linear embedding Science 290 2323 2326 10.1126/science.290.5500.2323 (Pubitemid 32041578)
48. Rozza, A., Lombardi, G., Re, M., Casiraghi, E., & Valentini, G. (2010). DDAG K-TIPCAC: an ensemble method for protein subcellular localization. In Proceedings of European conference on machine learning PKDD-(SUEMA) workshop (pp. 75-84).
49. Rozza, A., Lombardi, G., Rosa, M., & Casiraghi, E. (2010). O-IPCAC and its application to EEG classification. In Proceedings of workshop on applications of pattern analysis (WAPA) (pp. 4-11).
50. Rozza, A., Lombardi, G., Casiraghi, E., & Campadelli, P. (2012). Novel Fisher discriminant classifiers. Pattern Recognition (in press).
51. Rozza, A., Lombardi, G., Rosa, M., Casiraghi, E., & Campadelli, P. (2011). IDEA: intrinsic dimension estimation algorithm. In Proceedings of international conference on image analysis and processing (ICIAP) (Vol. 6978, pp. 433-442).
52. J. B. Tenenbaum V. Silva J. C. Langford 2000 A global geometric framework for nonlinear dimensionality reduction Science 290 2319 2323 10.1126/science.290.5500.2319 (Pubitemid 32041577)
53. M. E. Tipping C. M. Bishop 1997 Probabilistic principal component analysis Journal of the Royal Statistical Society. Series B 61 Part 3 611 622
54. Vapnik, V. (1998). Statistical learning theory. New York: Wiley.
55. P. J. Verveer R. P. W. Duin 1995 An evaluation of intrinsic dimensionality estimators IEEE Transactions on Pattern Analysis and Machine Intelligence 17 81 86 10.1109/34.368147
56. Wang, Q., Kulkarni, S. R., & Verdú, S. (2006). A nearest-neighbor approach to estimating divergence between continuous random vector. In IEEE international symposium on information theory (ISIT2006) (pp. 242-246). (Pubitemid 351244090)
57. H. Zou T. Hastie R. Tibshirani 2004 Sparse principal component analysis Journal of Computational and Graphical Statistics 15 262 286