Nonlinear estimators and tail bounds for dimension reduction in using cauchy random projections

Journal of Machine Learning Research

Volumn 8, Issue , 2007, Pages 2497-2532

  Ping, Li ^a   Hastie, Trevor J ^b   Church, Kenneth W ^c  

^a Department of Obstetrics Gynecology   (United States)

^b STANFORD UNIVERSITY   (United States)

^c MICROSOFT RESEARCH   (United States)

Author keywords

Cauchy random projections; Dimension reduction; Johnson Lindenstrauss (JL) lemma; l1 norm

Indexed keywords

ASYMPTOTIC ANALYSIS; DATA MINING; INFORMATION RETRIEVAL; MAXIMUM LIKELIHOOD ESTIMATION; RANDOM PROCESSES;

CAUCHY RANDOM PROJECTIONS; DIMENSION REDUCTION; GEOMETRIC MEAN ESTIMATORS; INVERSE GAUSSIAN; TAIL BOUNDS;

NONLINEAR SYSTEMS;

EID: 35748971790     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (70)

1. Dimitris Achlioptas. Database-friendly random projections: Johnson-Lindenstrauss with binary coins. Journal of Computer and System Sciences, 66(4):671-687, 2003.
2. Dimitris Achlioptas. Database-friendly random projections. In PODS, pages 274-281, Santa Barbara, CA, 2001.
3. Charu C. Aggarwal and Joel L. Wolf. A new method for similarity indexing of market basket data. In SIGMOD, pages 407-418, Philadelphia, PA, 1999.
4. Nir Ailon and Bernard Chazelle. Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform. In STOC, pages 557-563, Seattle, WA, 2006.
5. Charles Antle and Lee Bain. A property of maximum likelihood estimators of location and scale parameters. SIAM Review, 11(2):251-253, 1969.
6. Rosa Arriaga and Santosh Vempala. An algorithmic theory of learning: Robust concepts and random projection. Machine Learning, 63(2): 161-182, 2006.
7. Rosa Arriaga and Santosh Vempala. An algorithmic theory of learning: Robust concepts and random projection. In FOCS, pages 616-623, New York, 1999.
8. Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. Models and issues in data stream systems. In PODS, pages 1-16, Madison, WI, 2002.
9. Maurice S. Bartlett. Approximate confidence intervals, II. Biometrika, 40(3/4):306-317, 1953.
10. Rabi N. Bhattacharya and Jayanta K. Ghosh. On the validity of the formal edgeworth expansion. The Annals of Statistics, 6(2):434-451, 1978.
11. 1. Journal of ACM, 52(2):766-788, 2005.
12. 1. In FOCS, pages 514-523, Cambridge, MA, 2003.
13. Olivier Chapelle, Patrick Haffner, and Vladimir N. Vapnik. Support vector machines for histogram-based image classification. IEEE Trans. Neural Networks, 10(5):1055-1064, 1999.
14. Herman Chernoff. A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. The Annals of Mathematical Statistics, 23(4):493-507, 1952.
15. Raj S. Chhikara and J. Leroy Folks. The Inverse Gaussian Distribution: Theory, Methodology, and Applications. Marcel Dekker, Inc, New York, 1989.
16. Graham Cormode and S. Muthukrishnan. Estimating dominance norms of multiple data streams. In ESA, pages 148-160, 2003.
17. Graham Cormode, Mayur Datar, Piotr Indyk, and S. Muthukrishnan. Comparing data streams using hamming norms (how to zero in). In VLDB, pages 335-345, Hong Kong, China, 2002.
18. Graham Cormode, Mayur Datar, Piotr Indyk, and S. Muthukrishnan. Comparing data streams using hamming norms (how to zero in). IEEE Transactions on Knowledge and Data Engineering, 15 (3):529-540, 2003.
19. Francisco Jose De. A. Cysneiros, Sylvio Jose R dos Santos, and Gass M. Cordeiro. Skewness and kurtosis for maximum likelihood estimator in one-parameter exponential family models. Brazilian Journal of Probability and Statistics, 15(1):85-105, 2001.
20. Sanjoy Dasgupta and Anupam Gupta. An elementary proof of a theorem of Johnson and Lindenstrauss. Random Structures and Algorithms, 22(1):60-65, 2003.
21. Herbert A. David. Order Statistics. John Wiley & Sons, Inc., New York, NY, second edition, 1981.
22. Inderjit S. Dhillon and Dharmendra S. Modha. Concept decompositions for large sparse text data using clustering. Machine Learning, 42(1-2):143-175, 2001.
23. Bradley Efron, Trevor Hastie, Iain Johnstone, and Robert Tibshirani. Least angle regression. The Annals of Statistics, 32(2):407-499, 2004.
24. Eugene F. Fama and Richard Roll. Some properties of symmetric stable distributions. Journal of the American Statistical Association, 63(323):817-836, 1968.
25. Eugene F. Fama and Richard Roll. Parameter estimates for symmetric stable distributions. Journal of the American Statistical Association, 66(334):331-338, 1971.
26. 1-difference algorithm for massive data streams. In FOCS, pages 501-511, New York, 1999.
27. William Feller. An Introduction to Probability Theory and Its Applications (Volume II). John Wiley & Sons, New York, NY, second edition, 1971.
28. Silvia L. R Ferrari, Denise A. Botter, Gauss M. Cordeiro, and Francisco Cribari-Neto. Second and third order bias reduction for one-parameter family models. Stat. and Prob. Letters, 30:339-345, 1996.
29. Ronald A. Fisher. Two new properties of mathematical likelihood. Proceedings of the Royal Society of London, 144(852):285-307, 1934.
30. Peter Frank! and Hiroshi Maehara. The Johnson-Lindenstrauss lemma and the sphericity of some graphs. Journal of Combinatorial Theory A, 44(3):355-362, 1987.
31. Jerome H. Friedman. Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5): 1189-1232, 2001.
32. Hans U. Gerber. From the generalized gamma to the generalized negative binomial distribution. Insurance:Mathematics and Economics, 10(4):303-309, 1991.
33. Izrail S. Gradshteyn and Iosif M. Ryzhik. Table of Integrals, Series, and Products. Academic Press, New York, fifth edition, 1994.
34. Gerald Haas, Lee Bain, and Charles Antle. Inferences for the Cauchy distribution based on maximum likelihood estimation. Biometrika, 57(2):403-408, 1970.
35. Monika R. Henzinger, Prabhakar Raghavan, and Sridhar Rajagopalan. Computing on Data Streams. American Mathematical Society, Boston, MA, USA, 1999.
36. David V. Hinkley. Likelihood inference about location and scale parameters. Biometrika, 65(2): 253-261, 1978.
37. Philip Hougaard. Survival models for heterogeneous populations derived from stable distributions. Biometrika, 73(2):387-396, 1986.
38. Peter J. Huber. Robust Statistics. Wiley, New York, NY, 1981.
39. Piotr Indyk. Stable distributions, pseudorandom generators, embeddings, and data stream computation. Journal of ACM, 53(3):307-323, 2006.
40. Piotr Indyk. Stable distributions, pseudorandom generators, embeddings and data stream computation. In FOCS, pages 189-197, Redondo Beach, CA, 2000.
41. Piotr Indyk. Algorithmic applications of low-distortion geometric embeddings. In FOCS, pages 10-33, Las Vegas, NV, 2001.
42. Piotr Indyk and Rajeev Motwani. Approximate nearest neighbors: Towards removing the curse of dimensionality. In STOC, pages 604-613, Dallas, TX, 1998.
43. Jens Ledet Jensen. Saddlepoint Approximations. Oxford University Press, New York, 1995.
44. William B. Johnson and Joram Lindenstrauss. Extensions of Lipschitz mapping into Hilbert space. Contemporary Mathematics, 26:189-206, 1984.
45. ly. Acta. Math., 149:71-85, 1982.
46. Jerry F. Lawless. Conditional confidence interval procedures for the location and scale parameters of the Cauchy and logistic distributions. Biometrika, 59(2):377-386, 1972.
47. l. Geometric And Functional Analysis, 14(4):745-747, 2004.
48. α (0 < α ≤ 2) norm. In KDD, San Jose, CA, 2007.
49. α (0 < α ≤ 2) using stable random projections. In SODA, 2008.
50. Ping Li and Kenneth W Church. A sketch algorithm for estimating two-way and multi-way associations. Computational Linguistics, 33(3):305-354, 2007.
51. Ping Li and Kenneth W Church. Using sketches to estimate associations. In HLT/EMNLP, pages 708-715, Vancouver, BC, Canada, 2005.
52. Ping Li, Trevor J. Hastie, and Kenneth W. Church. Improving random projections using marginal information. In COLT, pages 635-649, Pittsburgh, PA, 2006a.
53. Ping Li, Trevor J. Hastie, and Kenneth W. Church. Very sparse random projections. In KDD, pages 287-296, Philadelphia, PA, 2006b.
54. Ping Li, Debashis Paul, Ravi Narasimhan, and John Cioffi. On the distribution of SINR for the MMSE MIMO receiver and performance analysis. IEEE Trans. Inform. Theory, 52(1):271-286, 2006c.
55. Ping Li, Kenneth W. Church, and Trevor J. Hastie. Conditional random sampling: A sketch-based sampling technique for sparse data. In NIPS, pages 873-880, Vancouver, BC, Canada, 2007a.
56. 1 using Cauchy random projections. In COLT, 2007b.
57. Gabor Lugosi. Concentration-of-measure inequalities. Lecture Notes, 2004.
58. J. Huston McCulloch. Simple consistent estimators of stable distribution parameters. Communications on Statistics-Simulation, 15(4):1109-1136, 1986.
59. Thomas K. Philips and Randolph Nelson. The moment bound is tighter than Chernoff's bound for positive tail probabilities. The American Statistician, 49(2):175-178, 1995.
60. V. Seshadri. The Inverse Gaussian Distribution: A Case Study in Exponential Families. Oxford University Press Inc., New York, 1993.
61. Thomas A. Severini. Likelihood Methods in Statistics. Oxford University Press, New York, 2000.
62. Gregory Shakhnarovich, Trevor Darrell, and Piotr Indyk, editors. Nearest-Neighbor Methods in Learning and Vision, Theory and Practice. The MIT Press, Cambridge, MA, 2005.
63. Leonard. R. Shenton and Kimiko O. Bowman. Higher moments of a maximum-likelihood estimate. Journal of Royal Statistical Society B, 25(2):305-317, 1963.
64. Alexander Stehl and Joydeep Ghosh. A scalable approach to balanced, high-dimensional clustering of market-baskets. In HiPC, pages 525-536, Bangalore, India, 2000.
65. Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of Royal Statistical Society B, 58(1):267-288, 1996.
66. Maurice C. K. Tweedie. Statistical properties of inverse Gaussian distributions. I. The Annals of Mathematical Statistics, 28(2):362-377, 1957a.
67. Maurice C. K. Tweedie. Statistical properties of inverse Gaussian distributions. II. The Annals of Mathematical Statistics, 28(3):696-705, 1957b.
68. Santosh Vempala. The Random Projection Method. American Mathematical Society, Providence, RI, 2004.
69. Ji Zhu, Saharon Rosset, Trevor Hastie, and Robert Tibshirani. 1-norm support vector machines. In NIPS, Vancouver, BC, Canada, 2003.
70. Vladimir M. Zolotarev. One-dimensional Stable Distributions. American Mathematical Society, Providence, RI, 1986.