Linear-time training of nonlinear low-dimensional embeddings

Journal of Machine Learning Research

Volumn 33, Issue , 2014, Pages 968-977

  Vladymyrov, Max ^a   Carreira Perpiñán, Miguel Á ^a  

^a Merced   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CURVE FITTING; STOCHASTIC SYSTEMS;

FAST MULTIPOLE METHOD; GRADIENT DESCENT; LOW DIMENSIONAL; MEAN CURVATURE; NONCONVEX OPTIMIZATION; OBJECTIVE FUNCTIONS; SPECTRAL METHODS; STOCHASTIC NEIGHBOR EMBEDDING;

OPTIMIZATION;

EID: 84955439530     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (35)

1. J. Barnes and P. Hut. A hierarchical O(N log N) force-calculation algorithm. Nature, 324(6096):446-449, Dec. 4 1986.
2. G. D. Battista, P. Eades, R. Tamassia, and I. G. Tollis. Graph Drawing: Algorithms for the Visualization of Graphs. Prentice-Hall, 1999.
3. B. J. C. Baxter and G. Roussos. A new error estimate of the fast Gauss transform. SIAM J. Sci. Comput., 24(1), 257-259 2002.
4. M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6): 1373-1396, June 2003.
5. M. Á. Carreira-Perpiñán. The elastic embedding algorithm for dimensionality reduction. In J. Furnkranz and T. Joachims, editors, Proc. of the 27th Int. Conf. Machine Learning (ICML 2010), pages 161-174, Haifa, Israel, June 21-25 2010.
6. M. Á. Carreira-Perpihán and Z. Lu. The Laplacian Eigenmaps Latent Variable Model. In M. Meilə and X. Shen, editors, Proc. of the 11th Int. Workshop on Artificial Intelligence and Statistics (AISTATS 2007), pages 59-66, San Juan, Puerto Rico, Mar. 21-24 2007.
7. N. de Freitas, Y. Wang, M. Mahdaviani, and D. Lang. Fast Krylov methods for N-body learning. In Y. Weiss, B. Schölkopf, and J. Platt, editors, Advances in Neural Information Processing Systems (NIPS), Volume 18. MIT Press, Cambridge, MA, 2006.
8. W Fong and E. Darve. The black-box fast multipole method. J. Comp. Phys., 228(23):8712-8725, Dec. 10 2009.
9. J. H. Friedman, J. L. Bentley, and R. A. Finkel. An algorithm for finding best matches in logarithmic expected time. ACM Trans. Mathematical Software, 3(3):209-226, 1977.
10. T. M. J. Fruchterman and E. M. Reingold. Graph drawing by force-directed placement. Software: Practice and Experience, 21(11): 1129-1164, Nov. 1991.
11. A. G. Gray and A. W. Moore. 'N-body' problems in statistical learning. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems (NIPS), Volume 13, pages 521-527. MIT Press, Cambridge, MA, 2001.
12. L. Greengard and V. Rokhlin. A fast algorithm for particle simulations. J. Comp. Phys., 73(2):325-348, Dec. 1987.
13. L. Greengard and J. Strain. The fast Gauss transform. SIAM J. Sci. Stat. Comput., 12(1):79-94, Jan. 1991.
14. G. Hinton and S. T. Roweis. Stochastic neighbor embedding. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems (NIPS), Volume 15, pages 857-864. MIT Press, Cambridge, MA, 2003.
15. Y. Hu. Efficient and high-quality force-directed graph drawing. The Mathematica Journal, 10(1):37-71, 2005.
16. D. Lee, A. Gray, and A. Moore. Dual-tree fast Gauss transforms. In Y. Weiss, B. Schölkopf, and J. Platt, editors, Advances in Neural Information Processing Systems (NIPS), Volume 18, pages 747-754. MIT Press, Cambridge, MA, 2006.
17. G. Loosli, S. Canu, and L. Bottou. Training invariant support vector machines using selective sampling. In L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, editors, Large Scale Kernel Machines, Neural Information Processing Series, pages 301-320. MIT Press, 2007.
18. A. Moore, J. Schneider, and K. Deng. Efficient locally weighted polynomial regression predictions. In D. H. Fisher, editor, Proc. of the 14th Int. Conf. Machine Learning (ICML'97), pages 236-244, Nashville, TN, July 6-12 1997.
19. A. Quigley and P. Eades. FADE: Graph drawing, clustering, and visual abstraction. In J. Marks, editor, Proc. 8th Int. Symposium on Graph Drawing (GD 2000), pages 197-210, Colonial Williamsburg, VA, Sept. 20-23 2000.
20. V. C. Raykar and R. Duraiswami. Fast optimal bandwidth selection for kernel density estimation. In Proc. of the 2006 SIAM Int. Conf. Data Mining (SDM 2006), pages 524-528, Bethesda, MD, Apr. 20-22 2006.
21. V. C. Raykar and R. Duraiswami. The improved fast Gauss transform with applications to machine learning. In L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, editors, Large Scale Kernel Machines, Neural Information Processing Series, pages 175-202. MIT Press, 2007.
22. S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290 (5500):2323-2326, Dec. 22 2000.
23. J. K. Salmon and M. S. Warren. Skeletons from the treecode closet. J. Comp. Phys., 111(1): 136-155, Mar. 1994.
24. H. Samet. Foundations of Multidimensional and Metric Data Structures. Morgan Kaufmann, 2006.
25. Y. Shen, A. Y Ng, and M. Seeger. Fast Gaussian process regression using KD-trees. In Y. Weiss, B. Schölkopf, and J. Platt, editors, Advances in Neural Information Processing Systems (NIPS), Volume 18, pages 1225-1232. MIT Press, Cambridge, MA, 2006.
26. J. C. Spall. Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. John Wiley & Sons, 2003.
27. J. Strain. The fast Gauss transform with variable scales. SIAM J. Sci. Stat. Comput, 12(5): 1131-1139, Sept. 1991.
28. L. J. P. van der Maaten. Barnes-Hut-SNE. In Int. Conf. Learning Representations (ICLR 2013), Scottsdale, AZ, May 2-4 2013.
29. L. J. P. van der Maaten and G. E. Hinton. Visualizing data using t-SNE. J. Machine Learning Research, 9:2579-2605, Nov. 2008.
30. M. Vladymyrov and M. Á. Carreira-Perpiñán. Partial-Hessian strategies for fast learning of nonlinear embed-dings. In J. Langford and J. Pineau, editors, Proc. of the 29th Int. Conf. Machine Learning (ICML 2012), pages 345-352, Edinburgh, Scotland, June 26 - July 1 2012.
31. M. Vladymyrov and M. Á. Carreira-Perpiñán. Entropic affinities: Properties and efficient numerical computation. In S. Dasgupta and D. McAllester, editors, Proc. of the 30th Int. Conf. Machine Learning (ICML 2013), pages 477-485, Atlanta, GA, June 16-21 2013.
32. X. Wan and G. E. Karniadakis. A sharp error estimate for the fast Gauss transform. J. Comp. Phys., 219(1):7-12, Nov. 20 2006.
33. C. Yang, R. Duraiswami, N. A. Gumerov, and L. Davis. Improved fast Gauss transform and efficient kernel density estimation. In Proc. 9th Int. Conf. Computer Vision (ICCV'03), pages 464-471, Nice, France, Oct. 14-17 2003.
34. Z. Yang, J. Peltonen, and S. Kaski. Scalable optimization for neighbor embedding for visualization. In S. Dasgupta and D. McAllester, editors, Proc. of the 30th Int. Conf. Machine Learning (ICML 2013), pages 127-135, Atlanta, GA, June 16-21 2013.
35. L. Ying, G. Biros, and D. Zorin. A kernel-independent adaptive fast multipole algorithm in two and three dimensions. J. Comp. Phys., 196(2):591-626, May 20 2004.