Efficient multilevel eigensolvers with applications to data analysis tasks

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 32, Issue 8, 2010, Pages 1377-1391

  Kushnir, Dan ^a   Galun, Meirav ^b   Brandt, Achi ^b  

^a YALE UNIVERSITY   (United States)

^b WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

Clustering.; Eigenvalues and eigenvectors; Graph algorithms; Multigrid and multilevel methods; Segmentation

Indexed keywords

ALGEBRAIC MULTIGRIDS; DATA ANALYSIS; DIMENSIONALITY REDUCTION; EIGEN-PROBLEMS; EIGENSOLVERS; EIGENVALUE PROBLEM; EIGENVALUES AND EIGENVECTORS; EIGENVECTORS; ERROR FUNCTION; GRAPH ALGORITHMS; INTERPOLATION SCHEMES; ITERATIVE RELAXATIONS; LANCZOS ALGORITHM; MASSIVE SYSTEMS; MULTI-GRID; MULTIGRID SOLVER; MULTILEVEL METHOD; ORDER OF MAGNITUDE;

CLUSTER ANALYSIS; CLUSTERING ALGORITHMS; IMAGE SEGMENTATION;

EIGENVALUES AND EIGENFUNCTIONS;

EID: 77953806813     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2009.147     Document Type: Article

References (46)

1. D. Kushnir, M. Galun, and A. Brandt, "Fast Multiscale Clustering and Manifold Identification," Pattern Recognition, vol.39, no.10, pp. 1876-1891, Oct. 2006.
2. I.S. Dhillon, Y. Guan, and B. Kulis, "Weighted Graph Cuts without Eigenvectors: A Multilevel Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.29, no.11, pp. 1944-1957, Nov. 2007.
3. G.H. Golub and C.F. Van Loan, Matrix Computations. John Hopkins Univ. Press, 1989.
4. A. Brandt, S. McCormick, and J. Ruge, "Algebraic Multigrid (AMG) for Automatic Multigrid Solution with Application to Geodetic Computations," report, Inst. for Computational Studies, 1982.
5. U. Trottenberg, A. Schuller, and C. Oosterlee, Multigrid. Acdemic Press, 2001.
6. A. Brandt, "Algebraic Multigrid Theory: The Symmetric Case," Applied Math. and Computation, vol.19, no.23, pp. 23-56, July 1986.
7. A. Brandt, "Multiscale Scientific Computation: Review 2001," Multiscale and Multiresolution Methods: Theory and Applications, Springer-Verlag, 2002.
8. A. Brandt, "Multilevel Adaptive Solutions to Boundary Value Problems," Math. of Computation, vol.31, no.138, pp. 333-390, Apr. 1977.
9. A. Brandt and D. Ron, "Multigrid Solvers and Multiscale Optimization Strategies," Multilevel Optimization and VLSICAD, Kluwer, 2003.
10. S. Kaczmarz, "Angenaherte Auflosung von Systemen Linearer Gleichungen," Bull. Acad. Polon. Sci. et Lett. A, pp. 355-357, 1937.
11. C. Fowlkes, S. Belongie, F. Chung, and J. Malik, "Spectral Grouping Using the Nystrom Method," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.26, no.2, pp. 214-225, Feb. 2004.
12. T. Cour, F. Benezit, and J. Shim, "Spectral Segmentation with Multiscale Graph Decomposition," Proc. IEEE Intl. Conf. Computer Vision and Pattern Recognition, 2005.
13. Y. Ng Andrew, M. Jordan, and Y. Weiss, "On Spectral Clustering: Analysis and an Algorithm," Advances in Neural Information Processing Systems, vol.14, pp. 849-856, MIT Press, Dec. 2001.
14. J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.22, no.8, pp. 888-905, Aug. 2000.
15. Code obtained from the URL http://www.cis.upenn.edu/jshi/software, 2008.
16. F. Chung, "Spectral Graph Theory," Am. Math. Soc, 92, 1997.
17. M. Belkin and P. Niyogi, "Laplacian Eigenmaps for Dimensionality Reduction and Data Representation," Neural Computation, vol.15, no.6, pp. 1373-1396, June 2003.
18. C. Chennubhotla and A.D. Jepson, "Hierarchical Eigensolver for Transition Matrices in Spectral Methods," Advances in Neural Information Processing Systems, vol.17, p. 273-280, MIT Press, Dec. 2005.
19. A. Brandt, S. McCormick, and J. Ruge, "Multigrid Methods for Differential Eigenproblems," SIAM J. Scientific and Statistical Computing, vol.4, no.2, pp. 244-260, June 1983.
20. S. McCormick, "A Mesh Refinement Method for Ax = XAx," Math. of Computation, vol.36, no.154, pp. 485-498, Apr. 1981.
21. S. Costiner and S. Taasan, "Adaptive Multigrid Techniques for Large-Scale Eigenvalue Problems: Solutions of the Schrodinger Problem in Two and Three Dimensions," Physical Rev. E, vol.51, no.4, pp. 3704-3717, Apr. 1995.
22. O. Livne and A. Brandt, "O(NlogN) Multilevel Calculation of N Eigenfunctions," Multiscale Computational Methods in Chemistry and Physics, IOS Press, 2001.
23. I. Livshits, "An Algebraic Multigrid Wave-Ray Algorithm to Solve Eigenvalue Problems for The Helmholtz Operator," Numerical Linear Algebra with Applications, vol.11, nos. 2-3, pp. 229-239, Mar. 2004.
24. I. Livshits, "One-Dimensional Algorithm for Finding Eigenbasis of the Schrodinger Operator," SIAM J. Scientific Computing, vol.30, no.1, pp. 416-440, Nov. 2007.
25. YA. Erlangga, C.W. Oosterlee, and C. Vuik, "A Novel Multigrid Based Preconditioner for Heterogeneous Helmholtz Problems," SIAM J. Scientific Computing, vol.27, No.4, pp. 1471-1492, 2006.
26. H.C. Elman, OG Ernst, and D.P. Oleary, "A Multigrid Method Enhanced by Krylov Subspace Iteration for Discrete Helmholtz Equations," SIAM J. Scientific Computing, vol.23, no.4, pp. 1291-1315, 2001.
27. A. Borzi and G Borzi, "Algebraic Multigrid Methods for Solving Generalized Eigenvalue Problems," Int'l J. Numerical Methods in Eng., vol.65, no.8, pp. 1186-1196, Sept. 2005.
28. U. Hetmaniuk, "A Rayleigh Quotient Minimization Algorithm Based on Algebraic Multigrid," Numerical Linear Algebra with Applications, vol.14, pp. 563-580, 2007.
29. J. Mandel and S. McCormick, "A Multilevel Variational Method for Au =\Bu on Composite Grids," J. Computational Physics, vol.80, pp. 442-452, 1989.
30. M. Brezina, T. Manfeuffel, S. McCormick, J. Ruge, G. Sanders, and P. Vassilevski, "Smoothed Aggregation-Based Eigensolvers," Numerical Linear Algebra with Applications, vol.15, pp. 249-269, 2008.
31. P. Vanek, J. Mandel, and M. Brezina, "Algebraic Multigrid by Smoothed Aggregation for Second and Fourth Order Elliptic Problems," Computing, vol.56, pp. 179-196, 1996.
32. M. Brezina, R. Falgout, S. MacLachlan, T. Manfeuffel, S. McCormick, and J. Ruge, "Adaptive Smoothed Aggregation (aSA)," SIAM J. Scientific Computing, vol.25, pp. 1896-1920, 2004.
33. M. Brezina, R. Falgout, S. MacLachlan, T. Manteuffel, S. McCormick, and J. Ruge, "Adaptive Smoothed Aggregation (aSA) Multigrid," SIAM Rev., vol.47, no.2, pp. 317-346, 2005.
34. M. Brezina, R. Falgout, S. MacLachlan, T. Manfeuffel, S. McCormick, and J. Ruge, "Adaptive Algebraic Multigrid," SIAM J. Scientific Computing, Vol.27, no.4, pp. 1261-1286, 2006.
35. H. de Sterck, T. Manteuffel, S. McCormick, Q. Nguyen, and J. Ruge, "Multilevel Adaptive Aggregation for Markov Chains, with Application to Web Ranking," SIAM J. Scientific Computing, vol.30, pp. 2235-2262, 2008.
36. Y. Takahashi, "A Lumping Method for Numerical Calculations of Stationary Distributions of Markov Chains," Research Report B-18, Dept. of Information Sciences, Tokyo 23 Inst. of Technology, 1975.
37. G. Horton and S.T. Leutenegger, "A Multi-Level Solution Algorithm for Steady-State Markov Chains," Proc. ACM SIGMETRICS, pp. 191-200, 1994.
38. S.T. Leutenegger and G Horton, "On the Utility of the Multi-Level Algorithm for the Solution of Nearly Completely Decomposable Markov Chains," Numerical Solution of Markov Chains, W. Stewart, ed., pp. 425-443, Kluwer Publishers, 1995.
39. U.R Krieger, "Numerical Solution of Large Finite Markov Chains by Algebraic Multigrid Techniques," Numerical Solution of Markov Chains, W. Stewart, ed., pp. 403-424, Kluwer Publishers, 1995.
40. C. Isensee and G Horton, "A Multi-Level Method for the Steady State Solution of Markov Chains," Simulation und Visualisierung, SCS European Publishing House, 2004.
41. S. Yu and J. Shi, "Multiclass Spectral Clustering," Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
42. R.B. Lehoucq, D.C. Sorensen, and C. Yang, ARPACK User Guide, SIAM Publications, 1998.
43. M. de Berg, M. van Kreveld, M. Overmars, and O. Schwarzkopf, Computational Geometry: Algorithms and Applications. Springer-Verlag, 1997.
44. Code obtained from the URL http://www.autonlab.org/autonweb/2.html, 2009.
45. J.A. Hartigan and M.A. Wong, "A k-Means Clustering Algorithm," Applied Statistics, vol.28, no.1, pp. 100-108, 1979.
46. A. Brandt, "Principles of Systematic Upscaling," Bridging the Scales in Science and Eng., J. Fish, ed., Oxford Univ. Press, 2008.