Stochastic gradient descent on riemannian manifolds

IEEE Transactions on Automatic Control

Volumn 58, Issue 9, 2013, Pages 2217-2229

  Bonnabel, Silvere ^a  

^a MINES ParisTech   (France)

Author keywords

Nonlinear identification; Riemannian geometry; stochastic approximation

Indexed keywords

COVARIANCE MATRICES; GRADIENT DESCENT ALGORITHMS; NONLINEAR IDENTIFICATIONS; RIEMANNIAN GEOMETRY; RIEMANNIAN MANIFOLD; STOCHASTIC APPROXIMATIONS; STOCHASTIC GRADIENT DESCENT; STOCHASTIC GRADIENT DESCENT ALGORITHM;

ALGORITHMS; COST FUNCTIONS; COVARIANCE MATRIX; GEOMETRY;

TIME VARYING NETWORKS;

EID: 84883004604     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2013.2254619     Document Type: Article

References (37)

1. P. A. Absil, R. Mahony, and R. Sepulchre, Optimization Algorithms on Matrix Manifolds. Princeton, NJ, USA: Princeton Univ. Press, 2007
2. B. Afsari, "Riemannian Lp center of mass: Existence, uniqueness, and convexity," Proc. Amer.Math. Soc., vol. 139, pp. 655-673, 2011
3. S. I. Amari, "Neural computation," in Natural Gradient Works Efficiently in Learning. Cambridge, MA, USA: MIT Press, 1998
4. M. Arnaudon, C. Dombry, A. Phan, and L. Yang, "Stochastic algorithms for computing means of probability measures," Stochast. Process. Their Applicat., vol. 122, pp. 1437-1455, 2012
5. F. Barbaresco, "Innovative tools for radar signal processing based on Cartan's geometry of symmetric positive-definite matrices and information geometry," in Proc. IEEE Radar Conf. RADAR'08, 2008, pp. 1-6
6. A. Benveniste, M. Goursat, and G. Ruget, "Analysis of stochastic approximation schemes with discontinuous and dependent forcing terms with applications to data communication algorithms," IEEE Trans. Autom. Control, vol. AC-25, no. 6, pp. 1042-1058, Dec. 1980 (Pubitemid 11466598)
7. S. Bonnabel, "Convergence des méthodes de gradient stochastique sur les variétés riemanniennes," in Proc. XIII ième Colloque GRETSI, Bordeaux, 2011
8. S. Bonnabel, G.Meyer, and R. Sepulchre, "Adaptive filtering for estimation for a low-rank positive semidefinite matrix," in Proc. Int. Symp. Math. Theory Netw. Syst. (MTNS), 2010
9. L. Bottou, "Online learning and neural networks," in Online Algorithms and Stochastic Approximations, D. Saad, Ed. Cambridge, U.K.: Cambridge Univ. Press, 1998
10. S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, "Randomized gossip algorithms," IEEE Trans. Inf. Theory, vol. 52, no. 6, pp. 2508-2530, Jun. 2006 (Pubitemid 43898906)
11. E. J. Candès and Y. Plan, "Tight oracle bounds for low-rank matrix recovery from aminimal number of random measurements," IEEE Trans. Inf. Theory, vol. 57, no. 4, pp. 2342-2359, Apr. 2009
12. D. Cordero-Erausquin, R. J. McCann, and M. Schmuckenschlager, "A Riemannian interpolation inequality a la Borell, Brascamp and Lieb," Invent. Math., vol. 146, pp. 219-257, 2001
13. T. A. Arias Edelman and A. S. T. Smith, "The geometry of algorithms with orthogonality constraints," SIAM J. Matrix Anal. Applicat., vol. 20, no. 2, pp. 303-353, 1998
14. J. Faraut and A. Koranyi, Analysis on Symmetric Cones. London, U.K.: Oxford Univ. Press, 1994
15. D. L. Fisk, "Quasi-martingales," Trans. Amer.Math. Soc., vol. 120, no. 3, pp. 369-389, 1965
16. N. J. Higham, Matrix Nearness Problems and Applications. Oxford, U.K.: Oxford Univ. Press, 1989
17. M. Journee, P.-A. Absil, F. Bach, and R. Sepulchre, "Low-rank optimization on the cone of positive semidefinite matrices," SIAM J. Optimiz., vol. 20, no. 5, pp. 2327-2351, 2010
18. B. Kulis, M. Sustik, and I. S. Dhillon, "Learning low-rank kernel matrices," in Proc. 23rd Int. Conf. Mach. Learn. (ICML), 2006
19. L. Ljung, "Analysis of recursive stochastic algorithms," IEEE Trans. Autom. Control, vol. AC-22, no. 4, pp. 551-575, Aug. 1977
20. L. Ljung, H. Hjalmarsson, and H. Ohlsson, "Four encounters with system identification," Eur. J. Control, vol. 17, no. 5, pp. 449-471, 2011
21. G.Meyer, "Geometric optimization algorithms for linear regression on fixed-rank matrices," Ph.D. dissertation, Univ. of Liège, Liège, Belgium, 2011
22. G. Meyer, S. Bonnabel, and R. Sepulchre, "Linear regression under fixed-rank constraints: A Riemannian approach," in Proc. 28th Int. Conf. Mach. Learn. (ICML), 2011
23. G. Meyer, S. Bonnabel, and R. Sepulchre, "Regression on fixed-rank positive semidefinite matrices: A Riemannian approach," J. Mach. Learn. Res. (JMLR), vol. 12, pp. 593-625, 2011
24. L. Moreau, "Stability of multiagent systems with time-dependent communication links," IEEE Trans. Autom. Control, vol. 50, no. 2, pp. 169-182, Feb. 2005 (Pubitemid 40319348)
25. E. Oja, Subspace Methods of Pattern Recognition. Hertfordshire, U.K.: Research Studies Press, 1983
26. E. Oja, "Principal components, minor components, and linear neural networks," Neural Netw., vol. 5, no. 6, pp. 927-935, 1992 (Pubitemid 23581250)
27. B. O'Neill, "Pure and applied mathematics," in Semi-Riemaniann Geometry. New York, NY, USA: Academic, 1983
28. X. Pennec, P. Fillard, and N. Ayache, "A Riemannian framework for tensor computing," Int. J. Comput. Vis., vol. 66, pp. 41-66, 2006 (Pubitemid 43004671)
29. H. Robbins and S. Monro, "A stochastic approximation method," Ann. Math. Statist., vol. 22, pp. 400-407, 1951
30. S. Sarlette, S. E. Tuna, V. D. Blondel, and R. Sepulchre, "Global synchronization on the circle," in Proc. 17th IFAC World Congr., 2008
31. A. Sarlette, S. Bonnabel, and R. Sepulchre, "Coordinated motion design on Lie groups," IEEE Trans. Autom. Control, vol. 55, no. 5, pp. 1047-1058, May 2010
32. S. Sepulchre, P. Derek, and N. E. Leonard, "Stabilization of planar collective motion with limited communication," IEEE Trans. Autom. Control, vol. 53, no. 3, pp. 706-719, Mar. 2008 (Pubitemid 351597178)
33. U. Shalit, D. Weinshall, and G. Chechik, "Online learning in the embedded manifold of low-rank matrices," J. Mach. Learn. Res. (JMLR), vol. 13, pp. 429-458, 2012
34. S. T. Smith, "Covariance, subspace, and intrinsic Cramer-Rao bounds," IEEE Trans. Signal Process., vol. 53, no. 5, pp. 1610-1629, May 2005
35. R. Tron, B. Afsari, and R. Vidal, "Average consensus on Riemannian manifolds with bounded curvature," in Proc. 50th IEEE Conf. Decision Control, 2011, pp. 7855-7862
36. J. N. Tsitskilis, D. P. Bertsekas, and M. Athans, "Distributed asynchronous deterministic and stochastic gradient optimization algorithms," IEEE Trans. Autom. Control, vol. AC-31, no. 9, pp. 803-812, Sep. 1986 (Pubitemid 16629805)
37. R.-G. Tsuda and K. M. Warmuth, "Matrix exponentiated gradient updates for online learning and Bregman projection," J. Mach. Learn. Res., vol. 36, no. 6, pp. 995-1018, 2005