Steepest descent algorithms for optimization under unitary matrix constraint

IEEE Transactions on Signal Processing

Volumn 56, Issue 3, 2008, Pages 1134-1147

  Abrudan, Traian E ^a   Eriksson, Jan Eriksson ^a   Koivunen, Visa ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

Array processing; Optimization; Source separation; Subspace estimation; Unitary matrix constraint

Indexed keywords

BLIND SOURCE SEPARATION; COMPUTATIONAL GEOMETRY; COMPUTER SIMULATION; CONSTRAINED OPTIMIZATION; CONVERGENCE OF NUMERICAL METHODS; MATRIX ALGEBRA; MIMO SYSTEMS;

LIE GROUP; RIEMANNIAN GEOMETRY; STEEPEST DESCENT ALGORITHMS; SUBSPACE ESTIMATION; UNITARY MATRIX CONSTRAINT;

SIGNAL PROCESSING;

EID: 40749116008     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.908999     Document Type: Article

References (54)

1. E. Polak, Optimization: Algorithms and Consistent Approximations. New York: Springer-Verlag, 1997.
2. J. H. Manton, "Optimization algorithms exploiting unitary constraints," IEEE Trans. Signal Process., vol. 50, pp. 635-650, Mar. 2002.
3. J. Cardoso and A. Souloumiac, "Blind beamforming for non Gaussian signals," Inst. Elect. Eng. Proc.-F, vol. 140, no. 6, pp. 362-370, 1993.
4. S. T. Smith, "Subspace tracking with full rank updates," in Conf. Rec. 31st Asilomar Conf. Signals, Syst. Comp., Nov. 2-5, 1997, vol. 1, pp. 793-797.
5. D. R. Fuhrmann, "A geometric approach to subspace tracking," in Conf. Rec. 31st Asilomar Conf. Signals, Syst. Comp., Nov. 2-5, 1997, vol. 1, pp. 783-787.
6. J. Yang and D. B. Williams, "MIMO transmission subspace tracking with low rate feedback," in Int. Conf. Acoust., Speech Signal Process. Philadelphia, PA, Mar. 2005, vol. 3, pp. 405-408.
7. W. Utschick and C. Brunner, "Efficient tracking and feedback of DL-eigenbeams in WCDMA," in Proc. 4th Europ. Pers. Mobile Commun. Conf., Vienna, Austria, 2001.
8. M. Wax and Y. Anu, "A new least squares approach to blind beamforming," in IEEE Int. Conf. Acoust., Speech, Signal Process. Apr. 21-24, 1997, vol. 5, pp. 3477-3480.
9. P. Stoica and D. A. Linebarger, "Optimization result for constrained beamformer design," in IEEE Signal Process. Lett., Apr. 1995, vol. 2, pp. 66-67.
10. Y. Nishimori, "Learning algorithm for independent component analysis by geodesic flows on orthogonal group," in Int. Joint Conf. Neural Netw., Jul. 10-16, 1999, vol. 2, pp. 933-938.
11. Y. Nishimori and S. Akaho, "Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold," Neurocomputing, vol. 67, pp. 106-135, Jun. 2005.
12. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley, 2001.
13. S. Fiori, A. Uncini, and F. Piazza, "Application of the MEC network to principal component analysis and source separation," in Proc. Int. Conf. Artif. Neural Netw., 1997, pp. 571-576.
14. M. D. Plumbley, "Geometrical methods for non-negative ICA: Manifolds, Lie groups, toral subalgebras," Neurocomput., vol. 67, pp. 161-197, 2005.
15. A. Cichocki and S.-I. Amari, Adaptive Blind Signal and Image Processing. New York: Wiley, 2002.
16. L.Wang, J. Karhunen, and E. Oja, "A bigradient optimization approach for robust PCA, MCA and source separation," in Proc. IEEE Conf. Neural Netw., 27 Nov.-1 Dec. 1995, vol. 4, pp. 1684-1689.
17. S. C. Douglas, "Self-stabilized gradient algorithms for blind source separation with orthogonality constraints," IEEE Trans. Neural Netw. vol. 11, pp. 1490-1497, Nov. 2000.
18. S.-I. Amari, "Natural gradient works efficiently in learning," Neural Comput., vol. 10, no. 2, pp. 251-276, 1998.
19. M. Nikpour, J. H. Manton, and G. Hori, "Algorithms on the Stiefel manifold for joint diagonalisation," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., 2002, vol. 2, pp. 1481-1484.
20. C. B. Papadias, "Globally convergent blind source separation based on a multiuser kurtosis maximization criterion," IEEE Trans. Signal Process., vol. 48, pp. 3508-3519, Dec. 2000.
21. P. Sansrimahachai, D. Ward, and A. Constantinides, "Multiple-input multiple-output least-squares constant modulus algorithms," in IEEE Global Telecommun. Conf., Dec. 1-5, 2003, vol. 4, pp. 2084-2088.
22. C. B. Papadias and A. M. Kuzminskiy, "Blind source separation with randomized Gram-Schmidt orthogonalization for short burst systems," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., May 17-21, 2004, vol. 5, pp. 809-812.
23. J. Lu, T. N. Davidson, and Z. Luo, "Blind separation of BPSK signals using Newton's method on the Stiefel manifold," in IEEE Int. Conf. Acoust., Speech, Signal Process., Apr. 2003, vol. 4, pp. 301-304.
24. A. Edelman, T. Arias, and S. Smith, "The geometry of algorithms with orthogonality constraints," SIAM J. Matrix Analysis Applicat., vol. 20, no. 2, pp. 303-353, 1998.
25. S. Fiori, "Stiefel-Grassman Flow (SGF) learning: Further results," in Proc. IEEE-INNS-ENNS Int. Joint Conf. Neural Netw., Jul. 24-27, 2000, vol. 3, pp. 343-348.
26. J. H. Manton, R. Mahony, and Y. Hua, "The geometry of weighted low-rank approximations," IEEE Trans. Signal Process., vol. 51, pp. 500-514, Feb. 2003.
27. S. Fiori, "Quasi-geodesic neural learning algorithms over the orthogonal group: A tutorial," J. Mach. Learn. Res., vol. 1, pp. 1-42, Apr. 2005.
28. D. G. Luenberger, "The gradient projection method along geodesics," Manage. Sci., vol. 18, pp. 620-631, 1972.
29. D. Gabay, "Minimizing a differentiable function over a differential manifold," J. Optim. Theory Applicat., vol. 37, pp. 177-219, Jun. 1982.
30. A. Iserles and A. Zanna, "Efficient computation of the matrix exponential by general polar decomposition," SIAM J. Numer. Anal. vol. 42, pp. 2218-2256, Mar. 2005.
31. I. Yamada and T. Ezaki, "An orthogonal matrix optimization by dual Cayley parametrization technique," in Proc. ICA, 2003, pp. 35-40.
32. C. Moler and C. van Loan, "Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later," SIAM Rev., vol. 45, no. 1, pp. 3-49, 2003.
33. S. Douglas and S.-Y. Kung, "An ordered-rotation kuicnet algorithm for separating arbitrarily-distributed sources," in Proc. IEEE Int. Conf. Independ. Compon. Anal. Signal Separat., Aussois, France, Jan. 1999, pp. 419-425.
34. C. Udriste, Convex Functions and Optimization Methods on Riemannian Manifolds. Mathematics and Its Applications. Boston, MA: Kluwer Academic, 1994.
35. M. P. do Carmo, Riemannian Geometry. Mathematics: Theory and Applications. Boston, MA: Birkhauser, 1992.
36. A. Knapp, Lie Groups Beyond an Introduction, Vol. 140 of Progress in Mathematics. Boston, MA: Birkhauser, 1996.
37. D. G. Luenberger,Linear and Nonlinear Programming. Reading, MA: Addison-Wesley, 1984.
38. S. T. Smith, "Optimization techniques on Riemannian manifolds," Fields Inst. Commun., Amer. Math. Soc., vol. 3, pp. 113-136, 1994.
39. R. W. Brockett, "Least squares matching problems," Linear Algebra Applicat., vol. 122/123/124, pp. 761-777, 1989.
40. R. W. Brockett, "Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems," Linear Algebra Applicat., vol. 146, pp. 79-91, 1991.
41. B. Owren and B. Welfert, "The Newton iteration on Lie groups," BIT Numer. Math., vol. 40, pp. 121-145, Mar. 2000.
42. P.-A. Absil, R. Mahony, and R. Sepulchre, "Riemannian geometry of Grassmann manifolds with a view on algorithmic computation," Acta Applicandae Mathematicae, vol. 80, no. 2, pp. 199-220, 2004.
43. S. G. Krantz, Function Theory of Several Complex Variables, 2nd ed. Pacific Grove, CA:WAdsworth and Brooks/Cole Advanced Books and Software, 1992.
44. S. Smith, "Statistical resolution limits and the complexified Cramér-Rao bound," IEEE Trans. Signal Process., vol. 53, pp. 1597-1609, May 2005.
45. D. H. Brandwood, "A complex gradient operator and its applications in adaptive array theory," in Inst. Elect. Eng. Proc., Parts F and H Feb. 1983, vol. 130, pp. 11-16.
46. Y. Yang, "Optimization on Riemannian manifold," in Proc. 38th Conf. Decision Contr., Phoenix, AZ, Dec. 1999, pp. 888-893.
47. N. J. Higham, "Matrix nearness problems and applications," in Applications of Matrix Theory, M. J. C. Gover and S. Barnett, Eds. Oxford, U.K.: Oxford Univ. Press, 1989, pp. 1-27.
48. A. Zanna and H. Z. Munthe-Kaas, "Generalized polar decomposition for the approximation of the matrix exponential," SIAM J. Matrix Anal. vol. 23, pp. 840-862, Jan. 2002.
49. E. Celledoni and A. Iserles, "Methods for approximation of a matrix exponential in a Lie-algebraic setting," IMA J. Numer. Anal., vol. 21, no. 2, pp. 463-488, 2001.
50. T. Abrudan, J. Eriksson, and V.Koivunen, "Optimization under unitary matrix constraint using approximate matrix exponential," in Conf. Rec. 39th Asilomar Conf. Signals, Syst. Comp. 2005, 28 Oct.-1 Nov. 2005.
51. P. Sansrimahachai, D.Ward, and A. Constantinides, "Blind source separation for BLAST," in Proc. 14th Int. Conf. Digital Signal Process., Jul. 1-3, 2002, vol. 1, pp. 139-142.
52. J. H. Manton, "On the role of differential geometry in signal processing," in Int. Conf. Acoust., Speech Signal Process., Philadelphia, PA, Mar. 2005, vol. 5, pp. 1021-1024.
53. P. Absil and K. A. Gallivan, Joint diagonalization on the oblique manifold for independent component analysis DAMTP, Univ. Cambridge, U.K., Tech. Rep. NA2006/01, 2006 [Online]. Available: http:// www.damtp.cam.ac.uk/user/na/reports.html
54. G. H. Golub and C. van Loan, Matrix Computations, 3rd ed. Baltimore, MD: The Johns Hopkins Univ. Press, 1996.