Parallel Algorithms for Constrained Tensor Factorization via Alternating Direction Method of Multipliers

IEEE Transactions on Signal Processing

Volumn 63, Issue 20, 2015, Pages 5450-5463

  Liavas, Athanasios P ^a   Sidiropoulos, Nicholas D ^b  

^a TECHNICAL UNIVERSITY OF CRETE   (Greece)

^b University of Minnesota   (United States)

Author keywords

PARAFAC model; parallel algorithms; Tensor decomposition

Indexed keywords

ALGORITHMS; ARRAY PROCESSING; ARTIFICIAL INTELLIGENCE; AUDIO SIGNAL PROCESSING; BIG DATA; CONSTRAINED OPTIMIZATION; FACTORIZATION; LEARNING SYSTEMS; OPTIMIZATION; PARALLEL ALGORITHMS; SCHEDULING ALGORITHMS; SIGNAL PROCESSING; TENSORS;

ALTERNATING DIRECTION METHOD OF MULTIPLIERS; CONSTRAINED OPTIMI-ZATION PROBLEMS; NUMERICAL EXPERIMENTS; PARAFAC; PARALLEL IMPLEMENTATIONS; STATE-OF-THE-ART APPROACH; TENSOR DECOMPOSITION; TENSOR FACTORIZATION;

ITERATIVE METHODS;

EID: 84941923446     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2015.2454476     Document Type: Article

References (36)

1. A. P. Liavas and N. D. Sidiropoulos, "Parallel algorithms for large-scale constrained tensor decomposition," presented at the IEEE ICASSP, Brisbane, Australia, Apr. 19-24, 2015.
2. N. D. Sidiropoulos, R. Bro, and G. Giannakis, "Parallel factor analysis in sensor array processing," IEEE Trans. Signal Process., vol. 48, no. 8, pp. 2377-2388, 2000.
3. N. D. Sidiropoulos, G. Giannakis, and R. Bro, "Blind PARAFAC receivers for DS-CDMA systems," IEEE Trans. Signal Process., vol. 48, no. 3, pp. 810-823, 2000.
4. D. Nion, K. Mokios, N. D. Sidiropoulos, and A. Potamianos, "Batch and adaptive PARAFAC-based blind separation of convolutive speech mixtures," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 6, pp. 1193-1207, 2010.
5. C. Fevotte and A. Ozerov, "Notes on nonnegative tensor factorization of the spectrogram for audio source separation: Statistical insights and towards self-clustering of the spatial cues," in Exploring Music Contents, ser. Lecture Notes in Computer Science, S. Ystad, M. Aramaki, R. Kronland-Martinet, and K. Jensen, Eds. Berlin, Germany: Springer, 2011, vol. 6684, pp. 102-115.
6. E. E. Papalexakis, C. Faloutsos, and N. D. Sidiropoulos, "Parcube: Sparse parallelizable tensor decompositions," in ECML/PKDD (1),ser. Lecture Notes in Computer Science, P. A. Flach, T. D. Bie, and N. Cristianini, Eds. New York, NY, USA: Springer, 2012, vol. 7523, pp. 521-536.
7. R. Bro and N. D. Sidiropoulos, "Least squares regression under unimodality and non-negativity constraints," J. Chemometrics, vol. 12, pp. 223-247, 1998.
8. A. Cichocki, D. Mandic, C. Caiafa, A.-H. Phan, G. Zhou, Q. Zhao, and L. De Lathauwer, "Tensor decompositions for signal processing applications: From two-way to multiway component analysis," IEEE Signal Process. Mag., vol. 32, no. 2, pp. 145-163, Feb. 2014.
9. R. Harshman, "Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multimodal factor analysis," UCLA Working Papers in Phonetics, vol. 16, pp. 1-84, 1970.
10. R. Harshman, "Determination and proof of minimum uniqueness conditions for PARAFAC-1," UCLA Working Papers in Phonetics, vol. 22, pp. 111-117, 1972.
11. J. Carroll and J. Chang, "Analysis of individual differences in multi-dimensional scaling via an N-way generalization of 'Eckart-Young' decomposition," Psychometrika, vol. 35, no. 3, pp. 283-319, 1970.
12. L. Tucker, "Some mathematical notes on three-mode factor analysis," Psychometrika, vol. 31, no. 3, pp. 279-311, Sep. 1966.
13. L. De Lathauwer, "Decompositions of a higher-order tensor in block terms - Part II: Definitions and uniqueness," SIAM J. Matrix Anal. Appl., vol. 30, no. 3, pp. 1033-1066, 2008.
14. A. Smilde, R. Bro, P. Geladi, and J. Wiley, Multi-Way Analysis With Applications in the Chemical Sciences. New York, NY, USA: Wiley, 2004.
15. C. Hillar and L.-H. Lim, "Most tensor problems are NP-hard," 2009 [Online]. Available: http://arxiv.org/abs/0911.1393
16. G. Tomasi and R. Bro, "A comparison of algorithms for fitting the PARAFAC model," Comput. Statist. Data Anal., vol. 50, no. 7, pp. 1700-1734, 2006.
17. E. Acar, D. M. Dunlavy, and T. G. Kolda, "A scalable optimization approach for fitting canonical tensor decompositions," J. Chemometrics vol. 25, no. 2, pp. 67-86, 2011 [Online]. Available: http://dx.doi.org/10.1002/cem.1335
18. L. Sorber, M. van Barel, and L. De Lathauwer, "Optimization-based algorithms for tensor decompositions: Canonical polyadic decomposition, decomposition in rank-(lr, lr 1) terms, and a new generalization," SIAM J. Optim., vol. 23, no. 2, pp. 695-720, 2013.
19. Hadoop Apache [Online]. Available: http://hadoop.apache.org/
20. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107-113, 2008.
21. U. Kang, E. E. Papalexakis, A. Harpale, and C. Faloutsos, "Gigatensor: scaling tensor analysis up by 100 times-algorithms and discoveries," in Proc. 18th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., 2012, pp. 316-324.
22. N. D. Sidiropoulos, E. E. Papalexakis, and C. Faloutsos, "A parallel algorithm for big tensor decomposition using randomly compressed cubes (PARACOMP)," presented at the IEEE ICASSP, Florence, Italy, May 4-9, 2014.
23. N. D. Sidiropoulos, E. E. Papalexakis, and C. Faloutsos, "Parallel randomly compressed cubes: A scalable distributed architecture for big tensor decomposition," IEEE Signal Process. Mag., vol. 31, no. 5, pp. 57-70, Sep. 2014.
24. A. de Almeida and A. Kibangou, "Distributed computation of tensor decompositions in collaborative networks," in Proc. IEEE 5th Int. Workshop Comput. Adv. Multi-Sensor Adapt. Process. (CAMSAP), Dec. 2013, pp. 232-235.
25. A. de Almeida and A. Kibangou, "Distributed large-scale tensor decomposition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), May 2014, pp. 26-30.
26. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimization and statistical learning via the alternating direction Method of Multipliers," Found. Trends Mach. Learn. vol. 3, no. 1, pp. 1-122, Jan. 2011 [Online]. Available: http://dx.doi.org/10.1561/2200000016
27. B. W. Bader and T. G. Kolda, "Efficient MATLAB computations with sparse and factored tensors," SIAM J. Scientif. Comput. vol. 30, no. 1, pp. 205-231, 2008 [Online]. Available: http://dx.doi.org/10.1137/060676489
28. T. G. Kolda and J. Sun, "Scalable tensor decompositions for multi-aspect data mining," in Proc. IEEE ICDM, 2008, pp. 363-372.
29. B. W. Bader et al., Matlab Tensor Toolbox. ver. 2.5, Jan. 2012 [Online]. Available: http://www.sandia.gov/tgkolda/TensorToolbox/
30. N. Ravindran, N. D. Sidiropoulos, S. Smith, and G. Karypis, "Memory-efficient parallel computation of tensor and matrix products for big tensor decomposition," presented at the Asilomar Conf. Signals, Syst., Comput., Nov. 3-5, 2014.
31. Y. Xu, W. Yin, Z. Wen, and Y. Zhang, "An alternating direction algorithm for matrix completion with nonnegative factors," Frontiers Math. China, vol. 51, no. 2, pp. 365-384, 2010.
32. A. P. Liavas and N. D. Sidiropoulos, "Parallel algorithms for constrained tensor factorization via the alternating direction method of multipliers," Tech. Univ. of Crete, Greece, Tech. Rep., 2014.
33. L. Xu, B. Yu, and Y. Zhang, "An alternating direction and projection algorithm for structure-enforced matrix factorization," 2013 [Online]. Available: http://www.caam.rice.edu/yzhang/reports/tr1311.pdf
34. R. Gemulla, E. Nijkamp, P. J. Haas, and Y. Sismanis, "Large-scale matrix factorization with distributed stochastic gradient descent," in Proc. 17th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (KDD'11), New York, NY, USA, 2011, pp. 69-77 [Online]. Available: http://doi.acm.org/10.1145/2020408.2020426
35. C. A. Andersson and R. Bro, The N-way Toolbox for Matlab. [Online]. Available: http://www.models.life.ku.dk/source/nwaytoolbox
36. L. Sorber, M. Van Barel, and L. De Lathauwer, Tensorlab. ver. v2.0 [Online]. Available: http://www.tensorlab.net/