Subspace learning and imputation for streaming big data matrices and tensors

IEEE Transactions on Signal Processing

Volumn 63, Issue 10, 2015, Pages 2663-2677

  Mardani, Morteza ^a   Mateos, Gonzalo ^b   Giannakis, Georgios B ^a  

^a University of Minnesota   (United States)

^b University of Rochester   (United States)

Author keywords

Low rank; matrix and tensor completion; missing data; streaming analytics; subspace tracking

Indexed keywords

ALGORITHMS; CLUSTERING ALGORITHMS; DATA HANDLING; DATA MINING; MAGNETIC RESONANCE; MATRIX ALGEBRA; SOCIAL NETWORKING (ONLINE); TENSORS;

CARDIAC MAGNETIC RESONANCE; LOW DIMENSIONAL STRUCTURE; LOW RANK; LOW-DIMENSIONAL SUBSPACE; MISSING DATA; SUBSPACE TRACKING; TENSOR COMPLETION; WEIGHTED LEAST SQUARES;

BIG DATA;

EID: 84928381098     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2015.2417491     Document Type: Article

References (45)

1. [Online]. Available: http://internet2.edu/observatory/archive/data-col-lections.html
2. E. Acar, D. M. Dunlavy, T. G. Kolda, and M. Mrup, "Scalable tensor factorizations for incomplete data," Chemometr. Intell. Lab. Syst., vol. 106, no. 1, pp. 41-56, Mar. 2011.
3. L. Balzano, R. Nowak, and B. Recht, "Online identification and tracking of subspaces from highly incomplete information," presented at the Allerton Conf. Commun., Control, Comput., Monticello, IL, USA, Sep. 2010.
4. L. Balzano, "On GROUSE and incremental SVD," presented at the 5th Workshop Comput. Adv. Multi-Sensor Adapt. Process., St. Martin, Dec. 2013.
5. L. Balzano and S. J. Wright, "Local convergence of an algorithm for subspace identification from partial data," 2013, arXiv preprint arXiv:1306.3391 [Online]. Available: http://arxiv.org/abs/1306.3391
6. J. A. Bazerque, G. Mateos, and G. B. Giannakis, "Rank regularization and Bayesian inference for tensor completion and extrapolation," IEEE Trans. Signal Process., vol. 61, no. 22, pp. 5689-5703, Nov. 2013.
7. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imag. Sci., vol. 2, pp. 183-202, Jan. 2009.
8. J. M. F. T. Berge and N. D. Sidiropoulos, "On uniqueness in CANDE-COMP/PARAFAC," Psychometrika, vol. 67, no. 3, pp. 399-409, Sep. 2002.
9. D. P. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA, USA: Athena Scientific, 1999.
10. S. Burer and R. D. Monteiro, "Local minima and convergence in low-rank semidefinite programming," Math. Programm., vol. 103, no. 3, pp. 427-444, Dec. 2005.
11. E. J. Candes and B. Recht, "Exact matrix completion via convex optimization," Found. Comput. Math., vol. 9, no. 6, pp. 717-722, Apr. 2009.
12. E. Candes and Y. Plan, "Matrix completion with noise," Proceedings of the IEEE, vol. 98, pp. 925-936, Jun. 2009.
13. Y. Chi, Y. C. Eldar, and R. Calderbank, "PETRELS: Subspace estimation and tracking from partial observations," presented at the IEEE Int. Conf. Acoust., Speech, Signal Process., Kyoto, Japan, Mar. 2012.
14. Y. Chi, Y. C. Eldar, and R. Calderbank, "PETRELS: Parallel subspace estimation and tracking using recursive least squares from partial observations," IEEE Trans. Signal Process., vol. 61, no. 23, pp. 5947-5959, Nov. 2013.
15. W. Dai, O. Milenkovic, and E. Kerman, "Subspace evolution and transfer (SET) for low-rank matrix completion," IEEE Trans. Signal Process., vol. 59, no. 7, pp. 3120-3132, Jul. 2011.
16. M. Fazel, "Matrix rank minimization with applications," Ph.D. dissertation, Stanford Univ., Stanford, CA, USA, 2002.
17. J. Feng, H. Xu, and S. Yan, "Online robust PCA via stochastic optimization," presented at the Adv. Neural Inf. Process. Syst., Lake Tahoe, NV, USA, Dec. 2013.
18. J. Finn, K. Nael, V. Deshpande, O. Ratib, and G. Laub, "Cardiac MR imaging: State of the technology," Radiology, vol. 241, no. 2, pp. 338-354, Nov. 2006.
19. S. Gandy, B. Recht, and I. Yamada, "Tensor completion and low-n-rank tensor recovery via convex optimization," Inverse Problems, vol. 27, no. 2, pp. 1-19, Feb. 2011.
20. H. Gao, "Prior rank, intensity and sparsity model (PRISM): A divide-and-conquer matrix decomposition model with low-rank coherence and sparse variation," presented at the SPIE Opt. Eng. Appl., San Diego, CA, USA, Aug. 2012.
21. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed. New York, NY, USA: Springer, 2009.
22. H. Kim, S. Lee, X. Ma, and C. Wang, "Higher-order PCA for anomaly detection in large-scale networks," presented at the 3rd Workshop Comput. Adv. Multi-Sensor Adapt. Process., Aruba, Dutch Antilles, Dec. 2009.
23. E. D. Kolaczyk, Statistical Analysis of Network Data: Methods and Models. New York, NY, USA: Springer, 2009.
24. T. G. Kolda and B. W. Bader, "Tensor decompositions and applications," SIAM Rev., vol. 51, no. 3, pp. 455-500, 2009.
25. J. Kruskal, "Three-way arrays: Rank and uniqueness of trilinear decompositions with application to arithmetic complexity and statistics," Linear Algebra Appl., vol. 18, no. 2, pp. 95-138, 1977.
26. A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E. D. Kolaczyk, and N. Taft, "Structural analysis of network traffic flows," presented at the ACM SIGMETRICS, New York, NY, USA, Jul. 2004.
27. J. Liu, P. Musialski, P. Wonka, and J. Ye, "Tensor completion for estimating missing values in visual data," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, pp. 208-220, Jan. 2013.
28. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, Jan. 2010.
29. M. Mardani, G. Mateos, and G. B. Giannakis, "Decentralized sparsity regularized rank minimization: Applications and algorithms," IEEE Trans. Signal Process., vol. 61, no. 21, pp. 5374-5388, Nov. 2013.
30. M. Mardani, G. Mateos, and G. B. Giannakis, "Dynamic anomalog-raphy: Tracking network anomalies via sparsity and low rank," IEEE J. Sel. Topics Signal Process., vol. 7, no. 11, pp. 50-66, Feb. 2013.
31. Y. Nesterov, "A method of solving a convex programming problem with convergence rate," Sov. Math. Doklady, vol. 27, pp. 372-376, 1983.
32. D. Nion and N. D. Sidiropoulos, "Adaptive algorithms to track the PARAFAC decomposition of a third-order tensor," IEEE Trans. Signal Process., vol. 57, no. 6, pp. 2299-2310, Jun. 2009.
33. B. Recht, M. Fazel, and P. A. Parrilo, "Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, Aug. 2010.
34. M. Signoretto, R. V. Plas, B. D. Moor, and J. A. K. Suykens, "Tensor versus matrix completion: A comparison with application to spectral data," IEEE Signal Process. Lett., vol. 18, pp. 403-406, Jul. 2011.
35. K. Slavakis and G. B. Giannakis, "Online dictionary learning from big data using accelerated stochastic approximation algorithms," presented at the Int. Conf. Acoust., Speech, Signal Process., Florence, Italy, May 2014.
36. K. Slavakis, G. B. Giannakis, and G. Mateos, "Modeling and optimization for Big Data analytics," IEEE Signal Process. Mag., vol. 31, no. 5, pp. 18-31, Sep. 2014.
37. V. Solo and X. Kong, Adaptive Signal Processing Algorithms: Stability and Performance. Englewood Cliffs, NJ, USA: Prentice-Hall, 1995.
38. P. Sprechmann, A. M. Bronstein, and G. Sapiro, "Real-time online singing voice separation from monaural recordings using robust low-rank modeling," presented at the Annu. Conf. Int. Soc. Music Inf. Retrieval, Porto, Portugal, Oct. 2012.
39. N. Srebro, J. Rennie, and T. S. Jaakkola, "Maximum-margin matrix factorization," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, Canada, Dec. 2004, pp. 1329-1336.
40. N. Srebro and A. Shraibman, "Rank, trace-norm and max-norm," in Proc. Learn. Theory, 2005, pp. 545-560, Springer.
41. J. Sun, D. Tao, and C. Faloutsos, "Beyond streams and graphs: Dynamic tensor analysis," in Proc. 12th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., Philadelphia, PA, USA, Aug. 2006, pp. 374-383.
42. M. Thottan and C. Ji, "Anomaly detection in IP networks," IEEE Trans. Signal Process., vol. 51, no. 8, pp. 2191-2204, Aug. 2003.
43. R. Tomioka, T. Suzuki, K. Hayashi, and H. Kashima, "Statistical performance of convex tensor decomposition," in Proc. Adv. Neural Inf. Process. Syst., Granada, Spain, Dec. 2011, pp. 972-980.
44. B. Yang, "Projection approximation subspace tracking," IEEE Trans. Signal. Process., vol. 43, pp. 95-107, Jan. 1995.
45. J. F. Yang and M. Kaveh, "Adaptive eigensubspace algorithms for direction or frequency estimation and tracking," IEEE Trans. Acoust., Speech, Signal Process., vol. 36, no. 2, pp. 241-251, Feb. 1988.