A deep matrix factorization method for learning attribute representations

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 39, Issue 3, 2017, Pages 417-429

  Trigeorgis, George ^a   Bousmalis, Konstantinos ^b   Zafeiriou, Stefanos ^a   Schuller, Bjorn W ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b GOOGLE INC   (United States)

Author keywords

deep semi NMF; Deep WSF; face classification; face clustering; matrix factorization; Semi NMF; semi supervised learning; unsupervised feature learning; WSF

Indexed keywords

DEEP LEARNING; FACTORIZATION; SUPERVISED LEARNING;

DEEP SEMI-NMF; DEEP WSF; FACE CLASSIFICATION; FACE CLUSTERING; MATRIX FACTORIZATIONS; SEMI- SUPERVISED LEARNING; SEMI-NMF; UNSUPERVISED FEATURE LEARNING;

MATRIX ALGEBRA;

ALGORITHM; ARTICLE; LEARNING;

EID: 85012894180     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2016.2554555     Document Type: Article

References (55)

1. P. Paatero, and U. Tapper, "Positive matrix factorization: A nonnegative factor model with optimal utilization of error estimates of data values, " Environmetrics, vol. 5, no. 2, pp. 111-126, 1994.
2. J.-P. Brunet, P. Tamayo, T. R. Golub, and J. P. Mesirov, "Metagenes and molecular pattern discovery using matrix factorization, " Proc. Nat. Academy Sciences, vol. 101, pp. 4164-4149, 2004.
3. K. Devarajan, "Nonnegative matrix factorization: An analytical and interpretive tool in computational biology, " PLoS Comput. Biol, vol. 4, 2008, Art. no. e1000029.
4. M. W. Berry, and M. Browne, "Email surveillance using non-negative matrix factorization, " Comput. Math. Org. Theory, vol. 11, pp. 249-264, 2005.
5. S. Zafeiriou, A. Tefas, I. Buciu, and I. Pitas, "Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification, " IEEE Trans. Neural Netw, vol. 17, no. 3, pp. 683-695, May 2006.
6. I. Kotsia, S. Zafeiriou, and I. Pitas, "A novel discriminant non-negative matrix factorization algorithm with applications to facial image characterization problems, " IEEE Trans. Inform. Forensics Security, vol. 2, no. 3, pp. 588-595, Sep. 2007.
7. F. Weninger, and B. Schuller, "Optimization and parallelization of monaural source separation algorithms in the openBliSSART toolkit, " J. Signal Process. Syst, vol. 69, pp. 267-277, 2012.
8. C. H. Ding, T. Li, and M. I. Jordan, "Convex and semi-nonnegative matrix factorizations, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 32, no. 1, pp. 45-55, Jan. 2010.
9. C. Cing, X. He, and H. Simon, "On the equivalence of nonnegative matrix factorization and spectral clustering, " in Proc. SIAM Data Mining Conf., 2005, pp. 606-610.
10. J. Herrero, A. Valencia, and J. Dopazo, "A hierarchical unsupervised growing neural network for clustering gene expression patterns, " Bioinformatics, vol. 17, pp. 126-136, 2001.
11. Y. Zhao, and G. Karypis, "Hierarchical clustering algorithms for document datasets, " Data Mining Knowl. Discovery, vol. 10, pp. 141-168, 2005.
12. S. J. Prince, J. Warrell, J. H. Elder, and F. M. Felisberti, "Tied factor analysis for face recognition across large pose differences, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 30, no. 6, pp. 970-984, Jun. 2008.
13. G. Tsoumakas, and I. Katakis, "Multi-label classification: An overview, " Int. J. Data Warehousing Mining, vol. 3, pp. 1-13, 2007.
14. Y. Zhang, and Z.-H. Zhou, "Multilabel dimensionality reduction via dependence maximization, " ACM Trans. Knowl. Discovery Data, vol. 4, 2010, Art. no. 14.
15. G. Trigeorgis, K. Bousmalis, S. Zafeiriou, and W. B. Schuller, "A Deep semi-NMF model for learning hidden representations, " in Proc. Int. Conf. Mach. Learn., 2014, vol. 32, pp. 1692-1700.
16. G. E.Hinton, and R. R. Salakhutdinov, "Reducing the dimensionality of datawith neural networks, " Science, vol. 313, pp. 504-507, 2006.
17. G. H. Golub, and C. Reinsch, "Singular value decomposition and least squares solutions, " Numerische Mathematik, vol. 14, pp. 403-420, 1970.
18. S. Wold, K. Esbensen, and P. Geladi, "Principal component analysis, " Chemometrics Intell. Laboratory Syst, vol. 2, pp. 37-52, 1987.
19. D. D. Lee, and H. S. Seung, "Algorithms for non-negative matrix factorization, " in Proc. Adv. Neural Inf. Process. Syst., 2001, pp. 556-562.
20. D. Cai, X. He, J. Han, and T. S. Huang, "Graph regularized nonnegative matrix factorization for data representation, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 33, no. 8, pp. 1548-1560, Aug. 2011.
21. J.-H. Ahn, S. Choi, and J.-H. Oh, "A multiplicative up-propagation algorithm, " in Proc. Int. Conf. Mach. Learn., 2004, p. 3.
22. S. Lyu, and X. Wang, "On algorithms for sparse multi-factor NMF, " in Proc. Adv. Neural Inf. Process. Syst., 2013, pp. 602-610.
23. A. Cichocki, and R. Zdunek, "Multilayer nonnegative matrix factorization, " Electron. Lett, vol. 42, pp. 947-948, 2006.
24. H. A. Song, and S.-Y. Lee, "Hierarchical data representation model - multi-layer NMF, " in Proc. Int. Conf. Learn. Representations, 2013, Art. no. 1301.6316.
25. H. Liu, Z.Wu, X. Li, D. Cai, and T. S.Huang, "Constrained nonnegative matrix factorization for image representation, " IEEE Trans. Pattern Anal.Mach. Intell, vol. 34, no. 7, pp. 1299-1311, Jul. 2012.
26. I. Kotsia, S. Zafeiriou, and I. Pitas, "Novel discriminant non-negative matrix factorization algorithm with applications to facial image characterization problems, " IEEE Trans. Inform. Forensics Security, vol. 2, no. 3, pp. 588-595, Sep. 2007.
27. M. Riesenhuber, and T. Poggio, "Hierarchicalmodels of object recognition in cortex, " Nature Neuroscience, vol. 2, pp. 1019-1025, 1999.
28. J. Malo, I. Epifanio, R. Navarro, and E. P. Simoncelli, "Nonlinear image representation for efficient perceptual coding, " IEEE Trans. Image Process, vol. 15, no. 1, pp. 68-80, Jan. 2006.
29. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators, " Neural Netw, vol. 2, pp. 359-366, 1989.
30. Y. Nesterov, "Gradient methods for minimizing composite objective function, " Math. Program., Springer, vol. 140, no. 1, pp. 125-161, 2013.
31. M. Li, T. Zhang, Y. Chen, and A. Smola, "Efficient mini-batch training for stochastic optimization, " in Proc. 20th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2014, pp. 661-670.
32. D. Kingma, and J. Ba, "Adam: A method for stochastic optimization, " presented at the Int. Conf. Learning Representations, San Diego, CA, USA, 2015.
33. D. M. Cvetkovic, M. Doob, and H. Sachs, Spectra of Graphs: Theory and Application. New York, NY, USA: Academic, 1980, vol. 413.
34. M. Belkin, and P. Niyogi, "Using manifold structure for partially labelled classification, " in Proc. Adv. Neural Inf. Process. Syst., 2002, pp. 328-340.
35. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, " J. Mach. Learn. Res, vol. 7, pp. 2399-2434, 2006.
36. Y. Hao, C. Han, G. Shao, and T. Guo, "Generalized graph regularized non-negative matrix factorization for data representation, " in Proc. Int. Conf. Inform. Technol. Softw. Eng., 2013, pp. 1-12.
37. M. Turk, and A. Pentland, "Eigenfaces for recognition, " J. Cognitive Neurosci, vol. 3, pp. 71-86, 1991.
38. S. Li, X. W. H. X. W. Hou, H. J. Z. H. J. Zhang, and Q. S. C. Q. S. Cheng, "Learning spatially localized, parts-based representation, " in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recog., 2001, vol. 1, pp. I-207-I-212.
39. T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression (PIE) database, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 25, pp. 1615-1618, 2003.
40. K.Messer, J.Matas, J. Kittler, J. Luettin, and G.Maitre, "XM2VTSDB: The extended M2VTS database, " in Proc. 2nd Int. Conf. Audio Video-Based Biometric Person Authentication, 1999, pp. 965-966.
41. D. Yi, Z. Lei, S. Liao, and S. Z. Li, "Learning face representation from scratch, " arXiv preprint arXiv: 1411.7923, 2014.
42. N. Guan, D. Tao, Z. Luo, and B. Yuan, "NeNMF: An optimal gradient method for nonnegative matrix factorization, " IEEE Trans. Signal Process, vol. 60, no. 6, pp. 2882-2898, Jun. 2012.
43. R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multipie, " Image Vision Comput, vol. 28, pp. 807-813, 2010.
44. C. Ding, T. Li, and M. I. Jordan, "Convex and semi-nonnegative matrix factorizations, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 32, no. 1, pp. 45-55, Jan. 2010.
45. N. Gillis, and A. Kumar, "Exact and heuristic algorithms for seminonnegative matrix factorization, " SIAM J. Matrix Anal. Appl, vol. 36, pp. 1404-1424, 2014.
46. C. Boutsidis, and E. Gallopoulos, "SVD based initialization: A head start for nonnegative matrix factorization, " Pattern Recog, vol. 41, pp. 1350-1362, 2008.
47. M. Belkin, and P. Niyogi, "Laplacian eigenmaps and spectral techniques for embedding and clustering, " in Proc. Adv. Neural Inf. Process. Syst., 2001, pp. 585-591.
48. W. Xu, X. Liu, and Y. Gong, "Document clustering based on nonnegative matrix factorization, " in Proc. 26th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2003, pp. 267-273.
49. H. W. Kuhn, "The Hungarian method for the assignment problem, " Naval Res. Logistics Quart, vol. 2, pp. 1-2, 1955.
50. Y. A. LeCun, L. Bottou, G. B. Orr, and K. R. Müller, "Efficient backProp, " Neural Netw., Tricks Trade, vol. 7700, pp. 9-48, 2012.
51. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layer-wise training of deep networks, " Proc. Adv. Neural Inf. Process. Syst., 2007, pp. 153-160.
52. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, andM. Pantic, "300 faces in-The-wild challenge: The first facial landmark Localization Challenge, " in Proc. Comput.Vision Pattern Recog., 2013, pp. 397-403.
53. C. Sagonas, G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, "A semi-Automatic methodology for facial landmark annotation, " in Proc. IEEE Conf. Comput. Vision Pattern Recog. Workshops, 2013, pp. 896-903.
54. S. Zafeiriou, "Discriminant nonnegative tensor factorization algorithms, " IEEE Trans Neural Netw, vol. 20, no. 2, pp. 217-235, Feb. 2009.
55. S. Zafeiriou, "Algorithms for nonnegative tensor factorization, " Tensors Image Process. Comput. Vision, vol. 1, pp. 105-124, 2009.