Multiplicative update rules for concurrent nonnegative matrix factorization and maximum margin classification

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 3, 2013, Pages 422-434

  Zoidi, Olga ^a   Tefas, Anastasios ^a   Pitas, Ioannis ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Joint optimization; Maximum margin classification; Nonnegative matrix factorization (NMF); Support vector machines (SVMs)

Indexed keywords

CLASSIFICATION METHODS; CLASSIFICATION PERFORMANCE; DIMENSIONALITY REDUCTION; JOINT OPTIMIZATION; MAXIMUM MARGIN; MULTIPLICATIVE UPDATES; NONNEGATIVE MATRIX FACTORIZATION; SUPPORT VECTOR MACHINE (SVMS);

OPTIMIZATION; SUPPORT VECTOR MACHINES;

FACTORIZATION;

ALGORITHM; AUTOMATED PATTERN RECOGNITION; CLASSIFICATION; COMPUTER ASSISTED DIAGNOSIS; HUMAN; PROCEDURES;

ALGORITHMS; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED;

EID: 84884944454     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2235461     Document Type: Article

References (43)

1. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, vol. 401, no. 6755, pp. 788-791, Oct. 1999.
2. P. Smaragdis and J. Brown, "Non-negative matrix factorization for polyphonic music transcription," in Proc. IEEE Workshop Appl. Signal Process. Audio Acoust., Oct. 2003, pp. 177-180.
3. K. Devarajan, "Nonnegative matrix factorization: An analytical and interpretive tool in computational biology," PLoS Comput. Biol., vol. 4, no. 7, p. e1000029, Jul. 2008.
4. W. Xu, X. Liu, and Y. Gong, "Document clustering based on non-negative matrix factorization," in Proc. 26th Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retr., 2003, pp. 267-273.
5. D. Guillamet, J. Vitrià, and B. Schiele, "Introducing a weighted non-negative matrix factorization for image classification," Pattern Recognit. Lett., vol. 24, no. 14, pp. 2447-2454, 2003. (Pubitemid 36911088)
6. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Advances in Neural Information Processing Systems 13. Cambridge, MA: MIT Press, Apr. 2001, pp. 556-562.
7. C.-J. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1596, Nov. 2007. (Pubitemid 350148414)
8. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Disc., vol. 2, pp. 121-167, Jun. 1998.
9. D. Geebelen, J. A. K. Suykens, and J. Vandewalle, "Reducing the number of support vectors of SVM classifiers using the smoothed separable case approximation," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 4, pp. 682-688, Apr. 2012.
10. S. Moon and H. Qi, "Hybrid dimensionality reduction method based on support vector machine and independent component analysis," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 5, pp. 749-761, May 2012.
11. D. Anguita, A. Ghio, L. Oneto, and S. Ridella, "In-sample and out-of-sample model selection and error estimation for support vector machines," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 9, pp. 1390-1406, Sep. 2012.
12. S. Zafeiriou, A. Tefas, I. Buciu, and I. Pitas, "Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 683-695, May 2006.
13. Y. Wang and Y. Jia, "Fisher non-negative matrix factorization for learning local features," in Proc. Asian Conf. Comput. Vis., 2004, pp. 27-30.
14. Y. Wang, Y. Jia, C. Hu, and M. Turk, "Non-negative matrix factorization framework for face recognition," Int. J. Pattern Recognit. Artif. Intell., vol. 19, no. 4, pp. 495-511, 2005. (Pubitemid 40997528)
15. I. Buciu, N. Nikolaidis, and I. Pitas, "Nonnegative matrix factorization in polynomial feature space," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 1090-1100, Jun. 2008.
16. C.-J. Lin, "Projected gradient methods for nonnegative matrix factorization," Neural Comput., vol. 19, no. 10, pp. 2756-2779, Oct. 2007.
17. I. Kotsia, S. Zafeiriou, and I. Pitas, "A novel discriminant non-negative matrix factorization algorithm with applications to facial image characterization problems," IEEE Trans. Inf. Forens. Security, vol. 2, no. 3, pp. 588-595, Sep. 2007. (Pubitemid 47290591)
18. Z. Liang, Y. Li, and T. Zhao, "Projected gradient method for kernel discriminant nonnegative matrix factorization and the applications," Signal Process., vol. 90, no. 7, pp. 2150-2163, 2010.
19. R. Zdunek and A. Cichocki, "Nonnegative matrix factorization with quadratic programming," Neurocomputing, vol. 71, nos. 10-12, pp. 2309-2320, 2008.
20. R. Zdunek and A. Cichocki, "Nonnegative matrix factorization with constrained second-order optimization," Signal Process., vol. 87, no. 8, pp. 1904-1916, 2007. (Pubitemid 46590394)
21. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Online nonnegative matrix factorization with robust stochastic approximation," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1087-1099, Jul. 2012.
22. P. O. Hoyer, "Non-negative matrix factorization with sparseness constraints," J. Mach. Learn. Res., vol. 5, pp. 1457-1469, Dec. 2004.
23. S. Li, X. W. Hou, H. J. Zhang, and Q. S. Cheng, "Learning spatially localized, parts-based representation," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 1. Jun. 2001, pp. 207-212.
24. B. Gao, W. Woo, and S. Dlay, "Variational regularized 2-D nonnegative matrix factorization," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 5, pp. 703-716, May 2012.
25. A. Pascual-Montano, J. Carazo, K. Kochi, D. Lehmann, and R. Pascual-Marqui, "Nonsmooth nonnegative matrix factorization (NSNMF)," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 3, pp. 403-415, Mar. 2006. (Pubitemid 43159636)
26. M. Heiler and C. Schnörr, "Learning sparse representations by non-negative matrix factorization and sequential cone programming," J. Mach. Learn. Res., vol. 7, pp. 1385-1407, Dec. 2006. (Pubitemid 44024595)
27. A. Cichocki, R. Zdunek, and S.-I. Amari, "Csiszár's divergences for non-negative matrix factorization: Family of new algorithms," in Independent Component Analysis and Blind Signal Separation (Lecture Notes in Computer Science), vol. 3889, J. Rosca, D. Erdogmus, J. Prãsncipe, and S. Haykin, Eds. Berlin, Germany: Springer-Verlag, 2006, pp. 32-39.
28. M. D. Gupta and J. Xiao, "Non-negative matrix factorization as a feature selection tool for maximum margin classifiers," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 2841-2848.
29. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Royal Stat. Soc. B, vol. 39, no. 1, pp. 1-38, 1977.
30. L. Saul, F. Pereira, and O. Pereira, "Aggregate and mixed-order Markov models for statistical language processing," in Proc. 2nd Conf. Empirical Methods Natural Lang. Process., 1997, pp. 81-89.
31. T. Hofmann, B. Schölkopf, and A. J. Smola, "Kernel methods in machine learning," Ann. Stat., vol. 36, no. 3, pp. 1171-1220, 2008.
32. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization." Nature, vol. 401, no. 6755, pp. 788-791, 1999.
33. A. Frank and A. Asuncion. (2010). UCI Machine Learning Repository [Online]. Available: http://archive.ics.uci.edu/ml
34. V. G. Sigillito, S. P. Wing, L. V. Hutton, and K. B. Baker, "Classification of radar returns from the ionosphere using neural networks," in Johns Hopkins APL Tech. Dig., vol. 10. 1989, pp. 262-266.
35. J. W. Smith, J. E. Everhart, W. C. Dickson, W. C. Knowler, and R. S. Johannes, "Using the ADAP learning algorithm to forecast the onset of diabetes mellitus," in Proc. Symp. Comput. Appl. Med. Care, 1988, pp. 261-265.
36. O. L. Mangasarian, W. N. Street, and W. H. Wolberg, "Breast cancer diagnosis and prognosis via linear programming," Operat. Res., vol. 43, pp. 570-577, Jan. 1995.
37. P. Ekman, "Universals and cultural differences in facial expressions of emotion," in Proc. Nebraska Symp. Motiat., vol. 19. 1971, pp. 207-283.
38. T. Kanade, J. Cohn, and Y. Tian, "Comprehensive database for facial expression analysis," in Proc. 4th IEEE Int. Conf. Autom. Face Gesture Recognit., Mar. 2000, pp. 46-53.
39. MOBISERV-AIIA Database. (2011) [Online]. Available: http://www.aiia.csd. auth.gr/MOBISERV-AIIA/index.html
40. E. Marami, A. Tefas, and I. Pitas, "Nutrition assistance based on skin color segmentation and support vector machines," in Man-Machine Interactions 2 (Advances in Intelligent and Soft Computing), vol. 103. Berlin, Germany: Springer-Verlag, 2011, pp. 179-187.
41. J. Davis, "Hierarchical motion history images for recognizing human motion," in Proc. IEEE Workshop Detect. Recognit. Events Video, 2001, pp. 39-46.
42. S. Nikitidis, N. Nikolaidis, and I. Pitas, "Multiplicative update rules for incremental training of multiclass support vector machines," Pattern Recognit., vol. 45, no. 5, pp. 1838-1852, 2012.
43. F. Sha, Y. Lin, L. K. Saul, and D. D. Lee, "Multiplicative updates for nonnegative quadratic programming," Neural Comput., vol. 19, pp. 2004-2031, Aug. 2007.