Online non-negative convolutive pattern learning for speech signals

IEEE Transactions on Signal Processing

Volumn 61, Issue 1, 2013, Pages 44-56

  Wang, Dong ^a   Vipperla, Ravichander ^b   Evans, Nicholas ^b   Zheng, Thomas Fang ^a  

^a TSINGHUA UNIVERSITY   (China)

^b EURECOM   (France)

Author keywords

convolutive NMF; Non negative matrix factorization; online pattern learning; sparse coding; speech processing; speech recognition

Indexed keywords

BATCH LEARNING; COMPUTING POWER; COMPUTING RESOURCE; CONVOLUTIVE NMF; INPUT DATAS; LEARNED PATTERNS; LOCAL MINIMUMS; NON-NEGATIVE SPARSE CODING; NONNEGATIVE MATRIX FACTORIZATION; ONLINE IMPLEMENTATION; ONLINE LEARNING; ONLINE LEARNING ALGORITHMS; PATTERN LEARNING; SPARSE CODING; SPEECH SIGNALS; TRAINING DATA;

FACTORIZATION; LEARNING ALGORITHMS; SPEECH PROCESSING; SPEECH RECOGNITION;

E-LEARNING;

EID: 84870974763     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2222381     Document Type: Article

References (53)

1. P. Paatero and U. Tapper, "Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, " Environmetr., vol. 5, no. 12, pp. 111-126, 1994.
2. P. Paatero, "Least squares formulation of robust non-negative factor analysis, " Chemometr. Intell. Lab. Syst., vol. 37, pp. 23-35, 1997. (Pubitemid 27242783)
3. D. D. Lee and H. Seung, "Learning the parts of objects by non-negative matrix factorization, " Nature, vol. 401, no. 6755, pp. 788-791, Oct. 1999.
4. D. D. Lee and H. Seung, "Algorithms for non-negative matrix factorization, " in Proc. NIPS, 2000, pp. 556-562.
5. D. L. Donoho and V. Stodden, "When does non-negative matrix factorization give a correct decomposition into parts?, " in Proc. NIPS 2003, 2003.
6. C. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization, " IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1596, Nov. 2007. (Pubitemid 350148414)
7. A. Shashua and T. Hazan, "Non-negative tensor factorization with applications to statistics and computer vision, " in Proc. Int. Conf. Mach. Learn. (ICML), 2005, pp. 792-799.
8. P. Smaragdis and J. C. Brown, "Non-negative matrix factorization for polyphonic music transcription, " in Proc. IEEE Workshop on Appl. Signal Process. Audio Acoust., 2003, pp. 177-180.
9. D. Guillamet, J. Vitrià, and B. Schiele, "Introducing a weighted non-negative matrix factorization for image classification, " Pattern Recogn. Lett., vol. 24, no. 14, pp. 2447-2454, 2003. (Pubitemid 36911088)
10. M. W. Berry, M. Browne, A. N. Langville, V. P. Pauca, and R. J. Plem-Mons, "Algorithms and applications for approximate non-negative matrix factorization, " Computat. Statist. Data Anal., vol. 52, no. 1, pp. 155-173, 2006.
11. W. Xu, X. Liu, and Y. Gong, "Document clustering based on nonnegative matrix factorization, " in Proc. SIGIR, 2003, pp. 267-273.
12. M. Welling and M. Weber, "Positive tensor factorization, " Pattern Recogn. Lett., vol. 22, no. 12, pp. 1255-1261, 2001. (Pubitemid 32784358)
13. I. S. Dhillon and S. Sra, "Generalized non-negative matrix approximations with Bregman divergences, " in Proc. Neural Inf. Proc. Syst., 2005, pp. 283-290.
14. R. Zdunek and A. Cichocki, "Non-negative matrix factorization with quasi-newton optimization, " in Proc. 8th Int. Conf. Artif. Intell. Soft Comput. ICAISC, 2006, pp. 870-879, Springer. (Pubitemid 44089876)
15. A. Cichocki, R. Zdunek, and S. Ichi Amari, "Csiszár's divergences for non-negative matrix factorization: Family of new algorithms, " in Lecture Notes in Computer Science (LNCS). New York: Springer, 2006, pp. 32-39. (Pubitemid 43998396)
16. C. Lin, "Projected gradient methods for non-negative matrix factorization, " Neural Computat., vol. 19, no. 10, pp. 2756-2779, 2004.
17. R. Zdunek and A. Cichocki, "Nonnegative matrix factorization with constrained second-order optimization, " Signal Process., vol. 87, no. 8, pp. 1904-1916, 2007. (Pubitemid 46590394)
18. C. FéVotte, N. Bertin, and J.-L. Durrieu, "Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis, " Neural Computat., pp. 793-830, 2009.
19. R. Kompass, "A generalized divergence measure for non-negative matrix factorization, " Neural Computat., vol. 19, no. 3, pp. 780-791, 2007.
20. H. Kim and H. Park, "Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method, " SIAM J. Matrix Anal. Appl., vol. 30, pp. 713-730, July 2008.
21. A. T. Cemgil, "Bayesian inference for non-negative matrix factorisation models, " Intell. Neurosci., vol. 2009, pp. 4:1-4:17, Jan. 2009.
22. M. N. Schmidt, O. Winther, and L. K. Hansen, "Bayesian non-negative matrix factorization, " in Proc. Int. Conf. Independ. Compon. Anal. Signal Separat., 2009, vol. 5441, pp. 540-547, ser. Lecture Notes in Computer Science (LNCS), Springer.
23. Z. Yang and E. Oja, "Linear and non-linear projective non-negative matrix factorization, " IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 734-749, 2010.
24. P. Smaragdis, "Convolutive speech bases and their application to supervised speech separation, " IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 1, pp. 1-12, Jan. 2007.
25. D. Fitzgerald and E. Coyle, "Shifted non-negative matrix factorisation for sound source separation, " in Proc. IEEE Conf. Statist. Signal Process., 2005.
26. P. Hoyer, "Non-negative matrix factorization with sparseness constraints, " J. Mach. Learn. Res., vol. 5, pp. 1457-1469, 2004.
27. M. Heiler and C. Schnorr, "Learning sparse representations by non-negative matrix factorization and sequential cone programming, " J. Mach. Learn. Res., vol. 7, pp. 1385-1407, 2006. (Pubitemid 44024595)
28. H. Kim and H. Park, "Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis, " Bioinformatics, vol. 23, no. 12, pp. 1495-1502, 2007. (Pubitemid 47244474)
29. P. D. O'Grady and B. A. Pearlmutter, "Convolutive non-negative matrix factorisation with a sparseness constraint, " in Proc. IEEE Int. Workshop on Mach. Learn. Signal Process. (MLSP 2006), Maynooth, Ireland, Sep. 2006, pp. 427-432.
30. T. Virtanen, "Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria, " IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 3, pp. 1066-1074, 2007.
31. W. Wang, "Convolutive non-negative sparse coding, " in Proc. IJCNN'08, 2008, pp. 3681-3684.
32. W. Wang, A. Cichocki, and J. A. Chamber, "A multiplicative algorithm for convolutive non-negative matrix factorization based on squared Euclidean distance, " IEEE Trans. Signal Process., vol. 57, no. 5, pp. 2858-2864, 2009.
33. C. D. Sigg, T. Dikk, and J. M. Buhmann, "Speech enhancement with sparse coding in learned dictionaries, " in Proc. ICASSP'10, 2010.
34. T. Virtanen, "Separation of sound sources by convolutive sparse coding, " in Proc. SAPA'2004, 2004.
35. W. Smit and E. Barnard, "Continuous speech recognition with sparse coding, " Comput. Speech Lang., vol. 23, no. 2, 2009.
36. G. Sivaram, S. Nemala, M. Elhilali, T. Tran, and H. Hermansky, "Sparse coding for speech recognition, " in Proc. ICASSP'10, 2010.
37. P. O'grady and B. Pearlmutter, "Convolutive non-negative matrix factorisation with a sparseness constraint, " in Proc. IEEE Workshop on Mach. Learn. Signal Process., Sept. 2006, pp. 427-432.
38. H. Robbins and S. Monro, "A stochastic approximation method, " Ann. Math. Statist., vol. 22, pp. 400-407, 1951.
39. J. Macqueen, "Some methods for classification and analysis of multi-variate observations, " in Proc. 5th Berkeley Symp. Math. Statist. Prob-abil., 1967, vol. 1, pp. 281-297.
40. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding, " J. Mach. Learn. Res., vol. 2010, no. 11, pp. 19-60, Jan. 2010.
41. S. S. Bucak and B. Gunsel, "Incremental subspace learning via non-negative matrix factorization, " Pattern Recogn., vol. 42, no. 5, pp. 788-797, 2009.
42. A. Lefevre, F. Bach, and C. Fevotte, "Online algorithms for non-negative matrix factorization with the Itakura-Saito divergence, " in Proc. IEEE Workshop on Appl. Signal Process. Audio and Acoust. (WASPAA), Mohonk, NY, Oct. 2011.
43. E. Vincent, C. Fevotte, and R. Gribonval, "Performance measurement in blind audio source separation, " IEEE Trans. Audio, Speech Lang. Process., vol. 14, no. 4, pp. 1462-1469, 2006. (Pubitemid 46547636)
44. H. Christensen, J. Barker, N. Ma, and P. Green, "The CHiME corpus: A resource and a challenge for computational hearing in multisource environments, " in Proc. Interspeech, 2010.
45. S. Young, G. Evermann, M. Gales, T. Hain, D. Kershaw, X. Liu, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. Woodland, The HTK Book (For Hidden Markov Model Toolkit Version 3.4), 2006.
46. R. Vipperla, S. Bozonnet, D. Wang, and N. Evans, "Robust speech recognition in multi-source noise environments using convolutive non-negative matrix factorization, " in Proc. CHiME: Workshop on Mach. Listen. Multisource Environ., 2011, pp. 74-79.
47. J. Jacques Fuchs, "On sparse representations in arbitrary redundant bases, " IEEE Trans. Inf. Theory, vol. 50, no. 6, pp. 134-1344, 2004.
48. D. Wang, R. Vipperla, N. Evans, and T. F. Zheng, "Online non-negative convolutive pattern learning for speech signals, " EURECOM, Tech. Rep. RR-11-261, 2011.
49. D. Fisk, "Quasi-martingales, " Trans. Amer. Math. Soc., vol. 120, no. 3, pp. 359-388, 1965.
50. A. W. Vander Vaart, Asymptotic Statistics. Cambridge, U.K.: Cambridge Univ. Press, 1998.
51. L. Bottou, "Online algorithms and stochastic approximations, " in Online Learning in Neural Networks, D. Saad, Ed. Cambridge, U.K.: Cambridge Univ. Press, 1998.
52. M. Métivier, Semi-martigales. New York: Walter de Gruyter, 1983.
53. D. Bertsekas, Nonlinear Programming. Belmont, NY: Athena Scientific, 1999.