Unsupervised neural network based feature extraction using weak top-down constraints

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2015-August, Issue , 2015, Pages 5818-5822

  Kamper, Herman ^a   Elsner, Micha ^b   Jansen, Aren ^c   Goldwater, Sharon ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b OHIO STATE UNIVERSITY   (United States)

^c JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

deep neural networks; top down constraints; Unsupervised feature extraction; zero resource speech processing

Indexed keywords

AUDIO SIGNAL PROCESSING; DYNAMIC PROGRAMMING; EXTRACTION; FEATURE EXTRACTION; SPEECH COMMUNICATION; SPEECH PROCESSING;

ACOUSTIC MODELLING; DISCRIMINATION TASKS; FEATURE EXTRACTOR; STANDARD COMPONENTS; STATE-OF-THE-ART SYSTEM; SUPERVISED SYSTEMS; TOPDOWN; UNSUPERVISED NEURAL NETWORKS;

DEEP NEURAL NETWORKS;

EID: 84946101387     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2015.7179087     Document Type: Conference Paper

References (26)

1. G. E. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition," IEEE Trans. Audio, Speech, Language Process., vol. 20, no. 1, pp. 30-42, 2012.
2. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, 2012.
3. D. Yu, M. Seltzer, J. Li, J.-T. Huang, and F. Seide, "Feature learning in deep neural networks-studies on speech recognition," in Proc. ICLR, 2013.
4. A. Jansen, K. Church, and H. Hermansky, "Towards spoken term discovery at scale with zero resources," in Proc. Interspeech, 2010.
5. C. Lee and J. R. Glass, "A nonparametric Bayesian approach to acoustic model discovery," in Proc. ACL, 2012.
6. H. Gish, M.-H. Siu, A. Chan, and B. Belfield, "Unsupervised training of an HMM-based speech recognizer for topic classification," in Proc. Interspeech, 2009.
7. Y. Zhang and J. R. Glass, "Unsupervised spoken keyword spotting via segmental DTW on Gaussian posteriorgrams," in Proc. ASRU, 2009.
8. F. Metze, X. Anguera, E. Barnard, M. Davel, and G. Gravier, "The spoken web search task at MediaEval 2012," in Proc. ICASSP, 2013.
9. M. D. Zeiler, M. Ranzato, R. Monga, M. Mao, K. Yang, Q. V. Le, P. Nguyen, A. Senior, V. Vanhoucke, J. Dean, and G. E. Hinton, "On rectified linear units for speech processing," in Proc. ICASSP, 2013.
10. L. Badino, C. Canevari, L. Fadiga, and G. Metta, "An autoencoder based approach to unsupervised learning of subword units," in Proc. ICASSP, 2014.
11. A. Park and J. R. Glass, "Unsupervised word acquisition from speech using pattern discovery," in Proc. ICASSP, 2006.
12. A. Jansen and B. Van Durme, "Efficient spoken term discovery using randomized algorithms," in Proc. ASRU, 2011.
13. O. Walter, T. Korthals, R. Haeb-Umbach, and B. Raj, "A hierarchical system for word discovery exploiting DTW-based initialization," in Proc. ASRU, 2013.
14. A. Jansen and K. Church, "Towards unsupervised training of speaker independent acoustic models," in Proc. Interspeech, 2011.
15. A. Jansen, S. Thomas, and H. Hermansky, "Weak top-down constraints for unsupervised acoustic model training," in Proc. ICASSP, 2013.
16. M. Hunt, S. M. Richardson, D. C. Bateman, and A. Piau, "An investigation of PLP and IMELDA acoustic representations and of their potential for combination," in Proc. ICASSP, 1991.
17. G. Synnaeve1, T. Schatz, and E. Dupoux, "Phonetics embedding learning with side information," in Proc. SLT, 2014.
18. Y. Bengio, "Learning deep architectures for AI," Found. Trends Mach. Learning, vol. 2, no. 1, pp. 1-127, 2009.
19. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
20. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layer-wise training of deep networks," in Proc. NIPS, 2007.
21. H. Sakoe and S. Chiba, "Dynamic programming algorithm optimization for spoken word recognition," IEEE Trans. Acoust., Speech, Signal Process., vol. 26, no. 1, pp. 43-49, 1978.
22. P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, "Extracting and composing robust features with denoising autoencoders," in Proc. ICML, 2008.
23. S. J. Young, G. Evermann, M. J. F. Gales, T. Hain, D. Kershaw, X. Liu, G. L. Moore, J. J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. C. Woodland, The HTK Book (for HTK Version 3. 4), Cambridge University Engineering Department, 2009.
24. I. J. Goodfellow, D. Warde-Farley, P. Lamblin, V. Dumoulin, M. Mirza, R. Pascanu, J. Bergstra, F. Bastien, and Y. Bengio, "Pylearn2: a machine learning research library," arXiv:1308. 4214, 2013.
25. C. Weng, D. Yu, S. Watanabe, and B.-H. Juang, "Recurrent deep neural networks for robust speech recognition," in Proc. ICASSP, 2014.
26. M. A. Carlin, S. Thomas, A. Jansen, and H. Hermansky, "Rapid evaluation of speech representations for spoken term discovery," in Proc. Interspeech, 2011