Using bidirectional lstm recurrent neural networks to learn high-level abstractions of sequential features for automated scoring of non-native spontaneous speech

2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings

Volumn , Issue , 2016, Pages 338-345

  Yu, Zhou ^a   Ramanarayanan, Vikram ^b,c   Suendermann Oeft, David ^b,c   Wang, Xinhao ^b,c   Zechner, Klaus ^a   Chen, Lei ^a   Tao, Jidong ^a   Ivanou, Aliaksei ^b,c   Qian, Yao ^b,c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b EDUCATIONAL TESTING SERVICE   (United States)

^c EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

Automatic speech scoring; non native speech; recurrent neural networks

Indexed keywords

ABSTRACTING; AGGREGATES; GRADING; RECURRENT NEURAL NETWORKS; SPEECH;

AUTOMATED RESPONSE; AUTOMATIC SPEECH; HIGH-LEVEL ABSTRACTION; LONG SHORT TERM MEMORY; MULTI LAYER PERCEPTRON; NON-NATIVE SPEECH; SPOKEN LANGUAGES; SPONTANEOUS SPEECH;

SPEECH RECOGNITION;

EID: 84964499542     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2015.7404814     Document Type: Conference Paper

References (28)

1. Zhen Wang and Alina A Davier, "Monitoring of scoring using the e-rater R automated scoring system and human raters on a writing test," ETS Research Report Series, vol. 2014, no. 1, pp. 1-21, 2014
2. Yigal Attali and Jill Burstein, "Automated essay scoring with e-rater R v. 2," The Journal of Technology, Learning and Assessment, vol. 4, no. 3, 2006
3. Klaus Zechner, Derrick Higgins, Xiaoming Xi, and David M Williamson, "Automatic scoring of non-native spontaneous speech in tests of spoken English," Speech Communication, vol. 51, no. 10, pp. 883-895, 2009
4. Thomas K Landauer, Darrell Laham, and PeterWFoltz, "Automated scoring and annotation of essays with the intelligent essay assessor," Automated essay scoring: A cross-disciplinary perspective, pp. 87-112, 2003
5. Catia Cucchiarini, Helmer Strik, and Lou Boves, "Quantitative assessment of second language learners fluency: Comparisons between read and spontaneous speech," the Journal of the Acoustical Society of America, vol. 111, no. 6, pp. 2862-2873, 2002
6. Horacio Franco, Harry Bratt, Romain Rossier, Venkata Rao Gadde, Elizabeth Shriberg, Victor Abrash, and Kristin Precoda, "Eduspeak R A speech recognition and pronunciation scoring toolkit for computer-aided language learning applications," Language Testing, vol. 27, no. 3, pp. 401-418, 2010
7. Jared Bernstein, Jian Cheng, and Masanori Suzuki, "Fluency and structural complexity as predictors of l2 oral proficiency," in Proceedings of Interspeech, 2010
8. XinhaoWang, Keelan Evanini, and Klaus Zechner, "Coherence modeling for the automated assessment of spontaneous spoken responses," in Proceedings of NAACL HLT, 2013, pp. 814-819
9. Derrick Higgins, Xiaoming Xi, Klaus Zechner, and David M. Williamson, "A three-stage approach to the automated scoring of spontaneous spoken responses," Computer Speech &Language, vol. 25, no. 2, pp. 282-306, 2011
10. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, "Imagenet classification with deep convolutional neural networks," in Proceedings of the advances in neural information processing systems, 2012, pp. 1097-1105
11. Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng, "Multimodal deep learning," in Proceedings of the 28th international conference on machine learning (ICML-11), 2011, pp. 689-696
12. Lei Chen and Klaus Zechner, "Applying rhythm features to automatically assess non-native speech," in Proceedings of the Interspeech, Florence, Italy, 2011
13. Lei Chen, Klaus Zechner, and Xiaoming Xi, "Improved pronunciation features for construct-driven assessment of non-native spontaneous speech," in Proceedings of the NAACL-HLT, Boulder, CO, USA, 2009
14. Lei Chen, Joel Tetreault, and Xiaoming Xi, "Towards using structural events to assess non-native speech," in Proceedings of the NAACL-HLT, Los Angeles, CA, USA, 2010
15. Lei Chen and Su-Youn Yoon, "Application of structural events detected on ASR outputs for automated speaking assessment," in Proceedings of Interspeech, Portland, OR, USA, 2012
16. Florian Eyben, Martin Wollmer, and Bjorn Schuller, "Opensmile: The munich versatile and fast open-source audio feature extractor," in Proceedings of the International Conference on Multimedia, New York, NY, USA, 2010, MM '10, pp. 1459-1462, ACM
17. Zhou Yu, David Gerritsen, Amy Ogan, Alan W Black, and Justine Cassell, "Automatic prediction of friendship via multi-model dyadic features," in Proceedings of SIGDIAL, 2013, pp. 51-60
18. Sepp Hochreiter and Jurgen Schmidhuber, "Long shortterm memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997
19. Mike Schuster and Kuldip K Paliwal, "Bidirectional recurrent neural networks," Signal Processing, IEEE Transactions on, vol. 45, no. 11, pp. 2673-2681, 1997
20. Alex Graves and Jurgen Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures," Neural Networks, vol. 18, no. 5, pp. 602-610, 2005
21. Alan Graves, Navdeep Jaitly, and Abdel-rahman Mohamed, "Hybrid speech recognition with deep bidirectional LSTM," in Proceedings of the Automatic Speech Recognition and Understanding (ASRU). IEEE, 2013, pp. 273-278
22. Yuchen Fan, Yao Qian, Fenglong Xie, and Frank K Soong, "TTS synthesis with bidirectional LSTM based recurrent neural networks," in Proceedings of Interspeech, 2014, pp. 1964-1968
23. Alex Graves, Supervised sequence labelling with recurrent neural networks, vol. 385, Springer, 2012
24. Anastassia Loukina, Klaus Zechner, Lei Chen, and Michael Heilman, "Feature selection for automated speech scoring," in Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, 2015, pp. 12-19
25. Christopher slette MøllerMBishop, Pattern recognition and machine learning, Springer, 2006
26. David Rumelhart, Geoffrey Hinton, and Ronald Williams, "Learning internal representations by error propagation," Tech. Rep., DTIC Document, 1985
27. Alex Smola and Bernhard Scholkopf, "A tutorial on support vector regression," Statistics and computing, vol. 14, no. 3, pp. 199-222, 2004
28. Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al., "Scikit-learn: Machine learning in python," The Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011