Dynamic time warping in phoneme modeling for fast pronunciation error detection

Computers in Biology and Medicine

Volumn 69, Issue , 2016, Pages 277-285

  Miodonska, Zuzanna ^a   Bugdol, Marcin D ^a   Krecichwost, Michal ^a  

^a SILESIAN UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

CAPT systems; DTW algorithm; Phoneme modeling; Pronunciation error detection; Word structure analysis

Indexed keywords

ERROR DETECTION; ERRORS; MARKOV PROCESSES; SPEECH RECOGNITION;

DTW ALGORITHMS; DYNAMIC TIME WARPING; DYNAMIC TIME WARPING ALGORITHMS; PHONEME MODELS; PHONETIC STRUCTURE; PRONUNCIATION ERROR DETECTION; PRONUNCIATION TRAININGS; WORD STRUCTURES;

HIDDEN MARKOV MODELS;

ARTICLE; DYNAMIC TIME WARPING; ERROR; FURTHEST SEGMENT SEARCH; HIDDEN MARKOV MODEL; MODEL; NORMALIZED FURTHEST SEGMENT SEARCH; NORMALIZED PHONEME DISTANCES THRESHOLDING; PHONEME; PRIORITY JOURNAL; PRONUNCIATION ERROR; SPEECH; TIME SERIES ANALYSIS; VARIATIONS OF THE WORD STRUCTURE; ALGORITHM; BIOLOGICAL MODEL; PHONETICS; POLAND; SPEECH DISORDERS;

ALGORITHMS; MODELS, BIOLOGICAL; PHONETICS; POLAND; SPEECH DISORDERS;

EID: 84957841078     PISSN: 00104825     EISSN: 18790534     Source Type: Journal    
DOI: 10.1016/j.compbiomed.2015.12.004     Document Type: Article

References (50)

1. Demenko G., Wagner A., Cylwik N. The use of speech technology in foreign language pronunciation training. Arch. Acoust. 2010, 35(5):309-330.
2. Y. Qin, G. Wang, A computer-aided Chinese pronunciation training program for English-speaking learners, in: 2014 International Conference on Asian Language Processing (IALP), 2014, pp. 154-157, . doi:10.1109/IALP.2014.6973499.
3. Su P.-H., Wu C.-H., Lee L.-S. A recursive dialogue game for personalized computer-aided pronunciation training. IEEE/ACM Trans. Audio Speech Lang. Process. 2015, 23(1):127-141. 10.1109/TASLP.2014.2375572.
4. T. Zhao, A. Hoshino, M. Suzuki, N. Minematsu, K. Hirose, Automatic Chinese pronunciation error detection using SVM trained with structural features, in: SLT, IEEE, Miami, Florida, 2012, ISBN: 978-1-4673-5125-6; . http://dx.doi.org/10.1109/SLT.2012.6424270.
5. H. Strik, K.P. Truong, F. de Wet, C. Cucchiarini, Comparing classifiers for pronunciation error detection, in: INTERSPEECH, ISCA, 2007, pp. 1837-1840.
6. S. Xu, J. Jiang, Z. Chen, B. Xu, Automatic pronunciation error detection based on linguistic knowledge and pronunciation space, in: ICASSP, IEEE, Taipei, Taiwan, 2009, pp. 4841-4844. http://dx.doi.org/10.1109/ICASSP.2009.4960715.
7. M.-S. Liang, J.-Y. Hung, R.-Y. Lyu, Y.-C. Chiang, Pronunciation error detection for computer assisted pronunciation teaching in mandarin, in: 6th International Symposium on Chinese Spoken Language Processing, 2008, ISCSLP [U+05F3]08, 2008, pp. 1-4, . doi:10.1109/CHINSL.2008.ECP.98.
8. W. Hu, Y. Qian, F.K. Song, Y. Wang, Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiers, Speech Commun. 67 (2015) 154-166, . http://dx.doi.org/10.1016/j.specom.2014.12.008.
9. M. Russell, R.W. Series, J.L. Wallace, C. Brown, A. Skilling, The STAR system: an interactive pronunciation tutor for young children, Comput Speech Lang. (2000) 161-175, . http://dx.doi.org/10.1006/csla.2000.0139.
10. G. Demenko, N. Cylwik, A. Wagner, Applying Speech and Language Technology to Foreign Language Education.
11. G. Demenko, A. Wagner, N. Cylwik, O. Jokisch, An audiovisual feedback system for acquiring l2 pronunciation and l2 prosody, in: SLaTE 2009.
12. N. Cylwik, A. Wagner, G. Demenko, The EURONOUNCE corpus of non-native polish for ASR-based pronunciation tutoring system, in: SLaTE, 2009.
13. H. Wang, X. Qian, H. Meng, Phonological modeling of mispronunciation gradations in L2 English speech of L1 Chinese learners, in: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2014, Florence, Italy, May 4-9, 2014, pp. 7714-7718, . doi:10.1109/ICASSP.2014.6855101.
14. S.M. Witt, Use of speech recognition in computer-assisted language learning, doctoral dissertation, University of Cambridge, November 1999.
15. Witt S.M., Young S.J. Phone-level pronunciation scoring and assessment for interactive language learning. Speech Commun. 2000, 30(2-3):95-108.
16. W. Hu, Y. Qian, F. Soong, A new neural network based logistic regression classifier for improving mispronunciation detection of L2 language learners, in: 2014 9th International Symposium on Chinese Spoken Language Processing (ISCSLP), 2014, pp. 245-249, . doi:10.1109/ISCSLP.2014.6936712.
17. K. Li, H. Meng, Mispronunciation detection and diagnosis in l2 English speech using multi-distribution deep neural networks, in: 2014 9th International Symposium on Chinese Spoken Language Processing (ISCSLP), 2014, pp. 255-259, . doi:10.1109/ISCSLP.2014.6936724.
18. H. Yuan, J. Xu, J. Zhao, J. Liu, Improve low-resource non-native mispronunciation detection with native speech by articulatory-based tandem feature, in: 2013 IEEE China Summit International Conference on Signal and Information Processing (ChinaSIP), 2013, pp. 127-131, . doi:10.1109/ChinaSIP.2013.6625312.
19. Liao H.-C., Guan Y.-H., Tu J.-J., Chen J.-C. A prototype of an adaptive Chinese pronunciation training system. System 2014, 45:52-66.
20. H. Franco, L. Ferrer, H. Bratt, Adaptive and discriminative modeling for improved mispronunciation detection, in: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp. 7709-7713, . doi:10.1109/ICASSP.2014.6855100.
21. Rabiner L., Juang B.-H. Fundamentals of Speech Recognition 1993, Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
22. Ge Z., Sharma S.R., Smith M.J.T. Improving mispronunciation detection using adaptive frequency scale. Comput. Electr. Eng. 2013, 39(5):1464-1472. 10.1016/j.compeleceng.2012.12.001.
23. Sahidullah M., Saha G. Design, analysis and experimental evaluation of block based transformation in MFCC computation for speaker recognition. Speech Commun. 2012, 54(4):543-565.
24. Koolagudi S.G., Rastogi D., Rao K.S. Identification of language using mel-frequency cepstral coefficients (MFCC). Proc. Eng. 2012, 38(0):3391-3398. International Conference on Modelling, Optimization and Computing.
25. Koniaris C., Salvi G., Engwall O. On mispronunciation analysis of individual foreign speakers using auditory periphery models. Speech Commun. 2013, 55(5):691-706.
26. Y.-B. Wang, L.-S. Lee, Toward unsupervised discovery of pronunciation error patterns using universal phoneme posteriorgram for computer-assisted language learning, in: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013, pp. 8232-8236, . doi:10.1109/ICASSP.2013.6639270.
27. A. Rosenberg, A. Colla, A connected speech recognition system based on spotting diphone-like segments-preliminary results, in: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP [U+05F3]87, vol. 12, 1987, pp. 85-88.
28. M. Saraclar, H.J. Nock, S. Khudanpur, Pronunciation modeling by sharing Gaussian densities across phonetic models. Comput. Speech Lang. (2000) 137-160, . http://dx.doi.org/10.1006/csla.2000.0140.
29. M. Saraclar, S. Khudanpur, Pronunciation change in conversational speech and its implications for automatic speech recognition, Comput. Speech Lang. (2004) 375-395.
30. O. Jokisch, A. Wagner, R. Sabo, R. Jaeckel, N. Cylwik, M. Rusko, A. Ronzhin, R. Hoffman, Multilingual speech data collection for the assessment of pronunciation and prosody training in a language learning system, in: Proceedings of Speech and Computer (SPECOM), vol. 12, 2009, pp. 515-520.
31. R.W.M. Ng, K. Hirose, Syllable: A self-contained unit to model pronunciation variation, in: ICASSP, IEEE, Kyoto, Japan, 2012, pp. 4457-4460. ISBN: 978-1-4673-0046-9; . http://dx.doi.org/10.1109/ICASSP.2012.6288909.
32. B. Ziółko, J. Gałka, S. Manandhar, R.C. Wilson, M. Ziółko, Triphone statistics for polish language, in: Z. Vetulani, H. Uszkoreit (Eds.), LTC, Lecture Notes in Computer Science, vol. 5603, Springer, Berlin, Heidelberg, 2007, pp. 63-73.
33. M. Liu, Z. Moore, L. Graham, S. Lee, A look at the research on computer-based technology use in second language learning: a review of the literature from 1990-2000, J. Res. Technol. Educ. 34(3) (2002).
34. O. Jokisch, U. Koloska, D. Hirschfeld, R. Hoffmann, Pronunciation learning and foreign accent reduction by an audiovisual feedback system., in: J. Tao, T. Tan, R.W. Picard (Eds.), ACII, Lecture Notes in Computer Science, vol. 3784, Springer, 2005, pp. 419-425.
35. B.M. Manhung, B. Mak, M. Siu, M. Ng, Y. cheung Tam, Y. chung Chan, K. wah Chan, K. yee Leung, S. Ho, F. ho Chong, J. Wong, J. Lo, Plaser: pronunciation learning via automatic speech recognition, in: Proceedings of the HLT-NAACL 2003 Workshop on Building Educational Applications using Natural Language Processing, 2003, pp. 23-29.
36. Atwell E., Howarth P., Souter C. The isle corpus. Italian and German spoken learners English. ICAME J. 2003, 27:5-18.
37. M. Bugdol, Z. Segiet, M. Krȩcichwost, Pronunciation error detection using dynamic time warping algorithm, in: E. Piȩtka, J. Kawa, W. Wiȩcławek (Eds.), Information Technologies in Biomedicine, vol. 4, Advances in Intelligent Systems and Computing, vol. 284, Springer International Publishing, Gliwice, 2014, pp. 345-354, ISBN: 978-3-319-06595-3, . doi:10.1007/978-3-319-06596-0_32.
38. Liu J., Cheng Q., Zheng Z., Qian M. A DTW-based probability model for speaker feature analysis and data mining. Pattern Recognit. Lett. 2002, 23(11):1271-1276. 10.1016/S0167-8655(02)00068-5.
39. Jing XinXing, Shi Xu, Speech recognition based on efficient DTW algorithm and its DSP implementation, Proc. Eng. 29 (2012) 832-836, 2012, International Workshop on Information and Electronics Engineering, . doi:10.1016/j.proeng.2012.01.050.
40. G. Demenko, M. Wypych, E. Baranowska, Implementation of grapheme-to-phoneme rules and extended SAMPA alphabet in polish text-to-speech synthesis, Poznań 7(17) (2003).
41. B. Ziolko, J. Galka, M. Ziolko, Polish phoneme statistics obtained on large set of written texts, Comput. Sci. 10(3).
42. J. Wells, The SAMPA Homepage, 〈〉. http://www.phon.ucl.ac.uk/home/sampa/index.html.
43. S. Kanters, C. Cucchiarini, H. Strik, The goodness of pronunciation algorithm: a detailed performance study, in: SLaTE 2009, 2009.
44. D.J. Bemdt, J. Clifford, Using dynamic time warping to find patterns in time series, KDD-94: AAAI Workshop on Knowledge Discovery in Databases, Seattle, Washington, pp. 359-370 (July 1994).
45. Rabiner L.R. Considerations in dynamic time warping algorithms for discrete word recognition. Acoust. Soc. Am. J. 1978, 63:79. 10.1121/1.2016831.
46. Salvador S., Chan P. Toward accurate dynamic time warping in linear time and space. Intell. Data Anal. 2007, 11(5):561-580.
47. H. Sakoe, S. Chiba, Dynamic programming algorithm optimization for spoken word recognition, in: Readings in Speech Recognition, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1990, pp. 159-165.
48. C. A. Ratanamahatana, E. Keogh, Everything you know about dynamic time warping is wrong, 3rd Workshop on Mining Temporal and Sequential Data, in conjunction with 10th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Seattle (2004).
49. S. Wydra, Zastosowanie parametryzacji miesznej w systemie rozpoznawania mowy polskiej, Technical report, Instytut Radioelektroniki, Politechnika Warszawska, Warszawa, 2006.
50. Y.-B. Wang, L. shan Lee, Improved approaches of modeling and detecting error patterns with empirical analysis for computer-aided pronunciation training, in: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012, pp. 5049-5052, . doi:10.1109/ICASSP.2012.6289055.