Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2016-May, Issue , 2016, Pages 5200-5204

  Trigeorgis, George ^a   Ringeval, Fabien ^b,c   Brueckner, Raymond ^c,d   Marchi, Erik ^c   Nicolaou, Mihalis A ^e   Schuller, Bjorn ^a,b,f   Zafeiriou, Stefanos ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF PASSAU   (Germany)

^c TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^d NUANCE COMMUNICATIONS   (United States)

^e UNIVERSITY OF LONDON   (United Kingdom)

^f EERING UG ^*  (Germany)

Author keywords

CNN; deep learning; emotion recognition; end to end learning; LSTM; raw waveform

Indexed keywords

EID: 84973293291     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2016.7472669     Document Type: Conference Paper

References (28)

1. J. Schmidhuber, "Deep learning in neural networks: An overview, " Neural Networks, vol. 61, pp. 85-117, January 2015.
2. G. Hinton, L. Deng, Y. Dong, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " Signal Processing Magazine, vol. 29, no. 6, pp. 82-97, November 2012.
3. A. Graves and N. Jaitly, "Towards end-to-end speech recognition with recurrent neural networks, " in Proc. ICML, Beijing, China, 2014, pp. 1764-1772.
4. B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, M. Mortillaro, H. Salamin, A. Polychroniou, F. Valente, and S. Kim, "The INTERSPEECH 2013 Computational Paralinguistics Challenge: Social signals, conflict, emotion, autism, " in Proc. INTERSPEECH, Lyon, France, August 2013, pp. 148-152, ISCA.
5. F. Ringeval et al., "AV+EC 2015-The First Affect Recognition Challenge Bridging Across Audio, Video, and Physiological Data, " in Proc. AVEC, Fabien Ringeval, Björn Schuller, Michel Valstar, Roddy Cowie, and Maja Pantic, Eds., Brisbane, Australia, October 2015, pp. 3-8, ACM.
6. N. Jaitly and G. Hinton, "Learning a better representation of speech sound waves using restricted Boltzmann machines, " in Proc. ICASSP, Prague, Czech Republic, May 2011, pp. 5884-5887, IEEE.
7. M. Bhargava and R. Rose, "Architectures for deep neural network based acoustic models defined over windowed speech waveforms, " in Proc. INTERSPEECH, Dresden, Germany, September 2015, pp. 6-10, ISCA.
8. T. Sainath, O. Vinyals, A. Senior, and H. Sak, "Convolutional, long short-term memory, fully connected deep neural networks, " in Proc. ICASSP, Brisbane, Australia, April 2015, pp. 4580-4584, IEEE.
9. T. Sainath, R. Weiss, A. Senior, K. Wilson, and O. Vinyals, "Learning the speech front-end with raw waveform cldnns, " in Proc. INTERSPEECH, Dresden, Germany, September 2015, pp. 1-5, ISCA.
10. D. Palaz, R. Collobert, and M. Magimai-Doss, "Estimating phoneme class conditional probabilities from raw speech signal using convolutional neural networks, " in Proc. INTERSPEECH, Lyon, France, August 2013, pp. 1766-1770, ISCA.
11. D. Palaz, M. Magimai-Doss, and R. Collobert, "Analysis of cnn-based speech recognition system using raw speech as input, " in Proc. INTERSPEECH, Dresden, Germany, September 2015, pp. 11-15, ISCA.
12. S. Dieleman and B. Schrauwen, "End-to-end learning for music audio, " in Proc. ICASSP, Florence, Italy, April 2014, pp. 7014-7018.
13. S. Sigtia, E. Benetos, and S. Dixon, "An end-to-end neural network for polyphonic music transcription, " arXiv, vol. arXiv: 1508. 01774, 2015.
14. B. Milde and C. Biemann, "Using representation learning and out-of-domain data for a paralinguistic speech task, " in Proc. INTERSPEECH, Dresden, Germany, September 2015, pp. 904-908, ISCA.
15. Q. Mao, M. Dong, Z. Huang, and Y. Zhan, "Learning salient features for speech emotion recognition using convolutional neural networks, " IEEE Transactions on Multimedia, vol. 16, no. 8, pp. 2203-2213, Dec 2014.
16. L. I-Kuei Lin, "A concordance correlation coefficient to evaluate reproducibility, " Biometrics, vol. 45, no. 1, pp. 255-268, March 1989.
17. H. G. Hirsch, P. Meyer, and H. W. Ruehl, "Improved speech recognition using high-pass filtering of subband envelopes, " in Proc. EUROSPEECH, Genoa, Italy, September 1991, pp. 413-416, ISCA.
18. R. Schlüter, L. Bezrukov, H. Wagner, and H. Ney, "Gammatone features and feature combination for large vocabulary speech recognition, " in Proc. ICASSP. April 2007, vol. 4, pp. 649-652, IEEE.
19. A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional lstm and other neural network architectures, " Neural Networks, IJCNN Special Issue, vol. 18, no. 5-6, pp. 602-610, July-August 2005.
20. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
21. F. Ringeval et al., "Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data, " Pattern Recognition Letters, vol. 66, pp. 22-30, November 2015.
22. F. Ringeval, A. Sonderegger, J. Sauer, and D. Lalanne, "Introducing the RECOLA Multimodal Corpus of Remote Collaborative and Affective Interactions, " in Proc. of EmoSPACE, FG, Shanghai, China, 2013.
23. F. Eyben et al., "The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing, " IEEE Transactions on Affective Computing, 2015, in press.
24. D. Kingma and J. Ba, "Adam: A method for stochastic optimization, " in Proc. ICLR, San Diego, USA, 2015.
25. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting, " Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929-1958, January 2014.
26. M. Kächele, P. Thiam, G. Palm, F. Schwenker, and M. Schels, "Ensemble methods for continuous affect recognition: Multimodality, temporality, and challenges, " in Proc. AVEC, Fabien Ringeval, Björn Schuller, Michel Valstar, Roddy Cowie, and Maja Pantic, Eds., Brisbane, Australia, October 2015, pp. 9-16.
27. S. Mariooryad and C. Busso, "Correcting time-continuous emotional labels by modeling the reaction lag of evaluators, " IEEE Transactions on Affective Computing, vol. 6, no. 2, pp. 97-108, April-June 2015.
28. K. Scherer, "Vocal communication of emotion: A review of research paradigms, " Speech Communication, vol. 40, no. 1-2, pp. 227-256, April 2003.