Deep learning our everyday emotions a short overview

Smart Innovation, Systems and Technologies

Volumn 37, Issue , 2015, Pages 339-346

  Schuller, Björn ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Affective computing; Deep learning; Emotion recognition; Neural networks

Indexed keywords

INTELLIGENT SYSTEMS; MEDICAL COMPUTING; NEURAL NETWORKS;

AFFECTIVE COMPUTING; AUTOMATIC EMOTION RECOGNITION; CONFIDENCE ESTIMATION; DEEP LEARNING; EMOTION RECOGNITION; FUNCTIONAL ACTIVITIES; MULTITASK LEARNING; REAL-LIFE APPLICATIONS;

SPEECH RECOGNITION;

EID: 84930936055     PISSN: 21903018     EISSN: 21903026     Source Type: Book Series    
DOI: 10.1007/978-3-319-18164-6_33     Document Type: Article

References (38)

1. Amer, M.R., Siddiquie, B., Richey, C., Divakaran, A.: Emotion Detection in Speech using Deep Networks. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2013), Florence, Italy. IEEE (2013)
2. Bennett, I.: Emotion detection device and method for use in distributed systems. US Patent 8,214,214 (July 3, 2012)
3. Bruckner, R., Schuller, B.: Likability Classification - A not so Deep Neural Network Approach. In: Proceedings of the INTERSPEECH 2012, 13th Annual Conference of the International Speech Communication Association, ISCA, Portland, OR, 4 pages (September 2012)
4. Bruckner, R., Schuller, B.: Hierarchical Neural Networks and Enhanced Class Posteriors for Social Signal Classification. In: Proceedings 13th Biannual IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2013, Olomouc, Czech Republic, 6 pages. IEEE (December 2013)
5. Bruckner, R., Schuller, B.: Being at Odds? - Deep and Hierarchical Neural Net-works for Classification and Regression of Conflict in Speech. In: Poggi, I., D'Errico, F., Vinciarelli, A. (eds.) Conflict and Negotiation: Social Research and Machine Intelligence. Computational Social Sciences. Springer, Heidelberg (2014)
6. Bruckner, R., Schuller, B.: Social Signal Classification Using Deep BLSTM Re-current Neural Networks. In: Proceedings of the 39th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2014, Florence, Italy, pp. 4856-4860. IEEE (May 2014)
7. Cibau, N.E., Albornoz, E.M., Rufiner, H.L.: Speech emotion recognition using a deep autoencoder. In: Proceedings of the XV Reunión de Trabajo en Procesamiento de la Información y Control (RPIC 2013), San Carlos de Bariloche (2013)
8. Coutinho, E., Deng, J., Schuller, B.: Transfer Learning Emotion Manifestation Across Music and Speech. In: Proceedings of the 2014 International Joint Conference on Neural Networks (IJCNN) as part of the IEEE World Congress on Computational Intelligence (IEEE WCCI), Beijing, China, p. 6. IEEE (July 2014)
9. Deng, J., Han, W., Schuller, B.: Confidence Measures for Speech Emotion Recog-nition: a Start. In: Fingscheidt, T., Kellermann, W. (eds.) Proceedings of Speech Communication; 10. ITG Symposium, Braunschweig, Germany, pp. 1-4. ITG, IEEE (2012)
10. Deng, J., Schuller, B.: Confidence Measures in Speech Emotion Recognition Based on Semi-supervised Learning. In: Proceedings of the INTERSPEECH 2012, 13th Annual Conference of the International Speech Communication Association, ISCA, Portland, OR, 4 pages. ISCA (September 2012)
11. Deng, J., Xia, R., Zhang, Z., Liu, Y., Schuller, B.: Introducing Shared-Hidden-Layer Autoencoders for Transfer Learning and their Application in Acoustic Emo-tion Recognition. In: Proceedings 39th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2014, Florence, Italy, May 2014, pp. 4851-4855. IEEE (2014)
12. Deng, J., Zhang, Z., Marchi, E., Schuller, B.: Sparse Autoencoder-based Feature Transfer Learning for Speech Emotion Recognition. In: Proc. 5th Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII 2013), Geneva, Switzerland, pp. 511-516. HUMAINE Association, IEEE (2013)
13. Deng, J., Zhang, Z., Schuller, B.: Linked Source and Target Domain Subspace Feature Transfer Learning - Exemplified by Speech Emotion Recognition. In: Proceedings 22nd International Conference on Pattern Recognition (ICPR 2014), Stockholm, Sweden, pp. 761-766. IAPR (August 2014)
14. Erhan, D., Bengio, Y., Courville, A., Vincent, P.-A.M.P., Bengio, S.: Why Does Unsupervised Pre-training Help Deep Learning? The Journal of Machine Learning Research 11, 625-660 (2010)
15. Esparza, J., Scherer, S., Schwenker, F.: Studying Self- and Active-Training Methods for Multi-feature Set Emotion Recognition. In: Schwenker, F., Trentin, E. (eds.) PSL 2011. LNCS, vol. 7081, pp. 19-31. Springer, Heidelberg (2012)
16. Eyben, F., Wollmer, M., Schuller, B.: A Multi-Task Approach to Continuous Five-Dimensional Affect Sensing in Natural Speech. ACM Transactions on Interactive Intelligent Systems, Special Issue on Affective Interaction in Natural Environments 2(1), 29 (2012)
17. Han, W., Li, H., Ruan, H., Ma, L., Sun, J., Schuller, B.: Active Learning for Dimensional Speech Emotion Recognition. In: Proceedings INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, Lyon, France, pp. 2856-2859. ISCA (August 2013)
18. Han, W., Zhang, Z., Deng, J., Wollmer, M., Weninger, F., Schuller, B.: Towards Distributed Recognition of Emotion in Speech. In: Proceedings 5th International Symposium on Communications, Control, and Signal Processing, ISCCSP 2012, Rome, Italy, pp. 1-4. IEEE (May 2012)
19. Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Im-proving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580 (2012)
20. Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527-1554 (2006)
21. Huang, C., Gong, W., Fu, W., Feng, D.: A Research of Speech Emotion Recognition Based on Deep Belief Network and SVM. Mathematical Problems in Engineering, Article ID 749604, 7 (2014)
22. Jirayucharoensak, S., Pan-Ngum, S., Israsena, P.: EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation. The Scientific World Journal, Article ID 627892, 10 (2014)
23. Kahou, S.E., Pal, C., Bouthillier, X., Froumenty Gulcehre, P., Memisevic, R., Vincent, P., Courville, A., Bengio, Y.: Combining Modality Specific Deep Neural Networks for Emotion Recognition in Video. In: Proceedings of the 15th ACM In-ternational Conference on Multimodal Interaction (ICMI 2013), Sydney, Australia, pp. 543-550. ACM (2013)
24. Kim, Y., Lee, H., Provost, E.M.: Deep learning for robust feature generation in audio-visual emotion recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2013), Vancouver, Canada. IEEE (2013)
25. Le, D., Provost, E.: Emotion recognition from spontaneous speech using Hidden Markov models with deep belief networks. In: 2013 IEEE Workshop on Proceedings Automatic Speech Recognition and Understanding (ASRU), pp. 216-221. IEEE, Olomouc (2013)
26. Li, L., Zhao, Y., Jiang, D., Zhang, Y., Wang, F., Gonzalez, I., Valentin, E., Sahli, H.: Hybrid Deep Neural Network - Hidden Markov Model (DNN-HMM) Based Speech Emotion Recognition. In: Proceedings Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII 2013). IEEE, Geneva (2013)
27. Maas, A., Hannun, A., Ng, A.: Rectifier Nonlinearities Improve Neural Network Acoustic Models. In: Proc. of ICML Workshop on Deep Learning for Audio, Speech, and Language Processing, WDLASL, Atlanta, GA, USA (June 2013)
28. Popovic, B., Ostrogonac, S., Delic, V., Janev, M., Stankovic, I.: Deep Architectures for Automatic Emotion Recognition Based on Lip Shape. Infotech-Jahorina 12, 939-943 (2013)
29. Sanchez-Gutierrez, M.E., Albornoz, E.M., Martinez-Licona, F., Rufiner, H.L., God-dard, J.: Deep Learning for Emotional Speech Recognition. In: Martinez-Trinidad, J.F., Carrasco-Ochoa, J.A., Olvera-Lopez, J.A., Salas-Rodríguez, J., Suen, C.Y. (eds.) MCPR 2014. LNCS, vol. 8495, pp. 311-320. Springer, Heidelberg (2014)
30. Schmidhuber, J.: Deep Learning in Neural Networks: An Overview. Technical Report IDSIA-03-14, IDSIA, Lugano, Switzerland (2014)
31. Schmidt, E.M., Kim, Y.E.: Learning Emotion-based Acoustic Features with Deep Belief Networks. In: Proceedings 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, pp. 65-68. IEEE (2011)
32. Stuhlsatz, A., Meyer, C., Eyben, F., Zielke, T., Meier, G., Schuller, B.: Deep Neu-ral Networks for Acoustic Emotion Recognition: Raising the Benchmarks. In: Pro-ceedings 36th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, pp. 5688-5691. IEEE, Prague (2011)
33. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.-A.: Extracting and composing robust features with denoising autoencoders. In: Proc. of ICML, New York, NY, USA, pp. 1096-1103 (2008)
34. Weninger, F., Eyben, F., Schuller, B.W., Mortillaro, M., Scherer, K.R.: On the Acoustics of Emotion in Audio: What Speech, Music and Sound have in Common. Frontiers in Psychology, Emotion Science, Special Issue on Expression of Emotion in Music and Vocal Communication 4 (Article ID 292), 1-12 (2013)
35. Zeiler, M., Ranzato, M., Monga, R., Mao, M., Yang, K., Le, Q.V., Nguyen, P., Senior, A., Vanhoucke, V., Dean, J., Hinton, G.: On Rectiied Linear Units for Speech Processing. In: ICASSP, Vancouver, Canada, May 2013, pp. 3517-3521. IEEE (2013)
36. Zhang, Z., Deng, J., Marchi, E., Schuller, B.: Active Learning by Label Uncertainty for Acoustic Emotion Recognition. In: Proceedings INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, Lyon, France, pp. 2841-2845. ISCA (August 2013)
37. Zhang, Z., Schuller, B.: Active Learning by Sparse Instance Tracking and Classi-ier Conidence in Acoustic Emotion Recognition. In: Proceedings INTERSPEECH 2012, 13th Annual Conference of the International Speech Communication Association, Portland, OR, p. 4. ISCA (September 2012)
38. Zhang, Z., Weninger, F., Wöllmer, M., Schuller, B.: Unsupervised Learning in Cross-Corpus Acoustic Emotion Recognition. In: Proceedings 12th Biannual IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2011, pp. 523-528. IEEE, Big Island (2011)