Automated facial coding: Validation of basic emotions and FACS AUs in facereader

Journal of Neuroscience, Psychology, and Economics

Volumn 7, Issue 4, 2014, Pages 227-236

  Lewinski, Peter ^a   Den Uyl, Tim M ^b   Butler, Crystal ^c  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b Vicarious Perception Technologies   (Netherlands)

^c POLYTECHNIC UNIVERSITY   (United States)

Author keywords

Action units; Basic emotions; Facereader; Facial expressions; FACs

Indexed keywords

ARTICLE; AUTOMATED PATTERN RECOGNITION; CLASSIFIER; COMPUTER PROGRAM; EMOTION; FACIAL EXPRESSION; FACING ACTION CODING SYSTEM; INTERRATER RELIABILITY; MEASUREMENT ACCURACY; PRIORITY JOURNAL;

EID: 84925411653     PISSN: 1937321X     EISSN: 2151318X     Source Type: Journal    
DOI: 10.1037/npe0000028     Document Type: Article

References (32)

1. Chentsova-Dutton, Y. E., & Tsai, J. L. (2010). Selffocused attention and emotional reactivity: The role of culture. Journal of Personality and Social Psychology, 98, 507-519. http://dx.doi.org/10.1037/a0018534
2. Chiu, M. H., Chou, C. C., Wu, W. L., & Liaw, H. (2014). The role of facial microexpression state (FMES) change in the process of conceptual conflict. British Journal of Educational Technology, 45, 471-486. http://dx.doi.org/10.1111/bjet.12126
3. Cook, I. A., Warren, C., Pajot, S. K., Schairer, D., & Leuchter, A. F. (2011). Regional brain activation with advertising images. Journal of Neuroscience, Psychology, and Economics, 4, 147-160. http://dx .doi.org/10.1037/a0024809
4. Cootes, T. F., & Taylor, C. J. (2004). Statistical models of appearance for computer vision. Imaging Science and Biomedical Engineering. University of Manchester, Manchester, UK.
5. Danner, L., Sidorkina, L., Joechl, M., & Duerrschmid, K. (2014). Make a face! Implicit and explicit measurement of facial expressions elicited by orange juices using face reading technology. Food Quality and Preference, 32, 167-172. http://dx.doi.org/10.1016/j.foodqual.2013.01.004
6. den Uyl, M., & van Kuilenburg, H. (2005). The FaceReader: Online facial expression recognition. In L. P. J. J. Noldus, F. Grieco, L. W. S. Loijens, & P. H. Zimmerman (Eds.), Proceedings of Measuring Behavior 2005, 5th International Conference on Methods and Techniques in Behavioral Research (pp. 589-590). Wageningen, The Netherlands: Noldus Information Technology.
7. de Wijk, R. A., He, W., Mensink, M. G., Verhoeven, R. H., & de Graaf, C. (2014). ANS responses and facial expressions differentiate between the taste of commercial breakfast drinks. PLoS ONE, 9, e93823. http://dx.doi.org/10.1371/journal.pone .0093823
8. Ekman, P., & Cordano, D. (2011). What is meant by calling emotions basic. Emotion Review, 3, 364-370. http://dx.doi.org/10.1177/1754073911410740
9. Ekman, P., & Friesen, W. V. (1978). Facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.
10. Ekman, P., Friesen, W. V., & Hager, J. C. (2002). Facial action coding system: The manual. Salt Lake City, UT: Research Nexus.
11. Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in facial displays of emotion. Science, 164, 86-88. http://dx.doi.org/10.1126/science.164.3875.86
12. Garcia-Burgos, D., & Zamora, M. C. (2013). Facial affective reactions to bitter-tasting foods and body mass index in adults. Appetite, 71, 178-186. http://dx.doi.org/10.1016/j.appet.2013.08.013
13. Goldberg, J. H. (2014). Measuring software screen complexity: Relating eye tracking, emotional valence, and subjective ratings. International Journal of Human-Computer Interaction, 30, 518-532. http://dx.doi.org/10.1080/10447318.2014.906156
14. Lajante, M., Droulers, O., Dondaine, T., & Amarantini, D. (2012). Opening the "black box" of electrodermal activity in consumer neuroscience research. Journal of Neuroscience, Psychology, and Economics, 5, 238-249. http://dx.doi.org/10.1037/a0030680
15. Le, Q. V., Ranzato, M., Monga, R., Devin, M., & Chen, K. (2013). Building high-level features using large scale unsupervised learning. Proceedings of the 29th IEEE International Conference on Machine Learning, 8595-8598. http://dx.doi.org/10.1109/ICASSP.2013.6639343
16. Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86, 2278-2324. http://dx.doi.org/10.1109/5.726791
17. Lewinski, P., Fransen, M. L., & Tan, E. S. H. (2014). Predicting advertising effectiveness by facial expressions in response to amusing persuasive stimuli. Journal of Neuroscience, Psychology, and Economics, 7, 1-14. http://dx.doi.org/10.1037/npe0000012
18. Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska directed emotional faces (KDEF) [CD ROM from Department of Clinical Neuroscience, Psychology section]. Karolinska, Sweden: Karolinska Institutet.
19. Micu, A. C., & Plummer, J. T. (2010). Measurable emotions: How television ads really work patterns of reactions to commercials can demonstrate advertising effectiveness. Journal of Advertising Research, 50, 137-153. http://dx.doi.org/10.2501/S0021849910091300
20. Nelson, N. L., & Russell, J. A. (2013). Universality revisited. Emotion Review, 5, 8-15. http://dx.doi .org/10.1177/1754073912457227
21. Noldus. (2014). FaceReader: Tool for automatic analysis of facial expression: Version 6.0 [Software]. Wageningen, The Netherlands: Noldus Information Technology B. V.
22. Olszanowski, M., Pochwatko, G., Kuklínski, K., Ścibor-Rylski, M., & Ohme, R. (2008, June). Warsaw set of emotional facial expression picturesvalidation study. Opatija, Croatia: EAESP General Meeting.
23. Ramsøy, T. Z., Friis-Olivarius, M., Jacobsen, C., Jensen, S. B., & Skov, M. (2012). Effects of perceptual uncertainty on arousal and preference across different visual domains. Journal of Neuroscience, Psychology, and Economics, 5, 212-226. http://dx.doi.org/10.1037/a0030198
24. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161-1178. http://dx.doi.org/10.1037/h0077714
25. Russell, J. A. (1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115, 102-141. http://dx.doi.org/10.1037/0033-2909.115.1.102
26. Terzis, V., Moridis, C. N., & Economides, A. A. (2012). The effect of emotional feedback on behavioral intention to use computer based assessment. Computers & Education, 59, 710-721. http://dx.doi.org/10.1016/j.compedu.2012.03.003
27. Terzis, V., Moridis, C. N., & Economides, A. A. (2013). Measuring instant emotions based on facial expressions during computer-based assessment. Personal and Ubiquitous Computing, 17, 43-52. http://dx.doi.org/10.1007/s00779-011-0477-y
28. Valstar, M. F., Mehu, M., Jiang, B., Pantic, M., & Scherer, K. (2012). Meta-analysis of the first facial expression recognition challenge. IEEE Transactions on Systems, Man, and Cybernetics Part B, Cybernetics, 42, 966-979. http://dx.doi.org/10.1109/TSMCB.2012.2200675
29. van der Schalk, J., Hawk, S. T., Fischer, A. H., & Doosje, B. (2011). Moving faces, looking places: Validation of the Amsterdam Dynamic Facial Expression Set (ADFES). Emotion, 11, 907-920. http://dx.doi.org/10.1037/a0023853
30. van Kuilenburg, H., Wiering, M., & den Uyl, M. (2005). A model-based method for facial expression recognition. In J. Gama, R. Camacho, P. Brazdil, A. Jorge, & L. Torgo (Eds.), Lectures notes in computer science: Vol. 3720. Machine learning: ECML 2005 (pp. 194-205). Berlin, Germany: Springer-Verlag. http://dx.doi.org/10.1007/11564096_22
31. Wexler, D. (1972). Method for unitizing protocols of descriptions of emotional states. Journal of Supplemental Abstracts Service, 2, 116.
32. With, S., & Delplanque, S. (2013, August). The Geneva Facial Action Toolbox (GeFACT) A Matlab ® toolbox for processing facial action units' time series. Poster presented at the 2013 biennial meeting of International Society for Research on Emotions, Berkeley, CA. Abstract retrieved from http://www.isre2013.org/online/core_routines/view_abstract_no.phpshow_close_window_yes&abstractno_78