Using affective parameters in a content-based recommender system for images

User Modeling and User-Adapted Interaction

Volumn 20, Issue 4, 2010, Pages 279-311

  Tkalčič, Marko ^a   Burnik, Urban ^a   Košir, Andrej ^a  

^a UNIVERSITY OF LJUBLJANA   (Slovenia)

Author keywords

Affective modeling; Content based recommender system; Emotion induction; IAPS; Item profile; Machine learning; Metadata; User profile; Valence arousal dominance

Indexed keywords

LEARNING SYSTEMS;

AFFECTIVE MODELING; CONTENT-BASED; EMOTION INDUCTION; IAPS; ITEM PROFILE; MACHINE-LEARNING; USER PROFILE; VALENCE-AROUSAL-DOMINANCE;

METADATA;

EID: 78049471510     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-010-9079-z     Document Type: Article

References (60)

1. G. Adomavicius A. Tuzhilin 2005 Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions IEEE. Trans. Knowl. Data Eng. 17 6 734 749 10.1109/TKDE.2005.99 (Pubitemid 40860454)
2. Ali, K., Van Stam, W.: TiVo: making show recommendations using a distributed collaborative filtering architecture. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 394-401. ACM, New York, NY, USA (2004)
3. Arapakis, I., Moshfeghi, Y., Joho, H., Ren, R., Hannah, D., Jose, J., Gardens, L.: Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. In: Proceedings of the IEEE International Conference on Multimedia and Expo, pp. 1440-1443 (2009)
4. Basu, C., Hirsh, H., Cohen, W.: Recommendation as classification: using social and content-based information in recommendation. In: Proceedings of the National Conference on Artificial Intelligence, pp. 714-720. Wiley, New York (1998)
5. A. Batliner S. Steidl C. Hacker E. Noth 2008 Private emotions versus social interaction: a data-driven approach towards analysing emotion in speech User Model. User-Adapt. Interact. J. Pers. Res. 18 1 175 206 10.1007/s11257-007-9039-4 10.1007/s11257-007-9039-4 (Pubitemid 351185636)
6. Berger H., Denk M., Dittenbach M., Pesenhofer A., Merkl D.: Photo-based user profiling for tourism recommender systems. In: Psaila, G., Wagner, R. (eds) E-Commerce and Web Technologies, vol. 4655, pp. 46-55. Springer, Berlin (2007)
7. Bradley, M.M., Lang, P.J.: The International Affective Picture System (IAPS) in the Study of Emotion and Attention, chap. 2. Series in Affective Science. Oxford University Press, Oxford (2007)
8. R. Burke 2002 Hybrid recommender systems: survey and experiments User Model. User-Adapt. Interact. 12 4 331 370 10.1023/A:1021240730564 1030.68607 10.1023/A:1021240730564
9. S. Carberry F. de Rosis 2008 Introduction to special issue on affective modeling and adaptation User Model. User-Adapt. Interact. 18 1 1 9 10.1007/s11257-007-9044-7
10. G. Caridakis K. Karpouzis M. Wallace L. Kessous N. Amir 2010 Multimodal users affective state analysis in naturalistic interaction J. Multimodal User Interfaces 3 1 49 66 10.1007/s12193-009-0030-8
11. Coan J., Allen J.: Handbook of Emotion Elicitation and Assessment. Oxford University Press, New York (2007)
12. C. Conati H. Maclaren 2009 Empirically building and evaluating a probabilistic model of user affect User Model. User-Adapt. Interact. 19 3 267 303 10.1007/s11257-009-9062-8
13. R. Cowie E.D. Cowie N. Tsapatsoulis G. Votsis S. Kollias W. Fellenz J. Taylor 2001 Emotion recognition in human-computer interaction IEEE Signal Process. Mag. 18 1 32 80 10.1109/79.911197 (Pubitemid 32287669)
14. Darwin C.: The Expression of the Emotions in Man and Animals. Oxford University Press, New York (1872)
15. S. D'Mello S. Craig A. Witherspoon B. McDaniel A. Graesser 2008 Automatic detection of learner's affect from conversational cues User Model. User-Adapt. Interact. J. Pers. Res. 18 1 45 80 10.1007/s11257-007-9037-6 10.1007/s11257-007-9037-6
16. Ekman P.: Basic emotions. In: Dalgleish, T., Power, T. (eds) Handbook of Cognition and Emotion, Wiley, New York (1999)
17. J.L. Fleiss 1971 Measuring nominal scale agreement among many raters Psychol. Bull. 76 5 378 382 10.1037/h0031619
18. L.R. Goldberg J.A. Johnson H.W. Eber R. Hogan M.C. Ashton C.R. Cloninger H.G. Gough 2006 The international personality item pool and the future of public-domain personality measures J. Res. Pers. 40 84 96 10.1016/j.jrp.2005.08. 007 (Pubitemid 43055663)
19. González, G., López, B., de la Rosa, J.L.L.: Managing emotions in smart user models for recommender systems. In: Proceedings of 6th International Conference on Enterprise Information Systems ICEIS 2004, vol. 5, pp. 187-194 (2004)
20. Grouplens Data Sets.: http://www.grouplensorg/node/12. Accessed Sept 2010
21. A. Hanjalic 2006 Extracting moods from pictures and sounds IEEE Signal Process. Mag. 23 2 90 10.1109/MSP.2006.1621452
22. Hastie T., Tibshirani R., Friedman J.H.: The Elements of Statistical Learning. Springer, New York (2001)
23. J. Herlocker J. Konstan L. Terveen J. Riedl 2004 Evaluating collaborative filtering recommender systems ACM Trans. Inf. Syst. 22 1 53 10.1145/963770.963772
24. S. Ioannou A. Raouzaiou V. Tzouvaras T. Mailis K. Karpouzis S. Kollias 2005 Emotion recognition through facial expression analysis based on a neurofuzzy network Neural Netw. 18 4 423 435 10.1016/j.neunet.2005.03.004 (Pubitemid 40922649)
25. Irun, M., Moltó Brotons, F.: Looking at pictures in North America and Europe: a cross-cultural study on the IAPS. In: Poster presented at the 1997 FEPS Meeting in Konstanz (1997)
26. Joho, H., Jose, J., Valenti, R., Sebe, N.: Exploiting facial expressions for affective video summarisation. In: Proceeding of the ACM International Conference on Image and Video Retrieval, pp. 1-8. ACM (2009)
27. Y. Kim B. Yum J. Song S. Kim 2005 Development of a recommender system based on navigational and behavioral patterns of customers in e-commerce sites Expert Syst. Appl. 28 2 381 393 10.1016/j.eswa.2004.10.017 (Pubitemid 40061301)
28. Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, vol. 2, no. 12, pp. 1137-1143. Morgan Kaufmann, San Mateo (1995)
29. Y. Koren R. Bell C. Volinsky 2009 Matrix factorization techniques for recommender systems Computer 42 8 30 37 ieeecomputersociety.org/10.1109/MC.2009. 263 10.1109/MC.2009.263
30. Lang, P., Bradley, M., Cuthbert, B.: International affective picture system (IAPS): affective ratings of pictures and instruction manual. Technical Report a-6. Technical Report, University of Florida, Gainesville, FL (2005)
31. Lehman E.L., Romano J.: Testing Statistical Hypotheses. Springer, New York (2005)
32. M.S. Lew N. Sebe C. Djeraba R. Jain 2006 Content-based multimedia information retrieval: state of the art and challenges ACM Trans. Multimed. Comput. 2 1 1 19 10.1145/1126004.1126005
33. Lichtenstein A., Oehme A., Kupschick S., Jürgensohn T.: Comparing Two Emotion Models for Deriving Affective States from Physiological Data, pp. 35-50. Springer-Verlag, Berlin (2008). doi: 10.1007/978-3-540-85099-1
34. McNee, S., Lam, S., Konstan, J., Riedl, J.: Interfaces for eliciting new user preferences in recommender systems. In: User Modeling 2003: 9th International Conference, UM 2003, Johnstown, PA, USA, June 22-26, 2003: Proceedings, pp. 178-187. Springer-Verlag, Berlin (2003)
35. S. McQuiggan B. Mott J. Lester 2008 Modeling self-efficacy in intelligent tutoring systems: an inductive approach User Model. User-Adapt. Interact. J. Pers. Res. 18 1 81 123 10.1007/s11257-007-9040-y 10.1007/s11257-007-9040-y
36. A. Mehrabian 1996 Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament Curr. Psychol. 14 4 261 292 10.1007/BF02686918 1714069
37. Nunes, M.A., Cerri, S., Blanc, N.: Improving recommendations by using personality traits in user profiles. In: Proceedings of I-KNOW'08 8th International Conference on Knowledge Management and Knowledge Technologies, pp. 92-100. Graz, Austria (2008)
38. M. Pantic A. Vinciarelli 2009 Implicit human-centered tagging IEEE Signal Process. Mag. 26 6 173 180 10.1109/MSP.2009.934186
39. Pazzani, M., Billsus, D.: Content-Based Recommendation Systems. The Adaptive Web, pp. 325-341. doi: 10.1007/978-3-540-72079-9 (2007)
40. Picard R.W.: Affective Computing. MIT Press, Cambridge (2000)
41. R. Plutchik 2001 The nature of emotions Am. Sci. 89 4 344 350
42. M. Pogačnik J. Tasič M. Meža A. Košir 2005 Personal content recommender based on a hierarchical user model for the selection of TV programmes User Model. User Adapt. Interact. 15 425 457 10.1007/s11257-005-4065-6 (Pubitemid 41721241)
43. K. Porayska-Pomsta M. Mavrikis H. Pain 2008 Diagnosing and acting on student affect: the tutor's perspective User Model. User-Adapt. Interact. J. Pers. Res. 18 1 125 173 10.1007/s11257-007-9041-x 10.1007/s11257-007-9041-x
44. J. Posner J.A. Russell B. Peterson 2005 The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology Dev. Psychopathol. 17 715 734 10.1017/S0954579405050340
45. Rashid, A., Albert, I., Cosley, D., Lam, S., McNee, S., Konstan, J., Riedl, J.: Getting to know you: learning new user preferences in recommender systems. In: Proceedings of the 7th International Conference on Intelligent User Interfaces, January, ACM, pp. 13-16 (2002)
46. R. Ribeiro S. Pompéia O. Bueno 2005 Comparison of Brazilian and American norms for the International Affective Picture System (IAPS) Revista Brasileira de Psiquiatria 27 208 215
47. Rottenberg J., Ray R.D., Gross J.J.: Emotion Elicitation Using Films, chap. 2. Oxford University Press, London (2007)
48. J. Scheirer R. Fernandez J. Klein R.W. Picard 2002 Frustrating the user on purpose: a step toward building an affective computer Interact. Comput. 14 2 93 118 10.1016/S0953-5438(01)00059-5
49. K. Scherer 2005 What are emotions? And how can they be measured? Soc. Sci. Inf. 44 4 695 10.1177/0539018405058216 (Pubitemid 41666684)
50. Schröeder, M., Baggia, P., Burkhardt, F., Oltramari, A., Pelachaud, C., Peter, C., Zovato, E.: Emotion markup language (emotionml) 1.0. W3C Working Draft 29 July 2010. http://www.w3.org/TR/2010/WD-emotionml-20100729/ (2010)
51. M.K. Shan F.F. Kuo M.F. Chiang S.Y. Lee 2009 Emotion-based music recommendation by affinity discovery from film music Expert. Syst. Appl. 36 4 7666 7674 10.1016/j.eswa.2008.09.042 10.1016/j.eswa.2008.09.042
52. Tkalčič, M., Tasič, J, Košir, A.: The LDOS-PerAff-1 corpus of face video clips with affective and personality metadata. In: Kipp M (ed.) Proceedings of the LREC 2010 Workshop on Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality (2010)
53. Verschuere, B., Crombez, G., Koster, E.: Cross Cultural Validation of the IAPS. Ghent University, Ghent, Belgium. http://users.ugent.be/~bvschuer/Iaps. pdf (2007)
54. Villon, O., Lisetti, C.: A user-modeling approach to build user's psycho-physiological maps of emotions using bio-sensors. In: The 15th IEEE International Symposium on Robot and Human Interactive Communication, 2006, ROMAN 2006, pp 269-276 (2006)
55. A. Vinciarelli M. Pantic H. Bourlard 2009 Social signal processing: survey of an emerging domain Image Vis. Comput. 27 12 1743 1759 10.1016/j.imavis.2008.11.007
56. Westen D.: Psychology: Mind, Brain and Culture. 2nd edn. Wiley, New York (1999)
57. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. 2nd edn. Morgan Kaufmann Series in Data Management Systems. Morgan Kaufmann, San Francisco (2005)
58. G. Yannakakis J. Hallam H. Lund 2008 Entertainment capture through heart rate activity in physical interactive playgrounds User Model. User-Adapt. Interact. J. Pers. Res. 18 1 207 243 10.1007/s11257-007-9036-7 10.1007/s11257-007-9036-7 (Pubitemid 351185637)
59. Yik M., Russell J.A., Ahn C.k., Fernandez Dols J.M., Suzuki N.: Relating the five-factor model of personality to a circumplex model of affect: a five-language study. In: McCrae, R.R., Allik, J. (eds) The Five-Factor Model of Personality Across Cultures, pp. 79-104. Kluwer Academic Publishers, New York (2002)
60. Z. Zeng M. Pantic G.I. Roisman T.S. Huang 2009 A survey of affect recognition methods: audio, visual, and spontaneous expressions IEEE Trans. Pattern Anal. Mach. Intell. 31 1 39 58 10.1109/TPAMI.2008.52 10.1109/TPAMI.2008.52