Aesthetics and emotions in images

IEEE Signal Processing Magazine

Volumn 28, Issue 5, 2011, Pages 94-115

  Joshi, Dhiraj ^a   Datta, Ritendra ^b   Fedorovskaya, Elena ^c   Luong, Quang Tuan ^d   Wang, James Z ^e   Li, Jia ^b   Luo, Jiebo ^a  

^a EASTMAN KODAK COMPANY   (United States)

^b PENNSYLVANIA STATE UNIVERSITY   (United States)

^c MOSCOW STATE UNIVERSITY   (Russian Federation)

^d University of Minnesota ^*

^e UNIVERSITY OF MINNESOTA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SIGNAL PROCESSING;

COMPUTATIONAL FRAMEWORK; COMPUTATIONAL INFERENCES; COMPUTATIONAL PROBLEM; REAL-WORLD; RESEARCH COMMUNITIES; VISUAL ARTS;

ELECTRICAL ENGINEERING;

EID: 85032751284     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2011.941851     Document Type: Article

References (117)

1. L. von Ahn, "Games with a purpose," IEEE Comput. Mag., vol. 39, no. 6, pp. 96-98, June 2006.
2. L. von Ahn and L. Dabbish, "Designing games with a purpose," Commun. ACM, vol. 51, no. 8, pp. 58-67, Aug. 2008.
3. I. Ara pakis, I. Konstas, and J. M. Jose, "Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance," in Proc. ACM Multimedia, 2009, pp. 461-470.
4. R. Arnhe im, Art and Visual Perception: A Psychology of the Creative Eye. Los Angeles: Univ. California Press, 1954.
5. O. Axels son, "Towards a psychology of photography: Dimensions underlying aesthetic appeal of photographs," Percept. Mot. Skills, vol. 105, no. 2, pp. 411-434, 2007.
6. M. Balabano vic and Y. Shoham, "Fab: Content-based, collaborative recommendation," Commun. ACM, vol. 40, no. 3, pp. 66-72, 1997.
7. B. Barry, "The mindful camera: Common sense for documentary videography," in Proc. ACM Multimedia, 2003, pp. 648-649.
8. B. Barry and G. Davenport, "Documenting life: Videography and common sense", in Proc. IEEE Int. Conf. Multimedia (ICME), 2003, pp. 197-200.
9. D. E. Berlyne, Aes thetics and Psychobiology. New York: Appleton-Century-Crofts, 1971.
10. N. Bianchi-Berthou ze, "K-dime: An affective image filtering system," IEEE Multimedia, vol. 10, no. 3, pp.103-106, 2003.
11. S. Bhattacharya, R. S ukthankar, and M. Shah, "A framework for photo-quality assessment and enhancement based on visual aesthetics," in Proc. ACM Multimedia, 2010, pp. 271-280.
12. M. Bressan, C. Cifarell i, and F. Perronnin, "An analysis of the relationship between painters based on their work," in Proc. IEEE ICIP, 2008, pp. 113-116.
13. I. E. Berezhnoy, E. O. Po stma, and H. J. van den Herik, "Computerized visual analysis of paintings," in Proc. 16th Int. Conf. Association History and Computing, 2005, pp. 28-32.
14. I. E. Berezhnoy, E. O. Postm a, and H. J. van den Herik, "Computer analysis of Van Gogh's complementary colors," Pattern Recognit. Lett., vol. 28, no. 6, pp. 703-709, 2007.
15. G. H. Bower, "Mood and memory," Amer. Psychol., vol. 36, no. 2, pp. 129-148, Feb. 1981.
16. C. Cerosaletti and A. Loui, "Measur ing the perceived aesthetic quality of photographic images," in Proc. 1st Int. Workshop Quality Multimedia Experience, 2009, pp. 47-52.
17. B. Cheng, B. Ni, S. Yan, and Q. Tian, "Learning to photograph," in Proc. ACM Multimedia, 2010, pp. 291-300.
18. S. Daly, "The visible differences predi ctor: An algorithm for the assessment of image fidelity," in Digital Image Hum. Vis. Cambridge, MA: MIT Press, 1993, pp. 179-206.
19. R. Datta, D. Joshi, J. Li, and J. Z. Wang, "Studying aesthetics in photographic images using a computational approach," in Proc. ECCV, 2006, pp. 288-301. (Pubitemid 44029685)
20. R. Data, D. Joshi, J. Li, and J. Z. Wang, "I mage retrieval: Ideas, influences, and trends of the new age," ACM Comput. Surv., vol. 40, no. 2, pp. 51-60, 2008.
21. R. Datta, J. Li, and J. Z. Wang, "Learning the consensus on visual quality for next generation image management," in Proc. ACM Multimedia, 2007, pp. 533-536.
22. R. Datta, J. Li, and J. Z. Wang, "Algorithmic inf erencing of aesthetics and emotion in natural images: An exposition," in Proc. ICIP, 2008, pp. 105-108.
23. B. C. Davis and S. Lazebnik, "Analysis of human att ractiveness using manifold kernel regression," in Proc. ICIP, 2008, pp. 109-112.
24. J. O'Doherty, J. Winston, H. Critchley, D. Perrett, D. M. Burt, and R. J. Dolan, "Beauty in a smile: The role of medial orbitofrontal cortex in facial attractiveness," Neuropsychologia, vol. 41, no. 2, pp. 147-155, 2003. (Pubitemid 35379692)
25. D. Dutton, The Art Instinct: Beauty, Pleasure, and Human Evolution. New York, NY: Bloomsbury Press, 2009.
26. J. Elkins, The Domain of Images. Ithaca, NY: Cornell Univ. Press, 1999.
27. J. Elkins, "Aesthetics and the two cultures: Why art and science should be allowed to go their separate ways," in Rediscovering Aesthetics: Transdisciplinary Voices from Art History, Philosophy, and Art Practice (Cultural Memory in the Present), F. Halsall, J. Jansen, and T. O'Connor, Eds. New York: Columbia Univ. Press, 2009, pp. 34-50.
28. Y. Eisenthal, G. Dror, and E. Ruppin, "Facial attractiveness : Beauty and the machine," Neural Comput., vol. 18, no. 1, pp. 119-142, 2006.
29. C. M. Falco, "Computer vision and art," IEEE Multimedia, vol. 1 4, no. 2, pp. 8-11, 2007. (Pubitemid 46681272)
30. Y. Fang, D. Geman, and N. Boujemaa, "An interactive system for men tal face retrieval," in Proc. ACM SIGMM Int. Workshop Multimedia Information Retrieval, 2005, pp. 193-200.
31. G. T. Fechner, "Zur experimentalen Ästhetik (On experimental aesthet ics)," Abhandlungen der Königlich Sächsischen Gesellschaft der Wissenschaften, vol. 9, pp. 555-635, 1871.
32. G. T. Fechner, "Vorschule der aesthetik," Breitkopf und Härtel, Leipzig, Breitkopf und Härtel, Leipzig, v ols. 1-2, 1876.
33. E. A. Fedorovskaya, C. Neustaedter, and W. Hao, "Image harmony for consumer im ages," in Proc. ICIP, 2008, pp. 121-124.
34. M. Freeman, The Photographer's Eye: Composition and Design for Better Digital Photos. Waltham, MA: Focal Press, Elsevier Inc., 2007.
35. S. Freud, The Interpretation of Dreams. New York: The Macmillan Company, 1913.
36. A. Gallagher, D. Joshi, J. Yu, and J. Luo, "Geo-location inference from image content and user tags," in Proc. IEEE Int. Workshop Internet Vision (CVPR), 2009, pp. 55-62.
37. A. Gallagher and T. Chen, "Using context to recognize people in consumer images," I PSJ Trans. Comput. Vis. Applicat., vol. 1, pp. 115-126, March 2009.
38. D. Goldberg, D. Nichols, B. M. Oki, and D. Terry, "Using collaborative filtering to w eave an information tapestry," Commun. ACM, vol. 35, no. 12, pp. 61-70, 1992.
39. N. Goodman, Languages of Art: An Approach to a Theory of Symbols, 2nd ed. Indianapolis, IN: Hackett Publishing Co., 1976.
40. C. Greenberg, Art and Culture Critical Essays. Boston, MA: Beacon Press, 1971.
41. J. Hays and A. Efros, "Scene completion using millions of photographs," ACM Trans. Graph ics, vol. 26, no. 2, 2007.
42. J. Hays and A. Efros, "IM2GPS: Estimating geographic information from a single image," in Proc. CVPR, 2008, pp. 1-8.
43. P. Hekkert, "Beauty in the eye of expert and non-expert beholders: A study in the appraisal of art," Amer. J. Psychol., vol. 109, no. 3, pp. 389-407, 1997. (Pubitemid 126417103)
44. J. Howe, "The rise of crowdsourcing," Wired Mag., vol. 14, no. 6, June 2006.
45. C. R. Johnson, Jr., E. Hendriks, I. J. Berezhnoy, E. Brevdo, S. M. Hughes, I. Daubechies, J. Li, E. Postma, and J. Z. Wang, "Image processing for artist identification: computerized analysis of Vincent van Gogh's painting brushstrokes," IEEE Signal Processing Mag. (Special Issue on Visual Cultural Heritage), vol. 25, no. 4, pp. 37-48, 2008.
46. H. Kawabata and S. Zeki, "Neural correlates of beauty," J. Neurophysiol., vol. 91, no. 4, pp. 1699-1705, April 200 4. (Pubitemid 38339340)
47. Y. Ke, X. Tang, and F. Jing, "The design of high-level features for photo quality assessment," in Proc. CVPR, 2006, pp. 419-426. (Pubitemid 44931359)
48. L. Kennedy and M. Naaman, "How flicker helps us make sense of the world: Context and content in community-contributed media collections," in Proc. ACM Multimedia, 2007, pp. 631-640.
49. L. Kennedy and M. Naaman, "Generating diverse and representative image search results for landmarks," in Pro c. 17th Int. Conf. World Wide Web, 2008, pp. 297-306.
50. U. Kirk, M. Skov, O. Hulme, M. S. Christensen, and S. Zeki, "Modulation of aesthetic value by semantic context : An fMRI study," Neuroimage, vol. 44, no, 3, pp. 1125-1132, Feb 2009.
51. K. Koffka, Gestalt Psychology. Orlando, FL: Harcourt Brace Jovanovic, 1935.
52. S. Kroner and A. Lattner, "Authentication of free hand drawings by pattern recognition methods," in Proc. IEEE IC PR, 1998, pp. 462-464.
53. A. Kushki, P. Androutsos, K. Plataniotis, and A. Venetsanopoulos, "Retrieval of images from artistic repositories u sing a decision fusion framework," IEEE Trans. Image Process., vol. 13, no. 3, pp. 277-292, 2004.
54. P. J. Lang, M. K. Greenwald, M. M. Bradley, and A. O. Hamm, "Looking at pictures: Affective, facial, visceral, and beh avioral reactions," Psychophysiology, vol. 30, no. 3, pp. 261-273, May 1993.
55. P. J. Lang, M. M. Bradley, and B. N. Cuthbert, "International affective pictur e system (IAPS): Technical, manual, and aff ective ratings," NIMH Center for the Study of Emotion and Attention, Gainsville, FL, 1997.
56. R. Latto, "The brain of the beholder," in The Artful Eye. New York, NY: Oxford Univ. Press, 1995, pp. 66-94.
57. C. C. Li and T. Chen, "Aesthetic visual quality assessment of paintings," IEEE J. Select. Topics Signal Process., vol. 3, no. 2, pp. 236-252, 2009.
58. J. Li and J. Z. Wang, "Studying digital imagery of ancient paintings by mixtures of stochastic models," IEEE Trans. Image Process ., vol. 13, no. 3, pp. 340-353, 2004.
59. Y. Liu, K. L. Schmidt, J. F. Cohn, and S. Mitra, "Facial asymmetry quantification for expression invariant human identification," in Proc. CVPR, 2003, pp. 198-204.
60. A. Louis, M. D. Wood, A. Scalise, and J. Birkelund, " Multidimensional image value assessment and rating for automated albuming and ret rieval," in Proc. ICIP, 2008, pp. 97-100.
61. P. J. Lu and P. J. Steinhardt, "Decagonal and quasi-crystalline tilings in medieval islamic architecture," Science, vol. 315, no. 5815, pp. 1106-1110, 23 Feb 2 007. (Pubitemid 46328017)
62. P. J. Lu, "Early precision compound machine from ancient China," Science, vol. 304, no. 5677, p. 1638, 11 June 2004. (Pubitemid 38765849)
63. J. Luo, A. Savakis, S. Etz, and A. Singhal, "On the application of Bayes networks to semantic understanding of consumer photographs," in Pro c. ICIP, 2000, pp. 512-515.
64. J. Luo, M. Boutell, and C. Brown, "Exploiting context for semantic scene content understanding," IEEE Signal Processing Mag. (Special Issue on Semantic Retrieval of Multimedia), vol. 23, no. 2, pp. 101-114, 2006.
65. J. Machajdik and A. Hanbury, "Affective image classification using features inspired by psychology and art theory," in Proc. ACM Multimedia, 2010 , pp. 83-92.
66. W. J. T. Mitchell, Iconology: Image Text and Ideology. Chicago, IL: Univ. Chicago Press, 1986.
67. W. J. T. Mitchell, Picture Theory. Chicago, IL: Univ. Chicago Press, 1994.
68. C. F. Nodien, P. J. Locher, and E. A. Krupinski, "The role of formal art training on perception and aesthetic judgment of art compositions," Leonar do, vol. 26, no. 3, pp. 219-227, 1993.
69. S. E. Palmer, "Aesthetic science: Human preferences for spatial composition," Keynote address in Proc. IS&T/SPIE Electronic Imaging Conf., 2009.
70. D. I. Perrett, K. A. May, and S. Yoshikawa, "Facial shape and judgments of female attractiveness," Nature, vol. 368, pp. 239-242, 17 March 1994. (Pubitemid 24108839)
71. G. Peters, "Aesthetic primitives of images for visualization," in Proc. IEEE Int. Conf. Information Visualization, 2007, pp. 316-325. (Pubitemid 47591756)
72. V. S. Ramachandran and W. Hirstein, "Science of art: A neurological theory of aesthetic experience," J. Consciousness Stud., vol. 6, no. 6/7, pp. 15-51, June/July 1999.
73. S. Ramanathan, H. Katti, R. Huang, T.-S. Chua, and M. Kankanhalli, "Automated localization of affective objects and actions in images via caption text-cum-eye g aze analysis," in Proc. ACM Multimedia, 2009, pp. 729-732.
74. R. N. Reber, N. Schwarts, and P. Winkielman, "Processing fluency and aesthetic pleasure: Is beauty in the perceiver's processing experience?" Pers. Social Psychol. Rev., vol. 8, no. 4, pp. 364-382, 2004. (Pubitemid 39665855)
75. D. Rockmore, S. Lyu, and H. Farid, "A digital technique for authentication in the visual arts," Int. Found. Art Res., vol. 8, no. 2, pp. 12-23, 2006.
76. A. Savakis, S. E tz, and A. Loui, "Evaluation of image appeal in consumer photography," in Proc. SPIE Human Vision and Electronic Imaging, 2000, pp. 111-120.
77. J. E. Scheib, S. W. Gangestad, and R. Thornhill, "Facial attractiveness, symmetry, and cues of good genes," Proc. Royal Soc. London, Biol Sci, vol. 266, no. 1431, pp. 1913-1917, 22 Sept 1999. (Pubitemid 29457076)
78. H. R. Sheikh, A. C. Bovik, and L. Cormack, "No-reference quality assessment using natural scene statistics: JPEG2000," IEEE Trans. Image Processing, vol. 14, no. 11, pp. 1918-1927, 2005. (Pubitemid 41646081)
79. B. Shevade, H. Sundaram, and L. Xie, "Modeling personal and social network context for event annotation in images," in Proc. Joint Conf. Digital Libraries, 2007, pp. 127-134. (Pubitemid 350157970)
80. P. Singh and B. Barry, "Teaching machines about everyday life," BT Technol J., vol. 22, no. 4, pp. 227-240, 2004.
81. A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain , "Contentbased image retrieval at the end of early years," IEEE Trans. Pattern Anal. Machine Intell., vol. 22, no. 12, pp. 1349-1380, 2000. (Pubitemid 32088465)
82. R. L. Solso, The Psychology of Art and the Evolution of the Conscious Brain. Cambridge, MA: MIT Press, 2003.
83. I. Stamos and P. Allen, "3-D model construction using range and image data," in Proc. CVPR, 2000, pp. 531-536.
84. D. Stork, "Computer vision and computer g raphics analysis of paintings and drawings: An introduction to the literature," in Proc Int. Conf. Computer Analysis of Images and Patterns (LNCS 5702). Berlin: Springer-Verlag, 2009, pp. 9-24.
85. X. Sun, H. Yao, R. Ji, and S. Liu, "Photo assessment based on computational visual attention model," in Proc. ACM Multimedia, 2009, pp. 541-544.
86. J. P. Swaddle and I. C. Cuth ill, "Asymmetry and human facial attractiveness: Symmetry may not always be beautiful," in Proc. Royal Soc. London, Biol Sci, vol. 261, no. 1360, pp. 111-116, 2 July 1995.
87. R. Taylor, A. P. Micolich, and D. Jones, "Fractal analysis of pollock's drip paintings," Nature, vol. 399, no. 6735, p. 422, 1999. (Pubitemid 29258838)
88. R. Taylor, "Pollock, Mondrian and the nature: Recent scientific investigations," Chaos Complexity Lett., vol. 1, no. 3, pp. 265-277, 2004.
89. A. Torralba , R. Fergus, and W. T. Freeman, "80 million tiny images: A large dataset for nonparametric object and scene recognition," IEEE Trans. Pattern Anal. Machine Intell., vol. 30, no. 11, pp. 1958-1970, 2008.
90. R. S. Ulrich and L. Gilpin, "Healing arts: Nutrition for the soul," in Putting Patients First: Designing and Practicing Patient-Centered Care. San Francisco: Jossey-Bass (Wiley), 2003, pp. 117-146.
91. R. Valenti, N. Sebe, and T. Gevers, "Facial expression recognition: A fully integrated approach," in Proc. Int. Workshop Visual and Multimedia Digital Libraries, 2007, pp. 125-130.
92. R. Valenti, A. Jaimes, and N. Sebe, "Sonify your face: Facial expressions for sound generation," in Proc. ACM Multimedia, 2010, pp. 1363-1372.
93. C. W. Valentine, The Experimental Psychology of Beauty. London: Methuen and Co. Ltd Publishers, 1962.
94. W. Wang and Q. He, "A survey on emotional semantic image retrieval," in P roc. ICIP, 2008, pp. 117-120.
95. X. J. Wang, L. Zhang, F. Jing, and W. Y. Ma, "Annosearch: Image auto-anno tation by search," in Proc. CVPR, 2006, pp. 1483-1490. (Pubitemid 44931495)
96. A. B. Watson, "Toward a perceptual video quality met ric," Proc. SPIE, vol. 3299, pp. 139-147, January 1998.
97. A. R. Willis and D. B. Cooper, "Computational reconstruction of ancient ar tifacts-From ruins to relics," IEEE Signal Processing Mag., vol. 25, no. 4, pp. 65-83, 2008.
98. A. S. Winston and G. C. Cupchik, "The evaluation of high art and popular art by naive and experienced viewers," Vis. Arts Res., vol. 18, no. 1, pp. 1-14, 1992.
99. R. Wollheim, Painting as an Art. Princeton, NJ: Princeton Un iv. Press, 1987.
100. J. Wypijewski, Ed., Painting by the Numbers: Komar and Melamid's Scientific Guide to Art. New York: Farrar, Straus and Giroux, 1997.
101. Y. Yang, M. S ong, N. Li, J. Bu, and C. Chen, "Visual attention analysis by pseudo gravitat ional field", in Proc. ACM Multimedia, 2010, pp. 553-556.
102. V. Yanulevskaya, J. C. van Gemert, K. Roth, A. K. Herbold, N. Sebe, and J. M. Geusebroek, "Emotional valence categorization using holistic image features," in Proc. ICIP, 2008, pp. 101-104.
103. D. W. Zaidel and J. A. Cohen, "The face, beauty, and symmetry: Perceiving asymmetry in beautiful faces," Int. J. Neurosci., vol. 115, no. 8, pp. 1165-1173, August 2005. (Pubitemid 40992417)
104. S. Zeki, Inner Vision: An Explorat ion of Art and the Brain. New York: Oxford Univ. Press, 1999.
105. A. Zunjarwad, H. Sundaram, and L. Xie, "Contextual wisdom: Social relations and correlations for multimedia event annotation," in Proc. ACM Mult imedia, 2007, pp. 615-624.
106. "Special issue on image processing f or cultural heritage," IEEE Trans. Image Processing, vol. 13, no. 3, 2004.
107. "Special issue on semantic retrieval of multimedia," IEEE Signal Processing Mag., vol. 23, no. 2, 2006.
108. "Special issue on visual cultural heritage," IEEE Signal Processing Mag., vol. 25, no. 4, 2008.
109. ACQUINE [Online]. Available: http://acquine.alipr.com
110. ALIPR [Online]. Available: http://alipr.com
111. DPChallenge [Online]. Available: http://w ww.dpchallenge.com
112. Encyclopedia Britannica [Online]. A vailable: http://www. britannica.com
113. Flickr [Online]. Available: http://www.flickr.com
114. Nadia Camera [On line]. Available: http://www.wired.com/gadgetlab/2010/ 07/nadia-camera-offers-opinion-of-your-terrible-photos/
115. Photo.net [Online]. Available: http://phot o.net
116. Terragalleria [Online]. Available: http://www.terragalleria.com
117. USA Today [Online]. Available: http://www.usatoday.com/tech/news/ techinnovations/2006-12-18-computer-feelings-x.htm