Deep neural networks are more accurate than humans at detecting sexual orientation from facial images

Journal of Personality and Social Psychology

Volumn 114, Issue 2, 2018, Pages 246-257

  Wang, Yilun ^a   Kosinski, Michal ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Computational social science; Facial morphology; Prenatal hormone theory; Privacy; Sexual orientation

Indexed keywords

ADULT; ANATOMY AND HISTOLOGY; FACE; FACIAL RECOGNITION; FEMALE; HUMAN; MALE; PERCEPTION; SEXUALITY;

ADULT; DEEP LEARNING; FACE; FACIAL RECOGNITION; FEMALE; HUMANS; MALE; SEXUALITY; SOCIAL PERCEPTION;

EID: 85041636493     PISSN: 00223514     EISSN: None     Source Type: Journal    
DOI: 10.1037/pspa0000098     Document Type: Article

References (66)

1. Allen, L. S., & Gorski, R. A. (1992). Sexual orientation and the size of the anterior commissure in the human brain. Proceedings of the National Academy of Sciences of the United States of America, 89, 7199-7202. http://dx.doi.org/10.1073/pnas.89.15.7199
2. Ambady, N., Hallahan, M., & Conner, B. (1999). Accuracy of judgments of sexual orientation from thin slices of behavior. Journal of Personality and Social Psychology, 77, 538-547. http://dx.doi.org/10.1037/0022-3514.77.3.538
3. Astley, S. J., Stachowiak, J., Clarren, S. K., & Clausen, C. (2002). Application of the fetal alcohol syndrome facial photographic screening tool in a foster care population. The Journal of Pediatrics, 141, 712-717. http://dx.doi.org/10.1067/mpd.2002.129030
4. Ballew, C. C., II, & Todorov, A. (2007). Predicting political elections from rapid and unreflective face judgments. Proceedings of the National Academy of Sciences of the United States of America, 104, 17948-17983. http://dx.doi.org/10.1073/pnas.0705435104
5. Berry, D. S. (1991). Accuracy in social perception: Contributions of facial and vocal information. Journal of Personality and Social Psychology, 61, 298-307. http://dx.doi.org/10.1037/0022-3514.61.2.298
6. Berry, D. S., & Brownlow, S. (1989). Were the physiognomists right? Personality correlates of facial babyishness. Personality and Social Psychology Bulletin, 15, 266-279. http://dx.doi.org/10.1177/0146167289152013
7. Bond, C. F., Jr., Berry, D. S., & Omar, A. (1994). The kernel of truth in judgments of deceptiveness. Basic and Applied Social Psychology, 15, 523-534. http://dx.doi.org/10.1207/s15324834basp1504_8
8. Borkenau, P., Brecke, S., Möttig, C., & Paelecke, M. (2009). Extraversion is accurately perceived after a 50-ms exposure to a face. Journal of Research in Personality, 43, 703-706. http://dx.doi.org/10.1016/j.jrp.2009.03.007
9. Brambilla, M., Riva, P., & Rule, N. O. (2013). Familiarity increases the accuracy of categorizing male sexual orientation. Personality and Individual Differences, 55, 193-195. http://dx.doi.org/10.1016/j.paid.2013.02.023
10. Brown, E., & Perrett, D. I. (1993). What gives a face its gender? Perception, 22, 829-840. http://dx.doi.org/10.1068/p220829
11. Bulygina, E., Mitteroecker, P., & Aiello, L. (2006). Ontogeny of facial dimorphism and patterns of individual development within one human population. American Journal of Physical Anthropology, 131, 432-443. http://dx.doi.org/10.1002/ajpa.20317
12. Chin, J., & Lin, L. (2017, June 26). China's all-seeing surveillance state is reading its citizens' faces. The Wall Street Journal. Retrieved from https://www.wsj.com
13. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542, 115-118. http://dx.doi.org/10 .1038/nature21056
14. Ferry, Q., Steinberg, J., Webber, C., FitzPatrick, D. R., Ponting, C. P., Zisserman, A., & Nellåker, C. (2014). Diagnostically relevant facial gestalt information from ordinary photos. eLife, 2014. http://dx.doi.org/ 10.7554/eLife.02020.001
15. Glaser, G. (2002). The nose: A profile of sex, beauty, and survival. New York, NY: Simon & Schuster.
16. Glimcher, M. E., Garcia, R. I., & Szabó, G. (1978). Organ culture of mammalian skin and the effects of ultraviolet light and testosterone on melanocyte morphology and function. The Journal of Experimental Zoology, 204, 229-237. http://dx.doi.org/10.1002/jez.1402040210
17. Halberstadt, A. G., Hayes, C. W., & Pike, K. M. (1988). Gender and gender role differences in smiling and communication consistency. Sex Roles, 19, 589-604. http://dx.doi.org/10.1007/BF00289738
18. Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning (Vol. 1). New York, NY: Springer series in statistics. http://dx.doi.org/10.1007/b94608
19. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. arXiv Preprint arXiv:1512.03385v1, 7, 171-180. http://dx.doi.org/10.3389/fpsyg.2013.00124
20. Hines, M. (2010). Sex-related variation in human behavior and the brain. Trends in Cognitive Sciences, 14, 448-456. http://dx.doi.org/10.1016/j.tics.2010.07.005
21. Hughes, S. M., & Bremme, R. (2011). The effects of facial symmetry and sexually-dimorphic facial proportions on assessments of sexual orientation. Journal of Social, Evolutionary, & Cultural Psychology, 5, 214-230. http://dx.doi.org/10.1037/h0099261
22. Jablonski, N. G., & Chaplin, G. (2000). The evolution of human skin coloration. Journal of Human Evolution, 39, 57-106. http://dx.doi.org/10.1006/jhev.2000.0403
23. Jannini, E. A., Blanchard, R., Camperio-Ciani, A., & Bancroft, J. (2010). Male homosexuality: Nature or culture? Journal of Sexual Medicine, 7, 3245-3253. http://dx.doi.org/10.1111/j.1743-6109.2010.02024.x
24. Jenkinson, J. (1997). Face facts: A history of physiognomy from ancient Mesopotamia to the end of the 19th century. The Journal of Biocommunication, 24, 2-7.
25. Jernigan, C., & Mistree, B. F. (2009). Gaydar: Facebook friendships expose sexual orientation. First Monday. Advance online publication. http://dx.doi.org/10.5210/fm.v14i10.2611
26. Jones, B. C., Hahn, A. C., Fisher, C. I., Wincenciak, J., Kandrik, M., Roberts, S. C.,. DeBruine, L. M. (2015). Facial coloration tracks changes in women's estradiol. Psychoneuroendocrinology, 56, 29-34. http://dx.doi.org/10.1016/j.psyneuen.2015.02.021
27. King, M., Semlyen, J., Tai, S. S., Killaspy, H., Osborn, D., Popelyuk, D., & Nazareth, I. (2008). A systematic review of mental disorder, suicide, and deliberate self harm in lesbian, gay and bisexual people. BMC Psychiatry, 8, 70. http://dx.doi.org/10.1186/1471-244X-8-70
28. Kosinski, M., Matz, S. C., Gosling, S. D., Popov, V., & Stillwell, D. (2015). Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist, 70, 543-556. http://dx.doi.org/10.1037/a0039210
29. Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences of the United States of America, 110, 5802-5805. http://dx.doi.org/10.1073/pnas.1218772110
30. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436-444. http://dx.doi.org/10.1038/nature14539
31. Lefevre, C. E., Lewis, G. J., Perrett, D. I., & Penke, L. (2013). Telling facial metrics: Facial width is associated with testosterone levels in men. Evolution and Human Behavior, 34, 273-279. http://dx.doi.org/10.1016/j.evolhumbehav.2013.03.005
32. Lefevre, C. E., Wilson, V. A. D., Morton, F. B., Brosnan, S. F., Paukner, A., & Bates, T. C. (2014). Facial width-to-height ratio relates to alpha status and assertive personality in capuchin monkeys. PLoS ONE, 9, e93369. http://dx.doi.org/10.1371/journal.pone.0093369
33. Little, A. C., Burriss, R. P., Jones, B. C., & Roberts, S. C. (2007). Facial appearance affects voting decisions. Evolution and Human Behavior, 28, 18-27. http://dx.doi.org/10.1016/j.evolhumbehav.2006.09.002
34. Lõhmus, M., Sundström, L. F., & Björklund, M. (2009). Dress for success: Human facial expressions are important signals of emotions. Annales Zoologici Fennici, 46, 75-80. http://dx.doi.org/10.5735/086.046.0108
35. Lombroso, C. (1911). Criminal man. New York, NY: GP Putnam.
36. Lu, C., & Tang, X. (2015). Surpassing human-level face verification performance on LFW with GaussianFace. In Blai Bonet & Sven Koenig (Eds.), Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (pp. 3811-3819). Austin, TX: AAAI Press; Retrieved fromhttp://arxiv.org/abs/1404.3840
37. Lubin, G. (2016, October 12). Facial-profiling could be dangerously inaccurate and biased, experts warn. Business Insider. Retrieved from http://www.businessinsider.de/
38. Lukaszewski, A. W., & Roney, J. R. (2011). The origins of extraversion: Joint effects of facultative calibration and genetic polymorphism. Personality and Social Psychology Bulletin, 37, 409-421. http://dx.doi.org/10.1177/0146167210397209
39. Lyons, M., Lynch, A., Brewer, G., & Bruno, D. (2014). Detection of sexual orientation ("gaydar") by homosexual and heterosexual women. Archives of Sexual Behavior, 43, 345-352. http://dx.doi.org/10.1007/s10508-013-0144-7
40. Macrae, C. N., & Bodenhausen, G. V. (2000). Social cognition: Thinking categorically about others. Annual Review of Psychology, 51, 93-120. http://dx.doi.org/10.1146/annurev.psych.51.1.93
41. Meyer-Bahlburg, H. F. L. (1984). Psychoendocrine research on sexual orientation. Current status and future options. Progress in Brain Research, 61, 375-398. http://dx.doi.org/10.1016/S0079-6123(08)64448-9
42. Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep face recognition. In X. Xie, M. W. Jones, & G. K. L. Tam (Eds.), Proceedings of the British Machine Vision Conference (p. 41.1). Swansea, UK: BMVA Press. http://dx.doi.org/10.5244/C.29.41
43. Penton-Voak, I. S., Pound, N., Little, A. C., & Perrett, D. I. (2006). Personality judgments from natural and composite facial images: More evidence for a "kernel of truth" in social perception. Social Cognition, 24, 607-640. http://dx.doi.org/10.1521/soco.2006.24.5.607
44. Porter, R. (Ed.), (2003). The Cambridge history of science: Eighteenthcentury science (Vol. 4). Cambridge, UK: Cambridge University Press.
45. Rhodes, G. (2006). The evolutionary psychology of facial beauty. Annual Review of Psychology, 57, 199-226. http://dx.doi.org/10.1146/annurev.psych.57.102904.190208
46. Riedweg, C. (2005). Pythagoras: His life, teaching, and influence. New York, NY: Cornell University Press.
47. Roberts, T., & Bruce, V. (1988). Feature saliency in judging the sex and familiarity of faces. Perception, 17, 475-481. http://dx.doi.org/10.1068/p170475
48. Rule, N. O., & Ambady, N. (2008). Brief exposures: Male sexual orientation is accurately perceived at 50 ms. Journal of Experimental Social Psychology, 44, 1100-1105. http://dx.doi.org/10.1016/j.jesp.2007.12.001
49. Rule, N. O., & Ambady, N. (2010). Democrats and republicans can be differentiated from their faces. PLoS ONE, 5, e8733. http://dx.doi.org/10.1371/journal.pone.0008733
50. Rule, N. O., Ambady, N., Adams, R. B., & Macrae, C. N. (2008). Accuracy and awareness in the perception and categorization of male sexual orientation. Journal of Personality and Social Psychology, 95, 1019-1028. http://dx.doi.org/10.1037/a0013194
51. Rule, N. O., Macrae, C. N., & Ambady, N. (2009). Ambiguous group membership is extracted automatically from faces. Psychological Science, 20, 441-443. http://dx.doi.org/10.1111/j.1467-9280.2009.02314.x
52. Samochowiec, J., Wänke, M., & Fiedler, K. (2010). Political ideology at face value. Social Psychological & Personality Science, 1, 206-213. http://dx.doi.org/10.1177/1948550610372145
53. Sell, R. L., Wells, J. A., & Wypij, D. (1995). The prevalence of homosexual behavior and attraction in the United States, the United Kingdom and France: Results of national population-based samples. Archives of Sexual Behavior, 24, 235-248. http://dx.doi.org/10.1007/BF01541598
54. Skerski, J. (2011). Tomboy chic: Re-fashioning gender rebellion. Journal of Lesbian Studies, 15, 466-479. http://dx.doi.org/10.1080/10894160.2011.532031
55. Skorska, M. N., Geniole, S. N., Vrysen, B. M., McCormick, C. M., & Bogaert, A. F. (2015). Facial structure predicts sexual orientation in both men and women. Archives of Sexual Behavior, 44, 1377-1394. http://dx.doi.org/10.1007/s10508-014-0454-4
56. Todorov, A., Mandisodza, A. N., Goren, A., & Hall, C. C. (2005). Inferences of competence from faces predict election outcomes. Science, 308, 1623-1626. http://dx.doi.org/10.1126/science.1110589
57. Todorov, A., Said, C. P., Engell, A. D., & Oosterhof, N. N. (2008). Understanding evaluation of faces on social dimensions. Trends in Cognitive Sciences, 12, 455-460. http://dx.doi.org/10.1016/j.tics.2008.10.001
58. Tskhay, K. O., & Rule, N. O. (2015). Emotions facilitate the communication of ambiguous group memberships. Emotion, 15, 812-826. http:// dx.doi.org/10.1037/emo0000077
59. Udry, J. R. (2000). Biological limits of gender construction. American Sociological Review, 65, 443-457. http://dx.doi.org/10.2307/2657466
60. Udry, J. R., & Chantala, K. (2006). Masculinity-femininity predicts sexual orientation in men but not in women. Journal of Biosocial Science, 38, 797-809. http://dx.doi.org/10.1017/S002193200500101X
61. United Nations Human Rights Council. (2015). Discrimination and violence against individuals based on their sexual orientation and gender identity. Retrieved from http://www.refworld.org/docid/5571577c4.html
62. Valentova, J. V., Kleisner, K., Havlícek, J., & Neustupa, J. (2014). Shape differences between the faces of homosexual and heterosexual men. Archives of Sexual Behavior, 43, 353-361. http://dx.doi.org/10.1007/s10508-013-0194-x
63. Whitehouse, A. J. O., Gilani, S. Z., Shafait, F., Mian, A., Tan, D. W., Maybery, M. T., Eastwood, P. (2015). Prenatal testosterone exposure is related to sexually dimorphic facial morphology in adulthood. Proceedings of the Royal Society B: Biological Sciences, 282, 20151351. http://dx.doi.org/10.1098/rspb.2015.1351
64. Zebrowitz, L. A. (1997). Reading faces: Window to the soul? New directions in social psychology. Boulder, CO: Westview Press.
65. Zebrowitz, L. A., & Collins, M. A. (1997). Accurate social perception at zero acquaintance: The affordances of a Gibsonian approach. Personality and Social Psychology Review, 1, 204-223. http://dx.doi.org/10.1207/ s15327957pspr0103_2
66. Zebrowitz, L. A., Collins, M., & Dutta, R. (1998). The relationship between appearance and personality across the life span. Personality and Social Psychology Bulletin, 24, 736-749. http://dx.doi.org/10.1177/01 46167298247006