Behavioral signal processing: Deriving human behavioral informatics from speech and language

Proceedings of the IEEE

Volumn 101, Issue 5, 2013, Pages 1203-1233

  Narayanan, Shrikanth ^a   Georgiou, Panayiotis G ^a  

^a University of Southern California   (United States)

Author keywords

Affective computing; behavior; computational psychology; computational social sciences; emotions; health applications; multimodal signal processing; natural language processing; speech understanding

Indexed keywords

BEHAVIORAL RESEARCH; DECISION MAKING; INFORMATICS; MODELING LANGUAGES; NATURAL LANGUAGE PROCESSING SYSTEMS; SOCIAL ASPECTS; SOCIAL SCIENCES COMPUTING; SPEECH; SPEECH COMMUNICATION;

AFFECTIVE COMPUTING; BEHAVIOR; COMPUTATIONAL PSYCHOLOGY; COMPUTATIONAL SOCIAL SCIENCE; EMOTIONS; MULTI-MODAL SIGNAL PROCESSING; NATURAL LANGUAGE PROCESSING; SPEECH UNDERSTANDING;

AUDIO SIGNAL PROCESSING;

EID: 84876686250     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2012.2236291     Document Type: Article

References (214)

1. M. Black, J. Chang, and S. Narayanan, "An empirical analysis of user uncertainty in problem-solving child-machine interactions," in Proc. Workshop Child Comput. Interaction, 2008.
2. S. Arunachalam, D. Gould, E. Andersen, D. Byrd, and S. Narayanan, "Politeness and frustration language in child-machine interactions," in Proc. Eurospeech Conf., Aalborg, Denmark, 2001, pp. 2675-2678.
3. S. Yildirim, C. Lee, S. Lee, A. Potamianos, and S. Narayanan, "Detecting politeness and frustration state of a child in a conversational computer game," in Proc. Eurospeech Conf., Lisbon, Portugal, 2005, pp. 2209-2212.
4. T. Zhang, M. Hasegawa-Johnson, and S. Levinson, "Cognitive state classification in a spoken tutorial dialogue system," Speech Commun., vol. 48, no. 6, pp. 616-632, 2006.
5. S. Yildirim, S. Narayanan, and A. Potamianos, "Detecting emotional state of a child in a conversational computer game," Comput. Speech Lang., vol. 25, no. 1, pp. 29-44, 2011.
6. K. Forbes-Riley and D. Litman, "Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor," Speech Commun., vol. 53, no. 9-10, pp. 1115-1136, 2011.
7. H. Pon-Barry and S. M. Shieber, "Recognizing uncertainty in speech," EURASIP J. Adv. Signal Process., 2011.
8. V. Petrushin, "Emotion in speech: Recognition and application to call centers," in Proc. Artif. Neural Netw. Eng., Nov. 1999, pp. 7-10.
9. C. M. Lee, S. Narayanan, and R. Pieraccini, "Recognition of negative emotions from the speech signal," in Proc. IEEE Workshop Autom. Speech Recognit. Understand., 2001, pp. 240-243.
10. C. M. Lee and S. Narayanan, "Towards detecting emotions in spoken dialogs," IEEE Trans. Speech Audio Process., vol. 13, no. 2, pp. 293-302, Mar. 2005.
11. C. Lord, S. Risi, L. Lambrecht, E. Cook, B. Leventhal, P. DiLavore, A. Pickles, and M. Rutter, "The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism," J. Autism Develop. Disorders, vol. 30, no. 3, pp. 205-223, 2000.
12. W. R. Miller and S. Rollnick, Motivational Interviewing: Preparing People for Change. New York: Guilford, 2002.
13. A. Vinciarelli, M. Pantic, and H. Bourlard, "Social signal processing: Survey of an emerging domain," Image Vis. Comput., vol. 27, pp. 1743-1759, 2009.
14. A. Gravano, R. Levitan, L. Willson, S. Beňuš, J. Hirschberg, and A. Nenkova, "Acoustic and prosodic correlates of social behavior," in Proc. 12th Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 97-100.
15. R. Ranganath, D. Jurafsky, and D. McFarland, "Detecting friendly, flirtatious, awkward, and assertive speech in speed-dates," Comput. Speech Lang., vol. 27, no. 1, pp. 89-115, Jan. 2013.
16. K. Hammond, "Psychology's scientific revolution: Is it in danger?" Cntr. Res. Judgment Policy, Tech. Rep., 1978.
17. E. Brunswik, Perception and the Representative Design of Psychological Experiments, 2nd ed. Los Angeles, CA: Univ. California Press, 1956.
18. K. Neuendorf, The Content Analysis Guidebook. Thousand Oaks, CA: Sage, 2002.
19. W. Ickes, "Empathic accuracy," J. Personality, vol. 61, no. 4, pp. 587-610, 1993.
20. G. Margolin, P. Oliver, E. Gordis, H. O'Hearn, A. Medina, C. Ghosh, and L. Morland, "The nuts and bolts of behavioral observation of marital and family interaction," Clin. Child Family Psychol. Rev., vol. 1, no. 4, pp. 195-213, 1998.
21. K. Audhkhasi, P. G. Georgiou, and S. Narayanan, "Reliability- weighted acoustic model adaptation using crowd sourced transcriptions," in Proc. InterSpeech Conf., 2011, pp. 3045-3048.
22. K. Audhkhasi, P. G. Georgiou, and S. Narayanan, "Accurate transcription of broadcast news speech using multiple noisy transcribers and unsupervised reliability metrics," in Proc. Int. Conf. Audio Speech Signal Process., 2011, pp. 4980-4983.
23. R. Waldinger, M. Schulz, S. Hauser, J. Allen, and J. Crowell, "Reading others' emotions: The role of intuitive judgments in predicting marital satisfaction, quality, and stability," J. Family Psychol., vol. 18, no. 1, pp. 58-71, 2004.
24. K. J. W. Baucom, B. Baucom, and A. Christensen, "Do the naBve know best? The predictive power of naBve ratings of couple interactions," Psychol. Assessment, 2013.
25. O. Holsti, Content Analysis for the Social Sciences and Humanities. Reading, MA: Addison-Wesley, 1969.
26. W. Scott, "Reliability of content analysis: The case of nominal scale coding," Public Opinion Quart., vol. 19, no. 3, pp. 321-325, 1955.
27. J. Cohen, "Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit," Psychol. Bull., vol. 70, no. 4, pp. 213-220, 1968.
28. J. Fleiss, "Measuring nominal scale agreement among many raters," Psychol. Bull., vol. 76, no. 5, pp. 378-382, 1971.
29. K. Krippendorff, "Estimating the reliability, systematic error and random error of interval data," Educ. Psychol. Meas., vol. 30, no. 1, pp. 61-70, 1970.
30. D. Snyder, R. Heyman, and S. Haynes, "Evidence-based approaches to assessing couple distress," Psychol. Assessment, vol. 17, no. 3, pp. 288-307, 2005.
31. K. Audhkhasi and S. S. Narayanan, "Data-dependent evaluator modeling and its application to emotional valence classification from speech," in Proc. InterSpeech Conf., Makuhari, Japan, 2010, pp. 2366-2369.
32. K. Audhkhasi and S. S. Narayanan, "A globally-variant locally-constant model for fusion of labels from multiple diverse experts without using reference labels," IEEE Trans. Pattern Anal. Mach. Intell., 2012, DOI: 10.1109/TPAMI.2012.139.
33. J. Ross, "The reliability, validity, and utility of self-assessment," Practical Assessment Res. Eval., vol. 11, no. 10, 2006.
34. K. Clements and A. Holtzworth-Munroe, "Aggressive cognitions of violent versus nonviolent spouses," Cogn. Therapy Res., vol. 32, no. 3, pp. 351-369, 2008.
35. S. Shiffman, A. A. Stone, and M. R. Hufford, "Ecological momentary assessment," Annu. Rev. Clin. Psychol., vol. 4, pp. 1-32, 2008.
36. G. Margolin, D. Chien, S. Duman, A. Fauchier, E. Gordis, P. Oliver, M. Ramos, and K. Vickerman, "Ethical issues in couple and family research," J. Family Psychol., vol. 19, no. 1, pp. 157-167, 2005.
37. National Research Council (U.S.) Committee, Computational technology for effective health care: Immediate steps and strategic directions on engaging the computer science research community in health care informatics, National Academies Press (U.S.), 2009.
38. V. Rozgic, C. Busso, P. Georgiou, and S. Narayanan, "Multimodal meeting monitoring: Improvements on speaker tracking and segmentation through a modified mixture particle filter," in Proc. IEEE 9th Workshop Multimedia Signal Process., 2007, pp. 60-65.
39. R. Cipolla and A. Pentland, Computer Vision for Human-Machine Interaction. Cambridge, U.K.: Cambridge Univ. Press, 1998.
40. S. Chu, S. Narayanan, C.-C. J. Kuo, and M. J. Mataric, "Where am I? Scene recognition for mobile robots using audio features," in Proc. Int. Conf. Multimedia Expo, 2006, pp. 885-888.
41. G. Potamianos and P. Lucey, "Audio-visual ASR from multiple views inside smart rooms," in Proc. IEEE Int. Conf. Multisensor Fusion Integration Intell. Syst., 2006, pp. 35-40.
42. M. Annavaram, N. Medvidovic, U. Mitra, S. Narayanan, G. Sukhatme, Z. Meng, S. Qiu, R. Kumar, G. Thatte, and D. Spruijt-Metz, "Multimodal sensing for pediatric obesity applications," in Proc. UrbanSense Conf., 2008, pp. 21-25.
43. A. Janin, J. Ang, S. Bhagat, R. Dhillon, J. Edwards, J. Macias-Guarasa, N. Morgan, B. Peskin, E. Shriberg, A. Stolcke, C. Wooters, and B. Wrede, "The ICSI meeting project: Resources and research," in Proc. Int. Conf. Acoust. Speech Signal Process., 2004, DOI: 10.1.1.63.1514.
44. AMI: Augmented Multi-Party Interaction. [Online]. Available: http://www.amiproject.org/
45. C. Busso, S. Hernanz, C. Chu, S. Kwon, S. Lee, P. Georgiou, I. Cohen, and S. Narayanan, "Smart room: Participant and speaker localization and identification," in Proc. Int. Conf. Acoust. Speech Signal Process., Philadelphia, PA, Mar. 2005, vol. 2, pp. 1117-1120.
46. V. Rozgíc, B. Xiao, A. Katsamanis, B. Baucom, P. G. Georgiou, and S. Narayanan, "A new multichannel multi modal dyadic interaction database," in Proc. InterSpeech Conf., 2010, pp. 27-30.
47. M. Black, D. Bone, M. Williams, P. Gorrindo, P. Levitt, and S. Narayanan, "The USC CARE corpus: Child-psychologist interactions of children with autism spectrum disorders," in Proc. 12th Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 1497-1500.
48. D. K. Oller, P. Niyogi, S. Gray, J. A. Richards, J. Gilkerson, D. Xu, U. Yapanel, and S. F. Warren, "Automated vocal analysis of naturalistic recordings from children with autism, language delay, and typical development," in Proc. Nat. Acad,. Sci. USA, 2010, DOI: http://10.1073/pnas. 1003882107.
49. A. Tsukada, M. Shino, M. Devyver, and T. Kanade, "Illumination-free gaze estimation method for first-person vision wearable device," in Proc. Int. Conf. Comput. Vis. Workshops, 2011, pp. 2084-2091.
50. R. Fletcher, K. Dobson, M. Goodwin, H. Eydgahi, O. Wilder-Smith, D. Fernholz, Y. Kuboyama, E. Hedman, M. Poh, and R. Picard, "iCalm: Wearable sensor and network architecture for wirelessly communicating and logging autonomic activity," IEEE Trans. Inf. Technol. Biomed., vol. 14, no. 2, pp. 215-223, Mar. 2010.
51. T. Chaspari, C.-C. Lee, and S. Narayanan, "Interplay between verbal response latency and physiology of children with autism during ECA interactions," in Proc. InterSpeech Conf., 2012.
52. E. Mower, M. Black, E. Flores, M. Williams, and S. Narayanan, "Design of an emotionally targeted interactive agent for children with autism," in Proc. IEEE Int. Conf. Multimedia Expo, 2011, DOI: 10.1109/ICME.2011. 6011990.
53. U. Mitra, B. Emken, S. Lee, M. Li, H. Vathsangam, D. Zois, M. Annavaram, S. Narayanan, D. Spruijt-Metz, and G. Sukhatme, "KNOWME: A case study in wireless body area sensor network design," IEEE Commun. Mag., vol. 50, no. 5, pp. 116-125, May 2012.
54. M. Li, V. Rozgic, G. Thatte, S. Lee, A. Emken, M. Annavaram, U. Mitra, D. Spruijt-Metz, and S. S. Narayanan, "Multimodal physical activity recognition by fusing temporal and cepstral information," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 18, no. 4, pp. 369-380, Aug. 2010.
55. A. Emken, M. Li, G. Thatte, S. Lee, M. Annavaram, U. Mitra, S. Narayanan, and D. Spruijt-Metz, "Recognition of physical activities in overweight Hispanic youth using knowme networks," J. Phys. Activity Health, vol. 9, no. 3, pp. 432-441, 2012.
56. J. Cassell, T. Bickmore, M. Billinghurst, L. Campbell, K. Chang, H. Vilhjalmsson, and H. Yan, "Embodiment in conversational interfaces: Rea," in Proc. Int. Conf. Human Factors Comput. Syst., Pittsburgh, PA, May 1999, pp. 520-527.
57. S. Narayanan and A. Potamianos, "Creating conversational interfaces for children," IEEE Trans. Speech Audio Process., vol. 10, no. 2, pp. 65-78, Feb. 2002.
58. J. Cassell and K. Ryokai, "Making space for voice: Technologies to support children's fantasy and storytelling," Pers. Ubiquitous Comput., vol. 5, no. 3, pp. 169-190, 2001.
59. W. Swartout, D. Traum, R. Artstein, D. Noren, K. B. Paul Debevec, J. Williams, A. Leuski, D. P. Shrikanth Narayanan, C. Lane, J. Morie, P. Aggarwal, M. Liewer, J.-Y. Chiang, J. Gerten, S. Chu, and K. White, "Virtual museum guides demonstration," in Proc. IEEE Spoken Lang. Technol. Workshop, 2010, pp. 163-164.
60. D. Wu, C. Courtney, B. Lance, S. S. Narayanan, M. Dawson, K. Oie, and T. D. Parsons, "Optimal arousal identification and classification for affective computing using physiological signals: Virtual reality stroop task," IEEE Trans. Affective Comput., vol. 1, no. 2, pp. 109-118, Jul.-Dec. 2010.
61. D. Traum, A. Roque, A. Leuski, P. Georgiou, J. Gerten, B. Martinovski, S. Narayanan, S. Robinson, and A. Vaswani, "Hassan: A virtual human for tactical questioning," in Proc. 8th SIGdial Workshop Discourse Dialogue, S. Keizer, B. Bunt, and T. Paek, Eds., Antwerp, Belgium, 2007, pp. 75-78.
62. X. Anguera Miro, S. Bozonnet, N. Evans, C. Fredouille, G. Friedland, and O. Vinyals, "Speaker diarization: A review of recent research," IEEE Trans. Audio Speech Lang. Process., vol. 20, no. 2, pp. 356-370, Feb. 2012.
63. C. Wooters and M. Huijbregts, "The ICSI RT07s speaker diarization system," Multimodal Technol. Percept. Humans, pp. 509-519, 2008.
64. G. Friedland, H. Hung, and C. Yeo, "Multi-modal speaker diarization of real-world meetings using compressed-domain video features," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2009, pp. 4069-4072.
65. V. Rozgic, K. Han, P. Georgiou, and S. Narayanan, "Multimodal speaker segmentation and identification in presence of overlapped speech segments," J. Multimedia, vol. 5, no. 4, pp. 322-331, 2010.
66. N. Checka, K. Wilson, M. Siracusa, and T. Darrell, "Multiple person and speaker activity tracking with a particle filter," in Proc. Int. Conf. Acoust. Speech Signal Process., 2004, vol. V, pp. 881-884.
67. D. Gatica-Perez, G. Lathoud, I. McCowan, J.-M. Odobez, and D. Moore, "Audio-visual speaker tracking with importance particle filters," in Proc. Int. Conf. Image Process., 2003, vol. III, pp. 25-28.
68. Z. Khan, T. Balch, and F. Dellaert, "MCMC-based particle filtering for tracking a variable number of interacting targets," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 11, pp. 1805-1918, Nov. 2005.
69. C. Busso, P. Georgiou, and S. Narayanan, "Real-time monitoring of participants' interaction in a meeting using audio-visual sensors," in Proc. Int. Conf. Acoust. Speech Signal Process., Honolulu, HI, Apr. 2007, pp. II-685-II-688.
70. S. Furui, "50 years of progress in speech and speaker recognition," Speech Commun., pp. 1-9, 2005.
71. P. G. Georgiou, M. P. Black, A. Lammert, B. Baucom, and S. S. Narayanan, "'That's aggravating, very aggravating': Is it possible to classify behaviors in couple interactions using automatically derived lexical features?" in Proc. Affective Comput. Intell. Interaction, Memphis, TN, 2011, pp. 87-96.
72. S. Kadambe and G. Boudreaux-Bartels, "Application of the wavelet transform for pitch detection of speech signals," IEEE Trans. Inf. Theory, vol. 38, no. 2, pp. 917-924, Mar. 1992.
73. P. Ghosh, A. Ortega, and S. Narayanan, "Pitch period estimation using multipulse model and wavelet transform," in Proc. InterSpeech Conf., Antwerp, Belgium, Aug. 2007, pp. 297-300.
74. D. Talkin, "A robust algorithm for pitch tracking (RAPT)," Speech Coding Synthesis, pp. 495-518, 1995.
75. T. Shimamura and H. Kobayashi, "Weighted autocorrelation for pitch extraction of noisy speech," IEEE Trans. Speech Audio Process., vol. 9, no. 7, pp. 727-730, Oct. 2001.
76. D. Wang and S. Narayanan, "Piecewise linear stylization of pitch via wavelet analysis," in Proc. 9th Eur. Conf. Speech Commun. Technol., 2005, pp. 3277-3280.
77. P. Ghosh and S. Narayanan, "Pitch contour stylization using an optimal piecewise polynomial approximation," IEEE Signal Process. Lett., vol. 16, no. 9, pp. 810-813, Sep. 2009.
78. C. d'Alessandro, P. Mertens et al., "Automatic pitch contour stylization using a model of tonal perception," Comput. Speech Lang., vol. 9, no. 3, p. 257, 1995.
79. A. Katsamanis, M. P. Black, P. G. Georgiou, L. Goldstein, and S. S. Narayanan, "SailAlign: Robust long speech-text alignment," in Proc. Very-Large-Scale Phonetics Workshop, Jan. 2011.
80. N. Morgan and E. Fosler-Lussier, "Combining multiple estimators of speaking rate," in Proc. Int. Conf. Acoust. Speech Signal Process., 1998, vol. 2, pp. 729-732.
81. D. Wang and S. Narayanan, "Robust speech rate estimation for spontaneous speech," IEEE Trans. Audio Speech Lang. Process., vol. 15, no. 8, pp. 2190-2201, Nov. 2007.
82. F. Tamburini and C. Caini, "An automatic system for detecting prosodic prominence in American English continuous speech," Int. J. Speech Technol., vol. 8, no. 1, pp. 33-44, 2005.
83. D. Wang and S. S. Narayanan, "An acoustic measure for word prominence in spontaneous speech," IEEE Trans. Audio Speech Lang. Process., vol. 15, no. 2, pp. 690-701, Feb. 2007.
84. A. Stolcke, E. Shriberg, R. Bates, M. Ostendorf, D. Hakkani, M. Plauche, G. Tur, and Y. Lu, "Automatic detection of sentence boundaries and disfluencies based on recognized words," in Proc. 5th Int. Conf. Spoken Lang. Process., 1998, DOI: 10.1.1.53.7127.
85. E. Shriberg, A. Stolcke, D. Hakkani-Tür, and G. Tür, "Prosody-based automatic segmentation of speech into sentences and topics," Speech Commun., vol. 32, no. 1, pp. 127-154, 2000.
86. D. Wang and S. Narayanan, "A multi-pass linear fold algorithm for sentence boundary detection using prosodic cues," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2004, vol. 1, pp. I-525-I-528.
87. K. Ross and M. Ostendorf, "Prediction of abstract prosodic labels for speech synthesis," Comput. Speech Lang., vol. 10, no. 3, pp. 155-185, 1996.
88. C. Wightman and M. Ostendorf, "Automatic labeling of prosodic patterns," IEEE Trans. Speech Audio Process., vol. 2, no. 4, pp. 469-481, Oct. 1994.
89. K. Chen, M. Hasegawa-Johnson, and A. Cohen, "An automatic prosody labeling system using ANN-based syntactic-prosodic model and GMM-based acoustic-prosodic model," in Proc. Int. Conf. Acoust. Speech Signal Process., 2004, vol. 1, pp. 509-512.
90. S. Ananthakrishnan and S. Narayanan, "Automatic prosodic event detection using acoustic, lexical, and syntactic evidence," IEEE Trans. Audio Speech Lang. Process., vol. 16, no. 1, pp. 216-228, Jan. 2008.
91. S. Ananthakrishnan and S. Narayanan, "An automatic prosody recognizer using a coupled multi-stream acoustic model and a syntactic-prosodic language model," in Proc. Int. Conf. Acoust. Speech Signal Process., 2005, pp. 269-272.
92. V. Rangarajan Sridhar, S. Bangalore, and S. Narayanan, "Exploiting acoustic and syntactic features for automatic prosody labeling in a maximum entropy framework," IEEE Trans. Audio Speech Lang. Process., vol. 16, no. 4, pp. 797-811, May 2008.
93. J. L. Austin, How to do Things With Words. Oxford, U.K.: Clarendon Press, 1962.
94. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Ess-Dykema, and M. Meteer, "Dialogue act modeling for automatic tagging and recognition of conversational speech," Comput. Linguist., vol. 26, no. 3, pp. 339-373, 2000.
95. D. Jurafsky, R. Bates, N. Coccaro, R. Martin, M. Meteer, K. Ries, E. Shriberg, A. Stolcke, P. Taylor, and C. V. Ess-Dykema, "Automatic detection of discourse structure for speech recognition and understanding," in Proc. IEEE Workshop Autom. Speech Recognit. Understand., Santa Barbara, CA, Dec. 1997, pp. 88-95.
96. A. Potamianos, S. Narayanan, and G. Riccardi, "Adaptive categorical understanding for spoken dialogue systems," IEEE Trans. Speech Audio Process., vol. 13, no. 3, pp. 321-329, May 2005.
97. V. Rangarajan Sridhar, S. Bangalore, and S. Narayanan, "Combining lexical, syntactic and prosodic cues for improved online dialog act tagging," Comput. Speech Lang., vol. 23, no. 4, pp. 407-422, 2009.
98. P. Amrhein, W. Miller, C. Yahne, M. Palmer, and L. Fulcher, "Client commitment language during motivational interviewing predicts drug use outcomes," J. Consult. Clin. Psychol., vol. 71, no. 5, pp. 862-878, 2003.
99. C.-C. Lee, M. P. Black, A. Katsamanis, A. Lammert, B. R. Baucom, A. Christensen, P. G. Georgiou, and S. Narayanan, "Quantification of prosodic entrainment in affective spontaneous spoken interactions of married couples," in Proc. InterSpeech Conf., 2010, pp. 793-796.
100. R. Levitan, A. Gravano, and J. Hirschberg, "Entrainment in speech preceding backchannels," in Proc. Annu. Meeting Assoc. Comput. Linguist., 2011, vol. 2, pp. 113-117.
101. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, "Emotion recognition in human-computer interaction," IEEE Signal Process. Mag., vol. 18, no. 1, pp. 32-80, Jan. 2001.
102. B. Schuller, A. Batliner, S. Steidl, and D. Seppi, "Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge," Speech Commun., vol. 53, pp. 1062-1087, 2011.
103. C. Busso, Z. Deng, S. Yildirim, M. Bulut, C. Lee, A. Kazemzadeh, S. Lee, U. Neumann, and S. Narayanan, "Analysis of emotion recognition using facial expressions, speech and multimodal information," in Proc. IEEE Int. Conf. Multimodal Interfaces, State College, PA, Oct. 2004, pp. 205-211.
104. A. Metallinou, S. Lee, and S. S. Narayanan, "Decision level combination of multiple modalities for recognition and analysis of emotional expression," in Proc. Int. Conf. Acoust. Speech Signal Process., Dallas, TX, 2010, pp. 2462-2465.
105. A. Metallinou, C. Busso, S. Lee, and S. S. Narayanan, "Visual emotion recognition using compact facial representations and viseme information," in Proc. Int. Conf. Acoust. Speech Signal Process., Dallas, TX, 2010, pp. 2474-2477.
106. A. Metallinou, M. Wollmer, A. Katsamanis, F. Eyben, B. Schuller, and S. Narayanan, "Context-sensitive learning for enhanced audiovisual emotion classification," IEEE Trans. Affective Comput., vol. 3, no. 2, pp. 184-198, Apr.-Jun. 2012.
107. M. Grimm, K. Kroschel, E. Mower, and S. Narayanan, "Primitives-based evaluation and estimation of emotions in speech," Speech Commun., vol. 49, no. 10-11, pp. 787-800, 2007.
108. E. Mower, A. Metallinou, C.-C. Lee, A. Kazemzadeh, C. Busso, S. Lee, and S. S. Narayanan, "Interpreting ambiguous emotional expressions," in Proc. Int. Conf. Affective Comput. Intell. Interaction, Amsterdam, The Netherlands, Sep. 2009, pp. 1-8, DOI: 10.1109/ACII.2009.5349500.
109. E. Mower, K. Han, S. Lee, and S. Narayanan, "A cluster-profile representation of emotion using agglomerative hierarchical clustering," in Proc. 11th Annu. Conf. Int. Speech Commun. Assoc., Makuhari, Japan, 2010, pp. 797-800.
110. E. Mower, M. Mataric, and S. Narayanan, "A framework for automatic human emotion classification using emotion profiles," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 5, pp. 1057-1070, Jul. 2011.
111. E. Mower and S. Narayanan, "A hierarchical static-dynamic framework for emotion classification," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2011, pp. 2372-2375.
112. C. Williams and K. Stevens, "Emotions and speech: Some acoustical correlates," J. Acoust. Soc. Amer., vol. 52, pp. 1238-1250, 1972.
113. K. Scherer, "Vocal communication of emotion: A review of research paradigms," Speech Commun., vol. 40, no. 1-2, pp. 227-256, Apr. 2003.
114. M. Bulut, S. Lee, and S. Narayanan, "Recognition for synthesis: Automatic parameter selection for resynthesis of emotional speech from neutral speech," in Proc. Int. Conf. Acoust. Speech Signal Process., Las Vegas, NV, Apr. 2008, pp. 4629-4632.
115. C. Busso, S. Lee, and S. Narayanan, "Analysis of emotionally salient aspects of fundamental frequency for emotion detection," IEEE Trans. Audio Speech Lang. Process., vol. 17, no. 4, pp. 582-596, May 2009.
116. B. Schuller, A. Batliner, D. Seppi, S. Steidl, T. Vogt, J. Wagner, L. Devillers, L. Vidrascu, N. Amir, and L. Kessous, "The relevance of feature type for automatic classification of emotional user states: Low level descriptors and functionals," in Proc. InterSpeech Conf., 2007, pp. 2253-2256.
117. C. Clavel, I. Vasilescu, L. Devillers, G. Richard, and T. Ehrette, "Fear-type emotion recognition for future audio-based surveillance systems," Speech Commun., vol. 50, no. 6, pp. 487-503, 2008.
118. M. Bulut, C. Busso, S. Yildirim, A. Kazemzadeh, C. M. Lee, S. Lee, and S. Narayanan, "Investigating the role of phoneme-level modifications in emotional speech resynthesis," in Proc. Eurospeech Conf., Lisbon, Portugal, Oct. 2005, pp. 801-804.
119. S. Yildirim, M. Bulut, C. Lee, A. Kazemzadeh, C. Busso, Z. Deng, S. Lee, and S. Narayanan, "An acoustic study of emotions expressed in speech," in Proc. 8th Int. Conf. Spoken Lang. Process., Jeju Island, Korea, 2004, pp. 2193-2196.
120. T. B̈anziger and K. Scherer, "The role of intonation in emotional expressions," Speech Commun., vol. 46, no. 3, pp. 252-267, 2005.
121. D. Ladd, K. Silverman, F. Tolkmitt, G. Bergmann, and K. Scherer, "Evidence for the independent function of intonation contour type, voice quality, and f0 range in signaling speaker affect," J. Acoust. Soc. Amer., vol. 78, no. 2, pp. 435-444, 1985.
122. F. Eyben, M. Ẅollmer, and B. Schuller, "Opensmile: The Munich versatile and fast open-source audio feature extractor," in Proc. Int. Conf. Multimedia, 2010, pp. 1459-1462.
123. C. Whissell, "The dictionary of affect in language," Emotion, Theory Res. Exp., vol. 4, pp. 113-131, 1989.
124. M. Bradley and P. Lang, Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings. Miami, FL: Univ. Florida, 1999.
125. N. Malandrakis, A. Potamianos, E. Iosif, and S. Narayanan, "Kernel models for affective lexicon creation," in Proc. 12th Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 2977-2980.
126. C. M. Lee, S. S. Narayanan, and R. Pieraccini, "Classifying emotions in human-machine spoken dialogs," in Proc. IEEE Int. Conf. Multimedia Expo, 2002, vol. 1, pp. 737-740.
127. C. Lee, S. Narayanan, and R. Pieraccini, "Combining acoustic and language information for emotion recognition," in Proc. 7th Int. Conf. Spoken Lang. Process., Denver, CO, Sep. 2002, pp. 873-876.
128. J. Liscombe, G. Riccardi, and D. Hakkani-Tür, "Using context to improve emotion detection in spoken dialog systems," in Proc. InterSpeech Conf., 2005, pp. 1845-1848.
129. C.-C. Lee, C. Busso, S. Lee, and S. Narayanan, "Modeling mutual influence of interlocutor emotion states in dyadic spoken interactions," in Proc. InterSpeech Conf., 2009, pp. 1983-1986.
130. C. Lee, E. Mower, C. Busso, S. Lee, and S. Narayanan, "Emotion recognition using a hierarchical binary decision tree approach," Speech Commun., vol. 53, pp. 1162-1171, 2011.
131. C. Busso, S. Lee, and S. Narayanan, "Using neutral speech models for emotional speech analysis," in Proc. InterSpeech Conf., Antwerp, Belgium, Aug. 2007, pp. 2225-2228.
132. M. Wöllmer, F. Eyben, S. Reiter, B. Schuller, C. Cox, E. Douglas-Cowie, and R. Cowie, "Abandoning emotion classes-towards continuous emotion recognition with modelling of long-range dependencies," in Proc. 9th Annu. Conf. Int. Speech Commun. Assoc., 2008, pp. 597-600.
133. M. Wollmer, B. Schuller, F. Eyben, and G. Rigoll, "Combining long short-term memory and dynamic Bayesian networks for incremental emotion-sensitive artificial listening," IEEE J. Sel. Top. Signal Process., vol. 4, no. 5, pp. 867-881, Oct. 2010.
134. M. Nicolaou, H. Gunes, and M. Pantic, "Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space," IEEE Trans. Affective Comput., vol. 2, no. 2, pp. 92-105, Apr.-Jun. 2011.
135. A. Metallinou, A. Katsamanis, Y. Wang, and S. Narayanan, "Tracking changes in continuous emotion states using body language and prosodic cues," in Proc. Int. Conf. Acoust. Speech Signal Process., 2011, pp. 2288-2291.
136. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Müller, and S. Narayanan, "Paralinguistics in speech and language: State-of-the-art and the challenge," Comput. Speech Lang., vol. 27, no. 1, pp. 4-39, 2012.
137. P. N. Juslin and K. R. Scherer, "Vocal expression of affect," in The New Handbook of Methods in Nonverbal Behavior Research, J. Harrigan, R. Rosenthal, and K. Scherer, Eds. Oxford, U.K.: Oxford Univ. Press, 2005, pp. 65-135.
138. J. M. Gottman and R. W. Levenson, "The social psychophysiology of marriage," Perspect. Marital Interaction, pp. 182-200, 1988.
139. J. T. Cacioppo, G. G. Berntson, J. F. Sheridan, and M. K. McClintock, "Multilevel integrative analyses of human behavior: Social neuroscience and the complementing nature of social and biological approaches," Psychol. Bull., vol. 126, no. 6, pp. 829-843, 2000.
140. A. M. Kring and D. Sloan, The Facial Expression Coding System (FACES): A Users Guide, Tech. Manual, 1991.
141. D. M. Sloan and A. M. Kring, "Measuring changes in emotion during psychotherapy: Conceptual and methodological issues," Clin. Psychol., Sci. Practice, vol. 14, no. 4, pp. 307-322, 2007.
142. A. L. Gonzales, J. T. Hancock, and J. W. Pennebaker, "Language style matching as a predictor of social dynamics in small groups," Commun. Res., vol. 37, no. 1, p. 3, 2010.
143. J. W. Pennebaker and C. K. Chung, "Expressive writing, emotional upheavals, and health," Found. Health Psychol., pp. 263-284, 2007.
144. R. E. Cromwell and D. H. L. Olson, Power in Families. New York: Sage, 1975.
145. R. Lakoff, "What you can do with words: Politeness, pragmatics and performatives," in Proc. Texas Conf. Performatives Presuppositions Implicatures, 1977, pp. 79-105.
146. G. Lakoff, "Pragmatics in natural logic," in Proc. Texas Conf. Performatives Presuppositions Implicatures, 1977, p. 107.
147. H. Sacks, E. Schegloff, and G. Jefferson, "A simplest systematics for the organization of turn-taking for conversation," Language, vol. 50, no. 4, pp. 696-735, 1974.
148. D. A. Coker and J. K. Burgoon, "The nature of conversational involvement and nonverbal encoding patterns," Human Commun. Res., vol. 13, no. 4, pp. 463-494, 1987.
149. K. Bousmalis, M. Mehu, and M. Pantic, "Spotting agreement and disagreement: A survey of nonverbal audiovisual cues and tools," in Proc. Affective Comput. Intell. Interaction, 2009, DOI: 10.1109/ACII.2009. 5349477.
150. E. Doohan, "Listening behaviors of married couples: An exploration of nonverbal presentation to a relational outsider," Int. J. Listening, vol. 21, no. 1, pp. 24-41, 2007.
151. A. Christensen, D. Atkins, S. Berns, J. Wheeler, D. Baucom, and L. Simpson, "Traditional versus integrative behavioral couple therapy for significantly and chronically distressed married couples," J. Consult. Clin. Psychol., vol. 72, no. 2, pp. 176-191, 2004.
152. J. Jones and A. Christensen, "Couples interaction study: Social support interaction rating system," Univ. California, Los Angeles, Tech. Manual, 1998.
153. C. Heavey, D. Gill, and A. Christensen, Couples Interaction Rating System 2 (CIRS2). Los Angeles, CA: Univ. California Press, 2002.
154. J. M. Gottman, M. J. Guralnick, B. Wilson, C. C. Swanson, and J. D. Murray, "What should be the focus of emotion regulation in children," Develop. Psychopathol., vol. 9, pp. 421-452, 1997.
155. B. R. Baucom, D. C. Atkins, L. E. Simpson, and A. Christensen, "Prediction of response to treatment in a randomized clinical trial of couple therapy: A 2-year follow-up," J. Consult. Clin. Psychol., vol. 77, no. 1, pp. 160-173, 2009.
156. M. Black, P. Georgiou, A. Katsamanis, B. Baucom, and S. Narayanan, "'You made me do it': Classification of blame in married couples' interaction by fusing automatically derived speech and language information," in Proc. InterSpeech Conf., Florence, Italy, Aug. 2011, pp. 89-92.
157. M. P. Black, A. Katsamanis, C.-C. Lee, A. Lammert, B. R. Baucom, A. Christensen, P. G. Georgiou, and S. S. Narayanan, "Automatic classification of married couples' behavior using audio features," in Proc. InterSpeech Conf., 2010, pp. 2030-2033.
158. P. K. Ghosh, A. Tsiartas, and S. S. Narayanan, "Robust voice activity detection using long-term signal variability," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 3, pp. 600-613, Mar. 2011.
159. B. Schuller, M. Wimmer, L. M̈osenlechner, C. Kern, D. Arsic, and G. Rigoll, "Brute-forcing hierarchical functionals for paralinguistics: A waste of feature space?" in Proc. Int. Conf. Acoust. Speech Signal Process., 2008, pp. 4501-4504.
160. C. C. Chang and C. J. Lin, LIBSVM: A library for support vector machines, 2001. [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm.
161. M. Black, A. Katsamanis, B. Baucom, C. Lee, A. Lammert, A. Christensen, P. Georgiou, and S. Narayanan, "Toward automating a human behavioral coding system for married couples? Interactions Using Speech Acoustic Features," Speech Commun., vol. 55, no. 1, pp. 1-21, 2012.
162. K. J. Williams-Baucom, D. C. Atkins, M. Sevier, K. A. Eldridge, and A. Christensen, "'You' and 'I' need to talk about 'us': Linguistic patterns in marital interactions," Pers. Relationships, vol. 17, no. 1, pp. 41-56, 2010.
163. C. A. Gubbins, L. M. Perosa, and S. Bartle-Haring, "Relationships between married couples' self-differentiation/individuation and Gottman's model of marital interactions," Contemporary Family Therapy, vol. 32, no. 4, pp. 383-395, 2010.
164. M. Kimura and I. Daibo, "Interactional synchrony in conversations about emotional episodes: A measurement by 'the between-participants pseudosynchrony experimental paradigm'," J. Nonverbal Behav., vol. 30, pp. 115-126, 2006.
165. J. Hirschberg, "Speaking more like you: Entrainment in conversational speech," presented at the 12th Annu. Conf. Int. Speech Commun. Assoc., Florence, Italy, 2011.
166. A. R. McGarva and R. M. Warner, "Attraction and social coordination: Mutual entrainment of vocal activity rhymes," J. Psycholinguist. Res., vol. 32, no. 3, pp. 335-354, 2003.
167. A. Nenkova, A. Gravano, and J. Hirschberg, "High frequency word
168. M. J. Richardson, K. L. Marsh, and R. C. Schmit, "Effects of visual and verbal interaction on unintentional interpersonal coordination," J. Exp. Psychol., Human Perception Performance, vol. 31, no. 1, pp. 62-79, 2005.
169. J. S. Pardo, "On phonetic convergence during conversational interaction," J. Acoust. Soc. Amer., vol. 119, pp. 2382-2393, 2006.
170. J. Edlund, M. Heldner, and J. Hirschberg, "Pause and gap length in face-to-face interaction," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc., 2009, pp. 2779-2782.
171. C.-C. Lee, A. Katsamanis, M. P. Black, B. Baucom, P. Georgiou, and S. Narayanan, "An analysis of PCA-based vocal entrainment measures in married couples' affective spoken interactions based vocal entrainment measures in married couples' affective spoken interactions," in Proc. InterSpeech Conf., Aug. 2011, pp. 3101-3104.
172. C. Lee, A. Katsamanis, M. Black, B. Baucom, P. Georgiou, and S. Narayanan, "Affective state recognition in married couples interactions using PCA-based vocal entrainment measures with multiple instance learning," in Proc. Int. Conf. Affective Comput. Intell. Interaction, 2011, pp. 31-41.
173. C.-C. Lee, A. Katsamanis, B. Baucom, and P. Georgiou, S. N. U. of Southern California"Using measures of vocal entrainment to inform outcome-related behaviors in marital conflicts," in Proc. Asia Pacific Signal Inf. Process. Assoc., 2012, pp. 1-5. [Online]. Available: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp= &arnumber= 6412027&isnumber=6411092.
174. P. Price, J. Tepperman, M. Iseli, T. Duong, M. Black, S. Wang, C. Boscardin, M. Heritage, P. David Pearson, S. Narayanan, and A. Alwan, "Assessment of emerging reading skills in young native speakers and language learners," Speech Commun., vol. 51, no. 10, pp. 968-984, 2009.
175. J. Mostow, G. Aist, C. Huang, B. Junker, R. Kennedy, H. Lan, D. T. Latimer, R. O'Connor, R. Tassone, B. Tobin, and A. Wierman, "4-month evaluation of a learner-controlled reading tutor that listens," in The Path of Speech Technologies in Computer Assisted Language Learning: From Research Toward Practice, F. N. F. V. M. Holland, Ed. New York: Routledge, 2008, pp. 201-219.
176. D. Litman and K. Forbes-Riley, "Predicting student emotions in computer-human tutoring dialogues," in Proc. 42nd Annu. Meeting Assoc. Comput. Linguist., 2004, pp. 351-358, article 351.
177. J. Tepperman, S. Lee, S. S. Narayanan, and A. Alwan, "A generative student model for scoring word reading skills," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 2, pp. 348-360, Feb. 2011.
178. A. Kazemzadeh, H. You, M. Iseli, B. Jones, X. Cui, M. Heritage, P. Price, E. Anderson, S. Narayanan, and A. Alwan, "Tball data collection: The making of a young children's speech corpus," in Proc. 9th Eur. Conf. Speech Commun. Technol., 2005, pp. 1581-1584.
179. M. Black, J. Tepperman, and S. Narayanan, "Automatic prediction of children's reading ability for high-level literacy assessment," IEEE Trans. Audio Speech Lang. Process., vol. 19, no. 4, pp. 348-360, May 2011.
180. M. Black and S. Narayanan, "Improvements in predicting children's overall reading ability by modeling variability in evaluators' subjective judgments," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2012, pp. 5069-5072.
181. S. D'Mello, A. Graesser, and R. Picard, "Toward an affect-sensitive autotutor," IEEE Intell. Syst., vol. 22, no. 4, pp. 53-61, Jul.-Aug. 2007.
182. A. Graesser and S. D'Mello, "Theoretical perspectives on affect and deep learning," New Perspectives Affect Learn. Technol., vol. 3, pp. 11-21, 2011.
183. M. Eskenazi, "An overview of spoken language technology for education," Speech Commun., vol. 51, no. 10, pp. 832-844, 2009.
184. D. Bone, M. P. Black, C.-C. Lee, M. E. Williams, P. Levitt, S. Lee, and S. Narayanan, "Spontaneous-speech acoustic-prosodic features of children with autism and the interacting psychologist," in Proc. InterSpeech Conf., 2012.
185. E. Prud'hommeaux, B. Roark, L. Black, and J. van Santen, "Classification of atypical language in autism," in Proc. Workshop Cogn. Model. Comput. Linguist., 2011, pp. 88-96.
186. American Psychiatric Association Task Force, Diagnostic and Statistical Manual of Mental Disorders: Dsm-iv-tr, 2000.
187. Centers for Disease Control and Prevention (CDC), National Center on Birth Defects and Developmental Disabilities. Autism Spectrum Disorders (ASDs), Mar. 2010. [Online]. Available: http://www.cdc.gov/ncbddd/autism/
188. G. Dawson, S. Rogers, J. Munson, M. Smith, J. Winter, J. Greenson, A. Donaldson, and J. Varley, "Randomized, controlled trial of an intervention for toddlers with autism: The early start Denver model," Pediatrics, vol. 125, no. 1, pp. 17-23, 2010.
189. K. Gotham, S. Risi, A. Pickles, and C. Lord, "The autism diagnostic observation schedule: Revised algorithms for improved diagnostic validity," J. Autism Develop. Disorders, vol. 37, no. 4, pp. 613-627, 2007.
190. S. Pepṕe, J. McCann, F. Gibbon, A. O'Hare, and M. Rutherford, "Receptive and expressive prosodic ability in children with high-functioning autism," J. Speech Lang. Hearing Res., vol. 50, no. 4, pp. 1015-1028, 2007.
191. L. Shriberg, R. Paul, J. McSweeny, A. Klin, D. Cohen, and F. Volkmar, "Speech and prosody characteristics of adolescents and adults with high-functioning autism and Asperger syndrome," J. Speech Lang. Hearing Res., vol. 44, no. 5, p. 1097, 2001.
192. M. C. Wallace, J. E. Cleary, E. H. Buder, W. Pettit, and D. K. Oller, "An acoustic inspection of vocalizations in young children with autism spectrum disorders," in Proc. Int. Meeting Autism Res., 2008.
193. E. Mower, C. Lee, J. Gibson, T. Chaspari, M. Williams, and S. Narayanan, "Analyzing the nature of ECA interactions in children with autism," in Proc. 12th Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 2989-2993.
194. T. Chaspari, E. M. Provost, A. Katsamanis, and S. Narayanan, "An acoustic analysis of shared enjoyment in ECA interactions of children with autism," in Proc. Int. Conf. Acoust. Speech Signal Process., Kyoto, Japan, 2012, pp. 4485-4488.
195. W. Miller, T. Moyers, D. Ernst, and P. Amrhein, Manual for the Motivational Interviewing Skill Code (MISC), 2003. [Online]. Available: http://casaa.unm.edu/download/misc.pdf.
196. B. Burke, C. Dunn, D. Atkins, and J. Phelps, "The emerging evidence base for motivational interviewing: A meta-analytic and qualitative inquiry," J. Cogn. Psychotherapy, vol. 18, no. 4, pp. 309-322, 2004.
197. D. Can, P. Georgiou, D. Atkins, and S. Narayanan, "A case study: Detecting counselor reflections in psychotherapy for addictions using linguistic features," in Proc. InterSpeech Conf., 2012.
198. B. Xiao, P. G. Georgiou, and S. Narayanan, "Analyzing the language of therapist empathy in motivational interview based psychotherapy," in Proc. Asia Pacific Signal Inf. Process. Assoc., 2012, pp. 1-4. [Online]. Available: http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6411762url= http%3A%2F%2Fieeexplore.ieee.org% 2Fstamp%2Fstamp.jsp%3Ftp%3D% 26arnumber%3D6411762.
199. G. Thatte, M. Li, S. Lee, A. Emken, S. Narayanan, U. Mitra, D. Spruijt-Metz, and M. Annavaram, "KNOWME an energy-efficient, multimodal body area network for physical activity monitoring," ACM Trans. Embedded Comput. Syst., vol. 11, no. S2, 2012, DOI: 10.1145/2331147. 2331158, article 48.
200. F. Enos and J. Hirschberg, "A framework for eliciting emotional speech: Capitalizing on the actors process," in Proc. 1st Int. Workshop Emotion: Corpora for Research on Emotion and Affect (Int. Conf. Lang. Resources Eval.), 2006, pp. 6-10.
201. T. B̈anziger and K. Scherer, "Using actor portrayals to systematically study multimodal emotion expression: The GEMEP corpus," Affective Comput. Intell. Interaction, vol. 4738, pp. 476-487, 2007.
202. C. Busso and S. Narayanan, "Recording audio-visual emotional databases from actors: A closer look," in Proc. Int. Conf. Lang. Resources Eval./Workshop Emotion: Corpora for Research on Emotion and Affect, Marrakech, Morocco, May 2008, pp. 17-22.
203. C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. Chang, S. Lee, and S. S. Narayanan, "IEMOCAP: Interactive emotional dyadic motion capture database," J. Lang. Resources Eval., vol. 42, no. 4, pp. 335-359, Dec. 2008.
204. A. Kazemzadeh, S. Lee, P. Georgiou, and S. Narayanan, "Emotion twenty questions: Toward a crowd-sourced theory of emotions," Affective Comput. Intell. Interaction, vol. 6975, pp. 1-10, 2011.
205. N. Ambady, F. Bernieri, and J. Richeson, "Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream," Adv. Exp. Soc. Psychol., vol. 32, pp. 201-271, 2000.
206. A. Katsamanis, J. Gibson, M. Black, and S. Narayanan, "Multiple instance learning for classification of human behavior observations," Affective Comput. Intell. Interaction, vol. 6974, pp. 145-154, 2011.
207. J. Gibson, A. Katsamanis, M. P. Black, and S. S. Narayanan, "Automatic identification of salient acoustic instances in couples' behavioral interactions using diverse density support vector machines," in Proc. InterSpeech Conf., Florence, Italy, 2011, pp. 1561-1564.
208. N. Mesgarani, M. Slaney, and S. Shamma, "Discrimination of speech from nonspeech based on multiscale spectro-temporal modulations," IEEE Trans. Audio Speech Lang. Process., vol. 14, no. 3, pp. 920-930, May 2006.
209. C. Siagian and L. Itti, "Rapid biologically-inspired scene classification using features shared with visual attention," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 2, pp. 300-312, Feb. 2007.
210. O. Kalinli and S. Narayanan, "Prominence detection using auditory attention cues and task-dependent high level information," IEEE Trans. Audio Speech Lang. Process., vol. 17, no. 5, pp. 1009-1024, Jul. 2009.
211. C. Busso and S. Narayanan, "Interrelation between speech and facial gestures in emotional utterances: A single subject study," IEEE Trans. Audio Speech Lang. Process., vol. 15, no. 8, pp. 2331-2347, Nov. 2007.
212. B. Xiao, V. Rozgic, A. Katsamanis, B. Baucom, P. G. Georgiou, and S. Narayanan, "Overlapped speech detection using long-term spectro-temporal similarity in stereo recording," in Proc. InterSpeech Conf., 2011.
213. C. C. Lee and S. Narayanan, "Predicting interruptions in dyadic spoken interactions," in Proc. Int. Conf. Acoust. Speech Signal Process., 2010, pp. 5250-5253.
214. C.-C. Lee, A. Katsamanis, M. P. Black, B. Baucom, A. Christensen, P. G. Georgiou, and S. S. Narayanan, "Computing vocal entrainment: A signal-derived PCA-based quantification scheme with application to affect analysis in married couple interactions," Comput. Speech Lang., 2012. [Online]. Available: http://dx.doi.org/10. 1016/j.csl.2012.06.006.