A realtime multimodal system for analyzing group meetings by combining face pose tracking and s peaker diarization

ICMI'08: Proceedings of the 10th International Conference on Multimodal Interfaces

Volumn , Issue , 2008, Pages 257-264

  Otsuka, Kazuhiro ^a   Araki, Shoko ^a   Ishizuka, Kentaro ^a   Fujimoto, Masakiyo ^a   Heinrich, Martin ^a   Yamato, Junji ^a  

^a Nippon Telegraph and Telephone Corporation   (Japan)

Author keywords

Face tracking; Fisheye lens; Focus of attention; Meeting analysis; Microphone array; Omnidirectional cameras; Realtime system; Speaker diarization

Indexed keywords

CAMERAS; DIRECTION OF ARRIVAL; INTERACTIVE COMPUTER SYSTEMS; LENSES; MICROPHONES; RADIO DIRECTION FINDING SYSTEMS; SENSORS; SPEECH RECOGNITION; THREE DIMENSIONAL; THREE DIMENSIONAL COMPUTER GRAPHICS; TRACKING (POSITION); VIDEO CAMERAS;

FACE TRACKING; FISHEYE LENS; FOCUS OF ATTENTION; MEETING ANALYSIS; MICROPHONE ARRAY; OMNIDIRECTIONAL CAMERAS; REALTIME SYSTEM; SPEAKER DIARIZATION;

FACE RECOGNITION;

EID: 60949097180     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1452392.1452446     Document Type: Conference Paper

References (24)

1. S. Araki, M. Fujimoto, K. Ishizuka, H. Sawada, and S. Makino. A DOA based speaker diarization system for real meetings. In Proc. HSCMA2008, pages 29-32, 2008.
2. M. Argyle. Bodily Communication - 2nd ed. Routledge, London and New York, 1988.
3. S. O. Ba and J.-M. Odobez. A study on visual focus of attention recognition from head pose in a meeting room. In Proc. MLMI2006, pages 75-87, 2006.
4. C. Busso, P. G. Georgiou, and S. S. Narayanan. Real-time monitoring of participants' interaction in a meeting using audio-visual sensors. In Proc. ICASSP2007, pages 685-688, 2007.
5. D. Douxchamps and N. Campbell. Robust real time face tracking for the analysis of human behaviour. In Proc. MLMI2007, pages 1-10, 2007.
6. M. Fujimoto, K. Ishizuka, and T. Nakatani. A voice activity detection based on the adaptive integration of multiple speech features and a signal decision scheme. In Proc. ICASSP2008, pages 4441-4444, 2008.
7. D. Gatica-Perez. Analyzing group interactions in conversations: a review. In Proc. IEEE Int. Conf. Multisensor Fusion and Integration for Intelligent Systems '06, pages 41-46, 2006.
8. D. Gatica-Perez, J.-M. Odobez, S. Ba, K. Smith, and G. Lathoud. Tracking people in meetings with particles. Technical Report IDIAP-RR 04-71, IDIAP, 2004.
9. A. Kendon. Some functions of gaze-direction in social interaction. Acta Psychologica, 26:22-63, 1967.
10. C. H. Knapp and G. C. Carter. The generalized correlation method for estimation of time delay. IEEE Trans. ASSP, 24(4):320-327, 1976.
11. L. Chen, et al. Vace multimodal meeting corpus. In Proc. MLMI2006, pages 40-51, 2006.
12. O. Mateo Lozano and K. Otsuka. Real-time visual tracker by stream processing. Journal of Signal Processing Systems, DOI 10.1007/s11265-008-0250-2, 2008.
13. O. Mateo Lozano and K. Otsuka. Simultaneous and fast 3D tracking of multiple faces in video by GPU-based stream processing. In Proc. ICASSP2008, pages 713-716, 2008.
14. Y. Matsusaka, H. Asoh, and F. Asano. Multi human trajectory estimation using stochastic sampling and its application to meeting recognition. In Proc. MVA2007, pages 16-18, 2007.
15. NIST Speech Group. Spring 2007 (RT-07) rich transcription meeting recognition evaluation plan. Technical Report rt07-meeting-eval-plan-v2, NIST, 2007.
16. K. Otsuka, Y. Takemae, J. Yamato, and H. Murase. A probabilistic inference of multiparty-conversation structure based on Markov-switching models of gaze patterns, head directions, and utterances. In Proc. ICMI'05, pages 191-198, 2005.
17. K. Otsuka and J. Yamato. Fast and robust face tracking for analyzing multiparty face-to-face meetings. In Proc. MLMI2008, 2008.
18. K. Otsuka, J. Yamato, and H. Murase. Conversation scene analysis with dynamic Bayesian network based on visual head tracking. In Proc. ICME'06, pages 949-952, 2006.
19. S. Renals, T. Hain, and H. Bourlard. Interpretation of multiparty meetings the AMI and AMIDA projects. In Proc. HSCMA2008, pages 115-118, 2008.
20. K. Smith, S. Schreiber, I. Potúcek, V. Beran, G. Rigoll, and D. Gatica-Perez. Real-time monitoring of participants' interaction in a meeting using audio-visual sensors. In Proc. MLMI2006, pages 88-101, 2006.
21. R. Stiefelhagen, J. Yang, and A. Waibel. Modeling focus of attention for meeting index based on multiple cues. IEEE Trans. Neural Networks, 13(4), 2002.
22. P. Viola and M. Jones. Robust real-time face detection. IJCV, 57(2):137-154, 2004.
23. M. Voit and R. Stiefelhagen. Tracking head pose and focus of attention with multiple far-field cameras. In Proc. ICMI2006, pages 281-286, 2006.
24. F. Wallhoff, M. Zobl, G. Rigoll, and I. Potucek. Face tracking in meeting room scenarios using omnidirectional views. In Proc. ICPR2004, 2004.